Lightning Talk: TerraKit: Standardising AI-Ready Geospatial Data... Rosie Lickorish & Romeo Kienzler

This talk introduces Terra Kit, an open source Python library designed for the curation of geospatial data sets. The speaker explains how traditional modeling techniques struggle with the vast and complex nature of geospatial data, whereas geospatial foundation models are pre-trained to provide a general purpose representation of this data. The session highlights the use of vision transformer architecture similar to that found in language models, which can effectively process earth observation data such as satellite imagery and climate data. Terra Kit simplifies the process of downloading and processing geospatial data through a few lines of code, making it accessible for users to build AI-ready datasets. The speaker demonstrates how Terra Kit connects with various data providers and emphasizes its capability to handle different modalities of data for fine-tuning models. The session concludes by showcasing the ease of use and powerful capabilities of Terra Kit in addressing the challenges of geospatial data processing.

Hello, thank you so much for joining us today. My name's Rosie and this is my colleague Romeo. We're here today to tell you about a new open source Python library called Terra Kit focused on geospatial data set curation. But before I tell you about Terra Kit, I'd like to just take a step back and tell you first a little bit about foundation models and why they are

changing our view of the world. While traditional modeling techniques are unable to handle the volume and complexity of geospatial data, geospatial foundation models are pre-trained to learn a general purpose representation of the data. They then be fine-tuned and any number of down downstream applications using a minimal number of labeled data points. And this is game-changing for geospatial modeling. These models use the same architecture, vision transformer architecture

as the language models we're all familiar with, but from this point onwards we're not talking about language models. We're talking about models trained on geospatial data. So that could be any kind of earth observation, satellite imaging from anywhere in the world. Could even be climate data or possibly even in situ measurements, anything that's got a geospatial element to it. So let's take you through a um a

quick example, quick look at how these geospatial foundation models can help us handle that massive amount of data using some of our open source tools that we've developed at IBM. Uh firstly with the Geo Studio where we're going to see in a moment, which is a tool for orchestrating and exploring geospatial models. With the Terra Kit library, the focus of today's talk, embedded and baked into this

um this the Geo Studio tool for the data preparation side of things. And Romeo is going to take you through that. Okay, now imagine you have something like a Google Maps which you can run privately with your data and your And we see here for example you can zoom in. This is a so-called Sentinel 2 tile. It's 5% of the area of India. It's live satellite data.

And on top of that tile you apply one of the pre-trained models and fine-tuned models. You see here the the size of the data set with the Himalayas in the background. And what you can do now is you can apply the model. And you see here in this example a crop map. So it's a semantic segmentation task and you see here colored in two different colors two

different crop types. And that way you have all at your fingertips without a single line of code. And this is something that would have been impossible or super challenging to do with a traditional geospatial modeling techniques. It would have taken months to handle this amount of data. But with foundation models it's it's hours or minutes even. And this particular one is using a a model we developed

in partnership with the European Space Agency uh called TerraMind uh that's focused on multimodal data. Flood detection disaster response calculating biodiversity are just a couple of examples of downstream applications that are benefiting from geospatial foundation models. Other examples include like we just saw uh segmentation of crop types greenhouse gas emission tracking detecting burn scars as well as identifying urban heat islands exploring where landslides are happening or

or maybe likely to take place. And also calculating biomass that can even let us understand where deforestation might be happening and potentially even exploring where there may be illegal mining taking With geospatial foundation models we can address the unique challenges and opportunities presented by the massive amount of earth observation data available to us. But, there's a problem. So, for example, ESA gives us 110 petabyte, NASA gives

us 130 petabyte. We have around 0.5 exabyte of data available in different formats, at different sources, at different clearance levels. So, what do we do, Rosie? Not to worry, Remi. I've got you covered, okay? So, that is exactly where Terrakit comes in. A new open-source Python library for allowing us to curate geospatial data sets. In just five lines of code, we can go from labels to an

AI-ready data set. Terrakit abstract abstracts away the API-specific details and enables users to download from any number of data providers. So, for example, uh it could be NASA Earth data, Sentinel Hub, or even the climate data from the Copernicus Climate Store. Terrakit makes discovering, processing, and retrieving geospatial data as easy as possible. So, let's take you through a specific example. Show you just how Terrakit can do

that data data processing pipeline for fine-tuning a foundation model. Here we have a burn scar outline from a particularly devastating wildfire event that took place in Portugal in 2025. 2024. So, we want to This is our This is our label. This is our starting point. So, how do we build up a data set from this starting point? Well, with Terrakit, we can easily connect using a one

of Terrakit's data connectors to, in this case, NASA Earth data to download a satellite image, an RGB image, um from NASA's archive. And that's just a single line of code with a specific config. Terrakit allows us to easily specify exactly how cloud-free we'd like that image to be and how many days before or after that event, depending on when the satellite image is available. Terrakit also gives

us detailed provenance information of where that data has been collected or downloaded and any processing that's taken place. But, we want to train a multimodal foundation model. So, what do we do next? Well, again, we do a slight change to our configuration and in that same line of code, we can interact with various different APIs. In this case, Sentinel Hub as well as NASA Sentinel archive AWS

archives to give us a couple of different modalities of data from that same label starting As well as that, Terrakit's extensible makes contributing access to new data connectors super simple with documentation, templates, abstract classes and also tests. So, that means that if you had a specific provider you wanted to access or interact with, Terrakit will make that super easy for you to do. The data is then

chips into subsections and packaged up ready for TorchGeo or Terratorch data loaders. From Sorry, over to you, Romeo. Okay. And so, now you have to download all the data and the labels and what you now want to do is you want to apply a geospatial foundation model, for example, Teramind, which we developed together with ESA. And this is the so-called multimodal foundation model, which allows you to

ingest different modalities like hyperspectral and um um radar data, for example, into the same model. For example, making use of the radar data if there is no cloud-free image available, you can use the radar data. And then, together with the label, you get a transferable fine-tuned model you can apply anywhere on the planet. And again, this isn't something that we could have done even just a few

years ago. By lowering the barrier barrier to entry, TerraKit is enabling a wider community of TorchGeo and TerraTorch users to interact with geospatial foundation models for Earth observation. that's particularly with some of the models that we have available on Hugging Face. So, for instance, the Prithvi Earth observation model developed in partnership with NASA, as well as our Granite geospatial ocean model, which I was super excited to

work on to develop to look at ocean color and look at develop fine-tuning for things like ocean primary productivity. So, we can tell how much carbon is being sequestered into the ocean. Here's a couple of cool things that some folks have done in the community with these models. So, for example, um our foundation models have been deployed on satellites and are being used to um identify and

detect and do cloud segmentation. They've also been um used for estimating ecosystem primary productivity and been found to um increase the accuracy of that by 20%. And finally, another example is actually detecting craters on the moon without explicitly being trained on that particular area. So, now we have this five lines of code in TerraKit and now we have a similar library which is called TerraTorch which gives

you the same five lines of code experience for fine-tuning models. So, it's using TorchTube behind the scenes and it's based on PyTorch Lightning and all you need to do is to specify configuration file, your data source, your labels, the models, and some parameters to the model, and you're done. So, we hope that we've persuaded you that geospatial foundation models are not only very cool, but also super

useful. And that with TerraKit, the headache of data processing is a thing of the past. So, thank you so much for listening. Um check out these both of our libraries, TerraKit as well as TerraTorch, uh add a star on GitHub, and um yeah, we are super excited to see you know what the community continues to develop yeah, as we bring this game-changing technology to to life. Thanks

so much for listening. Thank you. Yeah. Yeah, certainly the complexity is abstracted away to just a dot m file. So you put your keys for any of the data providers that need those into a dot m file and that's basically it. TerraKit takes care of the rest of it. The interaction is the same for any of those data providers and that's exactly what what TerraKit is giving

you. Thanks, great question. Any other questions? Okay. Oh, yeah. Yeah, exactly. That really challenging piece of of of processing and transforming. Yes, for the data providers that we have the data connectors we have we we have transformers that transformations that handle that. It's super challenging area and yes, that is exactly what TerraKit libraries is helping you with. Thank you. Thanks very much everyone for listening. Thank you.

Thank you.