# Climate Adaptation Digital Twin: climatological analysis of temperature in Germany This notebook will provide you guidance on how to access and use Climate Adaptation Digital Twin (Climate DT) [datasets](https://dev.earthdatahub.bopen.eu/collections/climate-dt-2/datasets/IFS-NEMO-SSP3-7.0-sfc-hourly-standard) on [Earth Data Hub](https://earthdatahub.destine.eu) (EDH). Access to these datasets is granted via [Destination Earth](https://platform.destine.eu/)’s Standard API keys, but restricted to users with [Upgraded Access](https://platform.destine.eu/access-policy-upgrade). In this tutorial we will learn how to use Xarray to access the [Climate DT standard resolution hourly data on single-level or surface, IFS-NEMO, SSP3-7.0](https://dev.earthdatahub.bopen.eu/collections/climate-dt-2/datasets/IFS-NEMO-SSP3-7.0-sfc-hourly-standard) dataset on EDH. However, you can easily replicate the procedure for any other dataset belonging to the same collection. If you are running this tutorial in your own environment, you can download the convenience modules costing.py and display.py from [DESP-UserWorkflowService-Templates/EarthDataHub](https://github.com/SercoSPA/DESP-UserWorkflowService-Templates/tree/main/EarthDataHub) # What you will learn: - how to access the dataset - select and reduce the data - plot the results --- In this notebook we set two goals: Our first goal is to plot the average projeceted 2 metre temperature in 2026 over Central Europe (Germany). Our second goal is to compute the 2 metre temperature climatology (monthly means and standard deviations) in Berlin for the 2020-2049 reference period. # Working with EDH data Datasets on EDH are typically very large and remotely hosted. Typical use imply a selection of the data followed by one or more reduction steps to be performed in a local or distributed Dask environment. The structure of a workflow that uses EDH data typically looks like this: 1. Data access 2. Data selection 3. Data reduction (optional) 4. Data download 5. Further operations and visualisation [Xarray](https://docs.xarray.dev/en/stable) and [Dask](https://www.dask.org) work together following a lazy principle. This means that when you access and manipulate a Zarr store the data is in not immediately downloaded and loaded in memory. Instead, Dask constructs a task graph that represents the operations to be performed. A smart user will first reduce the amount of data that needs to be downloaded and explicitly call `compute()` on it. Once the `compute()` operation is complete the data is loaded into memory and available for subsequent fast processing. ## 1. Data access ```python # Optional: ignore warnings import warnings warnings.filterwarnings('ignore') ``` To access any data on EDH, you need to obtain an API key. To obtain an API key you need to register to the [DestinE platform](https://platform.destine.eu). Then, you can go to [Earth Data Hub Account Settings - Quota & API keys](https://dev.earthdatahub.bopen.eu/account-settings) where you can find out your default API key or create others. Finally, you need to instruct your tools to use your API key when downloading the data. One way to do it, is to incorporate the API key into the dataset’s url. If you wish to follow this method, cut and paste your personal API key below: ⤵ ```python EDH_KEY = "your_EDH_API_key" #e.g. EDH_KEY="edh_key_MHOQO_Y_h4ywpaMQe4cndtydvbpU6ySBNARWprm5tgz" url = f"https://edh:{EDH_KEY}@api.earthdatahub.destine.eu/climate-dt-2/IFS-NEMO-SSP3-7.0-sfc-hourly-standard-v0.zarr" ``` There are more ways in which you can provide your API key to Xarray, especially if you plan to use the system frequently. Please refer to [https://earthdatahub.destine.eu/getting-started](https://earthdatahub.destine.eu/getting-started) for a complete overview. We are now ready to access the data with Xarray: ```python # Optional: ignore warnings import warnings warnings.filterwarnings('ignore') ``` ```python import xarray as xr xr.set_options(keep_attrs=True) ds = xr.open_dataset( url, chunks={}, engine="zarr", zarr_format=3, ) ds ``` ```text Size: 22TB Dimensions: (time: 306816, latitude: 512, longitude: 1025) Coordinates: * time (time) datetime64[ns] 2MB 2015-01-01 ... 2049-12-31T23:00:00 * latitude (latitude) float64 4kB -90.0 -89.65 -89.3 ... 89.3 89.65 90.0 * longitude (longitude) float64 8kB -180.0 -179.6 -179.3 ... 179.6 180.0 Data variables: (12/36) avg_ie (time, latitude, longitude) float32 644GB dask.array avg_iews (time, latitude, longitude) float32 644GB dask.array avg_inss (time, latitude, longitude) float32 644GB dask.array avg_ishf (time, latitude, longitude) float32 644GB dask.array avg_sdlwrf (time, latitude, longitude) float32 644GB dask.array avg_sdswrf (time, latitude, longitude) float32 644GB dask.array ... ... tciw (time, latitude, longitude) float32 644GB dask.array tclw (time, latitude, longitude) float32 644GB dask.array tcw (time, latitude, longitude) float32 644GB dask.array tcwv (time, latitude, longitude) float32 644GB dask.array u10 (time, latitude, longitude) float32 644GB dask.array v10 (time, latitude, longitude) float32 644GB dask.array Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 1003 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts ``` ⚠ At this point, no data has been downloaded yet, nor loaded in memory. ## 2. Data selection First, we perform a geographical selection corresponding to the Germany area. This greatly reduces the amount of data that will be downloaded from Earth Data Hub. ```python t2m = ds.t2m t2m_germany = t2m.sel(**{"latitude": slice(47, 55), "longitude": slice(5, 16)}) t2m_germany ``` ```text Size: 875MB dask.array Coordinates: * time (time) datetime64[ns] 2MB 2015-01-01 ... 2049-12-31T23:00:00 * latitude (latitude) float64 184B 47.03 47.38 47.73 ... 54.07 54.42 54.77 * longitude (longitude) float64 248B 5.273 5.625 5.977 ... 15.12 15.47 15.82 Attributes: (12/20) GRIB_paramId: 167 GRIB_dataType: fc GRIB_numberOfPoints: 196608 GRIB_typeOfLevel: heightAboveGround GRIB_stepUnits: 1 GRIB_stepType: instant ... ... GRIB_shortName: 2t GRIB_units: K long_name: 2 metre temperature units: K standard_name: air_temperature last_ingested_time_index: 306815 ``` Second, we further select the year 2026. This is again a lazy operation: ```python t2m_germany_2026 = t2m_germany.sel(time="2026") t2m_germany_2026 ``` ```text Size: 25MB dask.array Coordinates: * time (time) datetime64[ns] 70kB 2026-01-01 ... 2026-12-31T23:00:00 * latitude (latitude) float64 184B 47.03 47.38 47.73 ... 54.07 54.42 54.77 * longitude (longitude) float64 248B 5.273 5.625 5.977 ... 15.12 15.47 15.82 Attributes: (12/20) GRIB_paramId: 167 GRIB_dataType: fc GRIB_numberOfPoints: 196608 GRIB_typeOfLevel: heightAboveGround GRIB_stepUnits: 1 GRIB_stepType: instant ... ... GRIB_shortName: 2t GRIB_units: K long_name: 2 metre temperature units: K standard_name: air_temperature last_ingested_time_index: 306815 ``` Due to the chunked structure of the DataArray, xarray must download every chunk that includes a portion of the selected data. To estimate the size of the download, we can use the `costing.py` module. This must be done before we apply any reduction operation. ```python import costing costing.estimate_download_size(t2m_germany_2026) ``` ```text Estimated download size: 0.035 GiB (36.0 MiB) ``` ## 3. Data reduction Before downloading the data, we can perform further lazy operations. Here we compute the 2 metre temperature yearly average: ```python t2m_germany_2026_mean = t2m_germany_2026.mean(dim="time") t2m_germany_2026_mean ``` ```text Size: 3kB dask.array Coordinates: * latitude (latitude) float64 184B 47.03 47.38 47.73 ... 54.07 54.42 54.77 * longitude (longitude) float64 248B 5.273 5.625 5.977 ... 15.12 15.47 15.82 Attributes: (12/20) GRIB_paramId: 167 GRIB_dataType: fc GRIB_numberOfPoints: 196608 GRIB_typeOfLevel: heightAboveGround GRIB_stepUnits: 1 GRIB_stepType: instant ... ... GRIB_shortName: 2t GRIB_units: K long_name: 2 metre temperature units: K standard_name: air_temperature last_ingested_time_index: 306815 ``` ## 3. Data download This is the phase where we explicitly trigger the download of the data. To do so we will call `compute()` on the previously averaged temperature. The result will be small enough to easily fit into memory. Remember to assign the return of the `compute()` function to a new variable, so that the data is kept in memory. We can measure the time it takes: ```python %%time t2m_germany_2026_mean_computed = t2m_germany_2026_mean.compute() ``` ```text CPU times: user 1.08 s, sys: 311 ms, total: 1.39 s Wall time: 2.32 s ``` The data was very small, this didn’t take long! ## 5. Visualisation Finally, we can plot the 2026 average temperature on a map: ```python import display import matplotlib.pyplot as plt from cartopy import crs ``` ```python t2m_germany_2026_mean_computed_degC = t2m_germany_2026_mean_computed - 273.15 t2m_germany_2026_mean_computed_degC.attrs["units"] = "°C" ``` ```python display.map( t2m_germany_2026_mean_computed_degC, projection=crs.Miller(), vmax=30, vmin=-20, cmap="RdBu_r", title="Mean Surface Temperature, 2026" ); ``` ![image](https://earthdatahub.destine.eu/docs/_images/notebooks_insula-code-lab_tutorial-climate-dt-climatological-analysis-of-temperature-in-germany_27_0.png) ### 2 metre temperature climatology in Berlin, 2020-2049 The power of EDH is better showned when working with timeseries. We will now show how fast it is to compute the 2 metre temperature climatology (monthly mean and standard deviation) in Berlin, over the reference period 2020-2049. With legacy data distributon systems you would need to download the entire world temperature for the reference time period, extract the Berlin data and do your calculations. Alternatively the data provider might do a crop and merge operation for you, but chances are that this is very slow (internally the data provider still accesses the entire world temperature, you just don’t see it!). Thanks to Earth Data Hub this is not needed anymore. You only need to download the relevant chunks. Here, we select the closest data to Berlin: ```python t2m_Berlin_2020_2049 = t2m.sel(**{"latitude": 52.5, "longitude": 13.4}, method="nearest").sel(time=slice("2020", "2049")) t2m_Berlin_2020_2049 ``` ```text Size: 1MB dask.array Coordinates: * time (time) datetime64[ns] 2MB 2020-01-01 ... 2049-12-31T23:00:00 latitude float64 8B 52.66 longitude float64 8B 13.36 Attributes: (12/20) GRIB_paramId: 167 GRIB_dataType: fc GRIB_numberOfPoints: 196608 GRIB_typeOfLevel: heightAboveGround GRIB_stepUnits: 1 GRIB_stepType: instant ... ... GRIB_shortName: 2t GRIB_units: K long_name: 2 metre temperature units: K standard_name: air_temperature last_ingested_time_index: 306815 ``` This is already small enough to be computed: We estimate the cost of the download with de `costing.py` module. ```python costing.estimate_download_size(t2m_Berlin_2020_2049) ``` ```text Estimated download size: 0.809 GiB (828.0 MiB) ``` We explicitly trigger the download of the data. Remember to assign the return of the `compute()` function to a new variable, so that the data is kept in memory. ```python %%time t2m_Berlin_2020_2049_computed = t2m_Berlin_2020_2049.compute() ``` ```text CPU times: user 11.6 s, sys: 1.41 s, total: 13 s Wall time: 55.8 s ``` Now that the data is loaded in memory we can easily compute the climatology: ```python t2m_Berlin_2020_2049_computed_degC = t2m_Berlin_2020_2049_computed - 273.15 t2m_Berlin_2020_2049_computed_degC.attrs["units"] = "°C" ``` ```python t2m_Berlin_climatology_mean = t2m_Berlin_2020_2049_computed_degC.groupby("time.month").mean(dim="time") t2m_Berlin_climatology_std = t2m_Berlin_2020_2049_computed_degC.groupby("time.month").std(dim="time") ``` We can finally plot the climatology in Berlin for the 2020-2049 reference period ```python plt.figure(figsize=(10, 5)) t2m_Berlin_climatology_mean.plot(label="Mean", color="#3498db") plt.errorbar( t2m_Berlin_climatology_mean.month, t2m_Berlin_climatology_mean, yerr=t2m_Berlin_climatology_std, fmt="o", label="Standard Deviation", color="#a9a9a9" ) plt.title("Surface Temperature climatology in Berlin, 2020-2049") plt.xticks(t2m_Berlin_climatology_mean.month) plt.xlabel("Month") plt.ylabel("Surface Temperature [°C]") plt.legend() plt.grid(alpha=0.3) plt.show() ``` ![image](https://earthdatahub.destine.eu/docs/_images/notebooks_insula-code-lab_tutorial-climate-dt-climatological-analysis-of-temperature-in-germany_40_0.png)