ERA5 CDS upgrade (2024 Sep)

Climate Data Store (CDS)

Some time ago, an upgrade notice was officially released on the Copernicus Climate Change Service (C3S) website. The old interface was closed in September 2024. Currently, the download of ERA5 data is gradually upgraded from the old CDS interface system to the new CDS-Beta.

Climate Data Store

The Copernicus Climate Change Service (C3S) supports society by providing authoritative information about the past, present and future climate in Europe and the rest of the World.

• 15 May 2024 BETA version of the new CEMS Early Warning Data Store (EWDS). Your feedback is very useful for us. Please notice that access to the system might suffer some disruptions due to evolving updates.
• 16 Jun 2024 Check our informative page to best prepare yourself to use CDS-Beta.
• 16 Jun 2024 CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
• 16 Jun 2024 BETA version of the new CDS. Your feedback is very useful for us. Please notice that access to the system might suffer some disruptions due to evolving updates.
• 18 Jul 2024 Remember that you need to have an ECMWF account to use CDS-Beta. Your existing CDS credentials will not work in CDS-Beta!

Registration ECMWF account

registration address: https://www.ecmwf.int/ There is a Log in button in the upper right corner of the page, click on it directly!

CDS-beta account information supplement and terms upgrade

After successfully registering an ECMWF account, enter the page https://cds.climate.copernicus.eu/

We need to go to the upgraded CDS page to complete the information, including: - country, - affiliation, - thematic activities, - activities sectors, and

Then, three Accet Terms & Conditions. It’s all relatively simple! Finally, click Activate your profile to complete. Automatically return to the CDS-beta data customization home page.

Upgrade cdsapi library and .cdsapirc key

Note: We need to ensure that we have logged in to the CDS-beta account in advance!

Goto

https://cds.climate.copernicus.eu/how-to-api

Setup the CDS API personal access token

Here is how to setup the CDS API personal access token:

If you do not have an account yet, please register. If you are not logged in, please login. Once logged in, copy the code displayed below to the file - $HOME/.cdsapirc (in your Unix/Linux environment):

url: https://cds.climate.copernicus.eu/api
key: xxxxxx-xxxxx-xxxxx-xxxx-xxxx

Install the CDS API client

The CDS API client is a Python based library. It provides support for Python 3.

You can Install the CDS API client via the package management system pip, by running on Unix/Linux the command below.

$ conda install 'cdsapi>=0.7.0' # updated in future

and, (typing-extensions 4.9.0)

Version 0.7.0 or higher is required in order to be able to use the new data stores.

Use the CDS API client for data access

Once the CDS API client is installed, it can be used to request data from the datasets listed in the CDS, ADS, ECDS and CEMS Early Warning DS catalogues.

One must agree to the Terms of Use of a dataset before downloading any data out of it. This step must be done manually from the dataset page (at the bottom of the download form).

At the bottom of each dataset download form, press the "Show API request code" button to display a Python code snippet that can be used to run the manually built request. The API call must have this syntax:

import cdsapi

client = cdsapi.Client()

dataset = "<DATASET-SHORT-NAME>"
request = {
    <SELECTION-REQUEST>
}
target = "<TARGET-FILE>"

client.retrieve(dataset, request, target)

Example:

import cdsapi

client = cdsapi.Client()

dataset = 'reanalysis-era5-pressure-levels'
request = {
    'product_type': ['reanalysis'],
    'variable': ['geopotential'],
    'year': ['2024'],
    'month': ['03'],
    'day': ['01'],
    'time': ['13:00'],
    'pressure_level': ['1000'],
    'data_format': 'grib',
}
target = 'download.grib'

client.retrieve(dataset, request, target)

Issues

Recovering from connection error [HTTPSConnectionPool(host='cds.climate.copernicus.eu', port=443): Max retries exceeded with url: /api/retrieve/v1/processes/reanalysis-era5-pressure-levels/execute

• ValueError: Result not ready, job is accepted

and,

• https://forum.ecmwf.int/t/goodbye-legacy-climate-data-store-hello-new-climate-data-store-cds/6380/14

Anabelle
ECMWF Staff
Please be aware that as initially published in this topic, the cds-beta URL was planned to be discontinued and users were advised to update the URL in their .cdsapirc file by 20th October 2024.

We can confirm that the cds-beta URL is now discontinued. And this explains why some users are getting some SSL: CERTIFICATE_VERIFY_FAILED errors.

If you have not already done so, please update the URL in your .cdsapirc file as shown below:

url: https://cds.climate.copernicus.eu/api
key: xx-xx-xx-xx-xx

Alternatively, login [here](https://cds.climate.copernicus.eu/how-to-api) to check URL to use in .cdsapirc file.

Thank you

ECMWF Support
on behalf of the Data Stores team