To prepare the ERA5 GRIB file from the modified era5_with_signal.nc, you need to specify GRIB metadata such as dataDate, shortName, level, dataType, and typeOfLevel when converting era5_with_signal.nc back to GRIB format. The process involves using cdo and grib_set from ecCodes to properly define these GRIB headers.
Below is the detailed step-by-step guide:
Step 1: Check Metadata of the Original GRIB File
Inspect the original ERA5 GRIB file (era5.grib) to understand the required metadata.
1
grib_ls era5.grib
This will give you information about: - dataDate: The date of the data (e.g., 20250101 for January 1, 2025). - shortName: The short name of the variable (e.g., t2m for 2-meter temperature). - level: The model level, pressure level, or surface (e.g., 2 for surface data). - dataType: The type of data, such as an (analysis) or fc (forecast). - typeOfLevel: Specifies the level type (e.g., surface, isobaricInhPa, etc.).
Ensure you note these details.
Step 2: Convert era5_with_signal.nc to GRIB Format
Use cdo to convert the modified NetCDF file to GRIB format. You can specify metadata such as short names, levels, and GRIB options directly.
1
cdo -f grb copy era5_with_signal.nc era5_with_signal.grib
This creates a basic GRIB file, but it may not yet have all the required metadata (e.g., dataDate, shortName, typeOfLevel). You can fix those in the next steps.
Step 3: Assign GRIB Metadata
Use grib_set to modify GRIB metadata. First, inspect the GRIB file to see the current metadata:
1
grib_ls era5_with_signal.grib
If metadata like dataDate, shortName, or typeOfLevel is incorrect or missing, update them as follows:
Example 1: Set dataDate and shortName
Set the date and variable short name (e.g., for 2-meter temperature):
If your era5_with_signal.nc contains multiple variables (e.g., temperature t2m, wind speed u10, etc.), you need to split the file by variable, assign correct metadata, and merge them back into one GRIB file.
#!/bin/bash #------------------------------------------------# #Author: wpsze #Email： wpsze #date: 2025-02-06 08:41:43 #Version: 0.0 #Description: The purpose of the script #Copyright (C)： 2025 All rights reserved #------------------------------------------------#
#================= PL: t ==================================== cdo selname,t new_ERA5-0p25-PL-${yyyymmddhh}.nc t_nan.nc
cdo setmissval,-999 t_nan.nc t.nc
# cdo pack t.nc t_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy t.nc t_lev.grib cdo invertlev t_lev.grib t.grib
#================= PL: r ==================================== cdo selname,r new_ERA5-0p25-PL-${yyyymmddhh}.nc r_nan.nc
cdo setmissval,-999 r_nan.nc r.nc
# cdo pack r.nc r_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy r.nc r_lev.grib cdo invertlev r_lev.grib r.grib
#------------------------------------------------# #Author: wpsze #Email： wpsze #date: 2025-02-06 08:41:43 #Version: 0.0 #Description: The purpose of the script #Copyright (C)： 2025 All rights reserved #------------------------------------------------#
#================= PL: t ==================================== cdo selname,t new_ERA5-0p25-PL-${yyyymmddhh}.nc t_nan.nc
cdo setmissval,-999 t_nan.nc t.nc
# cdo pack t.nc t_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy t.nc t_lev.grib cdo invertlev t_lev.grib t.grib
#================= PL: r ==================================== cdo selname,r new_ERA5-0p25-PL-${yyyymmddhh}.nc r_nan.nc
cdo setmissval,-999 r_nan.nc r.nc
# cdo pack r.nc r_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy r.nc r_lev.grib cdo invertlev r_lev.grib r.grib
#================= PL: u ==================================== cdo selname,u new_ERA5-0p25-PL-${yyyymmddhh}.nc u_nan.nc
cdo setmissval,-999 u_nan.nc u.nc
# cdo pack u.nc u_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy u.nc u_lev.grib cdo invertlev u_lev.grib u.grib
#================= PL: v ==================================== cdo selname,v new_ERA5-0p25-PL-${yyyymmddhh}.nc v_nan.nc
cdo setmissval,-999 v_nan.nc v.nc
# cdo pack v.nc v_test.nc
# How to ensure the order of levels # encodeBMS_float : Missing value = NaN is unsupported! cdo -f grb copy v.nc v_lev.grib cdo invertlev v_lev.grib v.grib
NaN is not allowed in the grib standard, but you can change it with cdo:
1 2
[ram@thingol:~/tar/downloads]cdo setmissval,-9.e38 oscar_vel7403.nc t.nc cdo setmissval: Processed 2310724 values from 4 variables over 1 timestep ( 0.12s )
and then convert it to grib,
1 2 3
[ram@thingol:~/tar/downloads]cdo -f grb copy t.nc oscar_vel7403.grb cgribexDefLevel : Changed zaxis type from generic to pressure cdo copy: Processed 2310724 values from 4 variables over 1 timestep ( 0.09s )
Some lines on grib and netcdf:
GRIB1/GRIB2 are international standards for climate data sets defined by the WMO. Every data set has to have some identifiers (mostly integers), which describe what the data is (physical variable,unit,...). See here for a docu on the definitions. For example, NaN is not allowed for a missing value in the GRIB1 standard. In contrast to this Netcdf is self-explanatory: The user can defined, which name, unit or other (freely definable) attributes a certain variable should have. That's why it's not always possible to convert netcdf to grib.
NetCDF: Numeric conversion not representable
1 2 3 4 5 6 7 8 9 10 11 12 13 14
$ ncview sst.nc Ncview 2.1.7 David W. Pierce 29 March 2016 http://meteora.ucsd.edu:80/~pierce/ncview_home_page.html Copyright (C) 1993 through 2015, David W. Pierce Ncview comes with ABSOLUTELY NO WARRANTY; for details type `ncview -w'. This is free software licensed under the Gnu General Public License version 3; type `ncview -c' for redistribution details.
Note: no Ncview app-defaults file found, using internal defaults netcdf_fi_get_data: error on nc_get_vara_float call cdfid=65536 variable=sst start, count: [0]: 0 721 [1]: 0 1440 NetCDF: Numeric conversion not representable
Packing reduces the data volume by reducing the precision of the stored numbers. In NetCDF it is implemented using the attributes add_offset and scale_factor. CDO supports NetCDF files with packed data but can not automatically repack the data. That means the attributes add_offset and scale_factor are never changed. If you are using a CDO operator which change the range of the data you also have to take care that the modified data can be packed with the same add_offset and scale_factor. Otherwise the result could be wrong. You will get the following error message if some data values are out of the range of the packed datatype:
Error (cdf_put_vara_double) : NetCDF: Numeric conversion not representable
In this case you have to change the data type to single or double precision floating-point. This can be done with the CDO option -b F32 or -b F64.
As of CDO release 2.3.0, NetCDF data is always written out unpacked if the operator changes the range of the data. Use the new operator pack to repack the data if required:
cdo pack -<operator> infile outfile
1 2 3 4 5
$ cdo pack sst.nc sst_test.nc cdo pack: Processed 1038240 values from 1 variable over 1 timestep [0.29s 54MB].
-rw-rw-r-- 1 wpsze wpsze 8.0M Feb 6 11:16 sst.nc -rw-rw-r-- 1 wpsze wpsze 2.1M Feb 6 11:51 sst_test.nc
Packing reduces the data volume by reducing the precision of the stored numbers