Reducing UM-UKCA data output using flight-track simulation
PI: Dr Nathan Luke Abraham (National Centre for Atmospheric Science (NCAS) & University of Cambridge)
Technical staff: Dr Maria Russo (National Centre for Atmospheric Science (NCAS) & University of Cambridge)
Published : 2023-01-25
Our main goal for this project was to allow for easy comparison between atmospheric model data from the United Kingdom Chemistry and Aerosols model used as part of the Met Office Unified Model (UM-UKCA) and observed data measured from research aircrafts.
This task previously required output of large amount of model data (on average, for a climate resolution run, 3 Gb of storage would be required for each model diagnostic per month), of which only a small part was used, making it inefficient in terms of data storage. Post-processing of this large model dataset (consisting of global, hourly files) into model data defined onto specific flight-tracks, was also very time intensive and required the model user to have an in-depth knowledge of aircraft data, as well as technical and coding skills to handle the retrieving and post-processing of large model datasets.
In order to achieve the above goal, we have developed a stand-alone ‘flight-track simulator’ code which deals with all the required data processing as the model runs. This allows for a large reduction in the data storage requirement from 3 Gb per model variable per month to ~15 Mb per model variable per month (a 200 times reduction in data). The processed model data, defined onto the observed flight-track, is also directly available after the model completion, therefore making it a lot quicker and easier for a wider variety of users to benefit from this extra tool to evaluate UM-UKCA against research aircraft data (see plot below for an example of observed vs modelled ozone). This new model capability also lends itself to large research projects where model simulations are used in conjunction with measurement campaigns to address specific science questions.
Although our original aim was to use this code exclusively with UM-UKCA, the flight-track simulator code has been designed and developed as a stand-alone code which can be used with any other atmospheric and climate model, therefore benefiting the wider atmospheric science community.
Comparison between observed (left) and modelled (right) ozone.
Information about the code
The Met Office Unified Model (UM) is a numerical model of the atmosphere used for both weather and climate applications. The Unified Model is available for use outside the Met Office through a licensing agreement aimed at meeting the needs of the user. A number of research organisations and national meteorological services use the Unified Model in collaboration with the Met Office to undertake basic atmospheric process research, produce forecasts, develop the Unified Model code and build and evaluate earth system models. More information on the use of the Unified Model, as well as the opportunities and support available for collaboration, can be found on the Unified Model Partnership page. For access to model output contact the Met Office enquiries team.
For an example UM job with the flight track simulator embedded into the UM workflow, see u-cn586. The original job without the flight track simulator is u-cn535.
Download the required python libraries on ARCHER2. From the home directory, download and install miniconda and install python libraries:
$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.sh -b
$ PATH=~/miniconda3/bin:$PATH conda install mamba -n base -c conda-forge -y
$ PATH=~/miniconda3/bin:$PATH mamba install cf-python cf-plot udunits2 mpich esmpy iris mo_pack ipython -c ncas -c conda-forge -y
$ PATH=~/miniconda3/bin:$PATH conda install -c conda-forge cis
$ PATH=~/miniconda3/bin:$PATH conda install numpy==1.22.0 -c conda-forge
If necessary, cd into the ~/miniconda3/lib/python3.*/site-packages/cf/umread_lib/c-lib directory and type “make” to build the UM pp-format file reading routines needed by cf-python.
Modify .bash_profile (or .bashrc etc.) to include the following lines:
and log out and back in again
Add your local miniconda to $PATH for your flight_track_sim by adding the following to the suite.rc file of your Rose suite.
To follow in a few months – currently under embargo.