WELCOME to the PRACTICAL EXERCISES - CLM-Community

WELCOME 

to the 

PRACTICAL EXERCISES

1. For the practical exercises you have got a TUTORIAL, 

with which you can work most of the time on your own. 

2. There are 8 lessons, in which you will learn about 

1. Installation of the COSMO-Model package 

2. Preparing External, Initial and Boundary Data 

3. Running the COSMO-Model in NWP / CLM mode 

4. Visualizing COSMO-Model output using GrADS (NWP) 

5. Visualizing COSMO-Model output using ncview / cdo (CLM) 

6. Running Idealized Test Cases 

7. Troubleshooting for the COSMO-Model 

8. Running COSMO-ART 

07.-11.02.2011 COSMO-CLM Training 2011 2

• To do the practical exercises, we have got 16 terminals, 

but we are about 30 trainees. So you have to share your 

terminal. 

• You need some knowledge in working with Linux / 

Unix and how to use a text-editor. If you are not familiar 

with this, you should look for an experienced partner. 


Installation 

• In this lesson you will learn 

– how to login to and use DWD‘s supercomputers, 

the NEC SX-8R, SX-9 and its frontend, the hpc 

– what are the necessary compontents of the 

COSMO-Model software package 

– how these components can be installed on a 

computer 

– how jobs can be started 

– what to do with the reference data 


Logging in 

• To the PC 

– There is a plastic sheet on your desk with the 

• Login name (Benutzername) 

• Password (Kennwort) 

• And the Domain (Domäne) 

• Start Exceed /Cygwin 

• Start a Putty session 

– For setting up this session, see the paper sheet on your desk 

• Log in to hpc.dwd.de 

– Set the DISPLAY variable to the IP-Address of your PC 

(See also extra sheet on your desk) 


The Computers at DWD 

• The hpc 

– Is a Linux Cluster, that is only used for compilations, starting jobs on the 

SX-8 or SX-9, postprocessing. We run no models on the hpc 

– File systems on the hpc: There are 3 different filesystems: 

• /e/uhome/trngxxx: for source code and run-scripts 

• /e/uwork/trngxxx: for long-term model data 

• /e/uscratch/trngxxx: for short-term model data 

• The NEC SX-8 and SX-9 

– INT2LM and the COSMO-Model run on the NEC machines. 

– There are 3 different SX-clusters: the SX-8 (west), the SX-8 (east) and the 

SX-9 (east). All machines are used by DWD researchers and external 

collaborators. 

– For this course, the SX-8 machines should be used. 

– Each SX-8 machine has 7 nodes with 8 processors each. 


The Computers at DWD 

• The batch system NQSII: 

– Jobs are submitted to the SX-nodes with the batch system NQSII (Network 

Queuing System II). 

– The basic commands are: 

• nqsub jobname: to submit the job described in jobname. 

• nqstat @machine: to query the status of your batch jobs on a 

specific machine (sx8esiox, sx8wsiox, sx9esiox0). 

• nqstat -Ps @machine: to query the status of all jobs on a machine 

• nqdel jobnr@machine: to delete the job with the number jobnr 

from a machine. jobnr is given by nqstat. 

Some more information on DWD‘s computer system 

can be found in Appendix A of the Tutorial. 


NWP Exercises 

• Files and Data for these exercises can be 

accessed via 

/e/uhome/fegast3/SPRING_SCHOOL_2011/ 

data 

docu 

grads 

ideal 

source 

topo 

troubleshooting 


Files and Data 

• You should copy to your home directory: 

- from source/: 

DWD-libgrib1_110128.tar.gz 

int2lm_101210_1.15.tar.gz 

cosmo_110121_4.17_beta.tar.gz 

• You should copy to your work or scratch directory 

- from data/ 

reference_data.tar.gz 

• Unzip and de-tar all files and try to install the programs 

according to the Tutorial 1.2 and 1.3 


Installation 

Take 10-15 minutes to read Chapter 1 of the 

TUTORIAL 


Installing the 

GRIB1 Library

GRIB 

• GRIB is a WMO standard for packing and 

distributing meteorological fields. 

– GRIB1: GRIdded Binary: still used in the COSMO 

Software based on DWD libgrib1. 

– GRIB2: General Regularly-distributed Information 

in Binary form: will be implemented in the 

COSMO Software in the near future based on 

ECMWF Grib-API (freely available from 

http://www.ecmwf.int/products/data/software). 


The Directory Structure 

• tar xvf DWD-libgrib1_110128.tar 

gives the directory 

• DWD-libgrib1_110128 with subdirs 

– include: directory with include files 

– README_grib: File with some more details 

– source: subdirectory with 

• all source files 

• Makefile 

• Go to directory 

DWD-libgrib1_110128/source 

and take a look to the Makefile (using a texteditor 

of your choice) 


The Makefile 

• You have to set the following variables 

• LIBPATHpath where the library is written to 

• INCDIR 

• AR 

• FTN 

• FCFLAGS 

• F90FLAGS 

• CC 

• CCFLAGS 

path for the include directory 

archiving utility (usually ar, but sxar for NEC 

cross-compiler) 

Call to the Fortran compiler (choose SX-8) 

Fortran compiler flags 

Fortran 90 compiler flags 

Call to the C-compiler 

C compiler flags 

• Save the changes, close the text-editor and type: make 


Big Endian / Little Endian 

• Most (Unix) supercomputers today use big-endian processors. 

This means that the most significant (biggest) byte is stored first. 

• Typical PC (Intel) Processors (with Linux) do it the other way 

round and store the less significant (smallest) byte first. This is 

called little endian. 

• Grib, by definition, is big-endian! 

• Example: every Grib record starts with 4 bytes, that contain the 4 

ASCII signs: G R I B 

– Representation in big endian: G R I B 

– Representation in little endian: B I R G 

• On little endian machines, a byte-swapping has to be done when 

reading Grib-files. This is implemented in the Grib library and 

can be switched on by defining –D__linux__ 


DWD Specialities for GRIB 

There are two DWD Grib specials that have to be taken care of: 

• Controlwords between Grib Records: Every Grib record begins 

with „GRIB“ and ends with „7777“. DWD Grib files contain 8 

extra bytes (a controlword) between Grib records. For a proper 

treatment of these controlwords, an environment variable has to 

be set: LIBDWD_FORCE_CONTROLWORDS=1 

• ASCII Bitmaps: Not the full data, but only a subset of the GME 

data is sent as driving data. The mapping of the data to the grid is 

done by a bitmap in ASCII format (by GRIB definition it should 

be binary). To indicate the format, another environment variable 

has to be set: LIBDWD_BITMAP_TYPE=ASCII 



INT2LM


• tar xvf int2lm_yymmdd_x.y.tar gives 

the directory 

• int2lm_yymmdd_x.y with subdirs and files 

– DOCS some documentations 

– LOCAL examples for different compilers 

– Makefile, ObjDependencies, ObjFiles 

– Fopts to specify the Compiler Options 

– runxxx2yyy Run scripts to start the program 

– src directory with the source code 

– obj directory with object files 

– work 


Makefile / Fopts 

In LOCAL are several examples for Fopts-files for 

different machines. Choose and / or adapt a proper 

one. 

• F90 Fortran compiler call 

• COMFLGx Options for Fortran compiler 

• LDPAR Linker call for parallel programs 

• LDSEQ Linker call for sequential programs 

• LDFLAG Options for linking 

• LIB Directory and name of Grib library 

Type make exe or gmake –j 8 exe to create the binary 



COSMO-Model


• tar xvf cosmo_yymmdd_x.y.tar gives the directory 

• cosmo_yymmdd_x.y with subdirs and files 

– DOCS some documentations 

– LOCAL examples for different compilers 

– Makefile, ObjDependencies, ObjFiles 

– Fopts to specify the options 

– run_cosmo_xx Run scripts to start the program 

– src directory with the source code 

– obj directory with object files 

– work 


The Fopts File 

In LOCAL are several examples for Fopts-files for 

different machines. Choose and / or adapt a proper 

one. 

• F90 Fortran compiler call 

• COMFLGx Options for Fortran compiler 

• LDPAR Linker call for parallel programs 

• LDSEQ Linker call for sequential programs 

• LDFLAG Options for linking 

• LIB Directory and name of Grib library 

Type make exe or gmake –j 8 exe to create the binary 


Runscripts 

• Runscripts take over most of the tasks necessary to 

start a program. For the INT2LM and the COSMO- 

Model, these tasks are: 

- Set number of processors and directory names 

- Set batch commands for the machine to run on 

- Cat together the INPUT files for Namelist input 

- Start the program 

- Do some clean up after the run 

• See Appendix A of the Tutorial for some more details 


Runscript for INT2LM 

• In the reference data set (dir: int2lm_7_output_ascii) 

you will find the runscript run_gme_2_cosmo_7 

• Among others you have to check / modify 

• LM_EXT: directory with external parameters for COSMO-Model 

domain 

• IN_EXT: directory with external parameters for coarse grid 

domain 

• IN_DIR: directory with input data from coarse grid model 

• LM_DIR: directory with output from INT2LM 

• cd to a directory of your own 

• mpirun –np $NPY path_of_binary 


Runscript for COSMO 

• In the reference data set (dir: cosmo_7_output_ascii) 

you will find the runscript run_cosmo_7 

• Among others you have to check / modify 

• INIDIR: directory with initial fields for COSMO-Model 

• BD_DIR: directory with boundary fields for COSMO-Model 

• OUTPUT: directory with output data for COSMO-Model 

• RESTAR: not used right now 

• cd to a directory of your own 

• mpirun –np $NPY path_of_binary 


Preparing Initial and 

Boundary Data

Up to now 

• Now you know how to modify the script variables and 

mpirun settings. 

• IN_DIR, INIDIR, BD_DIR, LM_DIR, 

OUTPUT, … 

• cd 

• mpirun 

• One more remark: after termination of a job, a log-file is 

written in the directory where you started the run script, 

e.g. gme2eu.oxxxxx 

• This log-file would contain error messages, if the job 

terminates abnormally 


Coming up next 

• Now you will try to modify the Namelist Input 

for INT2LM 

• Chapter 2 explains some of the most important 

Namelist variables and gives some background 

– LMGRID: The Model Domain 

– GRID_IN: Coarse Grid Model Data 

– DATA: Specifying Data Characteristics 

– CONTRL: Parameters to Control the INT2LM 


Choosing a domain 

• You first need an external parameter data set. See 

/e/uhome/fegast3/SPRING_SCHOOL_2011/topo 

for example data sets 

• To choose the coordinates of a start and an end point, 

take a look to a picture! 

• Choose a resolution and then the size of the domain 


Domains on a Rotated Grid 

Rotated grid on the sphere 

and a regional domain 


Choosing a domain 

• The domain is specified in the Namelist group 

/LMGRID/ 

• You have to modify 

- startlat_tot, startlon_tot: coordinates 

of the lower left grid point in rotated (!) coordinates 

- pollat, pollon: geographical coordinates of 

the rotated North Pole 

- dlon, dlat: resolution (0.0625 ~ 7 km; 0.025 ~ 

2.8 km) 

- ie(lm)_tot, je(lm)_tot, ke(lm)_tot: 

size of the model grid in grid points 


GME Data 

• For the INT2LM we prepared data from the 

GME for 2 dates (see Appendix C) 

- 24.12.2010: snow fall on Christmas Eve 

- 07.02.2011: actual weather 


About Runscripts 

• About run-scripts to use: 

- rungme2eu: for GME input data and 7 km resolution for the 

COSMO-Model 

- runeu2de: for COSMO_EU input data and 2.8 km resolution for the 

(fine-grid) COSMO Model 

- run_cosmo_eu: 7 km resolution 

- run_cosmo_de: 2.8 km resolution 

• Check the number of processors, directory names and batch 

settings. 

- NPX=1, NPY=4/8, 

- Computation of N1,NODES: SX-9 has 16 processors, SX-8R has 

only 8 processors per node 

- N1=expr $NP + 15 => expr $NP + 7 

- NODES= expr $N1 \/ 16 => expr $N1 \/ 8 


Running the 

COSMO-Model

Up to now 

• When working with the reference data, you 

learned to modify some directories 

– INIDIR, BD_DIR, OUTPUT, … 

– cd 

– mpirun 


For the COSMO Runs 

• Now you have to modify the Namelist input, namely /LMGRID/ 

and /RUNCTL/ 

– To specify the same domain as in INT2LM 

– To adapt length of the forecast 

• Possible Runscripts 

- run_cosmo_eu: 7 km resolution 

- run_cosmo_de: 2.8 km resolution (for advanced studies) 

• Check the number of processors, directory names and batch 

settings. 

- NPX=1, NPY=4/8, 

- Computation of N1,NODES: SX-9 has 16 processors, SX-8R has only 

8 processors per node 

- N1=expr $NP + 15 => expr $NP + 7 

- NODES= expr $N1 \/ 16 => expr $N1 \/ 8 


The Grid Analysis 

and Display System 

GrADS

About GrADS 

• GrADS is a rather easy-to-use tool to visualize fields from 

meteorolgical models 

• GrADS implements a 4-dimensional data model 

• A dataset must be described by a descriptor file (with 

extension .ctl), which defines (among others) 

– dset grib_filename 

– index another_filename.idx 

– tdef 13 linear 12Z21dec2009 

– xdef 129 linear 1 1 

– ydef 161 linear 1 1 

– zdef 40 levels 

10281 10024 etc. 

– VARS 140 

HSURF 0, 8, 1, 0 ** height of orography 

• You have to run gribmap –v -i test_m40.ctl to 

create the index file. 


Some Caveats 

• Definition of zdef: The definition on the slide 

before was for 40 full levels. If data on the 41 half 

levels are in the same file, they cannot be displayed 

correctly. 

• Grib tables and numbers: Every I/O variable in the 

model must have a unique Grib table and element 

number. But GrADS cannot deal with several Grib 

tables. Therefore, variables from different tables with 

the same element number cannot displayed correctly 

(e.g. U and QI) 


The Reference Output 

• For the GrADS session you can use the Grib Output 

of the Reference Data set. But best is to rerun the 

model with the following changes in the group 

/GRIBOUT/: 

– hcomb=0.0,12.0,1.0 

– eliminate QI from the output list yvarml 

• Concatenate all the Grib files of the output together: 

cat lfff00000000c lfff*00 > 

all_lfff_refdata 

• There is an example descriptor file in 

/e/uhome/fegast3/…/grads/test_m40.ctl 

• Note: There is an error in the Tutorial: the files 

ref_model.grb and ref_model.ctl do not exist. 


Helpful Tools 

• grib2ctl: produces a descriptor file for a 

given dataset, e.g. 

grib2ctl lfff00000000 

• can have problems with multiple time steps 

• wgrib: can perform actions on grib files, e.g. 

wgrib lfff00000000c 

• can extract single fields out of a grib file (see 

Tutorial). 


Troubleshooting

Troubleshooting 

• The run-scripts you will test during this 

session are written for the reference data set. 

• You will find the scripts in 

/e/uhome/fegast3/…/troubleshooting 

• Modify the settings for the directories and the 

binary and try to run the programs 


int2lmrun_1 

• UP BITMIN, DATEI: … 

KANN NICHT GEOEFFNET WERDEN 

• For (some) GME data, the INT2LM uses 

bitmaps, to identify a special domain on the 

sphere. To use the proper bitmaps, an 

environment variable must be set 

• export LIBDWD_BITMAP_TYPE=ASCII 

• And the Namelist switches ybitmap_cat, 

ybitmap_lfn must be set 


int2lmrun_2 

• Undefined GME values: Bitmap 

not set properly 

• Then the specified domain does not fit to the 

domain defined by the bitmap. 

• Wrong setting: startlat_tot = -10.0 

instead of -5.0 


int2lmrun_3 

• There are differences between the Namelist 

input and the specifications in the data file. 

• Compare: upper right lat 

• Wrong setting: startlat_tot = -40.0 

instead of -5.0 


uneu2de 

• Error in cuegin 

• This means: The Grib records could not be 

read 

• A wrong external parameter file for the coarse 

grid is specified: 

lm_d5_07000_965x773.sso.mol.g1 

• The domain of this file does not fit to the 

domain specified for the coarse grid in 

/GRID_IN/ 


un_cosmo_test_1 

• FOR_E, FOR_D are missing 

• l_forest not set correctly, either in 

INT2LM or in COSMO-Model 


un_cosmo_test_2 

• T_SO missing? 

• lan_t_so0 not set correctly. With this 

switch it can be chosen, whether T_SO is used 

from an analysis or an (interpolated) forecast. 

• For your applications, usually 

lan_t_so0=.TRUE. 


THE END 

Thank you for your Attention

WELCOME to the PRACTICAL EXERCISES - CLM-Community

Create successful ePaper yourself

Delete template?

Save as template?