wiki:Testbed

Testing ORAC Code Updates

A Python 2.7 script has been produced to call the various ORAC executables and compare their outputs to those previously produced. It should be run before any commit to the repository to ensure there are no unforseen side effects. It can be found at http://proj.badc.rl.ac.uk/orac/browser#trunk/tools.

Regression test script

The primary script for testing changes to the ORAC code is regression.py. When called without arguments, it pre-processes six short segments of data, evaluates and merges liquid and ice phase cloud, and finally reports any changes in the files since the previous version of the ORAC code. These scripts are maintained by Adam Povey, who should be contacted if you have any issues.

Local defaults

You must create a local_defaults.py file in the script directory. An example is attached to this page. This defines where various auxilliary files are stored in your files system in order to minimise the number of arguments you normally pass the script.

Script arguments

The scripts has numerous optional arguments covering every option of the pre and post-processors and several other useful features. The most commonly used are:

Argument Short version Description
--help -h Print a full list of arguments and a description of their function.
--benchmark -B Produce a benchmark file for this current revision (i.e. don't compare the outputs).
--long -L Process the full data files, rather than just the short segments selected for regression testing.
--tests [] -t [] A list of regression tests to perform. The default tests are: DAYAATSRS, NITAATSRS, DAYMYDS, NITMYDS, DAYAVHRRS, NITAVHRRS. The full tests are: AATSR, DAYMYD, NITMYD, AVHRR.
--keep_driver -k Don't delete the driver files after processing.
--just_driver -J Don't call the executables; merely print the driver files to stdout.
--script_verbose -v Print the driver files and function calls to stdout.
--verbose -V Print full diagnostic information from all executables.
--revision [] -r [] The revision number used to label the files. This defaults to the svn revision number plus one, is written into the filenames and NCDF attributes, and is used to identify the previous data version when comparing.
--extra_lines [] Name of a file, the contents of which will be appended to the end of each main processing driver file. Used to set elements of Ctrl that aren't explicitly managed.
--chunking Chunk the post-processing to reduce memory overhead.
--clobber [] -C [] Controls which files are overwritten. 3=All (the default), 2=Pre-processing, 1=Pre and main processing, 0=None.
--timing Print how long each command took to run.
--sabotage Methodically invalidate channels until every possible combination of missing channels are available; a debugging mode.

If you wish to test the aerosol processing, the following arguments may be useful:

Argument Short version Description
--all_phases -A Process aerosol and cloud phases (rather than just cloud); requires two views.
--out_dir [] -o [] Directory to store output files.
--l1_land_mask Use the imager's land/sea mask, rather than that drawn from the USGS land-use database.
--use_oc Use the Ocean Colour CCI data product rather than assuming a constant chlorophyll concentration.

Regression test input data

An AATSR orbit, an AVHRR orbit, and day and night MODIS granules have been selected as test cases for ORAC. These were selected to contain cloud-covered land and sea on 20 June 2008. These data and all of the files necessary to process them can be found at http://proj.badc.rl.ac.uk/orac/browser#data (which can be stored separately from the main trunk on your local system). For the daytime MODIS scene, both a spectrally-subsetted and full LAADS version of the data are available.

Due to issues with the server timeout, it is not possible to place particularly large files into the repository. As such, two of the test files have been split into smaller segments. To reconstitute the original files, use the following commands:

cat AATSR.gz* | zcat > ATS_TOA_1PRUPA20080620_002337_000065272069_00345_32964_0666.N1
cat MODIS.gz* | zcat > MYD021KM.A2008172.0405.005.2009317014309.hdf

Regression test output data

The script will place the output files in the testoutput folder of the data directory set in the previous section. Each test is kept in a folder named after the test (e.g. 'DAYAATSRS' for the short daytime AATSR segment or 'AATSR' for the full orbit). The subsidiary output files are then further organised by:

  • pre) The pre-processor outputs (e.g. *.config.nc). Controlled by the --pre_dir argument.
  • main) The cloud-phase main processor outputs (WAT and ICE). Controlled by the --main_dir argument.
  • land and sea) The aerosol-phase main processor outputs (A70 to A79). Controlled by the --land_dir and --sea_dir arguments.

Stand-alone scripts

The python code can also be used independently of the regression tests to call all or part of the ORAC software package. The scripts are:

  • orac.py) Calls the pre, main, and post-processors in turn to evaluate a level 1B file (specified by the positional argument target). Most of the regression test optional arguments are available, in addition to:
Argument Short version Description
--phases [] -p [] The particle phases to be evaluated. Options are WAT, ICE, and A70-A79.
  • orac_preproc.py) Run the pre-processor on a given level 1B imager file. Useful optional arguments:
Argument Short version Description
--in_dir [] -i [] The directory in which the input file can be found. (This can be ommitted if the target argument is the full path for the file.)
--channel_ids [] -c [] The channel numbers to be included in the output file.
--limit X0 X1 Y0 Y1 -l [] A subset of the satellite swath to be output. (Indexed from 1.)
  • orac_main.py) Run the main processor on a set of pre-processed files sharing the given root filename. Useful optional arguments:
Argument Short version Description
--approach -a [] The retrieval approach to apply during retrieval. Options are CldWat, CldIce, AerOx, AerSw.
--land or --sea Process only land or sea pixels, respectively.
--cloud_only or --aerosol_only Process only cloudy or clear pixels, respectively.
  • orac_postproc.py) Run the postprocessor on a set of processed files sharing the given root filename. Useful optional arguments:
Argument Short version Description
--in_dir [] -i [] A list of directories in which the input files can be found.
--switch_phase When processing only cloud phases, check if the cloud top temperature is appropriate when selecting type.

Attachments