Changes between Version 2 and Version 3 of T02_CSML/Csmlscan


Ignore:
Timestamp:
24/01/08 15:04:16 (12 years ago)
Author:
domlowe
Comment:

First draft of csml scanning notes complete.

Legend:

Unmodified
Added
Removed
Modified
  • T02_CSML/Csmlscan

    v2 v3  
    6464 
    6565=== [features] section: === 
     66'''''type:''''' The type of feature, e.g. !GridSeries, !PointSeries, !RaggedSection. See the [http://proj.badc.rl.ac.uk/ndg/browser/TI02-CSML/trunk/documentation/CSML%20User%27s%20Manual%2C%20v2.doc?format=raw CSML User Manual] for more info on feature types. 
     67 
     68'''''number:''''' How many !GridSeries features there are. Should be 'one' or 'many'. In CSML, each phenomenon e.g. Temperature, Precipitation is a separate feature. So for model data this will usually be 'many'. Note: will probably deprecate this option in the future, it shouldn't have to go in a config file!! But for now it's there. 
     69 
    6670=== [files] section: === 
     71'''''root:''''' The root level of your dataset directory (the data to be scanned). The scanner will also scan sub directories of this directory. 
     72 
     73'''''mapping:''''' This is a phrase to describe the mapping between the CSML features and the data files. Csmlscan has the concept of a representative file. e.g. In a typical model run with timesteps in separate file, one (netcdf) file is typical of all the others. The various maps available are: 
     74 
     75''onetomany:    ''  
     76onetomany means one file per directory/subdirectory is the representative file 
     77 
     78''onetoseveral:    '' 
     79Like onetomany, but a directory may contain several representative files e.g. 5 files containing one (or more) feature(s), and then 5 files containing one (or more) different feature(s) etc. Need to examine the contents of each file to see if it's like another. Might be slow on large datsets as it examines the files individually. 
     80 
     81''onetoone:    '' 
     82onetoone means each feature is self contained within any individual file 
     83 
     84''oneonly:    '' 
     85oneonly means one file represents feature spanning multiple directories - assumes no file in toplevel directory, and then lots of subdirectories at next level containing files 
     86 
     87These patterns are quite common and hopefully cover most scanning cases. However as there are other possible combinations of features and files, there is a framework within csmlscan for defining new mappings in the [http://proj.badc.rl.ac.uk/ndg/browser/TI02-CSML/trunk/csml/csmllibs/csmlfeaturefilemap.py source code], with the actual details of the mappings defined [http://proj.badc.rl.ac.uk/ndg/browser/TI02-CSML/trunk/csml/csmllibs/csmlfiles.py here]. Defining a new mapping is a development task though, not something that can be configured at the config file level.  
     88 
     89Model data is typically a onetomany or onetoseveral mapping. 
     90 
     91'''''output:''''' The name of the csml output file that will be created. Make sure you have the correct write permissions for the location of the file. 
     92 
     93'''''printscreen:''''' Whether or not to print out the resulting csml at the command line. 1 for printing, 0 for no printing. 
    6794=== [spatialaxes] section: === 
     95'''''spatialstorage:''''' Whether to store the spatial info (e.g. Latitude, Longitude, Height) values inline in the CSML file, or to use CSML arraydescriptors to reference the original data files. Can be 'inline' or 'fileextract' 
    6896=== [values] section: === 
     97'''''valuestorage:''''' Whether to store the rangeset values (e.g. the Temperature measurements) in the CSML file or reference the original data files. Can be 'inline' or 'fileextract' 
    6998=== [time] section: === 
     99'''''timedimension''''' Exact name of the time variable in the data file(s) e.g. 'time1'. The time dimension is given special treatment when scanning so this information helps identify it. If a value is not given the scanner will attempt to work out which is the time dimension but this method may fail to correctly identify it. 
    70100 
     101'''''timestorage:''''' Whether to store the time values in the CSML file or reference the original data files. Can be 'inline' or 'fileextract' 
     102 
     103 
     104== Scanning == 
     105Now save your config file as ''myconfigfile''.cfg and run the scanner. 
     106 
     107{{{ 
     108csmlscan -c ''myconfigfile''.cfg 
     109}}} 
     110 
     111 
     112The output should be a csml document!  
     113 
     114== Problems? == 
     115Things to check are: 
     116 
     117 1. The config file is correct and can be read by the scanner. 
     118 1. The data sources you are trying to read are correctly mounted and you have read permissions. 
     119 1. The config file again. 
     120 
     121If it still doesn't work, then please report a bug, preferably with example data and config file.