Changeset 3352


Ignore:
Timestamp:
10/02/08 19:41:41 (12 years ago)
Author:
astephen
Message:

Got all versions so the re-factoring notes can be start of wiki page on changes made.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • nappy/trunk/nappy/Plans for re-factoring 2008.txt

    r3351 r3352  
    3434    * FFI 2110 is best for 2D columns for Excel etc. - Nappy must support this  
    3535 
     36-------------------- 
     37 
     38 
     39Dear Ag Stephans 
     40 
     41I found your nappy library for reading Ames Files at http://home.badc.rl.ac.uk/astephens/software/nappy/ 
     42For one of my projects I want to use that lib in a plugin for a wiki (http://moinmo.in) to give some feedback about submitted/attached data files. 
     43 
     44Your web page states last modified at Wed, 06 Apr 2005 13:44:53 GMT 
     45 As I have seen on http://home.badc.rl.ac.uk/astephens/software/nappy/USAGE.txt 
     46"[NOTE: We plan to implement a getVariableArray(var_name) to grab a specific variable from the above array.]" 
     47 
     48I like to ask if there is probably a further version of that lib available?  
     49 
     50cheers 
     51Reimar Bauer 
     52 
    3653---------------------- 
    3754 
     
    92109 
    93110We then need a set of nc_interface tests as well! Need NetCDF files to convert the other way as well. Need to be small. 
     111 
     112 
     113------- 
     114 
     115= Changes needed to make Nappy fit for purpose = 
     116 
     117 
     118 * Need error checking throughout - making it more robust but also checking on read will make Nappy a useful NASA Ames checker replacement. 
     119 
     120 * cdms interface should be in a separate part of the stack, not in core classes AND SUB-CLASSES 
     121 
     122 * Need to make an egg of it, but in the meantime we also need it installed as: 
     123 
     124   * nappy-0.2.3 
     125     * nappy 
     126     * bin 
     127     * nappy/nc_interface 
     128     * nappy/cdms_utils/ 
     129     * nappy/utils 
     130     * nappy/na_file 
     131     * nappy/contrib/aircraft 
     132     
     133 
     134 * GET REST FROM WIKI! 
     135 
     136 
     137=============== 
     138localRules.py --> REMOVE COMPLETELY 
     139localRules/blah - remove this and put stuff in a top-level config file, or even remove altogether. 
     140 
     141localRules/aircraft.py - Need to consider how this can be a sub-class of Cdms2NA so that we push all the odd code into separate modules. Need to encapstulate the differences into one or two methods that are small and can be overridden. 
     142 
     143====== 
     144bin/scanFAAM.py - put in contrib 
     145 
     146====== 
     147 
     148version.py - put in config file. 
     149 
     150====== 
     151 
     152general.py --> call it utils/xxxxx.py 
     153 
     154textParser --> utils/text_parser.py 
     155 
     156naError.py --> na_error/na_error.py 
     157 
     158naCore.py --> na_file/na_core.py 
     159 
     160listManipulator -_> utils/list_manipulator.py 
     161 
     162cdmsMap.py --> put in config file given simplicity 
     163 
     164Need utils/parse_config.py 
     165 
     166====== 
     167 
     168CDMS stuff is most of the mess 
     169============================== 
     170 
     1711. naToCdms.py holds: 
     172 
     173AbstractNAToCdms CLASS 
     174toCdmsFile 
     175createCdmsVariables 
     176toCdmsVariable 
     177createCdmsAuxVariables 
     178auxToCdmsVariable 
     179createCdmsAxes 
     180toCdmsAxes 
     181 
     1822. na2cdms.py: 
     183 
     184Command-line script 
     185 
     1863. bin/na2nc: 
     187 
     188Same as na2cdms.py ??? 
     189 
     1904. cdms2na.py is the mother of all modules: 
     191 
     192compareAxes --> areAxesIdentical(a,b) cdms_utils 
     193compareVariables --> areDomainsIdentical(v1, v2) cdms_utils 
     194isAuxAndVar --> isAuxVarAndVar  
     195arrayToList utils 
     196listOfListsCreator utils 
     197getBestName cdms_utils - need some advice and compare with Dom 
     198getMissingValue cdms_utils 
     199fixHeaderLengthNowDefunct # Can destroy 
     200flatten2DTimeData aircraft 
     201modifyNADictCopy - needs a better name as it is specific 
     202cdms2na - 200 lines of code to do main conversion, needs to be split out into other stuff. 
     203 * getVariableCollections(f and varlist) --> (ordered_vars, other_vars) 
     204 * buildNADicts() 
     205 * writeToOutputFiles() 
     206 
     207class CdmsToNABuilder --> NAContentCollector: (naDict, varIDs, varBin) 
     208__init__ --> sets everything up and runs it move some to --> analyse() 
     209analyseVariables 
     210defineNAVars 
     211defineNAAuxVars 
     212getAxisDefinition 
     213defineNAGlobals 
     214defineNAComments 
     215defineGeneralHeader 
     216_useLocalRule --> Remove this and put it all in aircraft contrib bit 
     217 
     218================= 
     219 
     220naToCdms.py 
     221=========== 
     222 
     223This is a sub-class of all NAFile objects. Bad idea. What we need is to: 
     224 
     225import convertor 
     226convertor.writeToNC(blah) 
     227convertor.convertToCdms(blah): (vars, global_atts) 
     228 
     229class NAToCdms 
     230 
     231toCdmsFile 
     232createCdmsVariables - does all 
     233toCdmsVariable - does each in turn 
     234CreateCdmsAuxVariables - does all 
     235auxToCdmsVariables - does each in turn 
     236createCdmsAxes - does all 
     237toCdmsAxes - does each in turn 
     238 
     239================ 
     240 
     241Have I broken textParser.py's main function - is it same in old and new. 
     242 
     243===== 
     244 
     245NAFile2010: 
     246_readData[12] are crying out for useful names 
     247 
     248GREP 
     249==== 
     250 
     251Need to do a lot of grepping for inconsistencies. 
     252 
     253The following need to be set to True or False (not yes,no): 
     254 * _normalizedX  
     255 * time_warning 
     256 
     257Global find and replace: 
     258 
     259floatFormat 
     260naDict 
     261===================== 
     262 
     263In naToCdms.py.NAToCdms.toCdmsAxis() there is a line naming the id (if too long) as: 
     264 
     265naAuxVariable.... 
     266 
     267 - need to find where else this string is used and replace all with "naIndVariable" with map-back! 
     268 
     269=================== 
     270 
     271Should we leave in the interactive time units checker in na_to_cdms.py - ask Charles 
     272 
     273===== 
     274cdms_map is not all done in the config file dict. 
     275 
     276Unit tests 
     277========== 
     278 
     2791001 appears to be done. Others are there as stubs but all need writing. 
     280 
     281We then need a set of nc_interface tests as well! Need NetCDF files to convert the other way as well. Need to be small. 
     282 
     283=============== 
     284 
     285= Changes needed to make Nappy fit for purpose = 
     286 
     287 
     288 * Need error checking throughout - making it more robust but also checking on read will make Nappy a useful NASA Ames checker replacement. 
     289 
     290 * cdms interface should be in a separate part of the stack, not in core classes AND SUB-CLASSES 
     291 
     292 * Need to make an egg of it, but in the meantime we also need it installed as: 
     293 
     294   * nappy-0.2.3 
     295     * nappy 
     296     * bin 
     297     * nappy/nc_interface 
     298     * nappy/cdms_utils/ 
     299     * nappy/utils 
     300     * nappy/na_file 
     301     * nappy/contrib/aircraft 
     302     
     303 
     304 * GET REST FROM WIKI! 
     305 
     306 
     307=============== 
     308localRules.py --> REMOVE COMPLETELY 
     309localRules/blah - remove this and put stuff in a top-level config file, or even remove altogether. 
     310 
     311localRules/aircraft.py - Need to consider how this can be a sub-class of Cdms2NA so that we push all the odd code into separate modules. Need to encapstulate the differences into one or two methods that are small and can be overridden. 
     312 
     313====== 
     314bin/scanFAAM.py - put in contrib 
     315 
     316====== 
     317 
     318version.py - put in config file. 
     319 
     320====== 
     321 
     322general.py --> call it utils/xxxxx.py 
     323 
     324textParser --> utils/text_parser.py 
     325 
     326naError.py --> na_error/na_error.py 
     327 
     328naCore.py --> na_file/na_core.py 
     329 
     330listManipulator -_> utils/list_manipulator.py 
     331 
     332cdmsMap.py --> put in config file given simplicity 
     333 
     334Need utils/parse_config.py 
     335 
     336====== 
     337 
     338CDMS stuff is most of the mess 
     339============================== 
     340 
     3411. naToCdms.py holds: 
     342 
     343AbstractNAToCdms CLASS 
     344toCdmsFile 
     345createCdmsVariables 
     346toCdmsVariable 
     347createCdmsAuxVariables 
     348auxToCdmsVariable 
     349createCdmsAxes 
     350toCdmsAxes 
     351 
     3522. na2cdms.py: 
     353 
     354Command-line script 
     355 
     3563. bin/na2nc: 
     357 
     358Same as na2cdms.py ??? 
     359 
     3604. cdms2na.py is the mother of all modules: 
     361 
     362compareAxes --> areAxesIdentical(a,b) cdms_utils 
     363compareVariables --> areDomainsIdentical(v1, v2) cdms_utils 
     364isAuxAndVar --> isAuxVarAndVar  
     365arrayToList utils 
     366listOfListsCreator utils 
     367getBestName cdms_utils - need some advice and compare with Dom 
     368getMissingValue cdms_utils 
     369fixHeaderLengthNowDefunct # Can destroy 
     370flatten2DTimeData aircraft 
     371modifyNADictCopy - needs a better name as it is specific 
     372cdms2na - 200 lines of code to do main conversion, needs to be split out into other stuff. 
     373 * getVariableCollections(f and varlist) --> (ordered_vars, other_vars) 
     374 * buildNADicts() 
     375 * writeToOutputFiles() 
     376 
     377class CdmsToNABuilder --> NAContentCollector: (naDict, varIDs, varBin) 
     378__init__ --> sets everything up and runs it move some to --> analyse() 
     379analyseVariables 
     380defineNAVars 
     381defineNAAuxVars 
     382getAxisDefinition 
     383defineNAGlobals 
     384defineNAComments 
     385defineGeneralHeader 
     386_useLocalRule --> Remove this and put it all in aircraft contrib bit 
     387 
     388================= 
     389 
     390naToCdms.py 
     391=========== 
     392 
     393This is a sub-class of all NAFile objects. Bad idea. What we need is to: 
     394 
     395import convertor 
     396convertor.writeToNC(blah) 
     397convertor.convertToCdms(blah): (vars, global_atts) 
     398 
     399class NAToCdms 
     400 
     401toCdmsFile 
     402createCdmsVariables - does all 
     403toCdmsVariable - does each in turn 
     404CreateCdmsAuxVariables - does all 
     405auxToCdmsVariables - does each in turn 
     406createCdmsAxes - does all 
     407toCdmsAxes - does each in turn 
     408 
     409================ 
     410 
     411Have I broken textParser.py's main function - is it same in old and new. 
     412 
     413===== 
     414 
     415NAFile2010: 
     416_readData[12] are crying out for useful names 
     417 
     418GREP 
     419==== 
     420 
     421Need to do a lot of grepping for inconsistencies. 
     422 
     423 * _normalizedX should be True|False not "yes","no" 
     424 
     425Global find and replace: 
     426 
     427floatFormat 
     428naDict 
Note: See TracChangeset for help on using the changeset viewer.