wiki:CDB

Version 41 (modified by lalibert, 8 years ago) (diff)

--

Climate Diagnostics Benchmark (CDB)

In support of ExArch Work Package 3: "Climate Diagnostics"

Draft Documentation

 Frédéric Laliberté and  Paul Kushner, 2011-2012, University of Toronto, Ontario, Canada.

The CDB was tested on the CICLAD compute server at the IPSL and on the login nodes at BADC.

Overview of scope and structure.

The CDB provides a simple framework to write ESG-oriented climate diagnostics scripts. It aims to:

  • Simplify the development of climate diagnostics on ESG servers
  • Benchmark a server-side computing framework
  • Help in the timely delivery of model intercomparisons

The first releases of the CDB are intended for use by climate researchers that have ssh access to an ESG node. In its simplest formulation, the CDB can be used remotely to create a subset of the ESG archive. This subset can then be transferred to a local compute node for further treatment. Because the CDB is portable, it can then be used locally for the processing of the ESG archive subset.

With its more advanced features, the CDB can be used remotely from the beginning to the end. For example, it can generate PBS submit scripts to simplify the distributed processing of ESG data on a cluster with a direct connection to an ESG node, as on the CICLAD server at IPSL.

It is a portable framework that makes it easier for climate scientists to use the vast amount of data on an ESG node. We present a simplified CDB flowchart at the end of this page.

Documentation

  1. Installation instructions
  2. Tutorial
  3. Features
    1. Finding available models
    2. Fixing an incomplete dataset
    3. Using an output in another computation
  4. Quality Assurance issues
  5. Notes on performance and benchmarking

This wiki is being maintained by  Frédéric Laliberté.

http://dl.dropbox.com/u/21787239/CDB_flowchart%20%283%29.png