Changes between Version 1 and Version 2 of DiscoveryWS


Ignore:
Timestamp:
06/01/12 12:32:11 (9 years ago)
Author:
sdonegan
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • DiscoveryWS

    v1 v2  
    1 = Discovery Web Services for the Revitalization Project = 
     1= NERC !DataGrid Discovery Web Service : NERC Upgrade based on MEDIN enhancements = 
     2 
     3TBC! 
     4 
     5[[PageOutline(1-2)]] 
     6 
     7The NERC !DataGrid (NDG) Discovery Web Service provides a search interface to metadata records harvested from collaborating data providers and is the backend server to which the NERC Data Discovery Service is a client. 
     8 
     9== Introduction == 
     10 
     11The Discovery Web Service is a presentation-free web service which acts as a search engine on top of the NDG Discovery metadata catalogue. This catalogue is dynamically populated by the [#harvesting harvesting] of metadata (by the [http://www.ceda.ac.uk CEDA] group at RAL) from a number of collaborating data providers, who make their metadata available in one of a number of [#SupportedMetadataFormats supported formats]. 
     12 
     13The search capability provided by the service enables full-text and spatio-temporal searches of catalogued metadata records and returns search results a defined XML structure, enabling search clients to be constructed by interested parties for their own purposes. The [http://ndg.nerc.ac.uk/discovery NERC Data Discovery Service] is one such client, as is the [http://www.edp.nerc.ac.uk Environmental Data Portal], among other examples. 
     14 
     15== Scope == 
     16 
     17This documentation covers functionality proposed for the improvements commissioned by MEDIN. Information on reporting and progress can be found [http://proj.badc.rl.ac.uk/ndg/wiki/DiscoveryMEDINImprovements here]. 
     18 
     19== Connectivity == 
     20 
     21Consumers may access the discovery service via SOAP. Client implementations should be generated from the WSDL at the following URIs: 
     22 
     23(URIs tbc pending implementation of proposed improvements) 
     24 
     25== XML Data Types == 
     26 
     27The XML documents used as request and response messages for each of the service operations (methods) are defined in the XML schema section of the [source:TI01-discovery/branches/ws-Discovery2-MEDIN/discoveryUpgrade/wsdl WSDL document]. The structure of each of these messages is discussed as part of the [#DiscoveryServiceMethods operation/method descriptions] below. Automatically-generated documentation for the schema showing the full structure is available [attachment:DiscoverySchema_xmlspydocumentation.doc here]. 
     28 
     29== Discovery Service Operations == 
     30 
     31The discovery service implements 4 operations, namely: 
     32  * getListNames 
     33  * getList 
     34  * doSearch 
     35  * doPresent 
     36 
     37=== getListNames operation === 
     38 
     39The discovery web service relies on several lists of valid terms which are specific to the functionality of this service. The reason for using these 2 "helper" operations rather than encoding these valid terms as <xs:enumeration> in the schema part of the WSDL, is so that future modifications to the service need not necessarily require the modification of the WSDL (which can be incovenient for clients already developed around a particular release of the WSDL). The getListNames operation simply returns the names of these lists, which can then be used in a subsequent call to the getList operation. 
     40 
     41The WSDL document defines the getListNamesRequest message as an empty <!GetListNames> element, so the request message should look like this (omitting the SOAP Envelope & Body parent elements): 
     42{{{ 
     43<m:GetListNames xmlns:m="urn:DiscoveryServiceAPI"/> 
     44}}} 
     45The getListNamesResponse message comprises a <!GetListNamesReturn> element, with child elements containing the names of the lists available for inspection: 
     46{{{ 
     47<sch:GetListNamesReturn xmlns:sch="http://medin.discovery.services.ndg/schema"> 
     48                        <sch:listNames> 
     49                                <sch:listName>TermTargetList</sch:listName> 
     50                                <sch:listName>PresentFormatList</sch:listName> 
     51                                <sch:listName>OrderByFieldList</sch:listName> 
     52                                <sch:listName>ScopeList</sch:listName> 
     53                                <sch:listName>SpatialOperatorList</sch:listName> 
     54                                <sch:listName>SpatialReferenceSystemList</sch:listName> 
     55                                <sch:listName>DateRangeTargetList</sch:listName> 
     56                                <sch:listName>TemporalOperatorList</sch:listName> 
     57                                <sch:listName>MetadataFormatList</sch:listName> 
     58                                <sch:listName>RecordDetailList</sch:listName> 
     59                        </sch:listNames> 
     60                </sch:GetListNamesReturn> 
     61}}} 
     62 
     63=== getList operation === 
     64 
     65The contents of each of the lists named by the getListNames operation are accessible by invoking a call to the getList operation, with the name of the list as the single argument, encoded as a getListRequest message, as defined in the WSDL : 
     66 
     67Request: 
     68{{{ 
     69<m:GetList xmlns:m="http://medin.discovery.services.ndg/schema"> 
     70      <m:listName>PresentFormatList</m:listName> 
     71</m:GetList> 
     72}}} 
     73 
     74Response: 
     75{{{ 
     76<sch:GetListReturn xmlns:sch="http://medin.discovery.services.ndg/schema"> 
     77                        <sch:list name="PresentFormatList"> 
     78                                <sch:listMember>DC</sch:listMember> 
     79                                <sch:listMember>DIF_v9.4</sch:listMember> 
     80                                <sch:listMember>MEDIN_v2.3.1</sch:listMember> 
     81                        </sch:list> 
     82                </sch:GetListReturn> 
     83}}} 
     84An explanation of the [#PresentFormatList PresentFormatList] is given later, in the context of the doPresent operation.  
     85 
     86=== doSearch operation === 
     87 
     88The doSearch operation performs a search against the NDG discovery database. Queries to this database are formulated from the doSearchRequest message, and forwarded to the database via private methods (i.e. the consumer of the web service is not able directly to interact with the database). 
     89 
     90Although outside the scope of the Discovery web service itself, it is worth explaining the structure of the NDG Discovery database which is searched by the service. This is populated from records harvested via [http://www.openarchives.org/pmh/ OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting)] from collaborating data providers. Records are harvested in the ISO19115-compliant metadata format agreed by the NERC Metadata Standards Working Group (MEDIN ISO v2.3.1) with legacy support for the previous format (DIF v9.4), and are tagged at ingest time with one or more "scope" keywords (listed in the scopeList list available from the getList operation). These enable the search to be restricted to particular communities, namely NERC, NERC_DDC (Designated Data Centres) and MEDIN (Marine Environmental Data Information Network). Limited quality control on ingested records is also applied at ingest time, but it is the responsibility of the data provider to ensure that metadata records are provided to sufficient quality to enable them to be visible in the system. 
     91 
     92The doSearchRequest message is shown in schema form below (click image to enlarge): 
     93 
     94[[Image(doSearchSchema.png, 400px)]] 
     95 
     96==== Choice of search criteria: <!TermSearch>, <!SpatialSearch> and <!TemporalSearch> ==== 
     97The <!SearchCriteria> element acts as a container enabling the selection of one or more of <!TermSearch>, <!SpatialSearch> and <!TemporalSearch>. Multiple <!TermSearch> elements, but only one of each of <!SpatialSearch> and <!TemporalSearch> may be included, so that the search has at least one component. If more than one of these three main types is specified, the resulting search combines the components in "AND" combination (i.e. metadata records should match the term search AND the spatial search AND the temporal search criteria). 
     98 
     99===== !TermSearch ===== 
     100!TermSearch is a full-text search invoked on a specific target field in the discovery database. !TermSearch elements must have an attribute named "id" with an integer value representing the order to be considered in complex multi termed searches, usually though "id" can be considered as the number of the !TermSearch element.  If more than one !TermSearch element is used then all subsequent !TermSearch elements (excluding the first) must also have an "operator" attribute describing the relationship between the !TermSearch element in question with preceding !TermSearchs (the order of these is given by the "id" attribute).  Acceptable values for the operator attribute are "AND", "OR" and "NOT" which are to be considered reserved words (for an exception see "Exact Phrase Search" paragraph below). 
     101 
     102Child elements <!Term> and <!TermTarget> should be populated as follows: 
     103  * <Term> : text term to search for. Whitespace separates component words, which are searched in "OR" combination unless the "+" symbol is used between them, in which case the words joined in this way are searched in "AND" combination.  The text "AND" may also be subsituted for "+". 
     104  * <!TermTarget> : target field name taken from the [#TermTargetList TermTargetList] list of valid term targets. 
     105 
     106An example of a single !TermSearch targeted at the !FullText term target: 
     107 
     108{{{ 
     109  <m:TermSearch> 
     110    <m:Term>snow + rain</m:Term> 
     111    <m:TermTarget>FullText</m:TermTarget> 
     112  </m:TermSearch> 
     113}}} 
     114 
     115means 
     116{{{ 
     117  Search for records where: 
     118    Full text contains "snow" AND "rain" 
     119}}} 
     120 
     121Other example of correct queries are 
     122{{{ 
     123  <m:Term>snow rain</m:Term>     ---> "snow OR rain" 
     124 
     125  <m:Term>snow AND rain</m:Term> ---> "snow AND rain" 
     126 
     127  <m:Term>snow + - rain</m:Term> ---> "snow AND NOT rain" 
     128 
     129  <m:Term>snow-rain</m:Term>     ---> "snow-rain" 
     130}}} 
     131 
     132 
     133Examples of incorrect queries are 
     134{{{ 
     135  <m:Term>- rain</m:Term> 
     136 
     137  <m:Term>snow - rain</m:Term> 
     138 
     139  <m:Term>snow OR ! rain</m:Term> ---> Not supported 
     140}}} 
     141We can extend the above example by adding an additional <!TermSearch> targeted at the Authors field: 
     142 
     143{{{ 
     144 
     145  <m:TermSearch id="1"> 
     146    <m:Term>snow + rain</m:Term> 
     147    <m:TermTarget>Abstract</m:TermTarget> 
     148  </m:TermSearch> 
     149  <m:TermSearch operator="AND" id="2"> 
     150    <m:Term>Lawrence</m:Term> 
     151    <m:TermTarget>Authors</m:TermTarget> 
     152  </m:TermSearch> 
     153}}}  
     154 
     155means 
     156{{{ 
     157  Search for records where: 
     158    abstract contains "snow" AND "rain" 
     159  AND 
     160    author contains "lawrence" 
     161 
     162}}} 
     163 
     164We can add further !TermSearches to extend this query: 
     165{{{ 
     166 
     167  <m:TermSearch id="1"> 
     168    <m:Term>snow + rain</m:Term> 
     169    <m:TermTarget>Abstract</m:TermTarget> 
     170  </m:TermSearch> 
     171  <m:TermSearch operator="AND" id="2"> 
     172    <m:Term>Lawrence</m:Term> 
     173    <m:TermTarget>Authors</m:TermTarget> 
     174  </m:TermSearch> 
     175  <m:TermSearch operator="OR" id="3"> 
     176    <m:Term>hail</m:Term> 
     177    <m:TermTarget>Abstract</m:TermTarget> 
     178  </m:TermSearch> 
     179}}} 
     180 
     181This would be interpreted to mean: 
     182{{{ 
     183Search for records where: 
     184    abstract contains ("snow" AND "rain") OR "hail" 
     185  AND 
     186    author contains "lawrence" 
     187}}} 
     188 
     189..and so on. 
     190 
     191 
     192===== Exact Phrase Search ===== 
     193 
     194If an "exact phrase" search is needed the sentence has to be wrapped with two double quotes. Consequently if the user is looking for looking for 'Snow and rain' should be translated to ""Snow and rain"" 
     195 
     196{{{ 
     197<m:Term>""Snow and rain""</m:Term> 
     198}}} 
     199 
     200===== Spatial searching : <!SpatialOperator> and <!BoundingBox> ===== 
     201The search may incorporate a spatial query to restrict results to those metadata records having spatial coverage(s) matching the search criteria defined by the <!BoundingBox> elements <!LimitNorth>, <!LimitSouth>, <!LimitEast> and <!LimitWest>. An optional element <!SpatialReferenceSystem> may be populated with an entry from the [#SpatialReferenceSystem SpatialReferenceSystem] list to specify an alternative spatial reference system (SRS) of the bounding box coordinates. (*Note : this feature is included for future development e.g. ability to supply spatial search coordinates in British National Grid coordinates. Currently the only supported SRS is EPSG:4326 (WGS84 lat/lon), and this will remain as the default if no SRS is specified.) 
     202 
     203When using SRS EPSG:4326 (default), values for <!LimitNorth>, <!LimitSouth>, <!LimitEast> and <!LimitWest> should be given in decimal degrees latitude and longitude. <!LimitNorth> and <!LimitSouth> must be in the range -90.0 to +90.0, with <!LimitNorth> greater than <!LimitSouth>. <!LimitWest> and <!LimitEast> must be in the range -180.0 to 180.0 and <!LimitEast> should be greater than <!LimitWest>. Bounding boxes that span the -180 degree meridian, or the poles, are not supported. 
     204 
     205An optional <!SpatialOperator> may be included, populated with a term from the [#SpatialOperatorList SpatialOperatorList], defining the comparison to be applied to spatial coverage(s) related to metadata records. The default value is "overlaps". Note that in the discovery index database, metadata records may contain several spatial coverages, so a match can occur if any of the spatial coverages related to the metadata item match the criteria specified in the spatial search.  
     206 
     207===== Temporal searching : <!DateRange> ===== 
     208The search may incoporate a temporal query to restrict results to those metadata records having temporal coverage(s) matching the search criteria specified within <!DateRange>. One or two <Date> elements may be specified, to represent either a single date, or a date range. Each <Date> element must contain a <!DateValue> element populated in the form YYYY-MM-DD, and optionally a <!TemporalOperator> element, populated with a value from [#TemporalOperatorList TemporalOperatorList], defining the semantic meaning of this date criterion in the search. In addition, an optional <!DateRangeTarget> element may be included, populated with a value from the [#DateRangeTargetList DateRangeTargetList], to enable searching of dates other than the default of "temporal coverage of data".  
     209Examples are shown below: 
     210 
     211{{{ 
     212<DateRange> 
     213  <Date> 
     214    <DateValue>2001-01-01</DateValue>     
     215  </Date> 
     216  <Date> 
     217    <DateValue>2002-02-03</DateValue>     
     218  </Date> 
     219</DateRange> 
     220}}} 
     221means 
     222{{{ 
     223  Find metadata records where the temporal coverage(s) of the data 
     224  overlaps the date range 2001-01-01 to 2002-02-03 inclusive. 
     225}}} 
     226 
     227{{{ 
     228<DateRange> 
     229  <Date> 
     230    <DateValue>2001-01-01</DateValue> 
     231    <temporalOperator>Before</temporalOperator>     
     232  </Date> 
     233  <Date> 
     234    <DateValue>2002-02-03</DateValue>     
     235    <temporalOperator>After</temporalOperator>     
     236  </Date> 
     237</DateRange> 
     238}}} 
     239means 
     240{{{ 
     241Find metadata records where the temporal coverage(s) of the data 
     242is outside of the date range 2001-01-01 to 2002-02-03 inclusive. 
     243}}} 
     244 
     245 
     246{{{ 
     247<DateRange> 
     248  <Date> 
     249    <DateValue>2001-01-01</DateValue> 
     250    <temporalOperator>OnOrbefore</temporalOperator>     
     251  </Date> 
     252</DateRange> 
     253}}} 
     254{{{ 
     255Find metadata records where the temporal coverage(s) of the data 
     256is on or before the date 2001-01-01. 
     257}}} 
     258 
     259{{{ 
     260<DateRange> 
     261  <Date> 
     262    <DateValue>2001-01-01</DateValue> 
     263    <temporalOperator>OnOrbefore</temporalOperator>     
     264  </Date> 
     265  <DateRangeTarget>LastRevisionDate</dateRangeTarget> 
     266</DateRange> 
     267}}} 
     268means 
     269{{{ 
     270Find metadata records where the last revision date of the data 
     271is on or before the date 2001-01-01. 
     272}}} 
     273 
     274==== Choice of Retrieve criteria: <!OrderBy>, <!RecordDetail> and <!MetadataFormat> ==== 
     275 
     276===== Ordering  ===== 
     277Ordering of the result set can optionally be requested by providing an <!OrderBy> element containing one or more <!OrderByField>s, each with an optional associated <!OrderByDirection> (default : descending). Available fields for use as an <!OrderByField> are listed in the [#OrderByFieldsList !OrderByFieldList]. 
     278 
     279 
     280===== !RecordDetail ===== 
     281The optional <!RecordDetail> element enables selection of the level of detail included in each result returned in the search result. Values must be one of those available from the [#RecordDetailList !RecordDetailList]. Default is !DocumentSimple, which simply returns the id of the document corresponding to the result. See [#SearchResults Search Results] section for further explanation of the structures returned in each of these cases. 
     282 
     283 
     284===== !MetadataFormat ===== 
     285The optional !MetadataFormat element enables selection of the XML format returned in the !DoSearch return !DocumentFull/Document element.  The default behavior is to return MEDIN_2.3 format metadata if !MetadataFormat is not used (but !RecordDetail is set to !DocumentFull).  Acceptable values for !MetadataFormat can be found in the [#MetadataFormatList !MetadataFormatList] 
     286 
     287 
     288 
     289 
     290==== Paging : <Start> and <!HowMany> ==== 
     291The optional elements <Start> and <!HowMany> control which records from the result set should be returned (although the total number of hits is always returned as a number to aid with paging in clients). If <Start> is omitted, the default value used is 1 (i.e. the first record). If <!HowMany> is omitted, '''all''' records are returned. 
     292 
     293 
     294==== Scope of search: <Scope> ==== 
     295The optional <Scope> element can be used to restrict the search to onr or more of the supported NDG Data Provider Groups, defined in NDG controlled vocabulary http://vocab.ndg.nerc.ac.uk/list/N010/0. Currently supported values from this vocabulary are these are given in the the [#ScopeList ScopeList] list accessible via the getList operation. 
     296If <Scope> is omitted, the search is not restricted in this way. 
     297 
     298 
     299==== !OriginalFormat ==== 
     300The optional <!MetadataFormat> element can be used to restrict the search to records in the discovery index whose original (harvested) representation was in the format specified. The format must be one of the values listed in the [#OriginalFormatList !OriginalFormatList] available from the getList operation. Default behaviour if this element is omitted is not to restrict the search in this way (and return results irrepsective of their harvested format). 
     301 
     302 
     303 
     304==== Search Results ==== 
     305 
     306The!DoSearchResponse message is defined in the WSDL as shown below (click image for larger version): 
     307 
     308[[Image(doSearchReturnSchema.png, 400px)]] 
     309 
     310The <!DoSearchReturn> element contains the following top-level elements: 
     311 
     312  Status:: 
     313    true if successful AND number of hits > 0, false otherwise (designed so that a client need only proceed to parse the rest of the message if results were successfully returned)  
     314  !StatusMessage:: 
     315    Textual information regarding success / failure / errors  
     316  !ResultId:: 
     317    reserved for future use  
     318  Hits:: 
     319    TOTAL number of hits returned  
     320  Documents:: 
     321    parent element for <!DocumentSimple>, <!DocumentBrief>, <!DocumentSummary> and <!DocumentFull> elements (as per choice in <!RecordDetail> within search request) containing returned results.  
     322 
     323A result where no hits were returned is shown below: 
     324{{{ 
     325<sch:DoSearchReturn xmlns:sch="http://medin.discovery.services.ndg/schema"> 
     326                        <sch:Status>true</sch:Status> 
     327                        <sch:StatusMessage>Search was successful but generated no results</sch:StatusMessage> 
     328                        <sch:ResultId>0</sch:ResultId> 
     329                        <sch:Hits>0</sch:Hits> 
     330</sch:DoSearchReturn> 
     331}}} 
     332 
     333A result where 2 hits were returned, with the <!RecordDetail> set to !DocumentSimple, is shown below: 
     334{{{ 
     335<sch:DoSearchReturn xmlns:sch="http://medin.discovery.services.ndg/schema"> 
     336                        <sch:Status>true</sch:Status> 
     337                        <sch:StatusMessage>Successful</sch:StatusMessage> 
     338                        <sch:ResultId>0</sch:ResultId> 
     339                        <sch:Hits>2</sch:Hits> 
     340                        <sch:Documents> 
     341                                <sch:DocumentSimple> 
     342                                        <sch:DocumentId>record1</sch:DocumentId> 
     343                                </sch:DocumentSimple> 
     344                                <sch:DocumentSimple> 
     345                                        <sch:DocumentId>record2</sch:DocumentId> 
     346                                </sch:DocumentSimple> 
     347                        </sch:Documents> 
     348</sch:DoSearchReturn> 
     349}}} 
     350 
     351If <!DocumentBrief> is specified as the !RecordDetail, a <!DocumentBrief> element is returned for each result, as outlined in the doSearchResponeMessage, above. This contains the <!DocumentId> element, containing the id of the document, but is accompanied by the additional element <Title>, containing the title from the metadata record, and a set of <!OrderedField> elements corresponding to the <!OrderByField>s used in the search request.  Within <!OrderedField> are 2 elements, <!MetricName> which is the name of the ordering field and <!MetricValue> which is the actual value by which this document is ordered.  These ordering fields are returned alongside the results so that a client can display the content of those fields which contributed to the resulting record ordering.  The <!AdditionalInformation> element holds child elements containing further detailed information (as requested) relating to the record in question. Where multiple values are associated with this element field within the database they are separated by a ";".  If there is no information available to fill these fields then they are left blank.  The purpose of this <!DocumentBrief> detail option is to enable clients to render a results list directly from the search response, without having immediately to invoke the doPresent operation to retrieve additional detail. 
     352 
     353Similarly, if <!DocumentSummary> is specified as the recordDetail, a <!DocumentSummary> element is returned for each result, as outlined in the doSearchResponseMessage, above. In addition to the content added by the <!DocumentBrief> option, <!DocumentSummary> includes the metadata abstract, and temporal and spatial information. For the temporal and spatial components of this <!DocumentSummary> the schema reuses the structures used for the search request, hence the optional temporalOperator are spatialOperator elements are redundant (and will be omitted) from the return context, however the dateRangeTarget element is useful as a contextual reminder of what the returned date pertains to (temporal coverage of data, last revision date of data, or ingestion date of metadata, etc.). 
     354 
     355Correspondingly the <!DocumentFull> element builds upon the <!DocumentBrief> and <!DocumentSummary> elements adding further information (including <!AdditionalInformation>) but importantly a <Document> element contains a CDATA representation of the original XML metadata with format corresponding to that chosen in <!MetadataFormat> (defaulting to MEDIN_2.3).   
     356For information, it should be noted that documents harvested (via OAI-PMH) are, at the time of ingest, stored in their native format but additionally converted to static copies of each of the supported return formats (listed in the [#PresentFormatList PresentFormatList].) When a document is requested for presenation in a particular format, the static copy (either the native copy or the generated copy in the requested format) is returned from the database. 
     357 
     358These 4 document types comprise a data type "Documents" which is dependant on the values supplied in <!RetrieveCriteria> and is also reused in the !DoPresent operation. 
     359 
     360 
     361=== !DoPresent operation === 
     362 
     363The !DoPresent operation provides a means of retrieving (presenting) one or more XML documents from the database.  The !DoPresent is distinguished from the !DoSearch in that is allows the specific request of documents according to ID with the !DoPresent response matching the order defined in the !DoPresent request.  The structure of the !DoPresent is closely related to that of the !DoSearch as most of the data types are reused. 
     364 
     365. The doPresentRequest message is defined as follows (click image to enlarge): 
     366 
     367[[Image(doPresentSchema.png, 400px)]] 
     368 
     369Within the <Documents> element, one or more <!DocumentId> elements should each contain the name of a document (in the form returned in the <!DocumentId> of the !DoSearchReturn message) to be retrieved.  The <!RetrieveCriteria> element again governs the level of detail and the XML format returned in the subsequent !DocumentFull//Document element.  The optional <!OrderBy> element is ignored when <!RetrieveCriteria> is used in a !DoPresent request. All documents returned by a single invocation of the doPresent operation are returned in the same format, i.e. the choice of presentFormat applies to the doPresent request and not individual documents. If <!MetadataFormat> is not defined then format defaults to MEDIN_2.3. 
     370 
     371 
     372==== !DoPresent response ==== 
     373 
     374The !DoPresentResponse message is defined in the WSDL as follows (click image to enlarge): 
     375 
     376[[Image(doPresentReturnSchema.png, 400px)]] 
     377 
     378The <DoPresentReturn> element contains the following top-level elements: 
     379 
     380  <Status>:: 
     381    true if there are any documents returned in the payload, false otherwise.  
     382  <!StatusMessage>:: 
     383    Textual information regarding success / failure / errors.  
     384  <Documents>:: 
     385    If some documents have been successfully returned, a <!Documents> element is present and consists of the same structure as the <!Documents> element in the !DoSearchReturn element depending on the values entered in the <!RetrieveCriteria> element in the !DoPresent request. 
     386 
     387The <Document> element, if present and populated, contains the retrieved document as an encapsulated string representation of the XML. Depending on the client used to display the payload document, it either appears contained within a <![CDATA[ ... ]]> construct, or as XML with the opening angle brackets "<" escaped as "&lt;". Most XML parsers should successfully parse the string to reconstruct the XML document, but it is returned in this form to avoid namespace issues.  
     388 
     389The following request / response sequence shows a successful doPresent operation: 
     390 
     391Request: 
     392{{{ 
     393<m:DoPresent xmlns:m="urn:DiscoveryServiceAPI"> 
     394    <m:Documents> 
     395        <m:DocumentId>badc.nerc.ac.uk__DIF__dataent_claus.xml</m:DocumentId> 
     396    </m:Documents> 
     397    <m:Format>DIF_v9.4</m:Format> 
     398</m:DoPresent> 
     399}}} 
     400 
     401Response: 
     402{{{ 
     403<DoPresentReturn xmlns="urn:DiscoveryServiceAPI"> 
     404    <Status>true</Status> 
     405    <StatusMessage>Success</StatusMessage> 
     406    <Documents> 
     407        <DocumentFull> 
     408           <DocumentId>badc.nerc.ac.uk__DIF__dataent_claus.xml</DocumentId> 
     409           <Title>Cloud Archive User Service data (CLAUS)</Title> 
     410           <Abstract>Global Brightness Temperature imagery from the  
     411Cloud Archive User Service project. This project produced a long  
     412time-series of global thermal infra-red imagery of the Earth using  
     413data from operational meteorological satellites, which was used in  
     414validating atmospheric General Circulation Models</Abstract> 
     415           <Document>&lt;DIF xmlns="http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/"  
     416xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 
     417&lt;Entry_ID>badc.nerc.ac.uk:DIF:dataent_claus&lt;/Entry_ID> 
     418 (.. truncated! ..) &lt;/DIF></Document> 
     419        </DocumentFull> 
     420    </Documents> 
     421</DoPresentReturn> 
     422}}} 
     423 
     424 
     425 
     426== Term Lists == 
     427The following lists and their contents are available via the getListNames and getList operations, but are repeated here with explanations of the list members. 
     428 
     429=== !TermTargetList === 
     430 
     431 !FullText:: 
     432   Target is a stored version of the original harvested XML metadata document 
     433 
     434 !Author:: 
     435   Target is the entry in the discovery index database corresponding to the author of the data described by the metadata. 
     436 
     437 !Parameter:: 
     438   Target is entries in the discovery index database corresponding to parameter keywords extracted from the metadata. 
     439 
     440 !ResourceType:: 
     441   ResourceType relates to MEDIN element 4 
     442 
     443 !TopicCategory:: 
     444   TopicCategory relates to MEDIN element 9 
     445 
     446 !Lineage:: 
     447   Lineage relates to MEDIN element 17 
     448 
     449 !PublicAccessLimits:: 
     450   PublicAccessLimits relates to MEDIN element 20 (Limitations on Public Access) 
     451 
     452 !DataOriginator:: 
     453   DataOriginator relates to MEDIN element 22.1 
     454 
     455 !DataFormat:: 
     456   DataFormat relates to MEDIN element 27 
     457 
     458 !AvailableDataFormats:: 
     459   AvailableDataFormats relates to MEDIN element 23 
     460 
     461 
     462Note that for targeting a search at a specific date based element (i.e. MetadataUpdateDate) the date must be entered in via the TemporalSearch and the date range target specified as MetadataUpdateDate 
     463 
     464 
     465=== !OrderByFieldList === 
     466 !TextRelevance:: 
     467   Ranking metric based on relevance of match to search term (metric derived by postgres text ranking function). 
     468 !DatasetStartDate::  
     469   The start date of the date range given for the temporal coverage of the metadata record. Records with no start date defined are treated as if their start date is later than that last record with a start date defined, hence appearing at the end of the results. 
     470 !DatasetEndDate::  
     471   The end date of the date range given for the temporal coverage of the metadata record. Records with no end date defined are treated as if their end date is later than that last record with a start date defined, hence appearing at the end of the results. 
     472 !DataCentre:: 
     473   The name of the data centre supplying the metadata record. In the case of records supplied in DIF format, this is the Data_Centre_Name/Short_Name field. In the case of other metadata formats, the most appropriate equivalent field is used as this index (e.g. "DistributorName" for MDIP format) 
     474 !DatasetResultsetPopularity:: 
     475   Measure of the popularity of a metadata record, related to how many times it has appeared in a result set in discovery searches. 
     476 !Proximity:: 
     477   The geographical proximity of the centre of the spatial coverage defined in the metadata record to the centre of the original search bounding box.  Where no spatial information was originally selected proximity is calculated against the centre of a 'global' bounding box (0N, 0E).  Metadata records with no spatial information originally defined are omitted from the search 
     478 !ProximityNearMiss:: 
     479   An ordering based on records that were not within the originally requested spatial extent, but that occur within an arbritrary 10% of the original bounding box extent.  Where records are present that satisfy this scenario, they are ordered according to proximity to the outside edge of the original bounding box.  This option is to give users an idea of datasets that were close to the original bounding box and still matched the search without having to redefine the original bounding box and original search terms.  
     480 !DatasetUpdateOrder:: 
     481   Date order based on the most recent update/edit by the original provider to the metadata record. 
     482 !DatasetOrder:: 
     483   Alphabetical order based on the name of the metadata record.  In DIF records this is the Entry_Title field and for MDIP the Title field. 
     484 !DiscoveryIngestDate:: 
     485   Date order based on the date of insertion of that record into the underlying database supporting the service.  Note that this differs from "datasetUpdateOrder" in that this records the actual date a record was placed in the database as opposed to the last edit date of that record. 
     486 !MEDINTermTarget.1*:: 
     487   One of fields specified by MEDIN for use as a termTarget : it should be possible to configure these for use as orderByFields, too. 
     488 
     489=== !ScopeList === 
     490 MEDIN:: 
     491   Marine Data Information Partnership (organisation replacing MDIP in previous API versions) 
     492 NERC_DDC:: 
     493   NERC Designated Data Centres  
     494 NERC:: 
     495   NERC (General) 
     496 DPPP:: 
     497   Data Portals Project Provider 
     498 
     499=== !SpatialOperatorList === 
     500 Overlaps (default):: 
     501   Return metadata records having one or more spatial coverages that spatially overlap the search bounding box.  
     502 !DoesNotOverlap:: 
     503   Return metadata records having no spatial coverages that spatially overlap the search bounding box. 
     504 Within:: 
     505   Return metadata records having one or more spatial coverages that are entirely within the search bounding box. 
     506 
     507=== !SpatialReferenceSystemList === 
     508 EPSG:4326:: 
     509   WGS84 Lat/Lon, in decimal degrees 
     510 
     511=== DateRangeTargetList === 
     512 !TemporalCoverage:: 
     513   The temporal coverage of the data described by the metadata record. 
     514 !LastRevisionDate:: 
     515   The data of last revision to the dataset, recorded in the metadata record. 
     516 !DiscoveryIngestDate:: 
     517   The date that the metadata record was (last) ingested into the discovery index. 
     518 !MetadataUpdateDate:: 
     519   The date that the metadata record was updated by the originating data centre/data provider to the Discovery service. 
     520 
     521=== TemporalOperatorList === 
     522 Equals:: 
     523   Target date value is the same as the date specified 
     524 !DoesNotEqual:: 
     525   Target date values does not equal the date specified 
     526 !OnOrBefore:: 
     527   Target date is equal to or before the date specified 
     528 !OnOrAfter:: 
     529   Target date is equal to or after the date specified 
     530 Before:: 
     531   Target date is before the date specified 
     532 After:: 
     533   Target date is after the date specified 
     534 
     535 
     536=== !OriginalFormatList === 
     537 
     538Note this is separate from MetadataFormatList,  OriginalFormatList lists values available to distinguish the original source format of a harvested metadata item. 
     539 
     540 DIF_9.4:: 
     541   GCMD DIF format (version 9.4) 
     542 MEDIN_2.3:: 
     543   Metadata format used by the Marine Environmental Data and Information Network, version 2.3 
     544 ISO19115:: 
     545   ISO19115 (Geographic Information: Metadata) encoded as ISO19139 XML. Currently an alias for MEDIN_v2.3.1, i.e. the MEDIN format is used as the implementation of ISO within the Discovery Service. 
     546 
     547 
     548 
     549=== !MetadataFormatList === 
     550 
     551Note that MetadataFormatList is used in the RetrieveCriteria element to define the format of xml returned in either the DocumentFull (doSearchReturn//DocumentFull//Document element or doPresentReturn//DocumentFull//Document element ).  This is distinct from OriginalFormatList which lists the accepted metadata input formats. 
     552 
     553 DC:: 
     554   Dublin Core format  
     555 DIF_9.4:: 
     556   GCMD DIF format (version 9.4) 
     557 MEDIN_2.3:: 
     558   Metadata format used by the Marine Environmental Data and Information Network, version 2.3.1 
     559 ISO19115:: 
     560   ISO19115 (Geographic Information: Metadata) encoded as ISO19139 XML. Currently an alias for MEDIN_2.3, i.e. the MEDIN format is used as the implementation of ISO within the Discovery Service. 
     561 
     562=== !RecordDetailList === 
     563 Id:: 
     564   Return a <!DocumentId> element containing the id of the matching metadata document, in a form that can be used as input to a doPresent request. 
     565 Brief:: 
     566   Return a <!DocumentBrief> element containing a <!DocumentId> element, the title and ordering fields. 
     567 Summary:: 
     568   Return a <!DocumentSummary> element containing content as per <!DocumentBrief> but additionally abstract, temporal and spatial information. 
     569