Ticket #566 (closed defect: fixed)

Opened 13 years ago

Last modified 11 years ago

[M] Discovery - supports doSpatioTemporal Search

Reported by: lawrence Owned by: lawrence
Priority: desirable Milestone: portal1
Component: discovery Version:
Keywords: WS-Discovery2 MDIP Cc:

Description (last modified by pmiller) (diff)

Should start by implementing standalone search, and then supporting doing a text search and scope limit on the result set.

Start with simple calendars. Start with simple lat/lon bounding box - added May 2007, working as of 29 Nov. 2007.

Change History

comment:1 Changed 13 years ago by mpritcha

  • Status changed from new to assigned

comment:2 Changed 12 years ago by selatham

  • Description modified (diff)

comment:3 Changed 12 years ago by mpritcha

  • Status changed from assigned to closed
  • Resolution set to fixed

spatioTemporalSearch, i.e. optional search on spatial / temporal coverage, is now part of doFullTextSearch (and will be part of doAuthorSearch, doParameterSearch) rather than a method in itself

comment:4 Changed 12 years ago by selatham

  • Status changed from closed to reopened
  • Resolution fixed deleted

Does the search do 'find something which is entirely contained within this bounding box'? (EDINA/MDIP want this sort of search).

comment:5 Changed 12 years ago by mpritcha

  • Status changed from reopened to closed
  • Resolution set to fixed

Added optional <spatialOperator/> element to WSDL. Valid values "overlaps", "doesNotOverlap" and "within". Example behaviour shown below: Observant among you will notice <temporalOperator/> in WSDL, too, but this isn't implemented yet.

Request 1: "within"

		<m:doSearch xmlns:m="urn:DiscoveryServiceAPI">
			<m:term>water</m:term>
			<m:termType>fullText</m:termType>
			<m:spatialOperator>within</m:spatialOperator>
			<m:BoundingBox>
				<m:LimitNorth>80</m:LimitNorth>
				<m:LimitSouth>20</m:LimitSouth>
				<m:LimitWest>-40</m:LimitWest>
				<m:LimitEast>40</m:LimitEast>
			</m:BoundingBox>
		</m:doSearch>

Response 1:

		<doSearchReturn xmlns="urn:DiscoveryServiceAPI">
			<status>true</status>
			<statusMessage>Success</statusMessage>
			<resultId>0</resultId>
			<hits>415</hits>
			<documents>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA47973RS2302.xml</document>
				...snipped


Request 2: "overlaps"

		<m:doSearch xmlns:m="urn:DiscoveryServiceAPI">
			<m:term>water</m:term>
			<m:termType>fullText</m:termType>
			<m:spatialOperator>overlaps</m:spatialOperator>
			<m:BoundingBox>
				<m:LimitNorth>80</m:LimitNorth>
				<m:LimitSouth>20</m:LimitSouth>
				<m:LimitWest>-40</m:LimitWest>
				<m:LimitEast>40</m:LimitEast>
			</m:BoundingBox>
		</m:doSearch>

Response 2:

		<doSearchReturn xmlns="urn:DiscoveryServiceAPI">
			<status>true</status>
			<statusMessage>Success</statusMessage>
			<resultId>0</resultId>
			<hits>467</hits>
			<documents>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA47973RS2302.xml</document>
				...snipped

Request 3: "doesNotOverlap"

		<m:doSearch xmlns:m="urn:DiscoveryServiceAPI">
			<m:term>water</m:term>
			<m:termType>fullText</m:termType>
			<m:spatialOperator>doesNotOverlap</m:spatialOperator>
			<m:BoundingBox>
				<m:LimitNorth>80</m:LimitNorth>
				<m:LimitSouth>20</m:LimitSouth>
				<m:LimitWest>-40</m:LimitWest>
				<m:LimitEast>40</m:LimitEast>
			</m:BoundingBox>
		</m:doSearch>

Response 3:

		<doSearchReturn xmlns="urn:DiscoveryServiceAPI">
			<status>true</status>
			<statusMessage>Success</statusMessage>
			<resultId>0</resultId>
			<hits>48</hits>
			<documents>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA22992RS2320.xml</document>
				...snipped

Request 4: <spatialOperator> omitted or empty : behaviour as per "overlaps"

		<m:doSearch xmlns:m="urn:DiscoveryServiceAPI">
			<m:term>water</m:term>
			<m:termType>fullText</m:termType>
			<m:BoundingBox>
				<m:LimitNorth>80</m:LimitNorth>
				<m:LimitSouth>20</m:LimitSouth>
				<m:LimitWest>-40</m:LimitWest>
				<m:LimitEast>40</m:LimitEast>
			</m:BoundingBox>
		</m:doSearch>

Response 4:

		<doSearchReturn xmlns="urn:DiscoveryServiceAPI">
			<status>true</status>
			<statusMessage>Success</statusMessage>
			<resultId>0</resultId>
			<hits>467</hits>
			<documents>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA47973RS2302.xml</document>
				...snipped

Request 5: Invalid value given for <spatialOperator> : spatial search ignored

		<m:doSearch xmlns:m="urn:DiscoveryServiceAPI">
			<m:term>water</m:term>
			<m:termType>fullText</m:termType>
			<m:spatialOperator>blah</m:spatialOperator>
			<m:BoundingBox>
				<m:LimitNorth>80</m:LimitNorth>
				<m:LimitSouth>20</m:LimitSouth>
				<m:LimitWest>-40</m:LimitWest>
				<m:LimitEast>40</m:LimitEast>
			</m:BoundingBox>
		</m:doSearch>

Response 5:
		<doSearchReturn xmlns="urn:DiscoveryServiceAPI">
			<status>true</status>
			<statusMessage>Success</statusMessage>
			<resultId>0</resultId>
			<hits>520</hits>
			<documents>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA47973RS2302.xml</document>
				<document>grid.bodc.nerc.ac.uk__DIF__PCDA48027RS6883.xml</document>
				...snipped

comment:6 Changed 12 years ago by pmiller

  • Status changed from closed to reopened
  • Resolution fixed deleted
  • Milestone changed from ReFactored_Discovery_WebServices to PROD Final

Discovery 'Spatial Coverage' search is not working properly! This has to be fixed before releasing NERC Discovery service.

I tried a search for MODIS satellite data covering the SW UK:  http://ndgbeta.badc.rl.ac.uk/discovery/?searchString=modis&textTarget=All&startDateYear=&startDateMon=&startDateDay=&endDateYear=&endDateMon=&endDateDay=&source=All&Search=Search&bboxN=52.6&bboxW=-5.9&bboxE=-2.7&bboxS=48.8

Some are sensible hits, some are not! Aegean Sea and Baltic Sea are too far east, Cape Verde is too far south.

What it looks like to me is an OR instead of an AND, ie. Dataset overlaps box latitude OR box longitude....!

While I am here, there is an awfully misleading description of the bounding box on the results page: Bounding Box: 51.3N,-6.2W,-3.4E,49.7S

To a scientist the NSEW suffixes make this mean: 51.3N, 6.2E, 3.4W, 49.7S!! Try: Latitude N: 49.7 to 51.3, Longitude E: -6.2, to -3.4

Before you ask, it is not just PML records, I searched for CTD records, the second listed ('Data from cruise CD97') is not within box:  http://ndgbeta.badc.rl.ac.uk/discovery/?searchString=ctd&textTarget=All&startDateYear=&startDateMon=&startDateDay=&endDateYear=&endDateMon=&endDateDay=&source=All&Search=Search&bboxN=51.3&bboxW=-6.2&bboxE=-3.4&bboxS=49.7

I have noted this in the NERC Discovery ticket:510.

comment:7 Changed 12 years ago by pmiller

  • Description modified (diff)

comment:8 follow-up: ↓ 9 Changed 11 years ago by selatham

Matt,Bryan, There is no default spatial operator in the wsdl. What is used in Discovery? Do we expect the default to be 'within'? What happend when you do overlaps?

comment:9 in reply to: ↑ 8 Changed 11 years ago by mpritcha

Replying to selatham:

Matt,Bryan, There is no default spatial operator in the wsdl. What is used in Discovery? Do we expect the default to be 'within'? What happend when you do overlaps?

See request 4 in examples above : default operator is "overlaps" ...doesn't explain Peter's problems, though, which I'm still looking into.

comment:10 Changed 11 years ago by selatham

  • Type changed from task to defect
  • Milestone changed from PROD Final to Replace Metadata Gateway

comment:11 Changed 11 years ago by mpritcha

  • Status changed from reopened to new
  • Owner changed from mpritcha to lawrence

This seems to be a problem with whatever composes the SOAP request to my WS. As I see it, the values for LimitSouth? and LimitEast? have been transposed (Bryan?)

Peter's first URL asks for a bounding box of

&bboxN=52.6&bboxW=-5.9&bboxE=-2.7&bboxS=48.8

And, via the handy Axis2  SOAPMonitor, I can see that the request arriving at my service looks like this, which is wrong:

<ns1:BoundingBox>
	<ns1:LimitNorth>52.600000</ns1:LimitNorth>
	<ns1:LimitSouth>-5.900000</ns1:LimitSouth>
	<ns1:LimitWest>-2.700000</ns1:LimitWest>
	<ns1:LimitEast>48.800000</ns1:LimitEast>
</ns1:BoundingBox>

Over to you Bryan! ;-)

comment:12 Changed 11 years ago by mpritcha

Actually, LimitSouth?, LimitWest? and LimitEast? are all affected!

comment:13 Changed 11 years ago by lawrence

Thanks Peter! Mea culpa on both charges ... fixed I believe (changesets 3059,3060). It's not live yet ... Can you please close the ticket when it's gone live (maybe a couple of days) and you're happy things are ok?

comment:14 Changed 11 years ago by lawrence

It should be ok now ... let me know if it is (or isn't) ... but don't close the ticket, I'll use this as a reminder to add the different type of geospatial searches.

comment:15 Changed 11 years ago by lawrence

  • Priority changed from blocker to desirable
  • Milestone changed from Replace Metadata Gateway to portal1

comment:16 Changed 11 years ago by pmiller

  • Description modified (diff)

Thanks Bryan, the bounding box appears to be working properly now.

comment:17 Changed 11 years ago by lawrence

  • Status changed from new to closed
  • Resolution set to fixed

Closing this as fixed, opening another one on temporal searching per se.

Note: See TracTickets for help on using tickets.