Ticket #881 (closed defect: wontfix)

Opened 14 years ago

Last modified 14 years ago

[M][WG] Missing related Data Entities on NDG Browse

Reported by: pmiller Owned by: lawrence
Priority: critical Milestone: PROD Final
Component: community Version:
Keywords: Cc:


I guess this for Bryan and Matt:

The Browse website is missing a key component in the MOLES record - the related 'Data Entities' that result from the Deployment. The best way to show this is by example:

 Data Entity HALOE L3A correctly shows the related DP Tool 'HALOE'.

But  DP Tool HALOE does not show any related Data Entities (should include HALOE L3A and a couple others).

In terms of debugging, I can see that the Entities are listed in the Stub-B under DeploymentSummary->dataEntityList but are not listed in a mysterious section DeploymentSummary->rawDeployments. So which list is the Browse code looking at?

This is absolutely critical to the Browse concept, for example our satellite data, as I was intending to have just certain 'Activities' discoverable, with smaller component datasets (e.g. years) accessible via browse to the related Data Entities.

I attach the relevant bits of my e-mail for further info.

Change History

comment:1 Changed 14 years ago by pmiller

From: Peter Miller <pim@pml.ac.uk>
To: NERC DataGrid Internal mailing list <nerc-datagrid-internal@ncas.ac.uk>
Subject: Re: [NDG-Internal] [M] LInking Moles to Difs
Date: Wed, 03 Oct 2007 12:24:19 +0100

Hi Bryan (or anyone else with advice),

I'll try to clarify this. Currently each PML dataset represents a year
for one region for one sensor, corresponding to one MOLES and one DIF
record. (We then have a granule for each month of data, representing
maybe 50-150 data files.)

As previously discussed at NDG meetings, this results in several hundred
records which may be too many for Discovery at present (until it can
automatically group similar records and order by relevance). So our next
stage is to only export DIFs for 'all-year' records (ie ALL data for one
region for one sensor). 

Steve's question is can we then have the single-year B/MOLES records
linked from this all-year D or B record? This would be analagous to
multiple CTDs from one cruise. I would prefer not to restructure our
datasets so that the entire date range are granules from a single
enormous MOLES record. Should we use some mechanism like project
'activity' to multiple datasets? Then the single years will appear as
'related entities'?

I looked on Discovery to see how other DP's have tackled this. It looks
to me like there may be some functionality missing in the B Browse web
code, as under the 'Related Entities' section I can see listed
Activities, Obs. Stns and DP Tools, but never Data Entities. E.g.:

This is a DP Tool 'HALOE', which lists several Data Entities in the
Stub-B under DeploymentSummary->dataEntityList. But these are not shown
as related entities on the web page. I see there is a section lower down
the XML: DeploymentSummary->rawDeployments which doesn't mention any
data entities, so maybe that is what is being listed. So my query here
is whether there is something wrong with this XML or with the web code
which is preventing the listing of the HALOE data entities here.

I understand your point that you cannot link to multiple B's from one
DIF record - that is fine as long as we can have a hierarchy of B's,
which I know was the main idea behind MOLES.

What you described about multiple bounding boxes within one DIF record
could be a later stage of dataset reduction, again we'd need to resolve
the same issue of linking a higher-level dataset to multiple smaller
datasets. Best not to tackle that one now if there is functionality
missing in the NDG code for handling multiple boxes.

Sorry for the long e-mail, hope it makes sense and you can help us.

comment:2 Changed 14 years ago by lawrence

  • Status changed from new to assigned

Hi Peter

I think I fixed the main bug this morning (coincidentally). It's fixed in changeset:2960 (not yet deployed).

Sorry I haven't replied to the main email, so I'll leave the ticket open for that.

comment:3 Changed 14 years ago by lawrence

  • Status changed from assigned to closed
  • Resolution set to worksforme

OK, I've read the email through, and believe that the related data entities sorts out the problem. Reopen the ticket if this isn't satisfactory!

comment:4 Changed 14 years ago by lawrence

  • Status changed from closed to reopened
  • Resolution worksforme deleted

bugger, no, I have that wrong ... it's not there ...

comment:5 Changed 14 years ago by lawrence

The current way it works is that for a DE, I parse the deployments for the related entities ...

That means, for any given b-record, I don't show related entities of the same type.

I wont be able to either, until the stub-B is modified to provide a name for them ...

It would be helpful if you could give me the URI of a record which does have related entities of the same type, and I'll follow through the stubB consequences.

comment:6 Changed 14 years ago by lawrence

(NB: for me, for later, I need to be looking at what stubB does to dgDataEntity/dgRelatedDataEntity

This is another case where MOLES has stupid overloading of element names as well ... there is no reason why all the types don't have the same element name ...

comment:7 Changed 14 years ago by pmiller

Hi Bryan,

Thanks for fixing that. I can now see the related Data Entities in Browse, and I believe that is all we need. We will convert our higher-level Entity records into Activities, then we can link to the components as related Data Entities.

You could envisage a Data Entity having related Entities, but that is not represented by the MOLES deployment concept. I don't think this issue is worthy of a design change at this stage.

Actually having said that, didn't we intend that there would be Activity-to-Activity relationships in MOLES, such that you could link e.g. a cruise Activity to the (multiple) project Activities it contributes to?

Currently there are no links shown on Browse between one Activity and any others. Please could Bryan (and Kevin?) comment on this before we close this ticket.

comment:8 Changed 14 years ago by pmiller

There was a flaw in my plan: "We will convert our higher-level Entity records into Activities, then we can link to the components as related Data Entities" Because Activities cannot discovered, only Data Entities. So Steve Rooney had the idea of turning it upside down - we now have Data Entities for the higher-level records, with the components as related Activities. Except the components are really Data Entities, listed in the Related Activities section...!

See e.g.  http://wwwdev.neodaas.ac.uk/projects/ndg/view/www.npm.ac.uk__NDG-B1__data_entity.531

This is nasty, but as Bryan said we cannot currently relate Entities to other Entities, until we change Stub-B and XQueries. So this will do for now, and it has vastly reduced the number of PML DIF records ;-).

Steve is now working on exporting our CSML records.

Still waiting for a response to my Activity-to-Activity question above.

comment:9 Changed 14 years ago by lawrence

As I said today, the acivity-to-activity depends on the stub-b query, and on my code, and as I said in a comment above, I'm not sure of either the stub-b or my rendering code. Have you got any examples where the underlying moles record does have related entities of the same type? If so, can you give me the id of the records, and I'll investigate what is happening ... and see if I can get the stub-b query and rendering working ...

comment:10 Changed 14 years ago by selatham

Peter - if your list of datasets and therefore DIFs are changing drastically, then it may be a good idea for me to delete all your records from discovery and start from scratch with your new ones. I won't do this until after the demo tomorrow.

comment:11 Changed 14 years ago by selatham

  • Milestone changed from PROD Step1 to PROD Final

comment:12 Changed 14 years ago by mggr

Comment by email (putting in ticket for the record):

Hi Sue,

Pete has asked me to reply on his behalf, in regards
to our changing datasets. These changes are only made
in our moles records and not our DIFs, therefore our
DIF records have not changed in any way. There will be
no need to repopulate them into discovery.


Steve Rooney 

comment:13 Changed 14 years ago by lawrence

  • Status changed from reopened to closed
  • Resolution set to wontfix

I understand this isn't a problem any more. If it is, reopen it.

Note: See TracTickets for help on using tickets.