[M] DIF-to-MOLES not picking up MDIP keyword

[selatham]

Not picking up MDIP keyword (possibly not any keywords)

[lawrence]

Don't understand this one, as both Matt and I have demonstrated it working. Can you give a specific example of what you typed where and what happened?

[selatham]

The DIF -to-MOLES run during ingest dosen't pick up on any Keywords tags in DIF. I have to post-add to the discovery MOLES. Assume this will be fixed inKev's re-factored version.

[ko23]

Yes, this will be parsing will be included, but remember that this was "broken as designed" and not a defect, as you were going to add the keywords as it was not clear whether they would be how they added until I suggested using the keywords field.

Note also that this doesn't help MDIP records as there's no general purpose keywords field in the schema (that I've noted at least), and Kieran hasn't responded to the suggestion of adding one (that I've seen).

Anyway, I have now a set of DIFs from NEODC that do include a term from the relevant vocab, and so can test it.

[ko23]

Whoops, wrong, there aren't aren't NEODC examples, indeed the BODC DIF example appears to be a MOLES record.

[lawrence]

OK, this is broken from the discovery interface now. It appears that it can work, but it requires a rather long query ... which times out from the wsgi interface.

Matt: are you confident that a query for bodc limited to mdip, works well, after a query to bodc, not limited in anyway?

[mpritcha]

Fixed the bottleneck, I think. Can you confirm it's quicker now, Bryan?

[lawrence]

Well, it's no longer timing out, at least while I was testing, but it's not obvious it's working :-). I was expecting a search on bodc limited to MDIP to only return one record.

Sue: Was that expectation correct?

[mpritcha]

I get 420 hits without the filter, 45 with. Does that correspond with the discovery moles docs? (Remember it's the moles docs that are used for the filter, so I'm assuming these are up to date in line with any keyword-containing original records).

[selatham]

As the moles docs are used in the scope query I would actually expect about 45 records. This is because I post-added the MDIP keyword to all BODC's discovery-moles, so that we had some test records. Using Kev latest code I am going to try ingest on neodcs DIFs (which now do have MDIP and NERC-DDC keywords).

[selatham]

I have successfully used Kev's re-factored code. Keywords are being picked up on.

(Discovery seems to be working with keywords now too.)