Use pyld as the json-ld parser for rdflib

[aucampia]

This is to track and discuss the work that @edmondchuc has started in #1836.

The change is essentially to make pyld the json-ld parser of RDFLib.

See also:

• Run all JSON-LD 1.1 tests #1833
• Run all tests from rdf 1.1 tests suite #1832
• Support for JSON-LD Framing in rdflib #1727

[aucampia]

@edmondchuc I have been thinking about this a bit.

Some important considerations for this:

• We should avoid or defer as much interface breaking as possible: It is not likely that we will get much significant rework of anything else, and ideally we should just have something that works behind our existing parser/serializer interface for now.
• We should avoid expanding our public interface as much as possible. I think we should possibly consider putting pyld in rdflib._contrib instead of rdflib.plugins - that way it does not quite become part of our public interface. We can of course change it later. For then exposing JSON-LD functionality (other than parse and serialize) we should possibly add rdflib.formats.jsonld and then put JSON-LD functions in there, but we can do this later and just call into rdflib._contrib.

I believe the first step you were trying was just to get it into our repo, and get the tests to pass, but even if not, this may not be a bad first step, integrate it verbatim, and then call into it.

I think we should ideally have a release where both the PyLD and existing JSON parser/serializer is available, and then once we are more or less happy with that, and possibly have some feedback, we can make another release.

I'm very happy to step up the release schedule as much as possible, and will do what I can to make this as automatic and uninvolved as possible.

[rob-metalinkage]

PYLD is somewhat broken - 19 open pull requests and no updates for 3 years - and substantive bugs such as digitalbazaar/pyld#143 that are showstoppers.

would need to resucitate the code management to adopt this.

[aucampia]

@rob-metalinkage it is hard to say this definitively, the RDFLib json-ld is more broken I think, we are not even running the whole test suite and the part we are running I'm not that sure about as it us using a rather old test runner, and we have many more open issues and bugs, some of them rather serious.

Integrating PyLD will definitely be a lot of work, and we will get baggage, but I think it is more realistic than fixing the RDFLib JSON-LD support.

We have spoken to people who worked on PyLD before and it is not likely that there will be further development in https://github.com/digitalbazaar/pyld - so we will have to fix it in our codebase or fork it, and that is also more effor than I think we have capacity for.

[rob-metalinkage]

That makes sense. I think then that we need to find sponsors for this work. Python and jsonld are too important to be ignored in combination.

Is there anyone out there who could cost a proposal and/or facilitate this. Has the community ever used gofundme or something to test the waters here?

[edmondchuc]

@aucampia thanks for creating this issue. I agree with what you've said above.

I think a good first step is to document how to use pyld with JSON-LD together, just so users are aware of the options available to them.

The pyld + RDFLib things I normally do:

Getting RDF into a framed JSON-LD

• Serialize an RDFLib graph as JSON-LD
• Alternatively, serialize the RDFLib graph as n-triples or n-quads and use the pyld processor' from_rdf function
• Use the frame function in the pyld processor with the JSON-LD data + frame and context as inputs

The above is the same for compaction and expansion by using the relevant function in the pyld processor.

Parsing JSON-LD with pyld and loading into an RDFLib graph

• To avoid some issues with the RDFLib's JSON-LD parser, we can use the pyld processor to parse it using the normalize function
• The result is an n-quads string. This can now be loaded into an RDFLib graph or dataset
• Alternatively, there's also a to_rdf function without the normalization algorithm. This is potentially faster.