Dear ANDS PoolParty user:
This is a heads-up to let you know that our PoolParty server will soon be upgraded to a new release of the PoolParty software. We're giving you more notice this time, because Release 5.6 includes a large number of changes from our current installation, some of which may have consequences for you – especially, if you make direct use of the PoolParty API, or if you use SPARQL to query vocabularies published through Research Vocabularies Australia (RVA).
We've put in a lot of work behind-the-scenes, in close collaboration with SWC, to make this particular upgrade go as smoothly as possible, and to be as transparent to you as possible. For example, SWC have provided us with access to a preview server running the very latest builds of PoolParty to enable us to make and test the necessary changes to our own code. However, you (and your "downstream" consumers) will notice some differences, depending on how your vocabularies are used. Please read on.
Release 22 of our software releases is scheduled for demo (demo.ands.org.au) on 6 December 2016, and RVA (vocabs.ands.org.au, "production") on 7 December 2016. SWC will upgrade our PoolParty server to Release 5.6, and migrate all existing vocabularies, on the evening of 6 December 2016. Availability of services will be as follows:
- 6 December, morning: demo release
At this point, publishing from PoolParty will not be available on demo.
- 6 December, evening: SWC will update our PoolParty instance to Release 5.6 and migrate existing vocabularies
During this time, the ANDS PoolParty server will be unavailable.
At this point, publishing from PoolParty will now work on demo, but not on RVA.
- 7 December, morning: RVA ("production") release
Once the release in completed, publishing from PoolParty will now work on both demo and RVA.
Our server currently runs PoolParty Release 5.3.4, and it will be upgraded to PoolParty Release 5.6. All of the major changes in question were made in Release 5.5; Release 5.6 is a minor release, with a small number of improvements and fixes. (In our ANDS Software Release 22 pre-release e-mail, we mentioned an upgrade to Release 5.5, but we have since arranged for an upgrade straight to Release 5.6, which corresponds to the version of the software we have actually been testing our own software against in the last few weeks.)
SWC have published a comprehensive set of release notes here: for Release 5.5: https://help.poolparty.biz/doc/white-papers-release-notes/poolparty-release-notes/release-notes-poolparty-5-5; for Release 5.6: https://help.poolparty.biz/doc/white-papers-release-notes/poolparty-release-notes/release-notes-poolparty-5-5/release-notes-poolparty-5-5-5-6-5-7-minor-bug-fix-releases. Please take a few minutes to read through these notes. Some points of particular interest are:
- the refreshed "look and feel" of the user interface;
- the improvements to resource IRI generation: there is now significantly enhanced flexibility;
- the improved support for SKOS matching properties: linking to arbitrary IRIs;
the changes to the "internal structure" of vocabularies: to be specific, how the underlying RDF is partitioned into named graphs. More specific detail on this is provided at a separate page: https://help.poolparty.biz/doc/poolparty-overview/repository-and-graph-structure-of-data-stored-in-poolparty
Please note: the release notes refer to some SKOS-XL functionality; ANDS's current licence does not provide access to that.
We now need to discuss the last dot point in more detail.
How the changes to the structure of the RDF might affect you
Already-published versions in RVA
The first thing to note is that we won't automatically re-publish any of your versions currently visible in RVA. So SPARQL and Linked Data endpoints of already-published versions will be unaffected. You will only start to notice changes in the presentation of your vocabularies in RVA when you re-publish (or publish new versions of) your PoolParty vocabularies.
PoolParty User interface
The changes to the structure of the RDF have no affect on the PoolParty user interface. Creating and editing of concepts works as usual.
DC Terms properties
SWC have now fixed a long-standing "issue" in the use of some properties in the DC Terms vocabulary. To be specific: until now, instances of the properties
dcterms:publisher were triples that had string literals as their objects. For example, your existing vocabulary might have a triple like this (in Turtle syntax):
A triple of this form is incorrect according to the specification of the DC Terms vocabulary: the object is not allowed to be a string literal; it must be a resource. In PoolParty 5.5, this is fixed by automatically creating resources where they are required. For example, the above triple becomes something like this (now in TriG syntax):
If you use PoolParty's own API to run SPARQL queries to extract DC Terms properties, you will need to modify your queries accordingly. See below, for further information about what happens to these triples when they are imported into RVA and made available through an RVA-hosted SPARQL endpoint.
Deleted/deprecated concept schemes and concepts
From Release 5.5, when you delete/deprecate a concept scheme or a concept, all triples associated with that resource are moved into a separate named graph. Please note the following points:
- The type triple (the triple with property
rdf:type) for the resource is no longer deleted.
- As before, triples that have property
skos:prefLabelare replaced with triples that have property
If you use PoolParty's own
export API call, and you specify an export format that doesn't support named graphs (e.g., RDF/XML, Turtle, etc.), you may not notice much difference (apart from the DC Terms properties mentioned above). However, one key difference is in the behaviour of deleted/deprecated concepts: whereas the triples for these were previously included in the output of an export call with
exportModules=concepts, from Release 5.5 they no longer are; you must now specify
exportModules=deprecatedConcepts to get these triples. Please see SWC's documentation page for the list of supported project modules: https://help.poolparty.biz/doc/developer-guide/general-information-on-the-poolparty-api/poolparty-project-modules. Note particularly that the name of an export module may or may not correspond exactly with the last component of the containing named graph.
What will happen to your PoolParty vocabularies during migration
The two key things that will happen to each of your vocabularies when the server is migrated to Release 5.6 are as explained above:
- the partitioning of the triples into the various named graphs;
- the transformation of the DC Terms properties to use resources instead of string literals.
Note that triples for any deleted/deprecated concepts are only moved into the
.../thesaurus/deprecated named graph; they are not otherwise modified or added to. In particular, type triples for deleted/deprecated concepts are not added during migration. (But they will be retained, for any current resources you delete subsequently using the user interface.)
How your vocabulary data will now be transformed during publication to RVA
In general, we have tried to make both the "user experience" for RVA publishers, and the end result of publication, look as much as possible as it did with previous PoolParty releases.
- In RVA's My Vocabs, when you use the function to "Add a new vocabulary from PoolParty", the metadata fields (title, description, etc.) will continue to be pre-populated as before. The "related entity" data – i.e., authors, contributors, and publishers – that is represented in PoolParty using DC Terms properties, is extracted correctly from the underlying RDF data.
- When you import and publish a PoolParty vocabulary, only content from the
.../thesaurus/deprecatednamed graphs, as well as only the necessary content related to DC Terms properties that is stored in the
.../thesaurus/usersnamed graph, is included. In other words, only the "real" vocabulary data is harvested. For deprecated resources, type triples are discarded during harvest. During import into RVA's RDF repository (Sesame), the triples are not stored in named graphs; they continue to be all stored in the "default" ("unnamed") graph.
- During harvest, the triples corresponding to DC Terms properties are "normalised" by replacing the SWC-specific property
<http://schema.semantic-web.at/users/username>with the more widely used
The example given above thus becomes:
What you should do
As an editor
Once the migration is complete, we will ask you to log in to PoolParty and check your vocabularies to make sure there are no issues. Please contact us (by email to email@example.com) if you notice anything unexpected.
You may not have to do anything else at this stage.
As a publisher
If you already have PoolParty vocabularies published in RVA, you won't need to republish them. However, if you wish to see what this would look like, we would encourage you, after the upgrade is complete, to make use of the ANDS "demo" server, at http://demo.ands.org.au/vocabs/. If you normally log in to My Vocabs on RVA using AAF Rapid Connect, you can also log in to demo with the same account. However, you may not already have the correct organisational affiliations on demo: you won't then be able to select the correct organisation from the "Owner" field when you add a vocabulary. If you have this problem, please contact firstname.lastname@example.org.
As a PoolParty API user
Take note of the changes to the structure of the RDF, and if you have any questions, please don't hesitate to ask.
We'd like to thank the team at IMOS (Natalia Atkins, Kim Finney, Sebastien Mancini) for collaborating with us during the planning of this upgrade.