Snow Owl v8.0.0

Snow Owl's biggest release yet comes with fully customizable resource model and complete resource management functionality; the fastest RF2 importer and exporter in the world; externally configurable per resource access management; latest Expression Constraint Language 1.6 features and many many more.

Breaking changes

The following changes in Snow Owl 8.0 might affect your applications and prevent them from operating normally. Before upgrading to 8.0, review these changes and take the described steps to mitigate the impact.

RF2 index format changes

To support sorting, filtering and field selection on the official SNOMED CT RF2 property names, Snow Owl 8 comes with an updated index representation where each and every SNOMED CT component index has correct, officially supported RF2 column names. Migrating from Snow Owl 7.x to 8.0 requires a full re-import of all terminology resource content.

API routing changes

Snow Owl 8 changes the old /admin and /snomed-ct/v3 Core and SNOMED CT REST API base path segments to /core and /snomedct, respectively. Make sure you update your REST API clients when you start your migration process to the new version. See PR #893 for more details.

Update endpoint changes

Some of the old SNOMED CT Component endpoints were using a POST /:componentType/updates style instead of PUT /componentType. Snow Owl 8 migrated all API endpoints using the former pattern to the new one. Make sure you update your REST API clients to the new pattern, if you have used any of the SNOMED CT component update API endpoints. For more details, please see PR #866.

Configuration changes

• Snow Owl does not support node-level configuration of certain SNOMED CT specific settings. These all have been moved to Code System resource specific configuration keys and can be configured for each SNOMED CT specific Code System separately (eg. for each Extension). The affected configuration settings are:
  • RF2 Export API configuration settings (#842)
  • Language alias configuration (snomed.languages) (#869)
  • Other options, such as defaultReasonerId, defaultQueryTypeRefsetId, etc.

Removed custom MRCM model

To support variety of use cases and easier adoption of SNOMED CT MRCM constraints, Snow Owl from version 8 fully supports not just MRCM member access and authoring, but evaluation of official and custom (maintained by NRCs and Extensions) MRCM constraints in the fastest way possible. The old custom MRCM model and all related API functionality has been removed in favor of the official MRCM tooling. See PRs #917 and #930 for more details.

Removed scroll based paging

Scroll API in Elasticsearch has been marked as deprecated until complete removal in the upcoming Elasticsearch 8.0 release. Thus, Snow Owl is no longer offering low-level Java and index APIs to access the terminology content via scroll contexts. For more information see PR #903.

Core

• Introducing Resource Management (#821, #886)
  • Any type of resources (when the appropriate model and API is available, plug-ins can provide their own specific models if they so desire)
  • Supported resources are Code System, Bundle (and other resources such as Value Sets and Concept Maps are available in pro versions)
  • Full REST API support to access (get, search), author (create, update, delete) and maintain (versioning, configuration, history) resources
• New Core API features:
  • New concept suggest API (#878, #882, #884, #921)
  • Field selection support (#858)
    • Restricting the number of returned fields of a representation to a set of properties is now possible on read endpoints (e.g GET /snomedct/SNOMEDCT-UK-CL/concepts?field=id)
    • By default the system will always return all properties with a non-null value, but this can be altered using the field query parameter.
  • Support includeNull query parameter to allow API consumers to select whether they would like to get all properties (with any value, including null) or just the non-null ones (default setting) (#857)
  • Support pretty query parameter to pretty print the JSON output to make it human-readable (#857)
  • Support multi-value expand parameters on all search/get endpoints (#855, fixes #659)
  • Support filtering versions by createdAt and version (#874, cdae9ac)
  • Support Forwarded headers on incoming HTTP requests when needed (cc7c544)
  • Support deletion of Code Systems via DELETE /codesystems/:id (d416c1e)
  • Use yyyy-MM-dd effective time format when creating new versions (27e34ab)
  • Increase branch path segment max length to 100 characters from the previous 50 characters (2fe892e)
• New Index features:
  • Reduce the default number_of_shards to one as Elasticsearch offers the same default, number_of_replicas still defaults to 0 to support initial loading scenarios better (fdd10ce, 6b1d462)
  • Allow fully customizable indices settings in snowowl.yml configuration file via repository.index.[indexName] configuration objects (#929)
  • Customizable word equivalence (aka synonym) filtering (#863)
    • An SO_HOME/configuration/analysis/synonym.txt file can be configured to allow Snow Owl to use the defined set of synonym rules when performing full-text searches. In case of remote Elasticsearch cluster, it is required to deploy the synonyms.txt on all cluster nodes
  • Support index-level search request caching (#931)
    • Any kind of search can be cached and retrieved faster than ever before. Plug-in maintainers and new tool developers can configure their preference of how and when they'd like to cache the results of a search request. By default Snow Owl will cache all search requests that are executed on versioned state.
  • Support numeric script sorting in index searches (8e51c6b)
• Deprecated branch review API endpoints and services have been removed (b79c8e1)

SNOMED CT

• New SNOMED CT API features:
  • Support dedicated isActiveMemberOf filter which return concepts that has an active member in any of the given list of refsetIds (#879)
  • Support searching SNOMED CT components by namespaceConceptId not just namespace (#881, #920)
  • Support exporting Snapshot RF2 content at specific times via resourceId@timestamp format, Full and Delta are not supported (#887)
  • Support exporting RF2 content between specific branch path range, like comparing branches (#907)
  • Support nested expansion of inactivationIndicator() properties, like associationTarget(), referenceSet() (#914)
  • Support sorting when expanding descriptions(), relationships(), inboundRelationships() on Concept API (7e2516a)
  • Expose memberOf and activeMemberOf components document fields on component domain representations (5928462)
  • Support component type (concept, description, relationships, member) based filtering on RF2 export endpoint (c8662e4)
  • Support multi-value query parameters on a lot of filter query parameters (5e5fc2a)
  • Support module, active and effectiveTime filters on GET /refsets endpoint (5e5fc2a)
• RF2 import improvements (#918)
  • Support smart auto-generation of delta content when importing Full releases (the existing component-based effective time based filtering is still there, but this makes the import much faster and easier to understand)
  • Support importUntil effective time property to import only a portion from an RF2 release
  • Disallow importing RF2 releases with createVersions enabled when the target branch is a CodeSystem's main development branch and it has unpublished changes
  • Automatically configure known language dialect aliases after successful RF2 import (#876)
• Consolidate RF2 property names on index documents to allow sorting, filtering on them using official RF2 column names (#856)
• Update ECL to 1.6 features (#837, #915)
  • The Query Language Draft implementation has been removed and extra unofficial features of it have been merged entirely into the ECL syntax. The SNOMED CT Concept search REST API endpoint's query parameter has also been removed. If you had any query language specific expressions, you might need to migrate them to the new ECL 1.6 syntax which includes all previously supported QL features.
• Deprecated SNOMED CT /imports and /exports endpoints have been removed (82938f6)

Security

• Improved authorization options to allow/restrict access to server resources. Access tokens describe the necessary scopes to give access to certain resources, either by:
  • directly denoting the resource ID in the scope
  • denoting the containing bundle ID in the scope (supports direct and indirect parent bundles)
  • or by allowing access to all resources (*)
• rootDn and rootDnPassword support from ldap security realm configuration have been removed (6483870)

FHIR

• Support CodeSystem, ValueSet and ConceptMap filtering by _id, name, title, system, url, version and other useful FHIR properties that are supported in the underlying resource representation (#909)
• Support for CapabilityStatement and OperationDefinition resources (#899)
• Read-only Bundle API support (#867)
• Experimental Batch API support (#867)
• Revised and improved all existing FHIR API endpoints to resolve all outstanding issues, such as (#416, #426, #435, #494, #561)
• Support after cursor based paging when expanding Value Sets (e9431f9)

Documentation

• Update REST API documentation with generic and SNOMED CT concept specific information (#906) (more will come in future 8.x releases)
• Upgrade REST API documentation to Springdoc and OAS3 (#845)
• Use rapidoc instead of Swagger UI (0948ef4, b78b71d, fix...

Snow Owl v7.17.6

Bugs/Improvements

• [core] fix incorrectly backported duplicate revision fix (#913, f6e3c31)
• [cli] add duplicate revision checker cli tool, see snowowl snomed revision command for details (8d9d1e5)

Snow Owl v7.17.5

Bugs/Improvements

• [core] backport fix for duplicate revision after synchronization (#913)
• [snomed] allow only snapshot RF2 export for point-in-time branches (#907)
• [snomed] add support for delta RF2 exports with branch path ranges (#907)
• [mrcm] make MRCM csv/json export format ordered, fix missing columns in CSV export (#910)

Snow Owl v7.17.4

Bugs/Improvements

• [core] report conflicts properly during code system synchronization (#888)
• [snomed] make source branch of MRCM export configurable (#890)
• [snomed] support validation of relationships with concrete values (#895)
• [releng] store source branch as label in docker image (235af78, 374dec5)

Snow Owl v7.17.3

Bugs/Improvements

• [snomed]: handle integer and decimal values as numbers and make sure they classify as is without type or value changes (#868)
• [snomed]: support nested branches when importing MRCM rules (#870)
• [api]: allow synchronization of empty extensions/branches without issues (#849)
• [api]: expose CodeSystem Sync and Complete API (#872, #873)

Snow Owl v7.17.2

Bugs/Improvements

• [build] improvements to b2i-ci's Jenkinsfile (skip deploy, downstream build configuration) (27de738, eba50fe)

Snow Owl v7.17.1

Bugs/Improvements

• [index] use real-time scrolling instead of snapshot scroll when creating compare between two branches (11de246)
• [snomed] validate incorrect partition and component identifiers when validating any SNOMED CT IDs during RF2 import (4bc59b1)
• [snomed] improve concept preferred description change processing performance (9cb727c)
• [api] apply alternate type rule for ComponentIdentifier type (9f50aeb)
• [api] return with proper error response when labeling up ECL expressions via POST /label-expressions API (#830)

Snow Owl v7.17.0

Core

• Code System API
  • Move to POST /upgrade endpoint (#808)
  • Support selecting Code System versions as sources when creating upgrades to let authors continue working on unpublished content without the need of fixing data at two places (#808, #822)
  • Create versions on non-working branches is now disabled (#799, #810)

SNOMED CT

• Support for SNOMED CT Concrete Value Specification (RF2 2021-07-31) (#811)
  • Add complete support for SNOMED International Concrete Value Specification, including import and export of the new der2_RelationshipConcreteValues RF2 file, searching concrete values using ECL expressions and editing then classifying concrete values.
• Add new ignoreMissingReferencesIn configuration option to RF2 file imports (#803)
  • This option can be used to define SNOMED CT Reference Set IDs when importing an RF2 archive where the selected Reference Sets might reference non-existent components. (eg. SNOMED CT UK Clinical Extension, UK Edition RF2 content often released with such reference set memberships)
• New POST /:path/label-expressions endpoint to label up one or more ECL expressions using the selected description type (FSN or PT) and locales (#814, #817)
• New validation rules (#801, #804)
  • Active terms should not contain double spaces
  • Active terms should be capitalized or have Entire term case sensitive case significance
  • Spacing around hyphens should be consistent in active terms
  • In active terms, spaces should not precede commas, colons, semicolons or full stops
  • Descriptions which are not Fully Specified Names should not contain semantic tags
  • The fully specified names of active concept should contain a bracketed suffix (semantic tag)
  • Terms should only contain ASCII characters
  • Fully specified names should not contain <, >, &, %, $, @, # characters
  • Synonyms should not contain $, @ characters
  • Brackets in active terms should balance and be sequentially valid (in correct order)
  • FSN terms should not duplicate other FSN terms, regardless of case
  • Synonyms should not duplicate other synonyms, regardless of case
  • Reference Sets should not contain retired concepts
  • Reference Sets should not contain retired descriptions

FHIR API

• Add initial support for FHIR Search Parameters, like _id, _name (#806)
• Add initial support for FHIR $validate-code operation endpoints (#816)

Bugs/Improvements

• [index] use segment diff to compute accurate base/head timestamps when merging empty branches (#807)
• [index] support indexing fields with type double (b275329)
• [index] don't create "null" strings when mapping results to String[] (c78a710)
• [index] support exlucing non-property type changes when comparing branches (#815, #818, #825)
• [index] cache rootField and fieldPaths to prevent recomputing them when processing larger commits (1cb9e6a)
• [index] compute property diffs only for FSNs during change processing to reduce memory pressure (fe6ed78)
• [core] improve concept map compare comment merge phase to avoid exponential complexity and never ending compares (67d20d6)
• [snomed] fixed an issue where importing released version of a component did not set the released flag to true after RF2 import (#802)
• [classification] commit classification relationship/concrete value changes in batches (#809)
• [quality] fix major and minor code smells (#819, #824) Thank you for the contribution @eranhash!

Dependencies

• Upgrade snomed-ecl to 1.4.3 (#826)

Snow Owl v7.16.2

Bugs/Improvements

• [core] support automatic merging of conflict-free nested objects and arrays (#790, #791)
• [snomed] register reactivated concepts for taxonomy updates (#792)
• [snomed] use "broken bone" icon for morphologic abnormalities (cfe08be)
• [api] disable versioning of upgrade codesystems (#789)
• [cli] improved formatting of 'snowowl codesystems' command (#761)

Dependencies

• Bump Eclipse Platform to 2020-09 (#779)

Snow Owl v7.16.1

Bugs/Improvements

• [index] support fields alias migration on index level (#783)
• [snomed] exclude inactive OWL Axioms when calculating concept's definition status (#785)
• [validation] consider moduleId when calculating reference set member duplication in SIMPLE, LANGUAGE, ASSOCIATION and ATTRIBUTE reference sets (#782)