feat: add support for jsonldContext member in subscriptions #1230

ranim-n · 2024-08-28T13:44:16Z

No description provided.

bobeal · 2024-08-28T16:15:37Z

some points are missing in the implementation.

for instance, in 5.8.1.4, it is said: The @context to be used for sending Notifications related to this Subscription shall be the one specified in the jsonldContext field. If not present, the jsonldContext field shall be initialized with the @context applicable for the Subscription (if @context is also not present in the Subscription, see clause 5.5.5). When the remote JSON-LD @context referenced by the jsonldContext field is not available implementations shall raise an error of type LdContextNotAvailable. If the remote JSON-LD @context referenced by the jsonldContext field is invalid, implementations shall raise an error of type BadRequestData.

(search for jsonldContext in the specification)

github-actions · 2024-08-28T19:44:44Z

Test Results

62 files ±0 62 suites ±0 1m 23s ⏱️ +2s
1 014 tests +4 1 014 ✅ +4 0 💤 ±0 0 ❌ ±0
1 053 runs +4 1 053 ✅ +4 0 💤 ±0 0 ❌ ±0

Results for commit d928b62. ± Comparison against base commit 2d2e0ff.

This pull request removes 190 and adds 38 tests. Note that renamed tests count towards both.

                                    { "id":…, withTemporalValues=true, withAudit=false, expectation={
                      "@id": "https://uri…
                      "@type": "@json",
                      …
                    "@value": "/A/B"
                    "@value": "/C/D"
                    "@value": 20
                    "…
                    {
                  "@type": "https://uri.etsi.org/ngsi-ld/DateTime",
…

com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [1] source={
    "attribute": {
        "type": "Property",
        "value": 12.0,
        "observedAt": "2024-04-14T12:34:56Z"
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM"
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "observedAt": "2024-04-14T12:34:56Z"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [1] source={
    "attribute": {
        "type": "Property",
        "value": 12.0,
        "observedAt": "2024-04-14T12:34:56Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "subAttribute": {
            "type": "Property",
            "value": "newSubAttributeValue"
        }
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "observedAt": "2024-04-14T12:34:56Z",
        "subAttribute": {
            "type": "Property",
            "value": "newSubAttributeValue"
        }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [2] source={
    "attribute": {
        "type": "Property",
        "value": { "en": "car", "fr": "voiture" }
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": { "fr": "vélo", "es": "bicicleta" }
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": { "en": "car", "fr": "vélo", "es": "bicicleta" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [2] source={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 1, "b": null, "c": 12.4 },
        "observedAt": "2022-12-24T14:01:22.066Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}, target={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 2, "b": "something" },
        "observedAt": "2023-12-24T14:01:22.066Z"
    }
}, expected={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 2, "b": "something" },
        "observedAt": "2023-12-24T14:01:22.066Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [3] source={
    "attribute": {
        "type": "Property",
        "value": [ "car", "voiture" ]
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": [ "vélo", "bicicleta" ]
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": [ "vélo", "bicicleta" ]
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [3] source={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "stellio"
    }
}, target={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}, expected={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [4] source={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:01"
    }
}, target={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:02"
    }
}, expected={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:02"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [5] source={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "en": "train", "fr": "train" }
    }
}, target={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "fr": "TGV", "es": "tren" }
    }
}, expected={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "en": "train", "fr": "TGV", "es": "tren" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [6] source={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 1, "b": "thing" }
    }
}, target={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 2, "c": "other thing" }
    }
}, expected={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 2, "b": "thing", "c": "other thing" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [7] source={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "stellio"
    }
}, target={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}, expected={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}
…

♻️ This comment has been updated with latest results.

ranim-n · 2024-08-29T13:20:28Z

some points are missing in the implementation.

for instance, in 5.8.1.4, it is said: The @context to be used for sending Notifications related to this Subscription shall be the one specified in the jsonldContext field. If not present, the jsonldContext field shall be initialized with the @context applicable for the Subscription (if @context is also not present in the Subscription, see clause 5.5.5). When the remote JSON-LD @context referenced by the jsonldContext field is not available implementations shall raise an error of type LdContextNotAvailable. If the remote JSON-LD @context referenced by the jsonldContext field is invalid, implementations shall raise an error of type BadRequestData.

(search for jsonldContext in the specification)

to check if a jsonld context is available or valid do I make an http request to it to make sure it is reachable or do I use it in compaction/expansion to test it... ?

bobeal · 2024-08-29T13:46:58Z

to check if a jsonld context is available or valid do I make an http request to it to make sure it is reachable or do I use it in compaction/expansion to test it... ?

Doing a compaction or expansion is too expensive. So option 1.

The JSON-LD library should already have some functions to do it (this is something it is doing every time we expand / compact). Maybe in https://github.com/filip26/titanium-json-ld/blob/main/src/main/java/com/apicatalog/jsonld/loader/DefaultHttpLoader.java?

ranim-n · 2024-08-30T11:56:06Z

to check if a jsonld context is available or valid do I make an http request to it to make sure it is reachable or do I use it in compaction/expansion to test it... ?

Doing a compaction or expansion is too expensive. So option 1.

The JSON-LD library should already have some functions to do it (this is something it is doing every time we expand / compact). Maybe in https://github.com/filip26/titanium-json-ld/blob/main/src/main/java/com/apicatalog/jsonld/loader/DefaultHttpLoader.java?

when updating a subscription there are no checks (see #1090 ) should I do a check for the jsonldContext as stated in 5.8.2.4 anyway ?

bobeal · 2024-08-30T13:29:41Z

to check if a jsonld context is available or valid do I make an http request to it to make sure it is reachable or do I use it in compaction/expansion to test it... ?

Doing a compaction or expansion is too expensive. So option 1.
The JSON-LD library should already have some functions to do it (this is something it is doing every time we expand / compact). Maybe in https://github.com/filip26/titanium-json-ld/blob/main/src/main/java/com/apicatalog/jsonld/loader/DefaultHttpLoader.java?

when updating a subscription there are no checks (see #1090 ) should I do a check for the jsonldContext as stated in 5.8.2.4 anyway ?

Start with the creation only. Will see later.

subscription-service/config/detekt/baseline.xml

shared/config/detekt/baseline.xml

thomasBousselin · 2024-09-03T12:57:49Z

did you look at the sonar issues?

subscription-service/src/test/resources/ngsild/subscription_full.json

shared/src/main/kotlin/com/egm/stellio/shared/util/JsonLdUtils.kt

bobeal · 2024-09-04T15:39:26Z

...cription-service/src/main/kotlin/com/egm/stellio/subscription/service/NotificationService.kt

@@ -49,9 +49,11 @@ class NotificationService(
                        AttributeRepresentation.SIMPLIFIED
                    else AttributeRepresentation.NORMALIZED

+                val context = it.jsonldContext?.toString()?.let { listOf(it) } ?: it.contexts


since context is a list, call it contexts

a bit easier to read: it.jsonldContext?.let { listOf(it.toString()) } ?: it.contexts

bobeal · 2024-09-04T15:51:09Z

...ion-service/src/test/kotlin/com/egm/stellio/subscription/service/NotificationServiceTests.kt

+            val notificationResult = notificationResults[0]
+            assertEquals(subscription.id, notificationResult.first.id)
+            assertEquals(subscription.id, notificationResult.second.subscriptionId)
+            assertEquals(1, notificationResult.second.data.size)
+            assertTrue(notificationResult.second.data[0].containsKey(NGSILD_NAME_TERM))
+            assertTrue(notificationResult.second.data[0].containsKey(MANAGED_BY_COMPACT_RELATIONSHIP))
+            assertEquals(APIC_COMPOUND_CONTEXT, notificationResult.second.data[0][JsonLdUtils.JSONLD_CONTEXT])
+            assertTrue(notificationResult.third)


no need to repeat the assertions of the previous test. here, you want to check what should be different when you have a specific jsonldContext.

I am also a bit surprised that the entity in the notification has the same keys than the entity in the previous test, whereas they are not compacted with the same contexts.

The keys are not the same. In the previous entity they are NGSILD_NAME_PROPERTY and MANAGED_BY_RELATIONSHIP) . In this entity they are NGSILD_NAME_TERM and MANAGED_BY_COMPACT_RELATIONSHIP

indeed, my bad

bobeal · 2024-09-04T16:12:38Z

shared/src/main/kotlin/com/egm/stellio/shared/util/JsonLdUtils.kt

+    fun checkJsonldContext(context: URI): Either<APIException, Unit> = either {
+        return try {
+            context.toString().toUri()
+            val options = DocumentLoaderOptions()
+            loader.loadDocument(context, options)
+            Unit.right()
+        } catch (e: JsonLdError) {
+            e.toAPIException(e.cause?.cause?.message).left()
+        } catch (e: BadRequestDataException) {
+            e.left()
+        }
+    }


what about:

fun checkJsonldContext(context: URI): Either<APIException, Unit> = either { return try { loader.loadDocument(context, DocumentLoaderOptions()) Unit.right() } catch (e: JsonLdError) { e.toAPIException(e.cause?.cause?.message).left() } catch (e: IllegalArgumentException) { BadRequestDataException(e.cause?.message ?: "Provided context is invalid: $context").left() } }

bobeal · 2024-09-06T08:53:34Z

@ranim-n don't forget the last comments in this PR

thomasBousselin · 2024-09-06T09:28:49Z

have you check getContextsForSubscription() function

...cription-service/src/main/kotlin/com/egm/stellio/subscription/service/SubscriptionService.kt

bobeal · 2024-09-06T16:59:54Z

LGTM

ranim-n · 2024-09-09T07:09:04Z

@thomasBousselin

sonarcloud · 2024-09-09T08:20:04Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
63.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

ranim-n added feature New feature or request subscription Relates to subscription API ngsild-1.7.1 labels Aug 28, 2024

ranim-n requested review from bobeal and thomasBousselin August 28, 2024 13:44

ranim-n self-assigned this Aug 28, 2024

ranim-n linked an issue Aug 28, 2024 that may be closed by this pull request

Add support for jsonldContext member in subscriptions #1186

Closed

Base automatically changed from refactor/prepare-handlers-for-csr to develop August 30, 2024 13:33

bobeal force-pushed the feature/support-for-jsonldContext-in-subscriptions branch from 4254e8b to ba3e0e9 Compare August 31, 2024 06:31

thomasBousselin reviewed Sep 2, 2024

View reviewed changes

subscription-service/config/detekt/baseline.xml Outdated Show resolved Hide resolved

shared/config/detekt/baseline.xml Outdated Show resolved Hide resolved

bobeal reviewed Sep 3, 2024

View reviewed changes

subscription-service/src/test/resources/ngsild/subscription_full.json Outdated Show resolved Hide resolved

bobeal reviewed Sep 3, 2024

View reviewed changes

shared/src/main/kotlin/com/egm/stellio/shared/util/JsonLdUtils.kt Outdated Show resolved Hide resolved

bobeal reviewed Sep 4, 2024

View reviewed changes

thomasBousselin approved these changes Sep 6, 2024

View reviewed changes

bobeal approved these changes Sep 6, 2024

View reviewed changes

bobeal reviewed Sep 6, 2024

View reviewed changes

...cription-service/src/main/kotlin/com/egm/stellio/subscription/service/SubscriptionService.kt Outdated Show resolved Hide resolved

thomasBousselin approved these changes Sep 9, 2024

View reviewed changes

ranim-n added 2 commits September 9, 2024 10:12

feat: add support for jsonldContext member in subscriptions

b608433

added jsonldContext initialization and validation

427eb59

ranim-n added 5 commits September 9, 2024 10:12

re-implementing jsonldContext without initialization

240616b

exceptions handling + some fixes

a1e06e8

minor fixes

cb2945d

adapting link header when sending notification to jsonldContext

5236964

minor fix

d928b62

ranim-n force-pushed the feature/support-for-jsonldContext-in-subscriptions branch from fcd1ce8 to d928b62 Compare September 9, 2024 08:12

ranim-n merged commit cf5b4f7 into develop Sep 9, 2024
10 checks passed

ranim-n deleted the feature/support-for-jsonldContext-in-subscriptions branch September 9, 2024 08:21

github-actions bot locked and limited conversation to collaborators Sep 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for jsonldContext member in subscriptions #1230

feat: add support for jsonldContext member in subscriptions #1230

ranim-n commented Aug 28, 2024

bobeal commented Aug 28, 2024

github-actions bot commented Aug 28, 2024 •

edited

Loading

ranim-n commented Aug 29, 2024

bobeal commented Aug 29, 2024

ranim-n commented Aug 30, 2024

bobeal commented Aug 30, 2024

thomasBousselin commented Sep 3, 2024

bobeal Sep 4, 2024

bobeal Sep 4, 2024

ranim-n Sep 5, 2024

bobeal Sep 5, 2024

bobeal Sep 4, 2024

bobeal commented Sep 6, 2024

thomasBousselin commented Sep 6, 2024

bobeal commented Sep 6, 2024

ranim-n commented Sep 9, 2024

sonarcloud bot commented Sep 9, 2024

feat: add support for jsonldContext member in subscriptions #1230

feat: add support for jsonldContext member in subscriptions #1230

Conversation

ranim-n commented Aug 28, 2024

bobeal commented Aug 28, 2024

github-actions bot commented Aug 28, 2024 • edited Loading

Test Results

ranim-n commented Aug 29, 2024

bobeal commented Aug 29, 2024

ranim-n commented Aug 30, 2024

bobeal commented Aug 30, 2024

thomasBousselin commented Sep 3, 2024

bobeal Sep 4, 2024

Choose a reason for hiding this comment

bobeal Sep 4, 2024

Choose a reason for hiding this comment

ranim-n Sep 5, 2024

Choose a reason for hiding this comment

bobeal Sep 5, 2024

Choose a reason for hiding this comment

bobeal Sep 4, 2024

Choose a reason for hiding this comment

bobeal commented Sep 6, 2024

thomasBousselin commented Sep 6, 2024

bobeal commented Sep 6, 2024

ranim-n commented Sep 9, 2024

sonarcloud bot commented Sep 9, 2024

Quality Gate passed

github-actions bot commented Aug 28, 2024 •

edited

Loading