Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"StackOverflowError" for specific schema - allOf #50

Open
shoffmeister opened this issue Sep 11, 2024 · 1 comment
Open

"StackOverflowError" for specific schema - allOf #50

shoffmeister opened this issue Sep 11, 2024 · 1 comment

Comments

@shoffmeister
Copy link

Running kwack on a specific JSON schema

java -jar ./target/kwack-0.11.0-jar-with-dependencies.jar --bootstrap-server=localhost:9192 --topic MYTOPIC --value-serde=MYTOPIC=json:@MYTOPIC_SCHEMA.json --key-serde=MYTOPIC=string

yields a StackOverflow error

SLF4J(I): Connected with provider of type [org.slf4j.reload4j.Reload4jServiceProvider]
log4j:WARN No appenders could be found for logger (io.kcache.kwack.KwackConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.StackOverflowError
        at java.base/java.util.Arrays.hashCode(Arrays.java:4499)
        at java.base/java.util.Objects.hash(Objects.java:133)
        at org.everit.json.schema.Schema.hashCode(Schema.java:268)
        at org.everit.json.schema.StringSchema.hashCode(StringSchema.java:158)
        at java.base/java.util.Objects.hashCode(Objects.java:103)
        at java.base/java.util.HashMap$Node.hashCode(HashMap.java:299)
        at java.base/java.util.AbstractMap.hashCode(AbstractMap.java:526)
        at java.base/java.util.Collections$UnmodifiableMap.hashCode(Collections.java:1540)
        at java.base/java.util.Arrays.hashCode(Arrays.java:4499)
        at java.base/java.util.Objects.hash(Objects.java:133)
        at org.everit.json.schema.ObjectSchema.hashCode(ObjectSchema.java:365)
        at java.base/java.util.ArrayList.hashCodeRange(ArrayList.java:595)
        at java.base/java.util.ArrayList.hashCode(ArrayList.java:582)
        at java.base/java.util.Arrays.hashCode(Arrays.java:4499)
        at java.base/java.util.Objects.hash(Objects.java:133)
        at org.everit.json.schema.CombinedSchema.hashCode(CombinedSchema.java:238)
        at java.base/java.util.Arrays.hashCode(Arrays.java:4499)
        at java.base/java.util.Objects.hash(Objects.java:133)
        at org.everit.json.schema.ReferenceSchema.hashCode(ReferenceSchema.java:159)
        at java.base/java.util.HashMap.hash(HashMap.java:338)
        at java.base/java.util.HashMap.put(HashMap.java:610)
        at java.base/java.util.HashSet.add(HashSet.java:221)
        at io.kcache.kwack.transformer.json.JsonTransformer.collectPropertySchemas(JsonTransformer.java:210)
        at io.kcache.kwack.transformer.json.JsonTransformer.allOfToConnectSchema(JsonTransformer.java:149)
        at io.kcache.kwack.transformer.json.JsonTransformer.messageToColumn(JsonTransformer.java:286)
        at io.kcache.kwack.transformer.json.JsonTransformer.messageToColumn(JsonTransformer.java:288)
@shoffmeister
Copy link
Author

Unfortunately, I have not (yet) managed to trim down the rather large MYTOPIC_SCHEMA.json into something that does not contain sensitive information and which is minimally reproducing.

I can characterize that schema though as

  • having quite a few "allOf" (cf allOfToConnectSchema(JsonTransformer.java:149) in the stack trace)
  • generally (presumably) being somewhat structurally sound and correct

As a stop-gap I have taken to https://github.com/wolverdude/genson/ to reverse-engineer a schema from the JSON data.

kwack does accept that reverse-engineered schema for processing, and the content that ends up in the DuckDB database matches my expectations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant