Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apache Camel Cartridge parses DFDL EDIFACT schema upon the first message arrival only #120

Open
uliSchuster opened this issue Jun 27, 2022 · 0 comments
Open

Apache Camel Cartridge parses DFDL EDIFACT schema upon the first message arrival only #120

uliSchuster opened this issue Jun 27, 2022 · 0 comments
Assignees
@cjmamo

Comments

@uliSchuster
Copy link

I am using the Smooks Camel Cartridge together with the EDIFACT cartridge to parse EDIFACT files. I observe that the Smooks processor does not seem to be fully initialized at first. The DFDL schema is parsed only once the first EDIFACT message is processed, as can be seen in the log output below. I would expect that the processor is initialized earlier, once it is created as part of the Camel context.

Here is my relevant configuration code (Kotlin):

object EdifactParserFactory {

    private const val schemaPostfix: String = "EDIFACT-Messages.dfdl.xsd"

    fun mkEdifactParser(
        edifactDirectory: EdifactDirectory,
        messageTypes: List<EdiEnergyMessageType>,
        camelContext: CamelContext
    ): SmooksProcessor {
        [..]
        val schemaUri = "/${edifactDirectory/${schemaPostfix}"
        val smooks = Smooks()
        val reader = EdifactReaderConfigurator(schemaUri).setMessageTypes(messageTypeList)
        reader.cacheOnDisk = true
        smooks.setReaderConfig(reader)
        val filterSettings = FilterSettings.newSaxNgSettings().setReaderPoolSize(5)
        smooks.setFilterSettings(filterSettings)
        smooks.setExports(Exports(StringResult::class.java))
        return SmooksProcessor(smooks, camelContext)
    }
}

For the following (simplified) Camel route:

from(direct("inbound"))
            .convertBodyTo(InputStream::class.java)
            .choice()
                .`when`(header(EDIFACT_DIRECTORY_HEADER).isEqualTo(EdifactDirectory.D04B)).to(direct("D04B-import"))
                .`when`(header(EDIFACT_DIRECTORY_HEADER).isEqualTo(EdifactDirectory.D07B)).to(direct("D07B-import"))
                .otherwise().to("log:error").stop()
            .end()

[..]

val edifactParser = EdifactParserFactory.mkEdifactParser(dir, messageTypes, camelContext)

[..]

from(direct("D04B-import")
      .process(edifactParser)

[..]
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.7.0)

2022-06-27 09:40:22 INFO  ReceiveTestInterchange:55 - Starting ReceiveTestInterchange using Java 17.0.3
[..]
2022-06-27 09:40:25 INFO  ReceiveTestInterchange:61 - Started ReceiveTestInterchange in 3.391 seconds (JVM running for 4.847)
[..]
2022-06-27 09:40:25 INFO  CamelSpringBootExecutionListener:104 - Initialized CamelSpringBootExecutionListener now ready to start CamelContext
2022-06-27 09:40:25 INFO  CamelAnnotationsHandler:404 - Starting CamelContext with name [camelContext].
2022-06-27 09:40:25 WARN  Scanner:78 - Not scanning classpath for ClassLoader 'jdk.internal.loader.ClassLoaders$AppClassLoader'.  ClassLoader must implement 'java.net.URLClassLoader'.
2022-06-27 09:40:25 WARN  Scanner:78 - Not scanning classpath for ClassLoader 'jdk.internal.loader.ClassLoaders$AppClassLoader'.  ClassLoader must implement 'java.net.URLClassLoader'.
2022-06-27 09:40:25 INFO  EdifactParserFactory:39 - Created Smooks EDIFACT parser from directory D04B for message(s) [INVOIC, CONTRL]
2022-06-27 09:40:26 WARN  Scanner:78 - Not scanning classpath for ClassLoader 'jdk.internal.loader.ClassLoaders$AppClassLoader'.  ClassLoader must implement 'java.net.URLClassLoader'.
2022-06-27 09:40:26 WARN  Scanner:78 - Not scanning classpath for ClassLoader 'jdk.internal.loader.ClassLoaders$AppClassLoader'.  ClassLoader must implement 'java.net.URLClassLoader'.
2022-06-27 09:40:26 INFO  EdifactParserFactory:39 - Created Smooks EDIFACT parser from directory D07B for message(s) [APERAK]
2022-06-27 09:40:26 INFO  AbstractCamelContext:3004 - Apache Camel 3.17.0 (camel-1) is starting
2022-06-27 09:40:26 INFO  AbstractCamelContext:3222 - Tracing is enabled on CamelContext: camel-1
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
[..]
2022-06-27 09:40:26 INFO  CamelSpringBootApplicationListener:147 - Starting CamelMainRunController to ensure the main thread keeps running
2022-06-27 09:40:26 INFO  MainSupport:68 - Apache Camel (Main) 3.17.0 is starting
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 WARN  SmooksProcessor:259 - Attachments module could not be found: attachments will not be propagated
2022-06-27 09:40:26 INFO  SmooksProcessor:266 - SmooksProcessor [configUri=null] Started
2022-06-27 09:40:26 INFO  AbstractCamelContext:3165 - Routes startup (total:6 started:6)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3170 -     Started D04B-import-route (direct://D04B-import)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3170 -     Started D07B-import-route (direct://D07B-import)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3170 -     Started edifact-inbound-transfer-route (direct://edifact-inbound-transfer)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3170 -     Started inbound-acknowledge-route (direct://inbound-acknowledge)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3170 -     Started inbound-route (direct://inbound)
2022-06-27 09:40:26 INFO  AbstractCamelContext:3194 - Apache Camel 3.17.0 (camel-1) started in 1s351ms (build:78ms init:943ms start:330ms)
2022-06-27 09:40:27 INFO  Tracing:264 - *--> [inbound-rout] [from[direct://inbound]           ] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: byte[], ...]
2022-06-27 09:40:27 INFO  Tracing:264 -      [inbound-rout] [direct://edifact-inbound-transfer] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: byte[], ...]
2022-06-27 09:40:27 INFO  Tracing:264 - ---> [edifact-inbo] [from[direct://edifact-inbound-tra] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: byte[], ...]
2022-06-27 09:40:27 INFO  Tracing:264 -      [edifact-inbo] [convertBodyTo[java.io.InputStream] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: byte[], ...]
2022-06-27 09:40:27 INFO  Tracing:264 -      [edifact-inbo] [choice[when[{header(EdifactDirect] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: org.apache.camel.converter.stream.ByteArrayInputStreamCache, Body: [Body is instance of org.apache.camel.StreamCache]]
2022-06-27 09:40:27 INFO  Tracing:264 -      [edifact-inbo] [direct://D04B-import             ] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: org.apache.camel.converter.stream.ByteArrayInputStreamCache, Body: [Body is instance of org.apache.camel.StreamCache]]
2022-06-27 09:40:27 INFO  Tracing:264 - ---> [D04B-import-] [from[direct:{{edifactDirectory}}-] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: org.apache.camel.converter.stream.ByteArrayInputStreamCache, Body: [Body is instance of org.apache.camel.StreamCache]]
2022-06-27 09:40:27 INFO  Tracing:264 -      [D04B-import-] [ref:{{edifactParser}}            ] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: org.apache.camel.converter.stream.ByteArrayInputStreamCache, Body: [Body is instance of org.apache.camel.StreamCache]]
2022-06-27 09:40:27 INFO  DataProcessorFactory:106 - Compiling and caching DFDL schema...
2022-06-27 09:40:27 INFO  DfdlSchema:117 - Saving compiled DFDL schema to .smooks/dfdl-cartridge/EDIFACT-Interchange-9800488b-61fc-3578-af0d-05483fa510ef-.dfdl.xsd.dat
2022-06-27 09:40:37 INFO  Tracing:264 -      [D04B-import-] [Processor@0x22657db1             ] Exchange[Id: 583FC68FED10EF5-0000000000000000, BodyType: String, ...]
[..]

As can be observed on the previous-to-last log entry, the DFDL schema is compiled (or loaded from cache) once the first camel exchange arrives at the Smooks processor only. (Plus, there are some other irregularities that I haven't been able to diagnose yet.). I am running on Kotlin 1.6.20 with JRE 17 as compile target.
@cjmamo cjmamo self-assigned this Dec 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cjmamo cjmamo

Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cjmamo @uliSchuster