Unable to run conformance test (XCA initiating gateway)

[elawende]

I want to run the conformance test for XCA initiating gateway, but unfortunately I can't get it to work. I keep getting errors during initialization or during the test run, before establishing a connection with the system under test.

I used version 7.8.0 with a clean deployment (default initialization of the external cache). It is deployed using Docker with the following dockerfile (the tomcat:9.0 base image is, as far as I can tell, based on Ubuntu):

FROM tomcat:9.0
RUN mkdir -p /path/to/my/external/cache
COPY xdstools7.8.0.war /usr/local/tomcat/webapps/xdstools7.war
COPY tomcat-users.xml /usr/local/tomcat/conf
EXPOSE 8080
CMD ["catalina.sh", "run"]

After deployment, I performed the following steps:

1. Created system configuration
2. Go to Conformance tests
3. Selected my system in the blue box (test context)
4. Go to XCA, non-affinity domain option.
5. Initialise the test environment

Initialisation with TLS fails with the following error (toolkit is unable to connect to its own simulator):

Initialization of default__community1 (section onedoc1) failed:
:15807: in default
onedoc1/submit_single_doc: 1 DocumentEntries 1 SubmissionSets 1 Associations 0 Folders0 Resources1: [ Using TLS ][ ]
2: [ Not using SAML ][ ]
3: [ Site: default__community1 ][ ]
4: [ Parameters: ][ ]
5: [ ...$testdata_home$: urn:oid:1.2.34.567.8.1 ][ ]
6: [ ...$patientid$: P0907114939.7^^^&1.3.6.1.4.1.21367.13.20.1000&ISO ][ ]
7: [ Starting ][ ]
8: [ ------------------------------------------------------------------------------------------------ ][ ]
9: [ Test: :15807: in default ][ ]
10: [ ------------------------------------------------------------------------------------------------ ][ ]
11: FAIL [ Status: Fail ][ ]
12: [ Section: 15807/onedoc1 Step: submit_single_doc ][ ]
13: [ Endpoint: https://localhost:8443/xdstools7/sim/default__community1/rep/prb ][ ]
14: [ Goals: ][ ]
15: FAIL [ SOAP Fault: Connection refused (stepId=submit_single_doc) ][ ]
16: FAIL [ No Message (stepId=submit_single_doc) ][ ]
17: FAIL [ Result was null (stepId=submit_single_doc) ][ ]
18: [ ------------------------------------------------------------------------------------------------ ][ ]
19: [ Finished ][ ]

When I try initialisation without TLS, initialisation seems to be ok. When I run the first test, it fails with the following internal error (cannot find log.xml):

tomcat_1  | Site is default__community1
tomcat_1  | Test: 12300
tomcat_1  | Sections: [finddocs]
tomcat_1  | Run section /usr/local/tomcat/webapps/xdstools7/toolkitx/testkit/tests/12300/finddocs/testplan.xml
tomcat_1  | INFO:       Step: finddocs 
tomcat_1  | INFO: endpoint coming from actors.xml 
tomcat_1  | INFO: Transaction = xcq Endpoint = https://localhost:8443/xdstools7/sim/default__community1/rg/xcq 
tomcat_1  | INFO:         Endpoint = https://localhost:8443/xdstools7/sim/default__community1/rg/xcq 
tomcat_1  | INFO: [<UseId testdir="../../12318/init" id="SubmissionSet01" step="init" section="AssignedPatientId" symbol="$patient_id$" />] 
tomcat_1  | INFO: Error: stepId=finddocs : Internal Error: finddocs: Could not find input file /path/to/my/external/cache/TestLogCache/default/12300/finddocs/../../12318/init/log.xml
tomcat_1  | Exception XdsInternalException
tomcat_1  |     gov.nist.toolkit.xdsexception.client.XdsInternalException: finddocs: Could not find input file /path/to/my/external/cache/TestLogCache/default/12300/finddocs/../../12318/init/log.xml
tomcat_1  |     at gov.nist.toolkit.testengine.transactions.BasicTransaction.compileUseIdLinkage(BasicTransaction.java:671)
tomcat_1  |     at gov.nist.toolkit.testengine.transactions.BasicTransaction.prepareMetadata(BasicTransaction.java:949)
tomcat_1  |     at gov.nist.toolkit.testengine.transactions.BasicTransaction.doRun(BasicTransaction.java:222)
tomcat_1  |     at gov.nist.toolkit.testengine.engine.StepContext.run(StepContext.java:487)

I tried version 7.3.4 but this version complained about an HTTP500 error on (almost?) every request. I also tried to deploy the toolkit on Windows, but could not get this running.

Is this a bug, or am I missing some deployment/configuration step?

[skbhaskarla]

This issue should be posted on the XDS Implementors Google group.

Only the standard installation of NIST XDS Toolkit is supported (if there are Docker specific issues then it requires additional troubleshooting on your part).

Please see the Installation guide here: https://github.com/usnistgov/iheos-toolkit2/wiki/installing

According to the Configure section, "When Toolkit starts for the first time it is likely that the configured External Cache location will not exist on your machine." So "/path/to/my/external/cache" is not realistic and is only used as a reminder to change the path and not intended to be used as-is. You should create a new external cache directory you are more likely to remember.

TLS requires additional configuration: https://github.com/usnistgov/iheos-toolkit2/wiki/installing#configuring-for-tls

In your the last part of your post, you mentioned you tried a test without TLS, but the log you provided seems to use TLS: https://localhost:8443/xdstools7/sim/default__community1/rg/xcq

You should be using the latest release, don't worry about older releases.

[elawende]

Hi Sunil,

Thanks for your reply. The request to join the XDS Implementors Google group was already sent, but since after one day I still don't have access, I posted it here.

I don't believe the issue is Docker-specific. I used it to make sure the environment is consistent and reproducible, and because I was not able to deploy on a local (Windows) installation of Tomcat (tried multiple versions of Tomcat and JDK). I'm using the latest version (7.8.0) on Tomcat 9.

The /path/to/my/external/cache exists, because I explicitly created it before deploying xdstools.

The page for TLS configuration contains a dead link (404) for server-side configuration: https://github.com/usnistgov/iheos-toolkit2/Tomcat-SSL-configuration-for-HTTPS

So unfortunately I'm not getting any further.

[skbhaskarla]

I sent a message to the XDS Implementors Google Group "admins" to check your membership request. (Toolkit users can benefit from your experience if you post it there instead, and you may get a faster reply since it reaches a broader audience.)

What problems or errors did you encounter when you tried to deploy toolkit on a local Windows installation?

Thanks for pointing out the dead link. After investigating this, it seems that the link only works when viewed from the GitHub wiki page editor preview mode but not when the page is published. The link is fixed now. (Remember the pages on TLS configuration are only for local testing purposes only. Some parameters such as the cipher suite will depend on Connectathon requirements and may change in future.)

You should try test toolkit without TLS first, why does it show the TLS (https) endpoint in your log when you are not using TLS? The "Could not find input file" error message seems to mean the Initialization Test 12318 requirement failed. You must pass the initialization before you can run the main conformance tests.

[skbhaskarla]

If you are using Java 9+ on Windows, then the following JDK Option may be required in your Tomcat bin\setenv.bat
SET "CATALINA_OPTS=%CATALINA_OPTS% --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"

(Java 8 does not require these options.)

[skbhaskarla]

Have you tried to use the Reset option before clicking the Initialize Testing Environment button in the Toolkit Conformance Tests page?

[elawende]

I have opened the discussion on the Google group, so let's continue here: https://groups.google.com/g/ihe-xds-implementors/c/ADEIwFgEV9A

I've tried again with your comments in mind. I get the same error, although now without TLS.

You say test 12318 must have failed, but this test is not listed on this conformance test. The tests for initialization are 15824 through 15826.