This project's goal is to simplify data analysis on HL7 FHIR by easing conversion of the hierarchical structure into a flat list for analysing R DataFrames.
If you're a lazy reader, you can also watch a short 5 minute presentation or a longer German presentation on Vimeo!
There are alternative ways to do this:
- Using Cerner Bunsen, you can load FHIR resources as DataFrames in Spark and query them using Spark-SQL. The main advantage is that you can use the full set of SQL operations like JOIN, etc., but You have to export your entire dataset first.
- Using the FHIR GraphQL API, you can use operators like "@flatten" to transform it into a flatter format. However, the result is still a JSON, and the GraphQL API must be supported by the server.
- fhircrackr and FHIRton, which are XML-based implementations for usage with R or Python.
This project is a simple WebServer, that connects to a FHIR server and forwards the requests. This way you can use all the FHIR REST API filtering options provided by the server. For each resource in the resulting bundle, a FHIRPath expression will be evaluated.
All parameters are forwarded as they are, except:
__columnsMust bename:expression,name2:expression2. You may add a list processing mode after the expression:@join(" ")concatenates the strings into a single cell with a delimiter of your choice@explodeWide($disc:expression,subcolumnname:subexpression,...)will create new columns for each element.
Inside the subexpression, use$thisto refer to the current element.$discis the expression that will be evaluated for the column name using the following schema:columnname.$disc.subcolumnname. Use%indexfor the current index of the element returned by the main expression.
You may leave the subcolumnname empty for one expression.@explodeLong(subcolumnname:subexpression,...)will create a new row for each element returned.
Inside the subexpression, use$thisto refer to the current element.
The resulting columns will be namedname.subcolumnname.
If you omit the subcolumns, the column itself will be exploded.
Please escape :in the column name with\:. In the FHIRPath expression, escape,with\,and@with\@!
__limitFhirExtinguisher automatically fetches Bundle pages until limit is met. This way you can limit the number of resources processed.__csvFormatChange the output CSV style. Supported are:Default,Excel,InformixUnload,InformixUnloadCsv,MySQL,PostgreSQLCsv,PostgreSQLText,RFC4180,TDFas provided by the Apache Commons CSV library, default isExcel.
The returned bundle(s) are evaluated against the FHIRPath expressions using the HAPI FHIRPath engine, and returned to the client.
You can download pre-compiled binaries in the Release-section of the Gitlab!
Requirements: Java 8, GUI tested with Firefox and Chrome
Use java -jar FhirExtinguisher-<version>-all.jar -f http://hapi.fhir.org/baseR4 -p 8080 to start the server on port
8080 on your local machine and connect to the public FHIR R4 server.
By default, FhirExtinguisher assumes, the server is FHIR R4. If you want to connect with a (D)STU3 server, please add
-v stu3 to the command line arguments.
Available command line options:
-f [url]FHIR server URL-v ["r4"|"stu3"]FHIR version of the server-a [username]:[password]Basic authentication credentials, if required by FHIR server-at [token]Bearer token, if required by the FHIR server-h "[header-name]: [value]"Add a custom header-p [portnumber]Port number on your local machine to open, e.g. with-p 8080, the GUI will be available athttp://localhost:8080/-t 30Set timeout for FHIR server in seconds (60 is default)-extAllow connections of non-localhost machines--no-conformance-checkDo not validate FHIR server URL at startup
To stop the FhirExtinguisher, press Ctrl+C in the command line window!
If you don't want to install Java on your local machine, you can use docker pull wwuimi/fhirextinguisher to pull a
docker image.
If you pulled the docker image from the docker hub use docker run --rm -p 127.0.0.1:8080:8080 --name fhirextinguisher wwuimi/fhirextinguisher -f <fhir-server-url> -p 8080 -ext to start the container.
If you build the image from the Dockerfile like its described in the Building section use
docker run --rm -p 127.0.0.1:8080:8080 --name fhirextinguisher fhirextinguisher -f <fhir-server-url> -p 8080 -ext to start the container.
Use docker stop fhirextinguisher to stop the container. You can use docker ps to check if container is
running.
You can create your own links using the specifications above. In R, you can use
data <- read.csv('http://localhost:8080/fhir/Patient?__limit=50&__columns=id:Patient.id@join(" ")') to always start
with a fresh version from the server. However, the FhirExtinguisher must be running during execution of the script.
You can create a link using the GUI and copy it into your R script.
You can use the GUI, which will be available under http://localhost:[portnumber]/, to download/export a CSV.
There, you can enter your query in the editor on the top using the FHIR Search API
and add the columns in the sidebar on the left.
Note that the table shown in the GUI is only a preview based on the first Bundle returned by the server. Note also, that the Import Resource... function is providing suggestions based on the Resource's StructureDefinintion, there is no guarantee of completeness. Array fields are inserted like single elements, so there might be unintended behaviour.
In the searchbar, you can use Ctrl+Space for autocompletion. This is based on the server conformance statement on HAPI FHIR using FHIR R4. Note that actual query options on your server might differ.
.type().nameGet the name of the type, e.g.Patient.deceased.type().nameis eitherdateTimeorboolean.resolve()resolves a reference to another resource.iif(gender = "female", "w", "m")if-else-expression.substring(0, 1)Return only the first letter of string
Custom functions introduced by FhirExtinguisher:
getIdPart()circumvents some inconveniences with HAPI FHIR'sIdTypeclassgetChildFields()returns a list of possible field names, but the expression until this point must return at least one elementstringify()returns a recursive serialization of all the element's fields
Requirements: Java 11, npm 6.13.x
Use ./gradlew shadowJar to compile the project. The resulting .jar file will be in /build/libs/.
At the first time, to compile the frontend, you have to run npm install (and eventually npm install --only=dev) in
the
/frontend folder, as well as npm run antlr4ts, since the gradle build script will only invoke webpack and copy the
files into the .jar file.
If you get Process 'command 'cmd'' finished with non-zero exit value 2, please execute "node_modules/.bin/webpack"
for the webpack error message.
Check out the repository, run docker build --tag fhirextinguisher .
You can execute node_modules\.bin\webpack-cli serve to start an automatically updating version of the frontend. If you
start the FhirExtinguisher using port 8081, even the backend will work.
Besides running the FhirExtinguisher on your local machine using the built-in Jetty engine, it is also possible to
deploy the FhirExtinguisher via Tomcat. Therefore, you can download the pre-compiled war file, open it as zip file and
edit the parameters in WEB-INF\classes\application.conf. Currently, there seems to be a bug in ktor, which prevents
the UI from loading if the FhirExtinguisher is not deployed on the root path. Also, having an Apache Web Server ahead of
Tomcat can cause some issues when using AJP, so I recommend using plain HTTP instead. I will update this text as well as
the Ktor version as soon as these issues are resolved.
- Johannes Oehm | (+49) 251 / 83-5 82 47 | [email protected]
- Oehm J, Storck M, Fechner M, Brix TJ, Yildirim K, Dugas M. FhirExtinguisher: A FHIR Resource Flattening Tool Using FHIRPath. Stud Health Technol Inform. 2021 May 27;281:1112-1113. doi: 10.3233/SHTI210369. PMID: 34042862.
Apache Licence 2.0
Supported by BMBF grant No. 01ZZ1802V (HiGHmed/Münster)