What does an ideal Fluid publication look like?

[RaoOfPhysics]

Motivation

A publication (scientific article, journalistic feature, interactive dashboard) containing Fluid code, henceforth “Fluid publication” or simply “publication”, should offer clearly defined advantages to existing publishing paradigms, which are already obvious in the case of PDFs.

These include, but are not limited to:

• Responsive format: see Tufte format for Quarto (see Use Quarto as the default publishing mechanism for Fluid? #5)
• Interactive graphics and tables – Fluid’s USP
  • Warning when load times are too long?
• Versioning information (see below)
• Pre-defined narrative chosen by author, but with narrative having the capacity to change if data selected change
• Possibility of exporting document with user-made changes to graphics/code (for peer review?) – floating button?

Optional features

• Hypothes.is integration?
  • To allow post-publication discussion

Versioning

There should be a metadata section with other information about the publication that lists the version details of (at least) the following, with links where applicable

• Source text: the “raw” text used to author the Fluid publication (see below)
• Source code – open source: part of “raw” text (i.e. code chunks) if part of publication workflow; else link to source code used to produce pre-processed data used in publication
• Fluid version: with link to server where Fluid is running (see below)
• Version of any Fluid packages: with link to config listing packages+versions
• Data version – open access: information on provenance (incl. info like date of publication) of data, including derived data, in case of additions/deletions/corrections
• Article itself, if narrative is updated after changes to data selection
  • Will button to export document show this info (colour-coded)
  • Investigate possibility of integrating with Solid, for readers to store own version of publication online

Fluid documents should be hashed, so they can be compared across different stores, using all the above versioning information. Ideally they should have persistent URLs/DOIs (per version, maybe).

Authoring

A suggested workflow would be to use GitHub Actions as part of the workflow.

If Quarto is chosen as the publication format, the text of the publication will be written in Markdown, with executable code chunks used for creating the visualisations and tables. The .qmd file can be edited in the user’s referred editor, with the code run and the output generated. However, only the .qmd file is pushed to GitHub, where Actions build the publication and upload it to GitHub pages. This will allow the raw source file(s) to be versioned.

The Fluid source should be specified in the first code chunk or in the YAML metadata, compulsorily, in order for the publication to be interactive once online.

Data source

If the (derived) datasets are small enough, they could be embedded within the document, using the <a href="base64 data" download="filename"> trick (see also the {downloadthis} package for R)

The editor should warn the user if the file is larger than some arbitrarily defined size, and should prompt them to store the data publicly, on Zenodo for example. All source datasets should be stored on Zenodo or a similar data repository anyway.

This is obviously only the case for instances where confidentiality, intellectual property and other related concerns do not prevent data from being stored publicly. Where data come from publicly available datasets (such as ones provided by government sources or IPCC etc.), this should not be an issue. Where there are concerns, a derived sanitised, suitable-for-public-consumption version of the data should be made available, without which the interactivity provided by Fluid will not be functional.

Fluid server

In order to be an interactive document, a Fluid publication must interface with a server hosting Fluid. We can consider a few different options, each with its own pros and cons (not fully explored here; perhaps deserving of a separate discussion).

• Binder is an existing tool, supported by the Jupyter community, which allows interactive versions of notebooks to be run online, but also provides a “server” with user-defined parameters that makes it possible for regular HTML pages to become interactive. However, there may be concerns about sustainability of the platform if we think of long-term availability of Fluid publications relying on it.
• Since Fluid is in Purescript, can we embed minified JS in the page?
• Ambitions and probably unnecessary, but would compiling Fluid to WebAssmbly be of any use?

---

Will update later with a sketch I have in mind.

[rolyp]

Thanks a lot for this, looks like a great start. I like the idea of focusing on where there are clear advantages vis-a-vis existing paradigms – I think this is a good guiding strategy. Added something about DOIs to the Versioning section, but feel free to edit/move as appropriate.

[RaoOfPhysics]

Yup, that makes sense! How do get DOIs is an open question. If you publish via GitHub Pages, you can get it via Zenodo integration, but those will point to Zenodo, not the publication.

[RaoOfPhysics]

See #28.