diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index cf56740..03c2a43 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -22,20 +22,30 @@ jobs: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install Quarto uses: quarto-dev/quarto-actions/setup@v2 + with: + version: 1.5.25 - name: Install Node - uses: actions/setup-node@v3 + uses: actions/setup-node@v4 with: - node-version: "latest" + node-version: 20 - name: Install Sverto docs npm dependencies run: npm install working-directory: ./docs + - name: Create docs _extensions folder + run: mkdir docs/_extensions + shell: bash + + - name: Copy Sverto extension into docs + run: cp -rf _extensions/sverto docs/_extensions/sverto + shell: bash + - name: Render sverto docs uses: quarto-dev/quarto-actions/render@v2 with: diff --git a/.gitignore b/.gitignore index c7aaf12..6aedf0e 100644 --- a/.gitignore +++ b/.gitignore @@ -18,4 +18,7 @@ # rendered documentation site /docs/_site/ /docs/.quarto/ -/docs/.sverto/ \ No newline at end of file +/docs/.sverto/ + +# sverto extension in docs site (we get it from project root on render) +/docs/_extensions/sverto \ No newline at end of file diff --git a/.quartoignore b/.quartoignore index 0632f33..642c9ea 100644 --- a/.quartoignore +++ b/.quartoignore @@ -1,6 +1,9 @@ README.md NEWS.md LICENSE +.luarc.json +package-lock.json docs/ .quarto/ -!.gitignore +.github/ +!.gitignore \ No newline at end of file diff --git a/Circles.svelte b/Circles.svelte index 96e7141..ee04f77 100644 --- a/Circles.svelte +++ b/Circles.svelte @@ -16,7 +16,7 @@ - @@ -24,7 +24,7 @@ {/each} diff --git a/NEWS.md b/NEWS.md index 3cf7997..93b2cda 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,17 @@ +## (Unreleased) Sverto 1.0.0 + +- Significant refactor of Sverto makes it easier to use and more compatible with Quarto's other features +- Use Sverto in a Quarto document by adding `sverto` to `filters` in the document frontmatter +- Add Svelte files to a document using the frontmatter key `sverto.use` +- No need for magic blocks anymore! +- When working in a website project, optionally use the `sverto` project type to cut down on duplicate Svelte compilation Quarto documents +- Works properly with Quarto includes +- Requires Quarto pre-release 1.5.25 or higher on Windows, but should work fine on Quarto 1.4 on macOS and Linux. + +## Sverto 0.0.3 + +- Migrated from [`360-info/sverto`](https://github.comn/360-info/sverto) to [`jimjam-slam/sverto`](htps://github.com/jimjam-slam/sverto). Old GitHub links are maintained. + ## Sverto 0.0.2 - Bump minimum Quarto version to 1.3.0. @@ -7,6 +21,6 @@ 1. avoid copying the `docs` folder in with the project template; and 2. include the `.gitignore` with the template -## 0.0.1 +## Sverto 0.0.1 - Initial release diff --git a/README.md b/README.md index 5081eab..9556587 100644 --- a/README.md +++ b/README.md @@ -4,14 +4,6 @@ Your Svelte components can seamlessly react to your ObservableJS code, making it quick and easy to build visuals that animate in response to [user inputs](https://observablehq.com/@observablehq/inputs?collection=@observablehq/inputs) or other changing data in your document. -## 💭 Why Sverto? - -[Quarto](https://quarto.org) helps users build beautiful documents regardless of their language of choice, and it encourages data analysts and scientists to explore web visualisation by making JavaScript accessible and easy to use. It makes interactive visualisations intuitive to write, but animated visuals are still a challenge that require either dipping into a high-level JavaScript library or learning a lower-level one like [d3](https://d3js.org). - -[Svelte](https://svelte.dev) is a framework for building web visualisations and apps in JavaScript. Svelte goes out of its way to make writing self-contained components, like charts, comfortable and intuitive. It has a great [playground environment](https://svelte.dev/repl/hello-world?version=3.55.1) for developing and testing components, but like many web frameworks, the experience is much more complex when you start developing locally. - -_Sverto aims to make it as easy to use Svelte components in Quarto documents as it is to work on them in the Svelte REPL: just write a `.svelte` file, add it to a Quarto document, and Sverto should take care of the rest._ - ## 📋 Prerequisites You'll need to install two things to run Sverto: @@ -35,31 +27,66 @@ npm install This will add the extension itself (which includes some project scripts) to the `_extension` folder, as well as a few other files. -> **Note:** Sverto depends on running [project pre-render scripts](https://quarto.org/docs/projects/scripts.html#pre-and-post-render), so you can't currently use it with single documents. +### 📦 What's in the box? + +When you use the Sverto template in a project, it creates some files for you: + +* [`example.qmd`](./example.qmd): an example Quarto doc that uses a Svelte component +* [`Circles.svelte`](./Circles.svelte): an example Svelte visualisation +* [`package.json`](./package.json): this is used to keep track of the dependencies of your Svelte components. **You should add this to version control.** +* `package-lock.json` is created once you run `npm install`. You should add this to version control. +* `node_modules/`: This folder is created once you rum `npm install`. Don't add it to version control. ## 🎉 Use -Here's the short way to add Svelte component you've written to a Quarto doc: +### Step 1: add Svelte to your document + +In the document frontmatter, add `sverto` to `filters`, and add one or more `.svelte` files to `sverto.use`: + +```yaml +--- +title: "My document" +filters: ["sverto"] +sverto: + use: + - example.svelte +--- +``` + +### Step 2: bring your Svelte component to life + +Use an [Observable JS](https://quarto.org/docs/interactive/ojs/) chunk to _construct_ your Svelte component. + +````js +```{ojs} +myChart = new example.default({ + target: document.querySelector("#chart") +}) +``` + +:::{#chart} +::: +```` -1. Add a magic placeholder block to your document with a [Quarto include](https://quarto.org/docs/authoring/includes.html) to the path to your Quarto doc, prefixed with `/.sverto/`. For example: +- the `target` is where it will appear. This needs to be an existing part of the document — you can put a [Pandoc div](https://quarto.org/docs/authoring/markdown-basics.html#divs-and-spans) right after this code, or put one anywhere else on the page +- `example` is the file name of your Svelte component, without the file extension - ``` - :::{} - {{< include /.sverto/example.qmd >}} - ::: - ``` +### Step 3: make your component reactive -2. Import your Svelte component in OJS with `Component = import_svelte("Component.svelte")` -3. Add a target block for your visual using `:::` and give it an `#id` -4. Instantiate the Svelte component with `myVisual = Component.default()` using some default props and your target block -5. Update the instantiated component with `myVisual.propName` -6. Render your Quarto website as usual with `quarto render` or `quarto preview`. +If your component has `props` that allow it to change or transition in response to other OJS code, you can update them by assigning the prop directly. -**To see this all in practice, check out [`example.qmd`](./example.qmd).** +For example, if we have a dataset called `myData` in OJS, and a year slider called `selectedYear`, we might change a prop called `chartData` whenever the user selects a new year like: -> **Note:** `quarto preview` won't "live reload" when you modify your Svelte component—but if you modify and save the Quarto doc that imports it, that will trigger a re-render. You may need to hard reload the page in your browser to see the updated Svelte component. +````js +```{ojs} +myChart.chartData = myData.filter(d => d.year == selectedYear) +``` +```` + +> ![NOTE] +> `quarto preview` won't "live reload" when you modify your Svelte component—but if you modify and save the Quarto doc that imports it, that will trigger a re-render. You may need to hard reload the page in your browser to see the updated Svelte component. > -> If you want to quickly iterate on the Svelte component and you aren't too concerned about the rest of your Quarto doc, you might find the [Svelte Preview](https://marketplace.visualstudio.com/items?itemName=RafaelMartinez.svelte-preview) extension for VSCode handy. +> If you want to quickly iterate on the Svelte component, you might find the [Svelte Preview](https://marketplace.visualstudio.com/items?itemName=RafaelMartinez.svelte-preview) extension for VSCode handy. ## 📦 What's in the box? @@ -79,14 +106,20 @@ As well as the project format, Sverto ships with document formats (the default i If you want to refer to other JavaScript libraries in your Svelte component (like d3, for example), add them to the project using `npm install package1 [package2 ...]`. For example: ``` -npm install d3-scale +npm install d3 ``` -## Use pre-compiled Svelte components +# 💭 Why Sverto? + +[Quarto](https://quarto.org) helps data scientists and analysts build beautiful documents regardless of their language of choice, and it encourages data analysts and scientists to explore web visualisation by making JavaScript accessible and easy to use. + +Quarto makes interactive charts intuitive to write, but animated ones are still a challenge that require either dipping into a high-level JavaScript library or learning a lower-level one like [d3](https://d3js.org). + +[Svelte](https://svelte.dev) is a framework for building charts, web visualisations and even apps in HTML, CSS and JavaScript. Svelte goes out of its way to make writing self-contained components, like charts, comfortable and intuitive. -If you'd prefer to compile your own Svelte components instead of letting this extension do it, you can skip steps 1 and 2 and simply refer to the compiled bundle with, for example, `Component = import("Component.js")` in an OJS block. +Svelte has a great [playground environment](https://svelte.dev/repl/hello-world?version=3.55.1) for developing and testing components, but like many web frameworks, the experience is much more complex when you start developing locally. -> **Note:** you must compile the Svelte component to an ES6 bundle, and you must enable accessors when compiling if you want to be able to update them from OJS. Refer to `_extensions/sverto/rollup.config.js` for guidance on configuring Rollup to do this. +_Sverto aims to make it as easy to build and use animated Svelte charts in Quarto documents as it is to work on them in the Svelte playground: just write a `.svelte` file, add it to a Quarto document, and Sverto takes care of the rest._ ## ❓ Issues diff --git a/_extensions/sverto/_extension.yml b/_extensions/sverto/_extension.yml index 1e08544..c211b56 100644 --- a/_extensions/sverto/_extension.yml +++ b/_extensions/sverto/_extension.yml @@ -1,20 +1,11 @@ title: Sverto -author: 360info -version: 0.0.2 -quarto-version: ">=1.3.0" +author: James Goldie +version: 1.0.0 +quarto-version: ">=1.4.0" contributes: + filters: + - sverto.lua project: project: type: website - pre-render: - - refresh.ts - - create-imports.lua - - compile-imports.ts - format: sverto-html - formats: - html: - filters: - - cleanup-transform.lua - revealjs: - filters: - - cleanup-transform.lua + pre-render: sverto-prerender.lua diff --git a/_extensions/sverto/cleanup-transform.lua b/_extensions/sverto/cleanup-transform.lua deleted file mode 100644 index 36afa2e..0000000 --- a/_extensions/sverto/cleanup-transform.lua +++ /dev/null @@ -1,6 +0,0 @@ --- replaces the doc body with just its first block (assumed to be the --- version of the doc with transformed .svelte -> .js refs) -Pandoc = function(doc) - doc.blocks = doc.blocks[1].content - return doc -end diff --git a/_extensions/sverto/compile-imports.ts b/_extensions/sverto/compile-imports.ts deleted file mode 100644 index 312b8fb..0000000 --- a/_extensions/sverto/compile-imports.ts +++ /dev/null @@ -1,15 +0,0 @@ -// check if _extensions/svero exists or _extensions/360-info/sverto - -import * as path from "https://deno.land/std/path/mod.ts"; - -const thisScript = path.fromFileUrl(import.meta.url); -const rollupConfig = path.join(path.dirname(thisScript), "rollup.config.js"); - -// run svelte compiler via rollup -// (prepend cmd /c to the command on windows) -const cmdPrefix = Deno.build.os == "windows" ? ["cmd", "/c"] : [] -const cmd = cmdPrefix.concat(["npm", "run", "build", rollupConfig]); - -// call rollup with the config file -const compileStep = Deno.run({ cmd }); -await compileStep.status(); diff --git a/_extensions/sverto/create-imports.lua b/_extensions/sverto/create-imports.lua deleted file mode 100644 index 4e6d462..0000000 --- a/_extensions/sverto/create-imports.lua +++ /dev/null @@ -1,294 +0,0 @@ --- create-imports: a project pre-render script that: --- * replaces svelte_import() ojs statements in qmd files, saving to /.sverto --- * writes svelte import paths to /.sverto s othey can be compiled - --- some content from quarto's qmd-reader.lua --- (c) 2023 rstudio, pbc. licensed under gnu gpl v2: --- https://github.com/quarto-dev/quarto-cli/blob/main/COPYRIGHT - --- return contents of named file -function read_file(name) - local file = io.open(name, "r") - if file == nil then - return "" - end - local contents = file:read("a") - file:close() - return contents -end - --- write content to named file path -function write_file(name, content) - local file = io.open(name, "w") - if file == nil then - return "" - end - file:write(content) - file:close() - return content -end - -function append_to_file(name, content) - local file = io.open(name, "a") - if file == nil then - return "" - end - file:write(content) - file:close() - return content -end - --- use mkdir (windows) or mkdir -p (*nix) to create directories --- from https://stackoverflow.com/a/14425862/3246758 -function get_path_sep() - return package.config:sub(1, 1) -end - --- create a folder recursively (mkdir on windows, mkdir -p on *nix) -function create_dir_recursively(path) - local path_separator = get_path_sep() - if path_separator == "\\" or path_separator == "\"" then - os.execute("mkdir " .. path) - else - -- macos/linux - os.execute("mkdir -p " .. path) - end -end - --- path_dir: extract the folder path from a file path --- from https://stackoverflow.com/a/9102300/3246758 -function path_dir(path) - return path:match("(.*".. get_path_sep() ..")") or "" -end - --- content following from quarto's qmd-reader.lua --- (c) 2023 rstudio, pbc. licensed under gnu gpl v2: --- https://github.com/quarto-dev/quarto-cli/blob/main/COPYRIGHT - -function random_string(size) - -- we replace invalid tags with random strings of the same size - -- to safely allow code blocks inside pipe tables - -- note that we can't use uppercase letters here - -- because pandoc canonicalizes classes to lowercase. - local chars = "abcdefghijklmnopqrstuvwxyz" - local lst = {} - for _ = 1,size do - local ix = math.random(1, #chars) - table.insert(lst, string.sub(chars, ix, ix)) - end - return table.concat(lst, "") -end - -function find_invalid_tags(str) - -- [^.=\n] - -- we disallow "." to avoid catching {.python} - -- we disallow "=" to avoid catching {foo="bar"} - -- we disallow "\n" to avoid multiple lines - - -- no | in lua patterns... - - -- (c standard, 7.4.1.10, isspace function) - -- %s catches \n and \r, so we must use [ \t\f\v] instead - - local patterns = { - "^[ \t\f\v]*(```+[ \t\f\v]*)(%{+[^.=\n\r]*%}+)", - "\n[ \t\f\v]*(```+[ \t\f\v]*)(%{+[^.=\n\r]+%}+)" - } - local function find_it(init) - for _, pattern in ipairs(patterns) do - local range_start, range_end, ticks, tag = str:find(pattern, init) - if range_start ~= nil then - return range_start, range_end, ticks, tag - end - end - return nil - end - - local init = 1 - local range_start, range_end, ticks, tag = find_it(init) - local tag_set = {} - local tags = {} - while tag ~= nil do - init = range_end + 1 - if not tag_set[tag] then - tag_set[tag] = true - table.insert(tags, tag) - end - range_start, range_end, ticks, tag = find_it(init) - end - return tags -end - -function escape_invalid_tags(str) - local tags = find_invalid_tags(str) - -- we must now replace the tags in a careful order. Specifically, - -- we can't replace a key that's a substring of a larger key without - -- first replacing the larger key. - -- - -- ie. if we replace {python} before {{python}}, Bad Things Happen. - -- so we sort the tags by descending size, which suffices - table.sort(tags, function(a, b) return #b < #a end) - - local replacements = {} - for _, k in ipairs(tags) do - local replacement - local attempts = 1 - repeat - replacement = random_string(#k) - attempts = attempts + 1 - until str:find(replacement, 1, true) == nil or attempts == 100 - if attempts == 100 then - print("Internal error, could not find safe replacement for "..k.." after 100 tries") - print("Please file a bug at https://github.com/quarto-dev/quarto-cli") - os.exit(1) - end - -- replace all lua special pattern characters with their - -- escaped versions - local safe_pattern = k:gsub("([%^%$%(%)%%%.%[%]%*%+%-%?])", "%%%1") - replacements[replacement] = k - local patterns = { - "^([ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern, - "(\n[ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern - } - - str = str:gsub(patterns[1], "%1" .. replacement):gsub(patterns[2], "%1" .. replacement) - end - return str, replacements -end - -function unescape_invalid_tags(str, tags) - for replacement, k in pairs(tags) do - -- replace all lua special replacement characters with their - -- escaped versions, so that when we restore the behavior, - -- we don't accidentally create a pattern - local result = k:gsub("([$%%])", "%%%1") - str = str:gsub(replacement, result) - end - return str -end - -local preprocess_qmd_filter = { - - -- search for `import_svelte("X.svelte")` refs in codeblocks and switch them - -- to `import("X.js")` - CodeBlock = function(block) - if block.classes:includes("{ojs}") then - - local svelte_import_syntax = - "import%_svelte%(\"([%w;,/%?:@&=%+%$%-_%.!~%*'%(%)#]+)%.svelte\"%)" - - local block_text = block.text - - -- get the qmd_path from disk - local current_qmd_path = read_file(".sverto/.sverto-current-qmd-folder") - - -- first, extract .svelte paths in import_svelte() statements - for svelte_path in block_text:gmatch(svelte_import_syntax) do - append_to_file(".sverto/.sverto-imports", - current_qmd_path .. svelte_path .. ".svelte\n") - end - - -- now change `import_svelte("X.svelte")` refs to `import("X.js")` - -- TODO - neaten up relative paths instead of assuming we're going from - -- /site_libs/quarto-ojs - block.text = block_text:gsub( - svelte_import_syntax, - "import(\"./../../" .. current_qmd_path .. "%1.js\")") - - end - return block - end, - - -- return the doc as a a whole unchanged... - -- except without the first block (the include statement) - Pandoc = function(doc) - local new_blocks = pandoc.Blocks({}) - for i, v in ipairs(doc.blocks) do - if i ~= 1 then - new_blocks:insert(v) - end - end - doc.blocks = new_blocks - return doc - - end -} - -create_dir_recursively(".sverto/") - --- collect the input qmd paths -in_file_string = os.getenv("QUARTO_PROJECT_INPUT_FILES") -in_files = {} -for in_file in string.gmatch(in_file_string, "([^\n]+)") do - table.insert(in_files, in_file) -end - --- transform each input qmd, saving the transformation in .sverto/[path] --- (write the identified .svelte files out to a file too!) -for key, qmd_path in ipairs(in_files) do - - -- before we read the file in with pandoc.read, let's read it in as a raw - -- string and do the quarto team's qmd-reader processing on it. THEN we can - -- read it with pandoc.read - - print(">>> PROCESSING " .. qmd_path) - - local raw_doc = read_file(qmd_path) - - -- store the current qmd_path on disk so the filter can access it - write_file(".sverto/.sverto-current-qmd-folder", path_dir(qmd_path)) - - -- escape invalid tags - local txt, tags = escape_invalid_tags(tostring(raw_doc)) - - -- some extension + format stuff that we can maybe ignore? - - -- for k, v in pairs(opts.extensions) do - -- extensions[v] = true - -- end - - -- if param("user-defined-from") then - -- local user_format = _quarto.format.parse_format(param("user-defined-from")) - -- for k, v in pairs(user_format.extensions) do - -- extensions[k] = v - -- end - -- end - - -- -- Format flavor, i.e., which extensions should be enabled/disabled - -- local flavor = { - -- format = "markdown", - -- extensions = extensions, - -- } - - local function restore_invalid_tags(tag) - return tags[tag] or tag - end - - -- NOW we read in with pandoc (hopefully ending up with real code blocks) - -- and restore them - -- local doc = pandoc.read(txt, flavor, opts):walk { - local doc = pandoc.read(txt, "markdown") - - local restored_doc = doc:walk { - CodeBlock = function (cb) - cb.classes = cb.classes:map(restore_invalid_tags) - cb.text = unescape_invalid_tags(cb.text, tags) - return cb - end - } - - -- local doc = pandoc.read(read_file(qmd_path)) - - -- pre-process the qmd, populating `svelte_files` in the process - -- local svelte_files = {} - local transformed_doc = restored_doc:walk(preprocess_qmd_filter) - create_dir_recursively(".sverto/" .. path_dir(qmd_path)) - write_file(".sverto/" .. qmd_path, pandoc.write(transformed_doc, "markdown")) - -end - --- write the output dir path temporarily (so rollup can use it) -write_file(".sverto/.sverto-outdir", os.getenv("QUARTO_PROJECT_OUTPUT_DIR")) - --- TODO - if there's no {{< import .sverto/file.qmd >}} block, add it? - diff --git a/_extensions/sverto/refresh.ts b/_extensions/sverto/refresh.ts deleted file mode 100644 index b6d96b1..0000000 --- a/_extensions/sverto/refresh.ts +++ /dev/null @@ -1,10 +0,0 @@ -// delete the temporary import block files and the temp render metadata - -if (Deno.env.get("QUARTO_PROJECT_RENDER_ALL") == "1") { - try { - await Deno.remove(".sverto/", { recursive: true }) - } catch { - - } -} - diff --git a/_extensions/sverto/rollup.config.js b/_extensions/sverto/rollup.config.js index e730781..ca08fbf 100644 --- a/_extensions/sverto/rollup.config.js +++ b/_extensions/sverto/rollup.config.js @@ -1,57 +1,69 @@ -import svelte from 'rollup-plugin-svelte'; -import commonjs from '@rollup/plugin-commonjs'; -import resolve from '@rollup/plugin-node-resolve'; -import { terser } from 'rollup-plugin-terser'; +import svelte from "rollup-plugin-svelte" +import commonjs from "@rollup/plugin-commonjs" +import resolve from "@rollup/plugin-node-resolve" +import terser from "@rollup/plugin-terser" -const fs = require('fs'); -const path = require('node:path'); +const path = require('node:path') // this is false when we run rollup with -w/--watch (never presently) const production = !process.env.ROLLUP_WATCH; -// get quarto project output directory and list of inputs -const quartoOutDir = fs.readFileSync('.sverto/.sverto-outdir', 'utf8'); +/* export an array of rollup configs - one for each input svelte file - using + additional command line args supplied from lua */ +export default cmd => { -const svelteImportListPath = '.sverto/.sverto-imports'; + const svelteInputPaths = + cmd["configSvelteInPaths"].split(":").filter(d => d != "") + + // if no svelte paths, bail out early + if (svelteInputPaths == undefined || svelteInputPaths.length == 0) { + console.log("No Svelte filtes found; skipping Svelte compilation") + process.exit(0) + } + + /* get quarto render dir from cmd line arg */ + const quartoRenderPath = cmd["configQuartoOutPath"] + if (quartoRenderPath == undefined) { + console.error( + "Error: supply a --configQuartoOutPath. Please report this to " + + "the Sverto developer.") + process.exit(1) + } + + return svelteInputPaths.map( + + svelteFile => ({ + input: svelteFile, + output: { + format: "es", + dir: path.join( + quartoRenderPath, + path.dirname(svelteFile)), + sourcemap: true + }, + plugins: [ + svelte({ + // css is added to the js bundle instead + emitCss: false, + compilerOptions: { + // required for ojs reactivity + accessors: true, + dev: !production, + } + }), + resolve({ + browser: true, + dedupe: ["svelte"] + }), + commonjs(), + production && terser() + ] + }) + + ) -// skip svelte compilation if there's nothing to compile -if (!fs.existsSync(svelteImportListPath)) { - console.log("ℹ No Svelte imports to process; skipping compilation"); - process.exit(); } - -// get the list of unique imports to compile -const svelteFiles = fs.readFileSync(svelteImportListPath, 'utf8') - .split("\n") - .filter(d => d !== ""); -const uniqueSvelteFiles = [... new Set(svelteFiles)] - -// we export an array of rollup configs: one for each input svelte file -export default uniqueSvelteFiles.map( - svelteFile => ({ - input: svelteFile, - output: { - format: "es", - dir: path.join( - quartoOutDir, - path.dirname(svelteFile)), - sourcemap: true - }, - plugins: [ - svelte({ - // css is added to the js bundle instead - emitCss: false, - compilerOptions: { - // required for ojs reactivity - accessors: true, - dev: !production, - } - }), - resolve({ - browser: true, - dedupe: ["svelte"] - }), - commonjs(), - production && terser() - ] - })); + + + + diff --git a/_extensions/sverto/sverto-prerender.lua b/_extensions/sverto/sverto-prerender.lua new file mode 100644 index 0000000..a9cb70e --- /dev/null +++ b/_extensions/sverto/sverto-prerender.lua @@ -0,0 +1,145 @@ + +-- return contents of named file +function read_file(name) + local file = io.open(name, "r") + if file == nil then + return "" + end + local contents = file:read("a") + file:close() + return contents +end + +-- get a prefix for calling npm run based on windows or *nix/macos +-- using path separator: i don't know pandoc.system.os() values +function get_cmd_prefix() + if pandoc.path.separator == "\\" then + return "cmd /c " + else + return "" + end +end + +-- create a folder recursively (mkdir on windows, mkdir -p on *nix) +-- function create_dir_recursively(path) +-- if pandoc.path.separator == "\\" or pandoc.path.separator == "\"" then +-- os.execute("mkdir " .. path) +-- else +-- -- macos/linux +-- os.execute("mkdir -p " .. path) +-- end +-- end + +-- file_exists: true if the file at `name` exists +-- from https://pandoc.org/lua-filters.html#building-images-with-tikz +function file_exists(name) + local f = io.open(name, 'r') + if f ~= nil then + io.close(f) + return true + else + return false + end +end + +-- offset a relative `svelte_path` to a .qmd `input_path`, or an absolute +-- `svelte_path` to the project path. then normalize. +function offset_svelte_path(svelte_path, input_path) + + -- offset from either input .qmd (if relative) or project dir (if absolute) + -- local offset_from = pandoc.system.get_working_directory() + local offset_from = "./" + if pandoc.path.is_relative(svelte_path) then + -- offset_from = pandoc.path.directory(input_path) + offset_from = pandoc.path.join({ + offset_from, + pandoc.path.directory(input_path) + }) + end + + -- join offset and svelt paths + local relative_path = pandoc.path.join({ + offset_from, + pandoc.utils.stringify(svelte_path) + }) + + -- normalize and return + local final_path = pandoc.path.normalize(relative_path) + return final_path +end + +input_paths = os.getenv("QUARTO_PROJECT_INPUT_FILES") + +-- walk input files, processing the meta to get the svelte files +-- (make relative paths relative to the input file!) + +svelte_paths = {} + +-- for _, input_path in ipairs(input_paths) do +for input_path in input_paths:gmatch("[^\n]+") do + + local doc = pandoc.read(read_file(input_path), "markdown") + + doc:walk { + Meta = function(m) + + -- confirm sverto.use is a string or list + if m.sverto == nil or m.sverto.use == nil then + return nil + end + + local sverto_use + if pandoc.utils.type(m.sverto.use) == "List" then + sverto_use = m.sverto.use + elseif type(m.sverto.use) == "string" then + sverto_use = { m.sverto.use } + else + print( + "Sverto error: sverto.use key should be either a string path or " .. "a list of string paths, not " .. pandoc.utils.type(m.sverto.use)) + return nil + end + + -- add each unique path, resolving relative project location + for _, svelte_path in ipairs(sverto_use) do + local offset_path = offset_svelte_path( + pandoc.utils.stringify(svelte_path), + input_path) + svelte_paths[offset_path] = offset_path + end + + end + } + +end + +-- now concatenate them with : and send them to the svelte compiler +svelte_path_string = "" +for _, svelte_path in pairs(svelte_paths) do + svelte_path_string = svelte_path_string .. svelte_path .. ":" +end + +-- finally, call the svelte compiler via rollup +rollup_config = "" +if file_exists("./_extensions/jimjam-slam/sverto/rollup.config.js") then + rollup_config = "./_extensions/jimjam-slam/sverto/rollup.config.js" +elseif file_exists("./_extensions/sverto/rollup.config.js") then + rollup_config = "./_extensions/sverto/rollup.config.js" +else + print("Error: no rollup config found. Is Sverto installed properly?") + os.exit(1) +end + +cmd = + get_cmd_prefix() .. + "npm run build " .. + rollup_config .. " -- " .. + '--configQuartoOutPath="' .. os.getenv("QUARTO_PROJECT_OUTPUT_DIR") .. '" ' .. + '--configSvelteInPaths="' .. svelte_path_string .. '"' + +local svelteResult = os.execute(cmd) + +if svelteResult == nil or svelteResult == true then + print("Sverto pre-render finished!") +else + print("Svelte compiler finished with code " .. tostring(svelteResult)) +end diff --git a/_extensions/sverto/sverto.lua b/_extensions/sverto/sverto.lua new file mode 100644 index 0000000..87c8cf6 --- /dev/null +++ b/_extensions/sverto/sverto.lua @@ -0,0 +1,113 @@ +-- append_to_file: append a string of `content` to the file with the path `name` +function append_to_file(name, content) + local file = io.open(name, "a") + if file == nil then + return "" + end + file:write(content) + file:close() + return content +end + +-- inject_svelte: a pandoc filter that extracts svelte files named in the doc +-- frontmatter and adds javascript blocks to import them as es modules +function inject_svelte_and_compile(m) + + if not quarto.doc.isFormat("html") then + quarto.log.warning("Sverto shortcode ignored for non-HTML output") + return nil + end + + -- no files to process? abort + if m.sverto == nil or m.sverto.use == nil then + quarto.log.warning("No Svelte files found. To use Sverto with this document, add a list of .svelte files to the document frontmatter under the `sverto.use` key.") + return nil + end + + -- abort if sverto.use is not a list of MetaInlines + local sverto_use + if pandoc.utils.type(m.sverto.use) == "List" then + sverto_use = m.sverto.use + elseif type(m.sverto.use) == "string" then + sverto_use = { m.sverto.use } + else + quarto.log.error( + "sverto.use should be Inlines, not " .. + pandoc.utils.type(m.sverto.use)) + end + + -- if pandoc.utils.type(m.sverto.use) ~= "List" then + -- quarto.log.error( + -- "Sverto error: sverto.use key should be either a string path or " .. + -- "a list of string paths, not " .. pandoc.utils.type(m.sverto.use)) + -- end + + local sveltePaths = "" + + -- either add text to start of body (and return nil), or return a rawblock + -- %s: obj_name + -- %s: file_name, adapted for output path (and .svelte => .js) + local svelteInitTemplate = [[ + + ]] + + for _, path in ipairs(m.sverto.use) do + -- this is where we process the path + + local in_path = pandoc.utils.stringify(path) + local in_dir = pandoc.path.directory(in_path) + local in_name = pandoc.path.filename(in_path) + local obj_name = pandoc.path.split_extension(in_name) + local compiled_path = pandoc.path.join({ + in_dir, + obj_name .. ".js" + }) + + -- add path to svelte compiler path list + sveltePaths = sveltePaths .. in_path .. ":" + + local svelteInsert = string.format(svelteInitTemplate, + compiled_path, obj_name) + + -- quarto.log.warning("INJECTION:") + -- quarto.log.warning(svelteInsert) + + quarto.doc.include_text("before-body", svelteInsert) + + -- now run the svelte compiler... if we're not in a project + if quarto.project.directory ~= nil then + quarto.log.info("Project found; deferring Svelte compilation to post-render script") + else + local svelteCommand = + "npm run build rollup.config.js -- " .. + '--configQuartoOutPath="./" ' .. + '--configSvelteInPaths="' .. sveltePaths .. '"' + local svelteResult = os.execute(svelteCommand) + quarto.log.warning("Svelte compiler finished with code " .. svelteResult) + + end + end + +end + +return { + Meta = inject_svelte_and_compile +} diff --git a/_quarto.yml b/_quarto.yml deleted file mode 100644 index aa82966..0000000 --- a/_quarto.yml +++ /dev/null @@ -1,2 +0,0 @@ -project: - type: sverto \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..c1a2164 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,9 @@ +# Sverto documentation website + +Render (getting the latest copy of Sverto) and preview using: + +```bash +cp -Rf ../_extensions/sverto _extensions/sverto && quarto render + +quarto preview +``` diff --git a/docs/_extensions/sverto/_extension.yml b/docs/_extensions/sverto/_extension.yml deleted file mode 100644 index 2522b6e..0000000 --- a/docs/_extensions/sverto/_extension.yml +++ /dev/null @@ -1,20 +0,0 @@ -title: Sverto -author: 360info -version: 0.0.1 -quarto-version: ">=1.2.0" -contributes: - project: - project: - type: website - pre-render: - - refresh.ts - - create-imports.lua - - compile-imports.ts - format: sverto-html - formats: - html: - filters: - - cleanup-transform.lua - revealjs: - filters: - - cleanup-transform.lua diff --git a/docs/_extensions/sverto/cleanup-transform.lua b/docs/_extensions/sverto/cleanup-transform.lua deleted file mode 100644 index 36afa2e..0000000 --- a/docs/_extensions/sverto/cleanup-transform.lua +++ /dev/null @@ -1,6 +0,0 @@ --- replaces the doc body with just its first block (assumed to be the --- version of the doc with transformed .svelte -> .js refs) -Pandoc = function(doc) - doc.blocks = doc.blocks[1].content - return doc -end diff --git a/docs/_extensions/sverto/compile-imports.ts b/docs/_extensions/sverto/compile-imports.ts deleted file mode 100644 index 312b8fb..0000000 --- a/docs/_extensions/sverto/compile-imports.ts +++ /dev/null @@ -1,15 +0,0 @@ -// check if _extensions/svero exists or _extensions/360-info/sverto - -import * as path from "https://deno.land/std/path/mod.ts"; - -const thisScript = path.fromFileUrl(import.meta.url); -const rollupConfig = path.join(path.dirname(thisScript), "rollup.config.js"); - -// run svelte compiler via rollup -// (prepend cmd /c to the command on windows) -const cmdPrefix = Deno.build.os == "windows" ? ["cmd", "/c"] : [] -const cmd = cmdPrefix.concat(["npm", "run", "build", rollupConfig]); - -// call rollup with the config file -const compileStep = Deno.run({ cmd }); -await compileStep.status(); diff --git a/docs/_extensions/sverto/create-imports.lua b/docs/_extensions/sverto/create-imports.lua deleted file mode 100644 index 4e6d462..0000000 --- a/docs/_extensions/sverto/create-imports.lua +++ /dev/null @@ -1,294 +0,0 @@ --- create-imports: a project pre-render script that: --- * replaces svelte_import() ojs statements in qmd files, saving to /.sverto --- * writes svelte import paths to /.sverto s othey can be compiled - --- some content from quarto's qmd-reader.lua --- (c) 2023 rstudio, pbc. licensed under gnu gpl v2: --- https://github.com/quarto-dev/quarto-cli/blob/main/COPYRIGHT - --- return contents of named file -function read_file(name) - local file = io.open(name, "r") - if file == nil then - return "" - end - local contents = file:read("a") - file:close() - return contents -end - --- write content to named file path -function write_file(name, content) - local file = io.open(name, "w") - if file == nil then - return "" - end - file:write(content) - file:close() - return content -end - -function append_to_file(name, content) - local file = io.open(name, "a") - if file == nil then - return "" - end - file:write(content) - file:close() - return content -end - --- use mkdir (windows) or mkdir -p (*nix) to create directories --- from https://stackoverflow.com/a/14425862/3246758 -function get_path_sep() - return package.config:sub(1, 1) -end - --- create a folder recursively (mkdir on windows, mkdir -p on *nix) -function create_dir_recursively(path) - local path_separator = get_path_sep() - if path_separator == "\\" or path_separator == "\"" then - os.execute("mkdir " .. path) - else - -- macos/linux - os.execute("mkdir -p " .. path) - end -end - --- path_dir: extract the folder path from a file path --- from https://stackoverflow.com/a/9102300/3246758 -function path_dir(path) - return path:match("(.*".. get_path_sep() ..")") or "" -end - --- content following from quarto's qmd-reader.lua --- (c) 2023 rstudio, pbc. licensed under gnu gpl v2: --- https://github.com/quarto-dev/quarto-cli/blob/main/COPYRIGHT - -function random_string(size) - -- we replace invalid tags with random strings of the same size - -- to safely allow code blocks inside pipe tables - -- note that we can't use uppercase letters here - -- because pandoc canonicalizes classes to lowercase. - local chars = "abcdefghijklmnopqrstuvwxyz" - local lst = {} - for _ = 1,size do - local ix = math.random(1, #chars) - table.insert(lst, string.sub(chars, ix, ix)) - end - return table.concat(lst, "") -end - -function find_invalid_tags(str) - -- [^.=\n] - -- we disallow "." to avoid catching {.python} - -- we disallow "=" to avoid catching {foo="bar"} - -- we disallow "\n" to avoid multiple lines - - -- no | in lua patterns... - - -- (c standard, 7.4.1.10, isspace function) - -- %s catches \n and \r, so we must use [ \t\f\v] instead - - local patterns = { - "^[ \t\f\v]*(```+[ \t\f\v]*)(%{+[^.=\n\r]*%}+)", - "\n[ \t\f\v]*(```+[ \t\f\v]*)(%{+[^.=\n\r]+%}+)" - } - local function find_it(init) - for _, pattern in ipairs(patterns) do - local range_start, range_end, ticks, tag = str:find(pattern, init) - if range_start ~= nil then - return range_start, range_end, ticks, tag - end - end - return nil - end - - local init = 1 - local range_start, range_end, ticks, tag = find_it(init) - local tag_set = {} - local tags = {} - while tag ~= nil do - init = range_end + 1 - if not tag_set[tag] then - tag_set[tag] = true - table.insert(tags, tag) - end - range_start, range_end, ticks, tag = find_it(init) - end - return tags -end - -function escape_invalid_tags(str) - local tags = find_invalid_tags(str) - -- we must now replace the tags in a careful order. Specifically, - -- we can't replace a key that's a substring of a larger key without - -- first replacing the larger key. - -- - -- ie. if we replace {python} before {{python}}, Bad Things Happen. - -- so we sort the tags by descending size, which suffices - table.sort(tags, function(a, b) return #b < #a end) - - local replacements = {} - for _, k in ipairs(tags) do - local replacement - local attempts = 1 - repeat - replacement = random_string(#k) - attempts = attempts + 1 - until str:find(replacement, 1, true) == nil or attempts == 100 - if attempts == 100 then - print("Internal error, could not find safe replacement for "..k.." after 100 tries") - print("Please file a bug at https://github.com/quarto-dev/quarto-cli") - os.exit(1) - end - -- replace all lua special pattern characters with their - -- escaped versions - local safe_pattern = k:gsub("([%^%$%(%)%%%.%[%]%*%+%-%?])", "%%%1") - replacements[replacement] = k - local patterns = { - "^([ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern, - "(\n[ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern - } - - str = str:gsub(patterns[1], "%1" .. replacement):gsub(patterns[2], "%1" .. replacement) - end - return str, replacements -end - -function unescape_invalid_tags(str, tags) - for replacement, k in pairs(tags) do - -- replace all lua special replacement characters with their - -- escaped versions, so that when we restore the behavior, - -- we don't accidentally create a pattern - local result = k:gsub("([$%%])", "%%%1") - str = str:gsub(replacement, result) - end - return str -end - -local preprocess_qmd_filter = { - - -- search for `import_svelte("X.svelte")` refs in codeblocks and switch them - -- to `import("X.js")` - CodeBlock = function(block) - if block.classes:includes("{ojs}") then - - local svelte_import_syntax = - "import%_svelte%(\"([%w;,/%?:@&=%+%$%-_%.!~%*'%(%)#]+)%.svelte\"%)" - - local block_text = block.text - - -- get the qmd_path from disk - local current_qmd_path = read_file(".sverto/.sverto-current-qmd-folder") - - -- first, extract .svelte paths in import_svelte() statements - for svelte_path in block_text:gmatch(svelte_import_syntax) do - append_to_file(".sverto/.sverto-imports", - current_qmd_path .. svelte_path .. ".svelte\n") - end - - -- now change `import_svelte("X.svelte")` refs to `import("X.js")` - -- TODO - neaten up relative paths instead of assuming we're going from - -- /site_libs/quarto-ojs - block.text = block_text:gsub( - svelte_import_syntax, - "import(\"./../../" .. current_qmd_path .. "%1.js\")") - - end - return block - end, - - -- return the doc as a a whole unchanged... - -- except without the first block (the include statement) - Pandoc = function(doc) - local new_blocks = pandoc.Blocks({}) - for i, v in ipairs(doc.blocks) do - if i ~= 1 then - new_blocks:insert(v) - end - end - doc.blocks = new_blocks - return doc - - end -} - -create_dir_recursively(".sverto/") - --- collect the input qmd paths -in_file_string = os.getenv("QUARTO_PROJECT_INPUT_FILES") -in_files = {} -for in_file in string.gmatch(in_file_string, "([^\n]+)") do - table.insert(in_files, in_file) -end - --- transform each input qmd, saving the transformation in .sverto/[path] --- (write the identified .svelte files out to a file too!) -for key, qmd_path in ipairs(in_files) do - - -- before we read the file in with pandoc.read, let's read it in as a raw - -- string and do the quarto team's qmd-reader processing on it. THEN we can - -- read it with pandoc.read - - print(">>> PROCESSING " .. qmd_path) - - local raw_doc = read_file(qmd_path) - - -- store the current qmd_path on disk so the filter can access it - write_file(".sverto/.sverto-current-qmd-folder", path_dir(qmd_path)) - - -- escape invalid tags - local txt, tags = escape_invalid_tags(tostring(raw_doc)) - - -- some extension + format stuff that we can maybe ignore? - - -- for k, v in pairs(opts.extensions) do - -- extensions[v] = true - -- end - - -- if param("user-defined-from") then - -- local user_format = _quarto.format.parse_format(param("user-defined-from")) - -- for k, v in pairs(user_format.extensions) do - -- extensions[k] = v - -- end - -- end - - -- -- Format flavor, i.e., which extensions should be enabled/disabled - -- local flavor = { - -- format = "markdown", - -- extensions = extensions, - -- } - - local function restore_invalid_tags(tag) - return tags[tag] or tag - end - - -- NOW we read in with pandoc (hopefully ending up with real code blocks) - -- and restore them - -- local doc = pandoc.read(txt, flavor, opts):walk { - local doc = pandoc.read(txt, "markdown") - - local restored_doc = doc:walk { - CodeBlock = function (cb) - cb.classes = cb.classes:map(restore_invalid_tags) - cb.text = unescape_invalid_tags(cb.text, tags) - return cb - end - } - - -- local doc = pandoc.read(read_file(qmd_path)) - - -- pre-process the qmd, populating `svelte_files` in the process - -- local svelte_files = {} - local transformed_doc = restored_doc:walk(preprocess_qmd_filter) - create_dir_recursively(".sverto/" .. path_dir(qmd_path)) - write_file(".sverto/" .. qmd_path, pandoc.write(transformed_doc, "markdown")) - -end - --- write the output dir path temporarily (so rollup can use it) -write_file(".sverto/.sverto-outdir", os.getenv("QUARTO_PROJECT_OUTPUT_DIR")) - --- TODO - if there's no {{< import .sverto/file.qmd >}} block, add it? - diff --git a/docs/_extensions/sverto/refresh.ts b/docs/_extensions/sverto/refresh.ts deleted file mode 100644 index e0bf777..0000000 --- a/docs/_extensions/sverto/refresh.ts +++ /dev/null @@ -1,11 +0,0 @@ -// delete the temporary import block files and the temp render metadata - -if (Deno.env.get("QUARTO_PROJECT_RENDER_ALL") == "1") { - console.log("Clearing .sverto/") - try { - await Deno.remove(".sverto/", { recursive: true }) - } catch { - - } -} - diff --git a/docs/_extensions/sverto/rollup.config.js b/docs/_extensions/sverto/rollup.config.js deleted file mode 100644 index e730781..0000000 --- a/docs/_extensions/sverto/rollup.config.js +++ /dev/null @@ -1,57 +0,0 @@ -import svelte from 'rollup-plugin-svelte'; -import commonjs from '@rollup/plugin-commonjs'; -import resolve from '@rollup/plugin-node-resolve'; -import { terser } from 'rollup-plugin-terser'; - -const fs = require('fs'); -const path = require('node:path'); - -// this is false when we run rollup with -w/--watch (never presently) -const production = !process.env.ROLLUP_WATCH; - -// get quarto project output directory and list of inputs -const quartoOutDir = fs.readFileSync('.sverto/.sverto-outdir', 'utf8'); - -const svelteImportListPath = '.sverto/.sverto-imports'; - -// skip svelte compilation if there's nothing to compile -if (!fs.existsSync(svelteImportListPath)) { - console.log("ℹ No Svelte imports to process; skipping compilation"); - process.exit(); -} - -// get the list of unique imports to compile -const svelteFiles = fs.readFileSync(svelteImportListPath, 'utf8') - .split("\n") - .filter(d => d !== ""); -const uniqueSvelteFiles = [... new Set(svelteFiles)] - -// we export an array of rollup configs: one for each input svelte file -export default uniqueSvelteFiles.map( - svelteFile => ({ - input: svelteFile, - output: { - format: "es", - dir: path.join( - quartoOutDir, - path.dirname(svelteFile)), - sourcemap: true - }, - plugins: [ - svelte({ - // css is added to the js bundle instead - emitCss: false, - compilerOptions: { - // required for ojs reactivity - accessors: true, - dev: !production, - } - }), - resolve({ - browser: true, - dedupe: ["svelte"] - }), - commonjs(), - production && terser() - ] - })); diff --git a/docs/_extensions/sverto/setupTypeScript.js b/docs/_extensions/sverto/setupTypeScript.js deleted file mode 100644 index 133658a..0000000 --- a/docs/_extensions/sverto/setupTypeScript.js +++ /dev/null @@ -1,121 +0,0 @@ -// @ts-check - -/** This script modifies the project to support TS code in .svelte files like: - - - - As well as validating the code for CI. - */ - -/** To work on this script: - rm -rf test-template template && git clone sveltejs/template test-template && node scripts/setupTypeScript.js test-template -*/ - -const fs = require("fs") -const path = require("path") -const { argv } = require("process") - -const projectRoot = argv[2] || path.join(__dirname, "..") - -// Add deps to pkg.json -const packageJSON = JSON.parse(fs.readFileSync(path.join(projectRoot, "package.json"), "utf8")) -packageJSON.devDependencies = Object.assign(packageJSON.devDependencies, { - "svelte-check": "^2.0.0", - "svelte-preprocess": "^4.0.0", - "@rollup/plugin-typescript": "^8.0.0", - "typescript": "^4.0.0", - "tslib": "^2.0.0", - "@tsconfig/svelte": "^2.0.0" -}) - -// Add script for checking -packageJSON.scripts = Object.assign(packageJSON.scripts, { - "check": "svelte-check --tsconfig ./tsconfig.json" -}) - -// Write the package JSON -fs.writeFileSync(path.join(projectRoot, "package.json"), JSON.stringify(packageJSON, null, " ")) - -// mv src/main.js to main.ts - note, we need to edit rollup.config.js for this too -const beforeMainJSPath = path.join(projectRoot, "src", "main.js") -const afterMainTSPath = path.join(projectRoot, "src", "main.ts") -fs.renameSync(beforeMainJSPath, afterMainTSPath) - -// Switch the app.svelte file to use TS -const appSveltePath = path.join(projectRoot, "src", "App.svelte") -let appFile = fs.readFileSync(appSveltePath, "utf8") -appFile = appFile.replace("