REPL: reloading ES6 modules

[borkdude]

What approaches would be feasible, while also allowing for tree-shaking by ES6 tooling?

[borkdude]

One possible approach: in npx cherry --dev mode we could emit for every var:

var foo = ...
var set_foo = ...

Then when evaluating (defn foo []) in the REPL, we construct foo in a temporary module and then call original_module.set_foo(foo).

One problem would be side-effecting code perhaps.

[borkdude]

Another approach might be: every .cljs namespaces defines a global object (only in development) to facilitate the REPL: evaluating in the REPL mutates the global object. The ES6 module simply references the global object.

One difficulty with this is that imports / exports are static once the module has been loaded by JS.

[lilactown]

I've thought about this before; I haven't tried it yet, but I think you don't have to map every public var to an export. Instead, you could export a mutable object at dev time. pseudo code below

/* @cherry-namespace: myapp_main */
import {exports as cljs_core} from "@cherry/core";
import {exports as myapp_feature} from "./feature.js";

function foo () {
  return cljs_core.inc(myapp_feature.magic_number);
}

global.myapp_main = { foo };

export const exports = global.myapp_main;

So essentially, during development you would import and export any CLJS source files using the above method, and reloads would update the global.myapp_main object. At release time, you could remove the global and change it to exporting each public var, which would probably help with tree shaking (again, haven't tested this).

The problem with this approach would be if you were mixing CLJS and other local ESM code at dev time, because the import would be different depending on whether you were in dev or release mode. I wonder if this could be solved by some sort of plugin for webpack, etc?

[jo-sm]

What may work is mimicking how JS tools handle hot module reloading, which is to use import.meta.hot and import.meta.accept, which you could then somehow hook into when loading a namespace in a CLJS REPL and rewrite the code. Here are the docs on how Vite handles it.

If this approach works, then it mean that the way you deal with HMR and REPL reloading would be basically identical, which would be great and make it simple to work with existing HMR setups in JS-land.

[borkdude]

Of note: the vite and nextjs examples in examples already work fine with hot reloading performed by respective frameworks. Just have to watch the .cljs files to compile them (see bb.edn) and then the JS framework does the rest.

[bhauman]

I'd like to chime in that it would be nice to preserve the Clojure semantics around reloading in general. This behavior being one of the killer features of Clojure. I.E. reloading a file redefines the vars globally and any side effecting code gets executed. I do think this is probably best accomplished via compiling in a dev format versus a prod format.

[borkdude]

Agreed.

Btw, I know it's not a REPL, but made a little browser playground here:

https://borkdude.github.io/cherry/index.html

[bhauman]

Here's an interesting approach to reloading: https://itnext.io/hot-reloading-native-es2015-modules-dc54cd8cca01

The interesting thing is calling the "import" function to reload a file using a different URL (using a query param) to force re-evaluation of the file. Thus the top level stuff gets evaled and if you place the module functions on a global context variable window.cherry.root.example.foo = function(a,b) {...} they will be reassigned on loading.

The reason to hook it on a global object is for debugging purposes as its easier to get at from a console.

AND you could also use this import hack to implement a REPL eval so that it is working in the correct module context.

[borkdude]

The interesting thing is calling the "import" function to reload a file using a different URL (using a query param) to force re-evaluation of the file.

It's also used in the playground example, so executing the same snippet twice will evaluate twice. The snippet is first compiled and then encoded as a data url and then imported by the browser. :)

[bhauman]

Very cool! I hadn't looked into this before!

[borkdude]

I think @lilactown's approach makes sense too:

#25 (comment)

[bhauman]

Yeah that looks good! I guess it goes without saying that all calls have to be indirect through the mutable export.

Actually I might place the module on a common root object and then in turn place that on the global object. To enable cleaning things by deleting a single object and reducing the possibility of name collisions.

An alternative would be to encapsulate the mutable exports in its own module and not make it global. The module could have a convenience method to attach the root object to the global object for debugging

[borkdude]

Actually I might place the module on a common root object and then in turn place that on the global object.

With module, do you mean the mutable object with vars, or an ES6 module?

Some pseudo-code would help me understand this better.

An alternative would be to encapsulate the mutable exports in its own module

Again, some pseudo-code would help, I think I'm not following exactly.

[bhauman]

Yeah I should have provided examples to start with.

Here's the above example with a root object as an intermediary to the global object. This helps reduce the likelihood global name collision and makes it simpler matter to clean things up for a full reset without hitting reload.

/* @cherry-namespace: myapp_main */
import {exports as cljs_core} from "@cherry/core";
import {exports as myapp_feature} from "./feature.js";

function foo () {
  return cljs_core.inc(myapp_feature.magic_number);
}
/* EDIT */
/* ROOT object to avoid name collisions and to aid with clean up of live env if necessary. */
global.cherry_env_root_object.myapp_main = { foo: foo };

export const exports = global.cherry_env_root_object.myapp_main;

Here's a further example where we encapsulate a mutable root object. Encapsulating the root object further reduces any chance of interference from whatever else is executing in the runtime env.

/* cherry_env_root */

const context = {cherry_env_root: {} };

export default  context.cherry_env_root;

/* @cherry-namespace: myapp_main */
import {exports as cljs_core} from "@cherry/core";
import {exports as myapp_feature} from "./feature.js";

/* EDIT import encapsulated root from module */
import {exports as cherry_root} from "./cherry_env_root.js";



function foo () {
  return cljs_core.inc(myapp_feature.magic_number);
}
/* EDIT */
/* ROOT object to avoid name collisions and to aid with clean up of live env if necessary. */
cherry_root.myapp_main = { foo: foo };

export const exports = cherry_root.myapp_main;

[borkdude]

Much clearer, thank you!

[borkdude]

For completeness, just spitting out more ideas:

One could emit vars as var foo_fn = {val: function() { } } so vars are always mutable, also from the outside (ES6-wise). calls to vars would be compiled as foo_fn.val(1, 2, 3) (a var deref, like in Clojure JVM). This comes with a little bit of performance overhead, but maybe not that much. This will solve the REPL and dynamic var problem I think.

The annoying bit is when calling these suckers from JS: you also have to do the .val indirection, which sucks. Darn.

Or, what if you had:

> var foo_fn = function(...args) { return foo_fn.val(...args) }
> foo_fn.val = function(...args) { return [...args] }
> foo_fn(1, 2, 3)
[ 1, 2, 3 ]

> foo_fn.val = function(...args) { return args[0] }
[Function (anonymous)]
> foo_fn(1, 2, 3)
1

This only works for functions though.

[borkdude]

Some progress from squint: https://twitter.com/borkdude/status/1577255756919115777
This will be ported to cherry as well (I'm planning to unify the compiler code of both projects)

[borkdude]

More news:

https://github.com/squint-cljs/squint#repl