Flux RADIUSS Tutorial Discussion Issue

[vsoch]

We are going to be running the Flux tutorials for RADIUSS again in August, and want to assemble a list of "cool new stuff since last time" to showcase. I thought an issue would be a good place to put that. You can find quick references to release notes here: https://flux-framework.org/releases/ and here are some ideas from me:

Features

• emoji job ids!!
• colorized output from event log

Flux SDKs

• Interactions with Python (JobWatcher class)?
• Interactions with Go?

Advanced?

• flux kvs usage?
• flux filemap?
• flux exec (should this be advertised more, or just a development command)?

Power

• Let's have an embedded video with slides / tutorial on using Variorum 📖 with the flux-power-monitor module from an LC cluster. Ping @tpatki and please bring me in if you want another person to converse with!

Ping @milroy and I'll post this in slack for more discussion.

[grondo]

Thank you for doing this again @vsoch and @milroy!

emoji jobids

This could be part of a larger discussion on Flux jobids and multiple representations depending on use context:
(bring on the flux job id command)

• jobids are 64 bit numbers called FLUIDs. Like UUIDs but are only unique within a given Flux instance. (flux job id --to=dec)
• hexadecimal for slightly more compact representation (flux job id --to=hex)
• F58 (base58) default encoding for most compact representation without easily mistaken characters
• dotted hexadecimal for use as a KVS key (flux job id --to=dothex and flux job id --to=kvs)
• a sequence of words for speaking over the phone, e.g. to a hotline staff member (flux job id --to=words)
• and of course emoji encoding (base576) (flux job id --to=emoji)

flux filename

I assume you mean flux filemap? That is a good idea for a tutorial topic. I don't have a good idea of an example for the tutorial though. Maybe run a batch job that downloads a parallel program source to /tmp, builds it locally, maps the build dir, then uses the shell stage-in plugin to run the built parallel program without need of a network filesystem?

flux exec

This doesn't hurt to have in a tutorial. Noting that this utility can be used with a single-user instance (batch/alloc jobs) is probably worth doing.

Other ideas:

• Description of Flux URIs, the flux uri utility, and supported "TARGETs" for flux uri and other commands like flux proxy TARGET, flux top TARGET, flux shutdown TARGET

• I can't remember if we already have a short tutorial for flux bulksubmit. I was thinking it would be good to adapt one of the GNU parallel tutorials using flux bulksubmit. You could demonstrate how bulksubmit could be used to parallelize and monitor a more traditional set of tasks (non HPC workload) under flux start on a single machine. Then we could show how the same set of tasks can be scaled up by using a multinode instance. Running the Flux testsuite under Flux is a good demonstration, though long.

• Demonstrate the flux batch/flux alloc --conf option, e.g. run an instance with a node exclusive match policy and exclude rank 0 via flux batch --conf=sched-fluxion-resource.match-policy=firstnodex --conf=resource.exclude=0 OPTIONS.. SCRIPT

• Demonstrate the --add-file= option for submission commands, perhaps in combination with multiline batch directives.

• Another interesting tutorial would be to demonstrate the resiliency of Flux batch jobs. Launch an instance with 4 brokers. Terminate one broker and show that the instance is still running with one "node" marked down.

[vsoch]

flux filename

Yeah that was a typo - translation error between brain and fingers ;) The cases I always use filemap for are when I have multiple physical nodes and need to send over a file to them. E.g., to start usernetes I have a join-command (secret) that is generated by the control-plane and then gets moved over.

I can't remember if we already have a short tutorial for flux bulksubmit

That's a good suggestion (I don't remember) but I don't know how to use it that well, so we should if we don't.

These suggestions are great - thanks @grondo !

[grondo]

Should we convert this issue to a discussion?

[vsoch]

Oh neat! That was easy. Look at all this threading goodness!