Design for commands: plumbing vs. user (ping Mario Brothers!)

[vsoch]

We want to break the entire Flux command spec into groups of "this is for plumbing" (Mario and Luigi) vs "this is for a user" (Peach!) For now I'm going to ignore flags because that is a deeper level of detail. Since a diff can color a line, I'm going to use the following convention:

+ user-command This is a user command
- plumber-command This is a plumber command
  unknown-command This is a command that I'm not sure if it's user or plumber (and maybe could be both?)

And that will draw our attention to the user commands, which is what we should focus on as top priority. And please excuse my terrible Nintendo references. I can't help it.

Please edit these as you need if you know an answer to the above. If there is discussion on something we can have it in a thread here.
I thought this would be better for discussion than for an issue for that. For the sections below, if I don't include sub-commands for a top level command, assume the sub-commands all inherit their parent classification.

Flux

$ flux --help

-   config             Manage/query Flux configuration
+   cron               Schedule tasks on timers and events
-   dmesg              manipulate broker log ring buffer
-   dump               Write KVS snapshot to portable archive
   env                Print the flux environment or execute a command inside it
   exec               Execute processes across flux ranks
   filemap            Map files into a Flux instance
-   {get,set,ls}attr   Access, modify, and list broker attributes
+   jobs               list jobs submitted to Flux
   pstree             display job hierarchies
-   uptime             Tell how long Flux has been up and running
-   shutdown           Shut down a Flux instance
   resource           list/manipulate Flux resource status
-   restore            Read KVS snapshot from portable archive
   kvs                Flux key-value store utility
+   mini               Minimal Job Submission Tool
+   job                Job Housekeeping Tool
-  overlay            Show flux overlay network status
-   ping               measure round-trip latency to Flux services
 +  proxy              Create proxy environment for Flux instance
   queue              Manipulate flux queues
   start              bootstrap a local Flux instance
 -  startlog           Show Flux instance start and stop times
+   top                Display Running Flux Jobs
   version            Display flux version information

A lot of the command group above are unknown for me - e,.g., I use flux start a lot in the various containers / Flux operator use cases, but I'm not sure if that is strongly user or strongly plumbing (or both?) Perhaps for the convention here, if we consider something to be both we should just call it a user command? In which case I'd also add env proxy and top. For subcommands like exec and filemap and ping I'm mixed because while a user could use it, I'm not convinced it would be common. For something like queue it also feels a bit rare, or perhaps on the plumbing side (I don't know). For proxy there is definitely a user use case, but it's more an advanced mode (but still should be under "user"). For top and pstree I suspect these would be useful for users, and (although I haven't used them) we would want to encourage their usage.

Flux Job

$ flux job --help

-  urgency         Set job urgency (0-31, HOLD, EXPEDITE, DEFAULT)
+   cancel          Cancel one or more jobs **this should be one command with --all)**
   cancelall       Cancel multiple jobs
   raise           Raise exception on one or more jobs
   raiseall        Raise an exception on multiple jobs.
+   kill            Send signal to one or more running jobs
   killall         Send signal to multiple running jobs
   attach          Interactively attach to job **this should be something that is more akin to logs
+   status          Wait for job(s) to complete and exit with largest exit code
+   submit          Run job
-  id              Convert jobid(s) to another form (the complexity of the job identifier should be removed so this is not needed)
-   eventlog        Display eventlog for a job
-   wait-event      Wait for an event 
+   info            Display info for a job
+   stats           Get current job stats (there are too  many options here to get job into, and none of them actually return complete job info)
-   namespace       Convert job ids to job guest kvs namespace names
+   wait            Wait for job(s) to complete.
-   memo            Post an RFC 21 memo to a job
-   taskmap         Utility function for working with job task maps
-   timeleft        Find remaining runtime for job or enclosing instance
+   purge           Purge the oldest inactive jobs

A lot of these might be wrong in my classification - please change! I also want to again direct attention to https://github.com/vsoch/flux-commands because I like the start of that organization for the above.

Flux Mini

$ flux mini --help

+    run                 run a job interactively
+    submit              enqueue a job
+    bulksubmit          enqueue jobs in bulk
+    batch               enqueue a batch script
+    alloc               allocate a new instance for interactive use

Flux Queue

$ flux queue --help

-   enable          Enable job submission
-   disable         Disable job submission
-   start           Start scheduling
-   stop            Stop scheduling
+   status          Get queue status
-   drain           Wait for queue to become empty.
-   idle            Wait for queue to become idle.

Let's a goooo!

[grondo]

We should probably use the latest version of flux to gather existing commands. e.g. flux pgrep/pkill and flux queue list are missing above.

[grondo]

There's also a lot of missing commands because they are not automatically included in flux help. We can assume that these were all intended to be "plumbing" commands, and they haven't caused any confusion since they went unnoticed.

[vsoch]

I totally didn't know that! Yes let's pretend they aren't there for the time being.

[trws]

I agree, but I think one thing that would be really good to get out of this is a help --all or help --advanced that lists all the things. We can certainly automatically put those things in advanced but it would be good to at least have them there (unless we really, really want to hide them for some reason).

[grondo]

I was thinking the same thing when completing the recent rework. (I noticed git has git help -a). This should not be too difficult to implement (we already have a list of builtin commands, then just walk FLUX_EXEC_PATH globbing for flux-* with exec bit set)

[trws]

Yeah exactly that, spack has something similar. For reference, default help:

❯ spack help
usage: spack [-hkV] [--color {always,never,auto}] COMMAND ...

A flexible package manager that supports multiple versions,
configurations, platforms, and compilers.

These are common spack commands:

query packages:
  list                  list and search available packages
  info                  get detailed information on a particular package
  find                  list and search installed packages

build packages:
  install               build and install packages
  uninstall             remove installed packages
  gc                    remove specs that are now no longer needed
  spec                  show what would be installed, given a spec

configuration:
  external              manage external packages in Spack configuration

environments:
  env                   manage virtual environments
  view                  project packages to a compact naming scheme on the filesystem.

create packages:
  create                create a new package file
  edit                  open package files in $EDITOR

system:
  arch                  print architecture information about this machine
  audit                 audit configuration files, packages, etc.
  compilers             list available compilers

user environment:
  load                  add package to the user environment
  module                generate/manage module files
  unload                remove package from the user environment

optional arguments:
  --color {always,never,auto}
                        when to colorize output (default: auto)
  -V, --version         show version number and exit
  -h, --help            show this help message and exit
  -k, --insecure        do not check ssl certificates when downloading

more help:
  spack help --all       list all commands and options
  spack help <command>   help on a specific command
  spack help --spec      help on the package specification syntax
  spack docs             open https://spack.rtfd.io/ in a browser

Help all:

❯ spack help --all
usage: spack [-hHdklLmbpvV] [--color {always,never,auto}] [-c CONFIG_VARS] [-C DIR] [--timestamp] [--pdb] [-e ENV | -D DIR | -E] [--use-env-repo]
             [--sorted-profile STAT] [--lines LINES] [--stacktrace] [--backtrace] [--print-shell-vars PRINT_SHELL_VARS]
             COMMAND ...

A flexible package manager that supports multiple versions,
configurations, platforms, and compilers.

Complete list of spack commands:

administration:
  clone                 create a new installation of spack in another prefix
  deprecate             Replace one package with another via symlinks
  make-installer        generate Windows installer
  mark                  mark packages as explicitly or implicitly installed
  reindex               rebuild Spack's package database
  test                  run spack's tests for an install
  test-env              run a command in a spec's test environment, or dump its environment to screen or file
  verify                Check that all spack packages are on disk as installed

query packages:
  list                  list and search available packages
  info                  get detailed information on a particular package
  find                  list and search installed packages
  dependencies          show dependencies of a package
  dependents            show packages that depend on another
  diff                  compare two specs
  graph                 generate graphs of package dependency relationships
  location              print out locations of packages and spack directories
  providers             list packages that provide a particular virtual package
  resource              list downloadable resources (tarballs, repos, patches, etc.)
  tags                  Show package tags and associated packages

build packages:
  fetch                 fetch archives for packages
  stage                 expand downloaded archive in preparation for install
  patch                 patch expanded archive sources in preparation for install
  restage               revert checked out package source code
  install               build and install packages
  uninstall             remove installed packages
  clean                 remove temporary build files and/or downloaded archives
  build-env             run a command in a spec's install environment, or dump its environment to screen or file
  ci                    manage continuous integration pipelines
  dev-build             developer build: build from code in current working directory
  gc                    remove specs that are now no longer needed
  log-parse             filter errors and warnings from build logs
  spec                  show what would be installed, given a spec

configuration:
  config                get and set configuration options
  external              manage external packages in Spack configuration
  mirror                manage mirrors (source and binary)
  repo                  manage package source repositories
  tutorial              set up spack for our tutorial (WARNING: modifies config!)

container:
  containerize          creates recipes to build images for different container runtimes

developer:
  blame                 show contributors to packages
  cd                    cd to spack directories in the shell
  commands              list available spack commands
  debug                 debugging commands for troubleshooting Spack
  license               list and check license headers on files in spack
  maintainers           get information about package maintainers
  pkg                   query packages associated with particular git revisions
  pydoc                 run pydoc from within spack
  python                launch an interpreter as spack would launch a command
  solve                 concretize a specs using an ASP solver
  style                 runs source code style checks on spack
  unit-test             run spack's unit tests (wrapper around pytest)
  url                   debugging tool for url parsing

environments:
  add                   add a spec to an environment
  change                change an existing spec in an environment
  concretize            concretize an environment and write a lockfile
  develop               add a spec to an environment's dev-build information
  env                   manage virtual environments
  remove (rm)           remove specs from an environment
  undevelop             remove specs from an environment
  view                  project packages to a compact naming scheme on the filesystem.

extensions:
  extensions            list extensions for package

create packages:
  create                create a new package file
  edit                  open package files in $EDITOR
  buildcache            create, download and install binary packages
  checksum              checksum available versions of a package
  gpg                   handle GPG actions for spack
  versions              list available versions of a package

system:
  arch                  print architecture information about this machine
  audit                 audit configuration files, packages, etc.
  bootstrap             manage bootstrap configuration
  compiler              manage compilers
  compilers             list available compilers

user environment:
  load                  add package to the user environment
  module                generate/manage module files
  unload                remove package from the user environment

optional arguments:
  --backtrace           always show backtraces for exceptions
  --color {always,never,auto}
                        when to colorize output (default: auto)
  --lines LINES         lines of profile output or 'all' (default: 20)
  --pdb                 run spack under the pdb debugger
  --print-shell-vars PRINT_SHELL_VARS
                        print info needed by setup-env.[c]sh
  --sorted-profile STAT
                        profile and sort by one or more of:
                        [calls, ncalls, cumtime, cumulative, filename, line, module]
  --stacktrace          add stacktraces to all printed statements
  --timestamp           Add a timestamp to tty output
  --use-env-repo        when running in an environment, use its package repository
  -C DIR, --config-scope DIR
                        add a custom configuration scope
  -D DIR, --env-dir DIR
                        run with an environment directory (ignore named environments)
  -E, --no-env          run without any environments activated (see spack env)
  -H, --all-help        show help for all commands (same as spack help --all)
  -L, --disable-locks   do not use filesystem locking (unsafe)
  -V, --version         show version number and exit
  -b, --bootstrap       use bootstrap configuration (bootstrap store, config, externals)
  -c CONFIG_VARS, --config CONFIG_VARS
                        add one or more custom, one off config settings.
  -d, --debug           write out debug messages (more d's for more verbosity: -d, -dd, -ddd, etc.)
  -e ENV, --env ENV     run with a specific environment (see spack env)
  -h, --help            show this help message and exit
  -k, --insecure        do not check ssl certificates when downloading
  -l, --enable-locks    use filesystem locking (default)
  -m, --mock            use mock packages instead of real ones
  -p, --profile         profile execution using cProfile
  -v, --verbose         print additional output during builds

more help:
  spack help --all       list all commands and options
  spack help <command>   help on a specific command
  spack help --spec      help on the package specification syntax
  spack docs             open https://spack.rtfd.io/ in a browser

There's also a somewhat terrifying "complete" option that dumps all the help output for the entire tree of commands and subcommands. Not going to post that here, but the first two at least I find I use regularly, and the grouping of commands helps more than I would have thought.