Getting serious about schemas

[stuartpb]

I've been lax about the schema for this for way too long, mostly because I couldn't be bothered to actually learn JSON Schema (figuring, from the brief experience that I've had with it, that it wasn't sufficiently flexible for my needs).

Anyway, I'm thinking now about how I actually want to do this going forward, and I've come to the conclusion that I should be using, and testing, with an actual schema, rather than the weird, incomplete, docs-derived schema I'm using now.

Some thoughts:

• I think it'd make sense to start drafting the proper-schema stuff in a separate repo, now that this project has its own org. I'll make a new issue to talk about repo-separation.
• The docs should live alongside the schema, with this repo becoming only for profiles.

[stuartpb]

I'm thinking there'll be a schema file that contains the ID of the schema that ll the profiles follow. This'll also make it easier to track drafts for branches, per some of the latest talk about doing that, with the "schema" file being something where you can point it at the draft that the files in the branch are tracking.

[stuartpb]

To learn / decide: how does a JSON Schema work with the plan for versioning (#7, ie. schemas being only tracked by semver major version)? How does forward compatibility work? Is it a flag in schemas? Outside of it?

[stuartpb]

This needs to reach out to #42, #43, and #111.

[stuartpb]

I'll be writing a custom validation tool for this, so I've been giving a little thought about how that'll work.

So, the syntax will be opws-validate [FILENAMES]..., where FILENAMES is the name of every file to be validated. If every profile is to be validated, we rely on shell globbing etc to pass them in via opws-validate profiles/*, rather than having any built-in globbing magic (so the glob module can be jettisoned).

Since this'll be the validator for everything in the OPWS dataset, it'll have to take legacies (#43) and potentially other document types (like stacks) in the future: my plan is that the validator will sniff this from the prefix of the directory (ie. profiles/whatever.foo.yaml is probably a profile).

For testing in other ways (ie. piping it in, a bunch of files in a working directory that's not structured like the tree root), I'm thinking it'll (complaining to STDERR) assume profiles by default, but also allow specifying which set of validations to run with flags like --profiles or --legacies, where every filename coming after one of those is explicitly interpreted as such (with --sniff to get back to the path-sniffing and complaining-on-assumption behavior).

There should also probably be, like, a strong --sniff, that goes from the end of a given filename (so X layers deep doesn't bother it if the filenames are all being provided as absolute paths) and fails validation if it doesn't match the pathspec (which is part of testing now), to be used in CI. Maybe this could be --tree or --names? (If, for some reason, a test was looking to validate names only, it'd use some flag name like --names-only, because that would be less useful, and as such, longer to type.)

[stuartpb]

... I think its no-flags behavior (which should allow opws-validate my-prospective-profile.yaml just fine) can be a kind of floaty thing for now (ie. it might do a "test against all schemas and succeed if any one of them match", which might become a mode-switch via --any), since what I'm really focusing on is the way to make sure it's not leaky for use in scripting.

Also, one thing that'll be necessary: a --tight flag that'll give it the "reject unspecified data" behavior the current tests have. I don't want to force this on validation by default, but I do want to have it be part of accepting a pull request.

[stuartpb]

Also, since -- won't work as a way of allowing hyphen-prefixed files in a stream where hyphen-prefixed arguments are needed to switch modes between files, I'm thinking --profiles= will be an accepted syntax to specify which mode to run a file on, with everything after = being interpreted as a single filename (and then the rest of the command line continuing like --profiles was specified).

That way, you can specify multiple ugly names by prefixing each of them with the applicable processing mode.

No, wait, that's dumb, if you have files like that, you can just specify them as ./whatever, never mind.

[stuartpb]

There'll have to be a style guide for the schemata (as first mentioned in #32, and probably alluded to in other issues) to serve as a sort of "meta-schema" dictating what the schema will ever include.

Or, really, just after migrating repos (#147), there'll have to be a new label in issues for meta concerns (the kind that are currently tagged with "bureaucracy" labels or "policy" milestones) like #96, #149, #200, #265, etc. Then, at some point, those can be collected into some kind of document or documents that may or may not be merged with the style guide that's currently in the wiki (which'll probably get broken up into a schema style guide and a profile style guide).

[stuartpb]

This is pretty much done now: I'll open up a new issue in opws-guidelines for the schemata style guide thing.