Decomposition proposal

[0bOne]

Like many developers, I struggle with regular expressions. They are handy and portable, but terse.
Disadvantages are:

• Hard to memorize and understand
• Lack of intentionality/readability leads to bugs and loopholes, particularly concerning in security scenarios
• Hard to convert between other types of business rules
• Regular expressions evaluate differently in different languages/platforms
• Regular expressions don't cover every eventuality

A good example is an email validator. I could simply use format="email" but that is open to interpretation and has several obscure regular expressions.

In a private project, I am working, on an extension to the schema that would allow an alternative method of pattern matching. If the json schema were extended with a "decomposition" object type, which would act as both a string and array. For example:

• "$schema": "...",
• "$id": "...",
• "title": "email format",
• "description": "our definition of a valid email address",
• "type": "composition",
• "minLength": 5,
• "maxLength": 320,
• "split": "@",
• "minItems": 2,
• "maxItems": 2,
• "prefixItems": [
  { "$ref": "$defs/localPart" },
  { "$ref": "$defs/domainPart" }
  ],
• etc

It is first evaluated as a string (minLength, maxLength)
Then it is split based on new property "split" (character, sequence, or null)
Then it is evaluated as an array (minItems, maxItems, items, prefixItems, [suffixItems!] )
Those could be recursively evaluated as string regexes or other compositions.

I believe this approach would allow complex strings to be evaluated as json schema objects, with very little change in the existing schema specification. One could argue that strings are not part of a json schema, which would be valid, but we already validate them with regular expressions - this would be just an alternative method: more explicit and intentional.

Of course, it needs more discussion to understand:

• whether there is already a specification component that does this that I missed
  (please don't respond and say regex's do this; read my opinion on their limitations above)
• whether it should be considered (purists might disagree, but the intent is flexibility over dogma)
• whether it adds enough value to be a consideration, or is a corner-case
• whether there are limitations not thought of
• whether those limitations can be overcome

[handrews]

You'd probably be interested in a similar proposal for array processing: json-schema-org/json-schema-spec#1323

I am personally skeptical of this sort of thing (both proposals) as it is moving too far in the direction of programming languages with the control flow (then again, I didn't want to add if/then/else either).

The correct question here is about the scope of what you can do with a JSON Schema keyword. These proposals substantially expand that scope, and it's not at all clear to me that the expansion is desirable when you take into account the additional processing complexity.

For example, the expanded capabilities needed to implement unevaluatedProperties and unevaluatedItems came at the cost of requiring implementations to track state that could previously be ignored, and as a result reduced the set of circumstances in which parts of schema evaluation can be handled in parallel. This was determined to be acceptable because the keywords were the solution to a problem that had persisted for years. That discussion ran 500+ github comments in total across multiple competing proposals before we decided to do it. These sorts of tradeoffs cannot be made lightly. There are tools other than JSON Schema that may be more appropriate.

[0bOne]

I don't disagree with any of that, but as a metaprogrammer, my quest is the need to encode all business rules as data.

I agree, there is extra processing complexity, but that is true of any metaprogramming approach, where the goal is to have compact code that compares business data with business rule metadata to do its work. The code is four times harder to write, but it's 20 times as efficient (approximately, but proven)

The challenge we have is that standards like JSON schema (and others, like REST) get us most of the way there, then we have to 'branch' the standard to get to the final 100%.

Once we branch, keeping up with the standard becomes impractical, so it's always better to try to extend the standard than write our own.

[jdesrosiers]

I'm more interested in non-homogenous array validation than sting validation, but I think solutions for one can apply to the other, so I'd be interested in reading a case-study about what you learn from implementing this as a vocabulary. Here's my advice.

Although technically not forbidden, it's not a good idea to extend the functionality of existing keywords such as type. I suggest creating new keywords as necessary. The type keyword is for asserting the JSON type of the instance value. I'm not entirely clear, but you seem to be using it more as an annotation to trigger a different way of evaluating the schema. (The exception for type values is "integer", which is mostly there because it always has been, not because it makes sense.)

It's not clear what syntax you plan to use, but keep in mind that JSON objects are unordered data structures. Most implementations preserve the order of properties when parsing, but you can't rely on that across implementations. If it's important that the keywords be processed in a specific order, you'll need to use an array.