We should consider this in a severely timeboxed way. Bear in mind design, which is not directly "on the nose" to this request.

Note that unquoted literals appear in other places than in keys. We previously reserved a bunch of the ASCII punctuation (which is the main consideration here) for future use via reserved-statement. Removing that from the syntax does not mean that we should pilfer the box for more of these characters. Things that spoof sigils in appearance are probably a Bad Idea.

For example, one of the characters not listed above is :, which is the function introducer and namespace separator. It can't be in an unquoted. # and / probably need to be avoided because of markup. And @ because of attributes.

On the other hand, square brackets and parens seems potentially useful as do some of the other junk.

This won't go in 46.1, so I'm going to change the labels. I am also adding resolve-candidate because I think we won't extend unquoted, but that's for the WG to decide.

One comment; broadening can be done in future versions, since it would be backwards compatible.

Broadening can be done, so long as it is done in a backwards-compatible way. It's a little tricky here, because of the uses of literals in the syntax. I haven't carefully reviewed the proposal recently enough to say one way or the other if there are sticky bits that I'd object to. I won't say that we would never do an extension (never is a long time), but I think it unlikely in the 2.0 timeframe (e.g. 46.1/47)

I would like to have a discussion on this in January.

I will add this to a meeting agenda in January. The WG asked me to close this last week, as the consensus was that we would not revisit the syntax in the 46.1 timeframe and didn't feel that our previous discussions about text literals would change later.

Our usual process would be to create a design doc (the onus is on the person proposing a change to document the proposal and compare/contrast it with the current design and other options), but let's discuss first.

I would like at least a clear statement in v47 that the literal syntax could be expanded in future versions to include more characters than just what are in name and the simple number-literal.

The literal syntax allows nearly any Unicode character. Do you mean unquoted-literal?

Yes, I meant unquoted = name / number-literal can be expanded.

I just cleaned up the description a bit.

1. Added restriction 3
2. Changed 'unquoted' to 'unquoted-literal' (I think the ABNF might have changed after I wrote the first version)
3. Added Notes at the end.

@eemeli , any comments?

We also need to reserve at least @ (attributes) and # & / (markup), and almost certainly = as it's used in options and attributes.

I would be much more comfortable with !, %, *, and ? also being reserved to allow for some future extensibility. ', ", (, ), [, and ] should be reserved to preclude misunderstandings.

Strictly speaking,

Reserving '@' is not necessary, because syntactically it can't cause ambiguity in the syntax (an attribute can't be in the same position as a literal). Same for '#' and '='.

The '=' because it is required that a literal be separated from an attribute or option identifier by a =. So with someOption=a=b you know that the option value starts after the first =, but it doesn't formally matter that it contains a =. However, it could be confusing to readers.

'/' on the other hand can cause ambiguity in the syntax, because of the following:
markup = "{" o "#" identifier *(s option) *(s attribute) o ["/"] "}"
The option could have a value that in unquoted, and if / is allowed as a final character in that value then it would collide. It could be allowed as a non-final. That's unfortunate, because it is the simplest character for a rational number. But can't be helped.

Disallowing (, ), [, and ] would be unfortunate, since they are the natural characters for open/closed ranges.

I don't see any math use for ' or ", so no particular reason to allow them in unquoted literals.

However, I admit it would be simpler if the only ASCII characters allowed in unquoted literals were [A-Za-z0-9-+_.].

Reserving '@' is not necessary, because syntactically it can't cause ambiguity in the syntax (an attribute can't be in the same position as a literal). Same for '#' and '='.

If # is not reserved, {#foo} could parse as either a literal-expression or as markup.

If @ is not reserved, {@foo} would look like an expression with only an attribute, and that would be confusing.

The '=' because it is required that a literal be separated from an attribute or option identifier by a =. So with someOption=a=b you know that the option value starts after the first =, but it doesn't formally matter that it contains a =. However, it could be confusing to readers.

Yes, and we need to keep in mind that we should not presume that people look at MF2 often. Not confusing readers is a high priority.

We'll also need to exclude * as unquoted-literal is also used in variant keys, where * is special.

proposal:

1. In ASCII, only [A-Za-z0-9-+_.]
2. Disallow all of the big blocks of ‘strange’ code points that XML forbids: controls, (unpaired) surrogates, private-use, noncharacters.

These are all immutable (Unicode Character Encoding Stability).

This also disallows the noncharacters that XML didn’t know about yet, before the noncharacter property was made immutable.

3. No whitespace, since variant uses that for separators between keys, and expressions use it to separate various components.

This could be done by just disallowing the “s” production characters, but that could be very confusing. {a b} looks too much like two items (the space is an A0 NO-BREAK SPACE). So it should be broadened to the Unicode Whitespace characters.

Unicode Whitespace is not guaranteed immutable, but has not changed for over a decade. Anyway, we would derive the code points as of now, so everything would be stable into the future.

I just made a census of characters inside and outside of name_char:

So if we include just the ASCII that unquoted-literal contains now (without bothering with the number syntax), and regularize it, we'd get:

[[A-Za-z0-9\-_.+-][^\p{Cc}\p{Cs}\p{Co}\p{NChar}\p{whitespace}\p{ascii}]]

That is,

• Only specific ASCII: A-Za-z0-9 - _ . + - (plus any others we want to 'release')
• All non-ASCII except:

That would be result in the following.

unquoted-char =
      %x2B          ; "+"      omit Cc %x0-1F, Whitespace " ", Ascii "!".."*"
    / %x2D-2E       ; "-".."." omit Ascii ","
    / %x30-39       ; "0".."9" omit Ascii "/"
    / %x41-5A       ; "A".."Z" omit Ascii ":".."@"
    / %x5F          ; "_"      omit Ascii "[".."^"
    / %x61-7A       ; "a".."z" omit Ascii "`"
    / %xA1-167F     ;          omit Cc %x7F-9F, Whitespace %xA0, Ascii "{".."~"
    / %x1681-1FFF   ;          omit Whitespace %x1680
    / %x200B-2027   ;          omit Whitespace %x2000-200A
    / %x202A-202E   ;          omit Whitespace %x2028-2029
    / %x2030-205E   ;          omit Whitespace %x202F
    / %x2060-2FFF   ;          omit Whitespace %x205F
    / %x3001-D7FF   ;          omit Whitespace %x3000
    / %xF900-FDCF   ;          omit Cs %xD800-DFFF, Co %xE000-F8FF
    / %xFDF0-FFFD   ;          omit NChar %xFDD0-FDEF
    / %x10000-1FFFD ;          omit NChar %xFFFE-FFFF
    / %x20000-2FFFD ;          omit NChar %x1FFFE-1FFFF
    / %x30000-3FFFD ;          omit NChar %x2FFFE-2FFFF
    / %x40000-4FFFD ;          omit NChar %x3FFFE-3FFFF
    / %x50000-5FFFD ;          omit NChar %x4FFFE-4FFFF
    / %x60000-6FFFD ;          omit NChar %x5FFFE-5FFFF
    / %x70000-7FFFD ;          omit NChar %x6FFFE-6FFFF
    / %x80000-8FFFD ;          omit NChar %x7FFFE-7FFFF
    / %x90000-9FFFD ;          omit NChar %x8FFFE-8FFFF
    / %xA0000-AFFFD ;          omit NChar %x9FFFE-9FFFF
    / %xB0000-BFFFD ;          omit NChar %xAFFFE-AFFFF
    / %xC0000-CFFFD ;          omit NChar %xBFFFE-BFFFF
    / %xD0000-DFFFD ;          omit NChar %xCFFFE-CFFFF
    / %xE0000-EFFFD ;          omit NChar %xDFFFE-DFFFF
                    ;          omit Co %xF0000-FFFFD %x100000-10FFFD, NChar %xEFFFE-EFFFF %xFFFFE-FFFFF %x10FFFE-10FFFF