ICU-22907 MF2: Finish updating spec tests and implement required test functions #3216
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
2680b67 to
c3902e3
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
c3902e3 to
5f55d7c
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
5f55d7c to
38500d3
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
catamorphism
changed the title
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
51d2915 to
43b0280
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
catamorphism
changed the title
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
43b0280 to
076ee46
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
076ee46 to
31abd17
Compare
|
Hooray! The files in the branch are the same across the force-push. 😃 ~ Your Friendly Jira-GitHub PR Checker Bot |
… functions Implement :test:format, :test:select, and :test:function, which are required by the new `pattern-selection.json` tests. Change the internal value representation in the formatter in order to support some of the test cases (binding the results of selectors to a variable).
catamorphism
force-pushed
the
spec-updates-plus-pattern-selection
branch
from
31abd17 to
99fe24c
Compare
|
Notice: the branch changed across the force-push!
~ Your Friendly Jira-GitHub PR Checker Bot |
|
@catamorphism and @srl295 What's going on with this one? Do we need new reviewers? Is this still happening? |
@richgillam I'm currently on medical leave, hopefully returning Feb. 24. In the meantime, it could be reviewed. I hope it's still happening, as it's necessary for spec compliance! |
srl295
approved these changes
Jan 27, 2025
| } | ||
| // 9. If the fails option is set, then | ||
| Formattable failsOpt; | ||
| if (options.getFunctionOption(UnicodeString("fails"), failsOpt)) { |
There was a problem hiding this comment.
This is the expensive constructor for UnicodeString, using default char* constructor.
I think a followon would be good to make these statics or something.
| UnicodeString failsString = failsOpt.getString(status); | ||
| if (U_SUCCESS(status)) { | ||
| // 9i. If its value resolves to the string 'always', then | ||
| if (failsString == u"always") { |
There was a problem hiding this comment.
- won't this construct a string every time?
- move these strings into constants?
| @@ -211,6 +211,60 @@ namespace message2 { | |||
|
|
|||
| TextSelector(const Locale& l) : locale(l) {} | |||
| }; | |||
|
|
|||
| // See https://github.com/unicode-org/message-format-wg/blob/main/test/README.md | |||
There was a problem hiding this comment.
For API (though this is internal?) this should perhaps go to something more permanent in the ICU user guide?
| // TODO doc comments | ||
| // Encapsulates either a formatted string or formatted number; | ||
| // more output types could be added in the future. | ||
| // Returns a new FunctionOptions |
There was a problem hiding this comment.
Suggested change
| // Returns a new FunctionOptions | |
| /** | |
| * The original FunctionOptions isn't usable after this call. | |
| * @returns A new, merged FunctionOptions. | |
| */ |
is 'other' usable after this?
Comment on lines
+1468
to
+1473
| // 3i. The character . U+002E Full Stop. | ||
| result += u"."; | ||
| // 3ii. The single decimal digit character representing the value | ||
| // floor((abs(Input) - floor(abs(Input))) * 10) | ||
| int32_t val = floor((abs(input) - floor(abs(input)) * 10)); | ||
| result += digitToChar(val, status); |
There was a problem hiding this comment.
seems like it should be a utility function w/ other ICU number formatting. COuld be future.
| } | ||
|
|
||
| ResolvedSelector::ResolvedSelector(FormattedPlaceholder&& val) : value(std::move(val)) {} | ||
| // Options in `this` take precedence | ||
| // `this` can't be used after mergeOptions is called |
There was a problem hiding this comment.
i added a comment, this might be good to note in the function declaration
| } else { | ||
| formatSelectors(context, *globalEnv, status, result); | ||
| // Check for errors/warnings -- if so, then the result of pattern selection is the fallback value | ||
| // See https://github.com/unicode-org/message-format-wg/blob/main/spec/formatting.md#pattern-selection |
There was a problem hiding this comment.
should link to tr35 - maybe current spec
There was a problem hiding this comment.
Suggested change
| // See https://github.com/unicode-org/message-format-wg/blob/main/spec/formatting.md#pattern-selection | |
| // See https://www.unicode.org/reports/tr35/tr35-messageFormat.html#pattern-selection |
| @@ -465,8 +469,42 @@ static double parseNumberLiteral(const FormattedPlaceholder& input, UErrorCode& | |||
| return result; | |||
| } | |||
|
|
|||
| static UChar32 digitToChar(int32_t val, UErrorCode errorCode) { | |||
There was a problem hiding this comment.
should be merged with or use other utility function in ICU
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR does everything that's needed in order to make the pattern-selection spec tests work.
This includes implementing the test-only standard functions,
test:function,test:formatandtest:select, specified in the test README file in the spec.This was tricky because the select-only
test:selectfunction is used in a composable way in these tests, which isn't something that appeared in previous tests. Previously, the code assumed that a named result of a selector could be used in a.match, but not used as an operand to another function.So in this PR, I implement a limited form of what will eventually (see draft PR #3228, which is still in design review) be a more coherent mechanism for function composition. I implemented just enough of it to make these tests work.
I added an
InternalValuetype used in the formatter, which is distinct from theFormattedPlaceholdertype that function handlers return. This makes it easier to guarantee that fallback (error) values are not passed to functions. AnInternalValueincludes a function name, operand, and options. Since at the time when a resolved value is named, it's unknown whether it will be used for formatting or selection,InternalValuehas both a formatting and a selection method, which can be called when the resolved value is eventually consumed by either a.matchor a pattern.I had to modify the
depstest.pyscript for the same reasons as before; now, code that manipulates variants is also found inmessageformat2_evaluation.cpp, but as before, it's never used in a way that could throw an exception.I'll be happy to explain more and/or add comments to the code as needed.
Checklist