Retrieving Reserved Keywords of the Ballerina Language

[rasika]

Is there anyway to expose LexerTerminals defined in https://github.com/ballerina-platform/ballerina-lang/blob/master/compiler/ballerina-parser/src/main/java/io/ballerina/compiler/internal/parser/LexerTerminals.java ?

This is required for many langserver features such completions, code-actions to get reserved keywords of the Ballerina language.

Key requirement is there's no way to skip suggesting reserved keywords as variables-names. not only for the variable names, when suggesting auto-imports we need to escape module-names that has reserved keywords. Even I believe keywords should be transparent for the completion suggestions as well. So that we don't need duplicate and maintain 'em in langserver level.

looping @sameerajayasoma @Kishanthan @hasithaa @lochana-chathura @rdulmina

[nadeeshaan]

Currently, we have a major blocker when it comes to the code actions where we generate variable names with the keywords. It is mandatory for the language server to get the keywords and also, as @rasika mentioned, we are keeping a list of preserved keywords (not all) in language server for snippet generation as well. We need to prioritise this

[lochana-chathura]

Another requirement from the LS side could be to reuse the identifier matching logic used in the parser. Refer to the comment #28798 (comment) by @rasika.

We can do the following from the parser's side to address this. I'm not very sure exposing this way is a good idea.

• We do the identifier matching here in the lexer.

  ballerina-lang/compiler/ballerina-parser/src/main/java/io/ballerina/compiler/internal/parser/BallerinaLexer.java

  Line 783 in 3171885

  private STToken processIdentifierOrKeyword(int startChar) {

• It is not merely an isIdentifier(String txt) method. We read the input sequence char by char using the attached CharReader to capture the identifier. So we cannot expose as it is.

• But, maybe we can simply write a method to boot up a parser and try to parse an identifier from a given string and then return whether it is valid.

• Also, currently, we don't have a place to expose such API.

[lochana-chathura]

I am +1 to have some space to expose API. I really have not much idea about what could be the negative consequences of doing so.

[rasika]

@sameerajayasoma @Kishanthan @hasithaa Can we get a SyntaxFacts implementation through the Compiler APIs for isKeyword(string text) and isIdentifier(string text) ? Seems like the implementation is straightforward.

[hasithaa]

@rasika, Even though isKeyword implementation is simple, I think that is not the correct way to handle this use case. Ballerina keywords are contextual, for example, where is not a keyword in every context.

So I would suggest keeping all possible keywords for now on LS side, until we figure out the proper API for this.

[rasika]

@hasithaa there's nothing wrong in calculating where as a permanent keyword. I think we can forget being contextual for the first phase. WDYT?

[hasithaa]

That is correct, But we can't add APIs randomly. We need to design this properly.

[lochana-chathura]

For anyone going through this discussion, this requirement was addressed with SyntaxInfo API via PR #32806