Add Power Query (.pq) language support

[mattmasson]

Adds Power Query language support (.pq and .pqm)

Description

The Power Query / M formula language is a DSL for the Power Query Editor experience in Microsoft's Power BI, Excel, and SQL Server Analysis Services, and other products. This change adds highlighting support for .pq and .pqm files.

The PQ language was previously submitted in August 2017 (#3802) and not accepted (Pending Popularity). At the time the search results came back with ~75 URLs; we now have over 700.

• Power Query M syntax reference
• MSDN Language Reference

Checklist:

• I am adding a new language.
  • The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • https://github.com/search?p=1&q=extension%3Apq+NOT+TOPSRC+NOT+%22%21include%22+NOT+%22%40include%22+NOT+gnuplot+NOT+%22%40define%22&type=Code
      • https://github.com/search?q=extension:pqm+NOT+nothack&type=Code
      • (github files using our original .m file extension) https://github.com/search?q=extension%3Am+Source+Table+let+in+NOT+%22%23import%22+NOT+%22func%22+NOT+%22%23include%22+created%3A%3E2015-01-01+NOT+tableView+-language%3AMatlab+-language%3AM+-language%3AMathematica&type=Code
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • Table.ChangeType.pqm
      • TripPin.pq
      • TripPin.query.pq
    • Sample license(s):
      • MIT
  • I have included a syntax highlighting grammar.

[lildude]

Looks like .pqm is still not popular enough so can’t be included in this PR. .pq might just be (I’ll check later) however you’re going to need to add a heuristic as this extension is not unique to PowerQuery as demonstrated by your complex search.

[pchaigno]

Looks like .pqm is still not popular enough so can’t be included in this PR.

👍

need to add a heuristic as this extension is not unique to PowerQuery as demonstrated by your complex search.

It could be sufficient to add Makefile.pq as a filename entry. Though I'm not sure those are exactly Makefile files (syntax seems a bit different?).

@mattmasson Also, any idea what these files are?

[lildude]

Looks like .pq isn't quite popular enough yet either:

Total files found: 648
Unique public user/repos: 137
Unique owners: 103

[mattmasson]

Yes, Makefile.pq seems to be common name for a specific kind of makefile. There also seems to be some .pq files that are gnuplot oriented.

Here is a better search that pulls in both .pqm and .pq files. Power Query code will always contain either "section" or "let" statements.

https://github.com/search?p=52&q=extension%3Apq+extension%3Apqm+-filename%3AMakefile.pq+section+OR+let&type=Code

@lildude, what is the minimum counts we should have before resubmitting?

@pchaigno, what else can I change in linguist to make sure the Makefile.pq and gnuplot files aren't misclassified?

[lildude]

@lildude, what is the minimum counts we should have before resubmitting?

We don't have a specific count beyond "In most cases we prefer that extensions be in use in hundreds of repositories before supporting them in Linguist." as stated in the CONTRIBUTING.md.

Technically speaking, "hundreds" would signal at least 200, else it'll only be "hundred" 😉, though that's probably still on the low side as it's barely into the "hundreds" range. It also depends on how well spread out across users this is; 1000 unique repos for 2 users is not widespread usage.

The biggest limitation to us specifying precise figures is there's no easy way for peeps to quantify usage as this isn't exposed within the search results and querying the API to gain this info hits rate and abuse limits. I'm hoping that when code search comes to GraphQL, we can write an app or script to query the results without hitting rate limiting thus getting us out of this situation.

[pchaigno]

Closing in favor of #4219. We'll monitor the popularity of the extensions there.

If you have improved search queries compared to those I used in #4219, please post them here and I'll update #4219.

[pchaigno]

The .pq file extension is now popular enough for inclusion (see #4219). @mattmasson Do you want me to reopen this pull request or would you prefer to resubmit as a new pull request?

[wanderleihuttel]

Hello Guys
Please include this pull request to sintax highlight for Power Query M.

[jimbrig]

any plans to add language references for *.pq or *.m extensions?

[lildude]

any plans to add language references for *.pq or *.m extensions?

There are never any plans to add a language as contributions are entirely community driven. Submit a PR and we're likely to add support for that language or extensions provided the extensions meet the minimum usage requirements.

[jimbrig]

thanks @lildude - I will have to decide if its worth the time and effort or not then I guess!