Skip to content

Commit

Permalink
c++/c# workflow file bug, kotlin/typescript support, and readme updat…
Browse files Browse the repository at this point in the history
…es (#109)
  • Loading branch information
joshjohanning authored Mar 24, 2023
1 parent bb0d086 commit 06844a1
Show file tree
Hide file tree
Showing 11 changed files with 58 additions and 34 deletions.
4 changes: 2 additions & 2 deletions .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ GITHUB_API_TOKEN=
GITHUB_ENTERPRISE=
GITHUB_ORG=

# If you are filtering by language, set the language here. Please use either: javascript, go, python, ruby, java
# If you are filtering by language, set the language here. Please use either: javascript, typescript, go, python, ruby, c#, c++, java, or kotlin
LANGUAGE_TO_CHECK=

# Specify what to enable. The default will be codescanning. An example is: E.G ENABLE_ON=codescanning,secretscanning,dependabot,dependabotupdates,pushprotection
# Specify what to enable. The default will be codescanning. An example is: ENABLE_ON=codescanning,secretscanning,dependabot,dependabotupdates,pushprotection,actions
ENABLE_ON=secretscanning

# Create Issue on Repository
Expand Down
64 changes: 33 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ There are two main actions this tool does:

**Part One:**

Goes and collects repositories that will have Code Scanning (CodeQL)/Secret Scanning/Dependabot Alerts/Dependabot Security Updates enabled. There are three main ways these repositories are collected.
Goes and collects repositories that will have Code Scanning (CodeQL)/Secret Scanning/Secret Scanning Push Protection/Dependabot Alerts/Dependabot Security Updates/Actions enabled. There are three main ways these repositories are collected.

- Collect the repositories where the primary language matches a specific value. For example, if you provide JavaScript, all repositories will be collected where the primary language is, Javascript.
- Collect the repositories to which a user has administrative access, or a GitHub App has access.
Expand All @@ -25,27 +25,26 @@ If you select option 1, the script will return all repositories in the language

**Part Two:**

Loops over the repositories found within the `repos.json` file and enables Code Scanning(CodeQL)/Secret Scanning/Dependabot Alerts/Dependabot Security Updates/Secret Scanning Push Protection.

If you pick Code Scanning:

- Loops over the repositories found within the `repos.json` file. A pull request gets created on that repository with the `codeql-analysis-${language}.yml` found in the `bin/workflows` directory. The `${language}` will be replaced at runtime with the primary language of the repository. For convenience, all pull requests made will be stored within the `prs.txt` file, where you can see and manually review the pull requests after the script has run.

If you pick Secret Scanning:

- Loops over the repositories found within the `repos.json` file. Secret Scanning is then enabled on these repositories.

If you pick Dependabot Alerts:

- Loops over the repositories found within the `repos.json` file. Dependabot Alerts is then enabled on these repositories.

If you pick Dependabot Security Updates:

- Loops over the repositories found within the `repos.json` file. Dependabot Security Updates is then enabled on these repositories.

If you pick Actions:

- Loops over the repositories found within the `repos.json` file. Actions is enabled on these repositories. This is useful if you want to ensure that the Code Scanning workflow can run and Actions isn't disabled.
Loops over the repositories found within the `repos.json` file and enables Code Scanning(CodeQL)/Secret Scanning/Secret Scanning Push Protection/Dependabot Alerts/Dependabot Security Updates.

- If you pick Code Scanning:
- Loops over the repositories found within the `repos.json` file. A pull request gets created on that repository with the `codeql-analysis-${language}.yml` found in the `bin/workflows` directory.
- The `${language}` will be replaced at runtime with the primary language of the repository.
- For convenience, all pull requests made will be stored within the `prs.txt` file, where you can see and manually review the pull requests after the script has run.
- If you pick Secret Scanning:
- Loops over the repositories found within the `repos.json` file. Secret Scanning is then enabled on these repositories.
- If you pick Push Protections:
- Loops over the repositories found within the `repos.json` file. Secret Scanning Push Protection is then enabled on these repositories.
- If you pick Dependabot Alerts:
- Loops over the repositories found within the `repos.json` file. Dependabot Alerts is then enabled on these repositories.
- If you pick Dependabot Security Updates:
- Loops over the repositories found within the `repos.json` file. Dependabot Security Updates is then enabled on these repositories.
- If you pick Actions:
- Loops over the repositories found within the `repos.json` file. Actions is then enabled on these repositories.
- This is useful if you want to ensure that the Code Scanning workflow can run and Actions isn't disabled.
- If you pick Create Issue:
- Loops over the repositories found within the `repos.json` file. An issue will be created with the [following text](./src/utils/text/issueText.ts).
- This alerts repository maintainers that a pull request for CodeQL was created, along with other helpful resources.

## Prerequisites

Expand Down Expand Up @@ -86,11 +85,11 @@ If you pick Actions:

6. Update the `GITHUB_ORG` value found within the `.env`. Remove the `XXXX` and replace that with the name of the GitHub Organisation you would like to use as part of this script. **NOTE**: If you are running this across multiple organisations within an enterprise, you can not set the `GITHUB_ORG` variable and instead set the `GITHUB_ENTERPRISE` one with the name of the enterprise. You can then run `yarn run getOrgs`, which will collect all the organisations dynamically. This will mean you don't have to hardcode one. However, for most use cases, simply hardcoding the specific org within the `GITHUB_ORG` variable where you would like this script to run will work.
7. Update the `LANGUAGE_TO_CHECK` value found within the `.env`. Remove the `XXXX` and replace that with the language you would like to use as a filter when collecting repositories. **Note**: Please make sure these are lowercase values, such as: `javascript`, `python`, `go`, `ruby`, etc.
7. Update the `LANGUAGE_TO_CHECK` value found within the `.env`. Remove the `XXXX` and replace that with the language you would like to use as a filter when collecting repositories. **Note**: Please make sure these are lowercase values, such as: `javascript`, `typescript`, `python`, `go`, `ruby`, `c#`, `c++`, `java`, or `kotlin`
8. Decide what you want to enable. Update the `ENABLE_ON` value to choose what you want to enable on the repositories found within the `repos.json`. This can be one or multiple values. If you are enabling just code scanning (CodeQL) you will need to set `ENABLE_ON=codescanning`, if you are enabling everything, you will need to set `ENABLE_ON=codescanning,secretscanning,pushprotection,dependabot,dependabotupdates,actions`. You can pick one, two or three. The format is a comma-seperated list.
9. **OPTIONAL**: Update the `CREATE_ISSUE` value to `true/false` depending on if you would like to create an issue explaining the purpose of the PR. We recommend this, as it will help explain why the PR was created; and give some context. However, this is optional. The text which is in the issue can be modified and found here: `./src/utils/text/`.
9. **OPTIONAL**: Update the `CREATE_ISSUE` value to `true/false` depending on if you would like to create an issue explaining the purpose of the PR. We recommend this, as it will help explain why the PR was created; and give some context. However, this is optional. The text which is in the issue can be modified and found [here](./src/utils/text/issueText.ts): `./src/utils/text/`.
10. **OPTIONAL**: If you are a GHES customer, then you will need to set the `GHES` env to `true` and then set `GHES_SERVER_BASE_URL` to the URL of your GHES instance. E.G `https://octodemo.com`.
Expand All @@ -111,20 +110,22 @@ The first step is collecting the repositories you would like to run this script
**OPTION 1** (Preferred)
```bash
yarn run getRepos // In the `.env` set the `LANGUAGE_TO_CHECK=` to the language. E.G `python`, `javascript`, `go`, etc.
# In the `.env` set the `LANGUAGE_TO_CHECK=` to the language. E.G.: `javascript`, `typescript`, `python`, `go`, `ruby`, `c#`, `c++`, `java`, or `kotlin`
yarn run getRepos # or npm run getRepos
```
When using GitHub Actions, we commonly find (especially for non-build languages such as JavaScript) that the `codeql-analysis.yml` file is repeatable and consistent across multiple repositories of the same language. About 80% of the time, teams can reuse the same workflow files for the same language. For Java, C++ that number drops down to about 60% of the time. But the reason why we recommend enabling Code Scanning at bulk via language is the `codeql-analysis.yml` file you propose within the pull request has the highest chance of being most accurate. Even if the file needs changing, the team reviewing the pull request would likely only need to make small changes. We recommend you run this command first to get a list of repositories to enable Code Scanning. After running the command, you are welcome to modify this file. Just make sure it's a valid JSON file if you do edit.
When using GitHub Actions, we commonly find (especially for non-build languages such as JavaScript) that the `codeql-analysis.yml` file is repeatable and consistent across multiple repositories of the same language. About 80% of the time, teams can reuse the same workflow files for the same language. For Java, C++ that number drops down to about 60% of the time. But the reason why we recommend enabling Code Scanning at bulk via language is the `codeql-analysis.yml` file you propose within the pull request has the highest chance of being most accurate. Even if the file needs changing, the team reviewing the pull request would likely only need to make small changes. We recommend you run this command first to get a list of repositories to enable Code Scanning. After running the command, you are welcome to modify the `./bin/repos.json` file. Just make sure it's a valid JSON file before saving.

This script only returns repositories where CodeQL results have not already been uploaded to code scanning. If any CodeQL results have been uploaded to a repositories code scanning feature, that repository will not be returned to this list. The motivation behind this is not to raise pull requests on repositories where CodeQL has already been enabled.

**OPTION 2**

```bash
yarn run getRepos // or npm run getRepos
# In the `.env` leave the `LANGUAGE_TO_CHECK=` empty to pull in all repos
yarn run getRepos # or npm run getRepos
```

Similar to step one, another automated approach is to enable by user access. This approach will be a little less accurate as the file will most certainly need changing between a Python project and a Java project (if you are enabling CodeQL), and the user's PAT you are using will most likely. But the file you propose is going to be a good start. After running the command, you are welcome to modify this file. Just make sure it's a valid JSON file if you do edit.
Similar to step one, another automated approach is to enable by user access (i.e., enable for all repositories the user/PAT has administrative access to). This approach will be a little less accurate as the `codeql-analysis.yml` file will most certainly need changing between a Python project and a Java project (if you are enabling CodeQL). But the file you propose is going to be a good start. After running the command, you are welcome to modify the `./bin/repos.json` file. Just make sure it's a valid JSON file before saving.
This script only returns repositories where CodeQL results have not already been uploaded to code scanning. If any CodeQL results have been uploaded to a repositories code scanning feature, that repository will not be returned to this list. The motivation behind this is not to raise pull requests on repositories where CodeQL has already been enabled.
Expand All @@ -145,14 +146,15 @@ Create a file called `repos.json` within the `./bin/` directory. This file needs
"enableDependabotUpdates": "boolean",
"enablePushProtection": "boolean",
"enableSecretScanning": "boolean",
"enableActions": "boolean",
"repo": "string <org/repo>",
}
]
}
]
```
As you can see, the object takes a number of boolean keys: `createIssue`, `enableCodeScanning`, `enableDependabot`, `enableDependabotUpdates`, `enablePushProtection`, and `enableSecretScanning`, along with a single string key, namely, `repo`. Set `repo` to the name of the repository name where you would like to run this script. Set `enableDependabot` to `true` if you would also like to enable Dependabot Alerts on that repo; set it to `false` if you do not want to enable Dependabot Alerts. The same goes for `enableDependabotUpdates` for Dependabot Security Updates, `enableSecretScanning` for Secret Scanning, `pushprotection` for Secret Scanning push protection, and `enableCodeScanning` for Code Scanning (CodeQL). Finally set `createIssue` to `true` if you would like to create an issue on the repository with the text found in the `./src/utils/text/issueText.ts` file to supplement the PR.
As you can see, the object takes a number of boolean keys: `createIssue`, `enableCodeScanning`, `enableDependabot`, `enableDependabotUpdates`, `enablePushProtection`, `enableSecretScanning`, and `enableActions` along with a single string key, namely, `repo`. Set `repo` to the name of the repository name where you would like to run this script. Set `enableDependabot` to `true` if you would also like to enable Dependabot Alerts on that repo; set it to `false` if you do not want to enable Dependabot Alerts. The same goes for `enableDependabotUpdates` for Dependabot Security Updates, `enableSecretScanning` for Secret Scanning, `pushprotection` for Secret Scanning push protection, `enableCodeScanning` for Code Scanning (CodeQL), and `enableActions` to enable Actions. Finally set `createIssue` to `true` if you would like to create an issue on the repository with the text found in the `./src/utils/text/issueText.ts` [file](./src/utils/text/issueText.ts) to supplement the PR.
**NOTE:** The account that generated the PAT needs to have `write` access or higher over any repository that you include within the `repos` key.
Expand Down Expand Up @@ -206,7 +208,7 @@ env:
APP_CLIENT_ID: ${{ secrets.GHAS_ENABLEMENT_APP_CLIENT_ID }}
APP_CLIENT_SECRET: ${{ secrets.GHAS_ENABLEMENT_APP_CLIENT_SECRET }}
APP_PRIVATE_KEY: ${{ secrets.GHAS_ENABLEMENT_APP_PRIVATE_KEY }}
ENABLE_ON: "codescanning,secretscanning,pushprotection,dependabot,dependabotupdates"
ENABLE_ON: "codescanning,secretscanning,pushprotection,dependabot,dependabotupdates,actions"
DEBUG: "ghas:*"
CREATE_ISSUE: "false"
GHES: "false"
Expand All @@ -218,7 +220,7 @@ jobs:
enable-security-javascript:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
repository: NickLiffen/ghas-enablement
- name: Get dependencies and configure
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["cpp"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-csharp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["csharp"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["go"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["java"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-javascript.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["javascript"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["python"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
2 changes: 2 additions & 0 deletions bin/workflows/codeql-analysis-ruby.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ jobs:
matrix:
language: ["ruby"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
Expand Down
4 changes: 3 additions & 1 deletion src/utils/commitFile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import { execFile as ImportedExec } from "child_process";

import { response, commands } from "../../types/common";

import { getcodeQLLanguage } from "./getcodeQLLanguage";

const execFile = util.promisify(ImportedExec);

inform(`Platform detected: ${platform}`);
Expand Down Expand Up @@ -44,7 +46,7 @@ export const commitFileMac = async (
const {
env: { LANGUAGE_TO_CHECK: language },
} = process;
let codeQLLanguage = language;
let codeQLLanguage = getcodeQLLanguage(language || "");
if (!codeQLLanguage && primaryLanguage != "no-language") {
codeQLLanguage = primaryLanguage;
}
Expand Down
6 changes: 6 additions & 0 deletions src/utils/getcodeQLLanguage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@ export const getcodeQLLanguage = (primaryLanguage: string): string => {
case "javascript":
codeQLLang = "javascript";
break;
case "typescript":
codeQLLang = "javascript"; // use javascript for typescript
break;
case "java":
codeQLLang = "java";
break;
case "kotlin":
codeQLLang = "java"; // use java for kotlin
break;
case "go":
codeQLLang = "go";
break;
Expand Down

0 comments on commit 06844a1

Please sign in to comment.