jacoco-report

A Github action that publishes the JaCoCo report as a comment in the Pull Request with customizable pass percentage for modified modules, files and the overall project. You can view the coverage of just the changed files in your pull request.

Usage

Pre-requisites

Create a workflow .yml file in your repositories .github/workflows directory. An example workflow is available below. For more information, reference the GitHub Help Documentation for Creating a workflow file.

Inputs

• paths - [required] Comma separated paths of the generated jacoco xml files (supports wildcard glob pattern)
• token - [required] Github personal token to add comments to Pull Request
• min-coverage-overall - [optional {default: 80%}] The minimum code coverage that is required to pass for overall project
• min-coverage-changed-files - [optional {default: 80%}] The minimum code coverage that is required to pass for changed files
• update-comment - [optional {default: false}] If true, updates the previous coverage report comment instead of creating new one. Requires title to work properly
• title - [optional] Title for the Pull Request comment
• skip-if-no-changes - [optional {default: false}] If true, comment won't be added if there is no coverage information present for the files changed
• pass-emoji - [optional {default: 🍏}] Emoji to use for pass status shown when 'coverage >= min coverage' (should be a Github supported emoji).
• fail-emoji - [optional {default: ❌}] Emoji to use for fail status shown when 'coverage < min coverage' (should be a Github supported emoji).
• continue-on-error - [optional {default: true}] If true, then do not fail the action on error, but log a warning
• debug-mode - [optional {default: false}] If true, run the action in debug mode and get debug logs printed in console

Outputs

• coverage-overall - The overall coverage of the project
• coverage-changed-files - The total coverage of all changed files

Example Workflow

name: Measure coverage

on:
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Run Coverage
        run: |
          chmod +x gradlew
          ./gradlew testCoverage

      - name: Add coverage to PR
        id: jacoco
        uses: madrapps/[email protected]
        with:
          paths: |
            ${{ github.workspace }}/**/build/reports/jacoco/prodNormalDebugCoverage/prodNormalDebugCoverage.xml,
            ${{ github.workspace }}/**/build/reports/jacoco/**/debugCoverage.xml
          token: ${{ secrets.GITHUB_TOKEN }}
          min-coverage-overall: 40
          min-coverage-changed-files: 60

Understanding the Coverage Report

• The "delta" (the negative value next to the coverage) represents the percentage of newly added or modified lines of code that are not covered by unit tests. It calculates the difference in test coverage based solely on the changes made in the current commit or pull request. For example, if 10 lines of code are modified or added, and 8 of those lines are covered by unit tests, the "delta" would be -20%, indicating 20% of the newly changed code is untested. However, the "delta" has limitations. It can never be positive, meaning if you add more unit tests to cover existing, unmodified code, this additional coverage is not reflected in the delta. The metric only considers lines directly changed in the current set of modifications, as there is no mechanism to track improvements in coverage for pre-existing code that hasn't been altered.

Example Project

For a working project refer to jacoco-playgound. Check out the PR's in the project to get an idea on how the report is shown on a pull request comment. For multi module gradle project, refer jacoco-android-playground

Example Cases

1. If you want to fail your workflow when the minimum coverage is not met

   You can write an additional step that uses the Outputs for the jacoco-report action and fail the workflow. Refer sample pull request and its workflow

   - name: Fail PR if overall coverage is less than 80%
     if: ${{ steps.jacoco.outputs.coverage-overall < 80.0 }}
     uses: actions/github-script@v6
     with:
       script: |
         core.setFailed('Overall coverage is less than 80%!')

2. If you don't want to add the coverage comment everytime you push a commit to a pull request, but update the existing coverage comment instead

   Set the update-comment input to true and also set a title input. Refer sample pull request and its workflow

   - name: Jacoco Report to PR
     id: jacoco
     uses: madrapps/[email protected]
     with:
       paths: ${{ github.workspace }}/build/reports/jacoco/testCoverage/testCoverage.xml
       token: ${{ secrets.GITHUB_TOKEN }}
       min-coverage-overall: 40
       min-coverage-changed-files: 60
       title: Code Coverage
       update-comment: true

3. If you have a multi-module project like android, with multiple modules each with its own jacoco report

   Set the paths input with wildcard glob pattern (as shown in the Example workflow). This will pick all the files matching the pattern. Ensure your pattern matches only one report per module, since for the same module, you could have debugCoverage.xml and releaseCoverage.xml. Refer sample pull request and its workflow

   - name: Jacoco Report to PR
     id: jacoco
     uses: madrapps/[email protected]
     with:
       paths: |
         ${{ github.workspace }}/**/build/reports/jacoco/**/prodNormalDebugCoverage.xml,
         ${{ github.workspace }}/**/build/reports/jacoco/**/debugCoverage.xml
       token: ${{ secrets.GITHUB_TOKEN }}
       min-coverage-overall: 40
       min-coverage-changed-files: 60

4. When you need to customize the pass/fail emojis or the style of title in the comment

   Set the pass-emoji and fail-emoji to a valid emoji supported in Github. You can add # to choose the style of title from H1 to H6. If you don't choose, the default is H3. Refer sample pull request and its workflow

   - name: Jacoco Report to PR
     id: jacoco
     uses: madrapps/[email protected]
     with:
       paths: ${{ github.workspace }}/build/reports/jacoco/testCoverage/testCoverage.xml
       token: ${{ secrets.GITHUB_TOKEN }}
       min-coverage-overall: 40
       min-coverage-changed-files: 60
       title: '# :lobster: Coverage Report'
       pass-emoji: ':green_circle:'
       fail-emoji: ':red_circle:'

Troubleshooting

1. If the PR is created by bots like dependabot, then the GITHUB_TOKEN won't have sufficient access to write the coverage comment. So add the appropriate permission to your job (as shown in the Example workflow). More information here.

Contributing

We welcome contributions, and if you're interested, have a look at the CONTRIBUTING document.

License

The scripts and documentation in this project are released under the MIT License