add landing page and rules website (#2310)

* web: index: add gif of capa running

* index: add screencast of running capa

produced via:

```
asciinema capa.cast
./capa Practical\ Malware\ Analysis\ Lab\ 01-01.dll_
<ctrl-d>
agg --no-loop --theme solarized-light capa.cast capa.gif
```

* web: index: start to sketch out style

* web: landing page

* web: merge rules website

* web: rules: update bootstrap and integrate rules

* web: rules: use pygments to syntax highlight rules

Use the Pygments syntax-highlighting library to parse
and render the YAML rule content. This way we don't have
to manually traverse the rule nodes and emit lists; instead,
we rely on the fact that YAML is pretty easy for humans
to read and let them consume it directly, with some text 
formatting to help hint at the types/structure.

* web: rules: use capa to load rule content

capa (the library) has routines for deserializing the YAML
content into structured objects, which means we can use tools
like mypy to find bugs. So, prefer to use those routines instead
of parsing YAML ourselves.

* web: rules: linters

Run and fix the issues identified by the following linters:

  - isort
  - black
  - ruff
  - mypy

* web: rules: add some links to rule page

Add links to the following external resources:

  - GitHub rule source in capa-rules repo
  - VirusTotal search for matching samples

* web: rules: accept ?q= parameter for initial search

Update the rules landing page to accept a HTTP
query parameter named "q" that specifies an initial 
search term to to pass to pagefind. This enables
external pages link to rule searches.

* web: rules: add link to namespace search

* web: rules: use consistent header

Import header from root capa landing page.

* web: rules: add umami script

* web: add initial whats new section, TODOs

* web: rules: remove old images

* changelog

* CI: remove temporary branch push event triggers

* Delete web/rules/public/css/bootstrap-4.5.2.min.css

* Delete web/rules/public/js/bootstrap-4.5.2.min.js

* Delete web/public/img/capa.cast

* Rename readme.md to README.md

* web: rules: add scripts to pre-commit configs

* web: rules: add scripts to pre-commit configs

* lints

* ci: add temporary branch push trigger to get incremental builds

* web: rules: assert start_dir must exist

* ci: web: rules: deep checkout so we can get rule history

* web: rules: check output of subprocess

* web: rules: factor out common CSS

* web: rules: fix header links

* web: rules: only index rule content, not surrounding text

* ci: web: remote temporary branch push trigger

.github/workflows/web-deploy.yml

@@ -2,7 +2,7 @@ name: deploy web to GitHub Pages
 
 on:
   push:
-    branches: [ master, "wb/webui-actions-1" ]
+    branches: [ master ]
     paths:
       - 'web/**'
 
@@ -22,6 +22,7 @@ concurrency:
 
 jobs:
   build-landing-page:
+    name: Build landing page
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
@@ -32,6 +33,7 @@ jobs:
           path: './web/public'
 
   build-explorer:
+    name: Build capa explorer web
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
@@ -63,12 +65,51 @@ jobs:
           name: explorer
           path: './web/explorer/dist'
 
+  build-rules:
+    name: Build rules site
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the repository
+        uses: actions/checkout@v4
+        with:
+          submodules: 'recursive'
+          # full depth so that capa-rules has a full history
+          # and we can construct a timeline of rule updates.
+          fetch-depth: 0
+      - name: Set up Python
+        uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
+        with:
+          python-version: '3.12'
+      - uses: extractions/setup-just@v2
+      - name: Install pagefind
+        uses: supplypike/setup-bin@v4
+        with:
+          uri: "https://github.com/CloudCannon/pagefind/releases/download/v1.1.0/pagefind-v1.1.0-x86_64-unknown-linux-musl.tar.gz"
+          name: "pagefind"
+          version: "1.1.0"
+      - name: Install dependencies
+        working-directory: ./web/rules
+        run: pip install -r requirements.txt
+      - name: Build the website
+        working-directory: ./web/rules
+        run: just build
+      - name: Index the website
+        working-directory: ./web/rules
+        run: pagefind --site "public"
+      # upload the build website to artifacts
+      # so that we can download and inspect, if desired.
+      - uses: actions/upload-artifact@v4
+        with:
+          name: rules
+          path: './web/rules/public'
+
   deploy:
+    name: Deploy site to GitHub Pages
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
     runs-on: ubuntu-latest
-    needs: [build-landing-page, build-explorer]
+    needs: [build-landing-page, build-explorer, build-rules]
     steps:
       - uses: actions/download-artifact@v4
         with:
@@ -78,6 +119,10 @@ jobs:
         with:
           name: explorer
           path: './public/explorer'
+      - uses: actions/download-artifact@v4
+        with:
+          name: rules
+          path: './public/rules'
       - name: Setup Pages
         uses: actions/configure-pages@v4
       - name: Upload artifact

.gitignore

@@ -126,3 +126,4 @@ Pipfile.lock
 .github/binja/binaryninja
 .github/binja/download_headless.py
 .github/binja/BinaryNinja-headless.zip
+justfile

.pre-commit-config.yaml

@@ -38,6 +38,7 @@ repos:
         -   "capa/"
         -   "scripts/"
         -   "tests/"
+        -   "web/rules/scripts/"
         always_run: true
         pass_filenames: false
 
@@ -55,6 +56,7 @@ repos:
         -   "capa/"
         -   "scripts/"
         -   "tests/"
+        -   "web/rules/scripts/"
         always_run: true
         pass_filenames: false
 
@@ -72,6 +74,7 @@ repos:
         -   "capa/"
         -   "scripts/"
         -   "tests/"
+        -   "web/rules/scripts/"
         always_run: true
         pass_filenames: false
 
@@ -90,6 +93,7 @@ repos:
         -   "capa/"
         -   "scripts/"
         -   "tests/"
+        -   "web/rules/scripts/"
         always_run: true
         pass_filenames: false
 
@@ -107,6 +111,7 @@ repos:
         -   "capa/"
         -   "scripts/"
         -   "tests/"
+        -   "web/rules/scripts/"
         always_run: true
         pass_filenames: false
 

CHANGELOG.md

@@ -4,6 +4,9 @@
 
 ### New Features
 
+- add landing page https://mandiant.github.io/capa/ @williballenthin #2310
+- add rules website https://mandiant.github.io/capa/rules @DeeyaSingh #2310
+
 ### Breaking Changes
 
 ### New Rules (0)

pyproject.toml

@@ -172,7 +172,8 @@ scripts = [
 [tool.deptry]
 extend_exclude = [
     "sigs",
-    "tests"
+    "tests",
+    "web",
 ]
 
 # dependencies marked as first party, to inform deptry that they are local

scripts/capa2sarif.py

@@ -84,8 +84,7 @@ def main() -> int:
     args = _parse_args()
 
     try:
-        with Path(args.capa_output).open() as capa_output:
-            json_data = json.load(capa_output)
+        json_data = json.loads(Path(args.capa_output).read_text(encoding="utf-8"))
     except ValueError:
         logger.error("Input data was not valid JSON, input should be a capa json output file.")
         return -1

web/public/.gitignore

@@ -0,0 +1 @@
+rules/