README.md · Secure CI/CD Pipeline

84 lines · markdown

# Secure CI/CD Pipeline
 
A GitHub Actions pipeline that gates every push and pull request on four security
checks before code is allowed to merge, then reports the run back to a SOC for
visibility. The sample app is a small Flask task API; the point of the repo is the
pipeline around it.
 
The checks run as separate jobs so a failure tells you exactly which gate tripped
instead of one long log you have to scroll through.
 
```mermaid
flowchart TD
    push[push / pull_request] --> lint[ruff lint]
    lint --> sast[Semgrep SAST]
    lint --> secrets[gitleaks secret scan]
    lint --> deps[pip-audit dependencies]
    sast --> test[pytest + coverage]
    secrets --> test
    deps --> test
    test --> notify[notify SOC webhook]
```
 
## The gates
 
| Job | Tool | What it stops |
|-----|------|---------------|
| `lint` | ruff | style + a security rule set (`S`) on top of the usual lint |
| `sast` | Semgrep | the OWASP/Flask rule packs plus four custom rules in `.semgrep/rules.yml` |
| `secrets` | gitleaks | committed credentials, full history scanned on PRs |
| `dependencies` | pip-audit | known-vulnerable pinned dependencies |
| `test` | pytest | regressions, with coverage reported |
| `notify-soc` | `scripts/notify_soc.py` | nothing - it posts the run outcome to the SOC |
 
`lint` runs first as a cheap fail-fast. The three security scans fan out in
parallel, `test` waits on all of them, and the SOC notification runs last with
`if: always()` so the SOC hears about failures too, not just green runs.
 
## Custom Semgrep rules
 
The packs catch the common cases; `.semgrep/rules.yml` adds the ones I kept seeing
slip through:
 
- Flask started with `debug=True` (Werkzeug debugger RCE)
- `subprocess` with `shell=True` on a non-literal argument
- `jwt.decode` with signature verification disabled
- binding to `0.0.0.0` without an explicit reason
 
The four rules running against a deliberately vulnerable file - every finding is
Blocking, so the SAST job fails the pipeline before the code can merge:
 
![Semgrep custom rules](docs/screenshots/01-semgrep-sast.png)
 
## SARIF and the Security tab
 
Semgrep emits SARIF that gets uploaded with `github/codeql-action/upload-sarif`, so
findings show up under the repo's Security tab and as inline PR annotations rather
than only in the job log.
 
The dependency gate is `pip-audit`, which fails the build on a pinned package with a
known advisory:
 
![pip-audit dependency scan](docs/screenshots/02-pip-audit-deps.png)
 
## Run it locally
 
```bash
make install
make all
```
 
`make all` runs the same gates the pipeline does. You need `semgrep` and `gitleaks`
on your PATH for the `sast` and `secrets` targets; everything else is pip-installed
by `make install`.
 
## SOC integration
 
The last job posts the run outcome - repo, commit, actor, status, and a link back to
the run - to a Shuffle webhook. In my lab that webhook feeds the
[SOC automation lab](../soc-automation-lab), so a failed security gate opens a case
in TheHive the same way a Wazuh alert does. Set the `SHUFFLE_WEBHOOK_URL` repository
secret to wire it up; without it the job no-ops instead of failing.
 
More detail in [docs/pipeline.md](docs/pipeline.md) and
[docs/soc-integration.md](docs/soc-integration.md).

1	# Secure CI/CD Pipeline
2
3	A GitHub Actions pipeline that gates every push and pull request on four security
4	checks before code is allowed to merge, then reports the run back to a SOC for
5	visibility. The sample app is a small Flask task API; the point of the repo is the
6	pipeline around it.
7
8	The checks run as separate jobs so a failure tells you exactly which gate tripped
9	instead of one long log you have to scroll through.
10
11	```mermaid
12	flowchart TD
13	push[push / pull_request] --> lint[ruff lint]
14	lint --> sast[Semgrep SAST]
15	lint --> secrets[gitleaks secret scan]
16	lint --> deps[pip-audit dependencies]
17	sast --> test[pytest + coverage]
18	secrets --> test
19	deps --> test
20	test --> notify[notify SOC webhook]
21	```
22
23	## The gates
24
25	\| Job \| Tool \| What it stops \|
26	\|-----\|------\|---------------\|
27	\| `lint` \| ruff \| style + a security rule set (`S`) on top of the usual lint \|
28	\| `sast` \| Semgrep \| the OWASP/Flask rule packs plus four custom rules in `.semgrep/rules.yml` \|
29	\| `secrets` \| gitleaks \| committed credentials, full history scanned on PRs \|
30	\| `dependencies` \| pip-audit \| known-vulnerable pinned dependencies \|
31	\| `test` \| pytest \| regressions, with coverage reported \|
32	\| `notify-soc` \| `scripts/notify_soc.py` \| nothing - it posts the run outcome to the SOC \|
33
34	`lint` runs first as a cheap fail-fast. The three security scans fan out in
35	parallel, `test` waits on all of them, and the SOC notification runs last with
36	`if: always()` so the SOC hears about failures too, not just green runs.
37
38	## Custom Semgrep rules
39
40	The packs catch the common cases; `.semgrep/rules.yml` adds the ones I kept seeing
41	slip through:
42
43	- Flask started with `debug=True` (Werkzeug debugger RCE)
44	- `subprocess` with `shell=True` on a non-literal argument
45	- `jwt.decode` with signature verification disabled
46	- binding to `0.0.0.0` without an explicit reason
47
48	The four rules running against a deliberately vulnerable file - every finding is
49	Blocking, so the SAST job fails the pipeline before the code can merge:
50
51	![Semgrep custom rules](docs/screenshots/01-semgrep-sast.png)
52
53	## SARIF and the Security tab
54
55	Semgrep emits SARIF that gets uploaded with `github/codeql-action/upload-sarif`, so
56	findings show up under the repo's Security tab and as inline PR annotations rather
57	than only in the job log.
58
59	The dependency gate is `pip-audit`, which fails the build on a pinned package with a
60	known advisory:
61
62	![pip-audit dependency scan](docs/screenshots/02-pip-audit-deps.png)
63
64	## Run it locally
65
66	```bash
67	make install
68	make all
69	```
70
71	`make all` runs the same gates the pipeline does. You need `semgrep` and `gitleaks`
72	on your PATH for the `sast` and `secrets` targets; everything else is pip-installed
73	by `make install`.
74
75	## SOC integration
76
77	The last job posts the run outcome - repo, commit, actor, status, and a link back to
78	the run - to a Shuffle webhook. In my lab that webhook feeds the
79	[SOC automation lab](../soc-automation-lab), so a failed security gate opens a case
80	in TheHive the same way a Wazuh alert does. Set the `SHUFFLE_WEBHOOK_URL` repository
81	secret to wire it up; without it the job no-ops instead of failing.
82
83	More detail in [docs/pipeline.md](docs/pipeline.md) and
84	[docs/soc-integration.md](docs/soc-integration.md).