CTI Detection Automation

Pulls indicators from live threat-intel feeds, deduplicates them, extracts the MITRE ATT&CK techniques behind them, turns the result into Wazuh detection rules and CDB lists, and emails an analyst for sign-off before anything goes live. No rule reaches the SIEM without a human approving it.

This is the piece that feeds the watchlists used by the SOC automation lab - the cti-malicious-ip, cti-malicious-domain, and cti-malware-hash lists that the Wazuh rules look up against are generated here.

What it does

• Collects from ThreatFox, Feodo Tracker, URLhaus, AlienVault OTX, and OpenPhish, covering IPs, domains, URLs, file hashes, phishing, exploit-kit and malware-download URLs, C2 infrastructure, and (optionally) leaked credentials.
• Normalizes every source into one indicator model and deduplicates across feeds - an IP seen in both ThreatFox and Feodo becomes one indicator carrying both sources, the higher confidence, and the union of techniques.
• Extracts and dedups TTPs: ATT&CK technique IDs come straight from OTX pulses and are inferred from malware family and threat type for the feeds that don't carry them, then collapsed into a ranked coverage list.
• Generates Wazuh rules: CDB lists per indicator type plus an XML ruleset whose list-lookup rules are tagged with the dominant ATT&CK techniques for that bucket.
• Gates on approval: every run produces a candidate bundle and emails the analyst a signed, time-limited review link. Rules are written to Wazuh only when the analyst approves.

flowchart LR
    F[ThreatFox / Feodo / URLhaus<br/>OTX / OpenPhish] --> N[normalize]
    N --> D[deduplicate]
    D --> C{confidence<br/>>= threshold}
    C --> T[extract + dedup TTPs]
    T --> G[generate CDB lists<br/>+ Wazuh rules]
    G --> M[email analyst<br/>signed review link]
    M -->|approve| W[promote to Wazuh]
    M -->|reject| X[discard]

The approval gate

The pipeline never deploys on its own. It writes the candidate to a staging directory and emails a review link. The analyst sees the diff against the last approved bundle, the indicator counts, and the ATT&CK coverage, then approves or rejects.

Review page	Bundle history

Short walkthrough of the flow: docs/media/cti-approval-walkthrough.mp4

The review link is an itsdangerous signed token with a TTL, so it can't be forged or replayed after it expires.

Run the demo

No API keys needed - the demo runs against the bundled fixtures:

make install
make demo      # generates a bundle and writes the approval email to output/emails/
make serve     # approval console on http://localhost:8080

Open the email under output/emails/, click through to the review page, and approve. The promoted lists and rules land in output/active/.

Run it for real

cp .env.example .env          # set CTI_APPROVAL_SECRET and feed API keys
cp config.example.yaml config.yaml
python -m cti.cli run -c config.yaml

ThreatFox and OTX need free API keys (THREATFOX_AUTH_KEY, OTX_API_KEY); Feodo, URLhaus, and OpenPhish are keyless. Set email.backend: smtp and the SMTP settings in config.yaml to send the approval mail to a real inbox. Point wazuh_etc_dir at the manager's /var/ossec/etc and approval will write the lists and rules straight into place.

Schedule it with the systemd timer in deploy/ (hourly by default) or any cron.

Layout

src/cti/feeds/     one connector per source, each with a pure parse()
src/cti/dedup.py   cross-feed merge
src/cti/ttp.py     ATT&CK extraction + coverage report
src/cti/rules.py   CDB list + Wazuh XML generation
src/cti/approval.py signed tokens + email rendering + SMTP
src/cti/pipeline.py orchestration, candidate staging, promotion
src/cti/web.py     Flask approval console
fixtures/          sample feed payloads for the demo and tests
tests/             pytest suite (feeds, dedup, ttp, rules, approval, web)

Tests

make test

Connectors are split into fetch and parse, so the suite runs the real parsing logic against fixtures with no network. Coverage includes cross-feed dedup, TTP extraction, rule generation well-formedness, token signing/expiry, and the full approve/reject path through the web app.

See docs/architecture.md for the data model and docs/approval-flow.md for the gate in detail.