Architecture

The shape, precise enough to code against. The companion pivots doc answers why blacklight took this shape; this one answers what the shape is.

This page tracks the v0.6.0 release-tag boundary. Authoritative source is DESIGN.md in the repo; this page mirrors it for browsable reading.

Vernacular shortcut. This page references milestones (M13, M14, M15, M16) and the "Path C" skill-delivery shape several times. M13 was the Day-5 hackathon milestone that retired the flat bl-skills memory store; M14 (Day 6 morning) added the LMD trigger / closed-loop substrate hook; M15 (Day 6 afternoon) corrected the live-API endpoint shapes; M16 (Day 6 evening) closed the curator-prescription → collector-execution loop with the session-event → memstore-pending bridge and the per-verb args translator. "Path C" is the resulting skill-delivery architecture (Skills primitive + reference Files), distinct from Path A (the retired memstore approach) and Path B (a deferred callable_agents hunter pattern). Full glossary on the docs index; timeline view on the Build Timeline page.

The pitch in one paragraph

blacklight is a portable bash Runner (bl) that turns any Linux host into an agent-directed incident-response surface. The Runner runs locally on bash 4.1+ with curl, awk, and jq, 26 numbered source parts under src/bl.d/ assemble into a single ~10,700-line bl binary via make bl. No daemons, no Python. Every investigation is a conversation between the operator, the Runner, and a Managed Agents session hosted in the operator's Anthropic workspace. That session, Opus 4.7, 1M context, six description-routed routing Skills + eight reference Files mounted at creation, decides what to look at next, authors the defensive payloads (ModSec rules, firewall entries, YARA sigs), and prescribes remediation (rogue cron stripping, .htaccess cleanup, quarantine). The Runner executes; the agent directs; the existing defensive primitives your host already runs (ModSec, APF, CSF, iptables, nftables, LMD, ClamAV, YARA) are the hands. Man-days of manual IR become agentic-minutes on the substrate the defender already owns.

Three named layers

The system is three named layers: Runner, Curator, Substrate. Each has its own surface, its own contract, and its own enforcement boundary.

Layer 01 · Runner · bl on the host

• Form: single ~10,700-line bash script, one file, assembled from 26 numbered parts under src/bl.d/ via make bl.
• Surface: nine verbs, bl observe, bl consult, bl run, bl defend, bl clean, bl case, bl setup, bl trigger, bl flush.
• Dependencies: bash 4.1+, curl, awk, jq, grep, sed, tar, gzip.
• Lifecycle: no daemons, no services. Invoked per operator thought; exits when done.
• Reaches the Curator via: HTTPS + API key.

Layer 02 · Curator · Managed Agent session (Anthropic-hosted)

• Agent: bl-curator (Opus 4.7, 1M context, Managed Agent).
• Environment: bl-curator-env (cloud sandbox). Per-session apt installs run via the agent's bash tool: apache2, libapache2-mod-security2, modsecurity-crs, yara, jq, zstd, duckdb, pandoc, weasyprint.
• Routing Skills: 6, description-routed, lazy-loaded.
• Reference Files: 8, mounted at session create as /skills/<basename>.
• Memory store: bl-case (rw), hypothesis, evidence, pending, history.
• Per-case Files: evidence bundles + closed-case briefs, hot-attached.
• Lifecycle: no local runtime process. The session lives in the Anthropic workspace; the Runner reaches it via HTTPS on every invocation.
• Emits to the Substrate via: step directives written to bl-case/pending/, executed by the Runner.

Layer 03 · Substrate · defensive primitives on the host

• Tools: apachectl + mod_security, APF, CSF, iptables, nftables, LMD, ClamAV, YARA, Apache / nginx logs, journalctl, crontab, find, stat, cat -v.
• Contract: blacklight directs the Substrate; it does not install, replace, or re-abstract it. The primitives predate blacklight and will outlive it.

Layer boundary rules

• Runner never decides what action to take. It executes what the Curator prescribes (gated by safety tiers) and reports results.
• Curator never touches the host filesystem or primitives directly. It reasons, authors, prescribes. It never applies.
• Substrate is untouched by blacklight source code. No new rule engines, no new manifests, no new wire formats. Only native usage of existing primitives.

Runtime flow

blacklight investigations are operator-agent conversations, not batch dossier analyses. The canonical flow proceeds in numbered turns; each turn names the actor and the work it does.

1. Operator runs bl consult --new --trigger <hit>.
2. Runner: preflight workspace, create case record, POST a session wake event.
3. Curator: session wakes, 8 reference Files mounted, routing Skills lazy-load, read bl-case/hypothesis, reason, emit 4 steps to bl-case/pending/s-01..04.json.
4. Runner: poll bl-case/pending, present the proposed steps to the operator:
   • s-01 observe log apache --around <hit> --window 6h
   • s-02 observe fs --mtime-cluster …
   • s-03 observe htaccess …
   • s-04 observe cron --user …
5. Operator confirms Accept? [Y/n]. Runner: exec each step locally, write result to bl-case/results/s-01..04.json, POST wake event.
6. Curator: read results, revise hypothesis, emit next batch:
   • s-05..07 observe (further evidence)
   • s-08 defend firewall <ip>
   • s-09 defend modsec <rule>
   • s-10 clean cron
   • s-11 clean htaccess
7. Runner: auto-exec read-only and auto-tier steps; stop on destructive, show diff, require --yes per step. Operator confirms each destructive step (Confirm cron removal? y). Runner: exec, write result.
8. Curator: when open_questions = 0, propose close_case.
9. Operator runs bl case close. Runner: archive brief to Files, write precedent pointer in bl-case, schedule firewall-block retire-sweep at T+30d.

The mechanical choice is async step-emit over polled memory-store files, not synchronous SSE tool-result. The agent writes proposed step JSON to bl-case/pending/<id>.json; the Runner consumes pending steps via two modes: a continuous poll loop used by bl consult foreground REPL (3-second tick, dedup-against-seen-set, exit on end_turn or --timeout), and an on-demand single-fetch via bl run --list for batched/async operator workflows. Both paths execute, write to bl-case/results/<id>.json, and send wake events.

This avoids SSE bidirectional plumbing in bash, makes the case memory a self-documenting audit log, and keeps bl a short-lived command per invocation. Polling overhead (~3-9s per loop tick) is invisible against agent reasoning time.

Path C primitives map (M13 realignment)

The Curator layer is built on four Anthropic Managed Agents primitives. M13 retired the pre-Path-C two-memstore layout (bl-skills read_only + bl-case read_write) in favor of the full primitive surface.

The retired bl-skills memory store was a flat read_only memstore with ~22 operator-voice files at ≤50 KB. Older docs that name it predate M13; see Skills Architecture for the Path C shape.

Command namespace

Nine runtime verbs plus one entry-dispatcher. All bash functions live in a single bl script (assembled from src/bl.d/NN-*.sh via make bl); dispatched by first argument from main() in 90-main.sh.

bl observe: read-only evidence extraction

Auto-runs (no confirm). Emits JSONL to stdout and appends structured output to the current case.

bl consult: session attach and case management

bl run: execute agent-prescribed step

bl defend: apply agent-authored payload

bl clean: destructive remediation, diff-confirmed

bl case: case lifecycle

bl setup: workspace bootstrap

See Setup Flow for the per-call API contract.

bl trigger: hook-fired case open

bl trigger lmd --scanid <id> [--session-file <path>] [--source-conf <path>] [--unattended] parses the LMD per-session TSV, computes a cluster fingerprint, and allocates a case via bl_consult_new --dedup. Degraded path on empty / unreadable TSV opens a stub case anyway.

bl flush: durable outbox drain

bl flush --outbox replays queued wake / signal_upload / action_mirror records when the workspace is reachable again. Idempotent; safe to invoke from cron.

Dependencies

Tier 1: always present on the host

• bash ≥ 4.1
• coreutils (ls, cat, stat, find, sort, uniq, head, tail, wc, sha256sum, tar, gzip)
• curl
• awk (mawk or gawk; prefer gawk for associative arrays)
• sed
• grep (GNU preferred for -F -f patterns.txt Aho-Corasick speed)

Tier 2: ship as bl deps

• jq: single static binary, ~3 MB, portable back to CentOS 6. Non-optional.
• zstd: optional, runtime-detected. Falls back to gzip.

Tier 3: curator sandbox only

Inside the Anthropic-hosted environment, provisioned at env creation:

• apache2 + libapache2-mod-security2 + modsecurity-crs: for apachectl -t pre-flight of synthesized ModSec rules
• yara: for on-sandbox signature testing
• duckdb: for agentic SQL over JSONL

None of these Tier-3 deps are installed on the fleet host. The sandbox has them; the host does not.

Explicitly NOT required

• No Python on any host
• No Docker (operator can use docker for demo fixtures; bl runs native)
• No systemd requirement
• No database (SQLite or otherwise). All state lives in memory stores, files, and a small local ledger
• No web server or local HTTP listener. bl is a command, not a service

Non-goals

• Fleet-scope orchestration. bl is per-host. Fleet propagation rides the operator's existing primitive (Puppet, Ansible, Salt, Chef, manual SSH). blacklight generates the payload; the operator propagates.
• Continuous posture monitoring daemon. blacklight is trigger-bound by design. Periodic sweeps and trajectory analysis are deferred future work.
• Web frontend or dashboard. The terminal REPL is the operator surface. A rendered HTML brief is a post-close artifact.
• Replacing defensive primitives. blacklight directs apachectl, apf, csf, iptables, nftables, maldet, clamscan, yara. It does not re-implement any of them.
• Cross-CVE threat intelligence sharing. Future work.
• Windows / BSD support. Future work. The current build is Linux only.

Glossary

• Case: an investigation; carries hypothesis, evidence, actions, precedent. One per incident.
• Step: a single action the agent prescribes. Has an action tier, a verb, typed arguments, and a reasoning field. Written to bl-case/pending/.
• Action tier: one of read-only, auto, suggested, destructive, unknown. Determines gate behavior.
• Routing Skill: a description-routed operator-voice behavior module uploaded to the Anthropic Skills primitive. Lazy-loaded by the curator when the description matches the current reasoning need. Six in Path C; reference content delivered alongside via Files API.
• Reference File: a workspace-scope markdown file mounted at session creation under /skills/<basename> (always present, not lazy-loaded). Eight in Path C.
• Trigger: the first signal that opens a case (e.g. maldet quarantine, auditd critical event, ModSec rule fire).
• Curator: the Managed Agents session that owns a case.
• Synthesizer: the curator's synthesize_defense custom-tool emit surface for authoring a defensive payload.
• Intent reconstructor: the curator's reconstruct_intent custom-tool emit surface for analyzing a malware sample.
• Precedent: a closed case accessible to future cases via the bl-case memory store (lives within the same memstore in v0.6.0, not a separate store).
• Defense: any applied change to host state that reduces attack surface (ModSec rule, firewall entry, scanner signature).
• Remediation: any applied change to host state that removes attacker presence (file quarantine, cron strip, .htaccess edit, process kill).