How Conduit is Shaped
The architecture that lets one consumer site read N project repos without coupling either side to the other · ~17 min read · Suggested by Bob technical
Conduit's architecture is a publisher-consumer split with a deliberately narrow contract between them. The publisher lives once, in the consumer repo. The state lives in each project repo. The cadence is per-artifact, not global. None of those choices were obvious; together they are what makes the system work at the second project as well as the first.
A useful test for any architecture: how painful is the second consumer? The first one is always cheap because the system was designed to fit it. The second one is where the design tells the truth — every coupling glossed over the first time becomes a real cost.
Conduit’s architecture is shaped by that test. The publisher logic lives in one place. The contract between publisher and consumer is small enough to fit in a YAML file. State is per-project. Cadence is per-artifact. Onboarding the second project (osprey-vantage, after osprey-strike) cost about fifteen minutes of YAML and a state file, not a re-implementation.
Two halves and a seam
Conduit is split into two halves: a publisher that lives in each project’s CI, and a consumer that is the static site. The seam between them is a repository_dispatch event carrying a structured bundle.
graph TD
subgraph Project Repo
T[timbers ledger] --> W[Publish workflow]
S[.conduit/state.json] --> W
W -->|uses:| A[Composite Action<br/>Constructured/conduit/.github/actions/publish@v1]
end
A -->|repository_dispatch<br/>project-artifact-bundle| C
subgraph Conduit Repo
C[Receive workflow] --> F[src/projects/<slug>/<type>/<name>.md]
F --> B[Eleventy build]
B --> D[Cloudflare Pages]
end
The shape rewards a careful look:
- The composite action is imported by the project repo but defined in the consumer repo. That is unusual. The natural impulse is to put the publisher in the project repo (because the publisher reads the project’s data) or in a shared infrastructure repo (because it’s reusable). Putting it in the consumer repo makes the consumer the owner of its own ingest contract, which turns out to be exactly the right boundary.When the consumer owns the publisher, schema and behavior change together. When they live in different repos, every change becomes a coordination problem across N+1 PRs.
- The
repository_dispatchis the only thing crossing the seam. No shared databases, no message queues, no scheduled pollers. A bundle is either dispatched or not; if it is, the consumer ingests it. - The project repo holds only its own state and configuration. No publisher scripts, no LLM templates, no conduit-aware logic of its own.
The next sections take each of those pieces and explain why it ended up that shape.
The bundle envelope is the contract
Everything in the system serializes through a single JSON envelope. The publisher produces it, the dispatcher delivers it, the consumer reads it. Once you have looked at the envelope, you have most of the system in your head:
{
"bundle_version": 1,
"repo": "osprey-strike",
"commit_sha": "...",
"since_sha": "...",
"generated_at": "2026-05-04T15:00:00Z",
"project": { "name": "Osprey Strike", "color": "...", "icon": "📞", "repo_url": "..." },
"artifacts": [
{ "type": "devblog", "path": "devblog/2026-05-04.md", "content": "...", "provenance": { "range": "...", "model": "...", "template": "..." } },
{ "type": "adr", "path": "adr/0007-restrict-eco-deletion.md", "content": "...", "number": 7, "title": "...", "provenance": { ... } }
]
}
That envelope is the version-stable interface. bundle_version: 1 is the only schema declaration. As long as the envelope is shaped this way, any publisher can drop into Conduit’s ingest path. As long as Conduit accepts envelopes shaped this way, any project repo can publish into it.
The bundle envelope is the architectural line in the sand. Everything above it (cadence rules, drafting, prompts) and everything below it (rendering, search, the static site) can change independently as long as the envelope shape stays stable.
The dispatch path is small enough to fit in a paragraph: the publisher constructs the envelope, builds a gh api POST to the consumer’s repository_dispatch endpoint with event_type: project-artifact-bundle and the envelope as the client_payload, and a workflow on the consumer side picks it up. There is no broker, no sustained connection, no retry queue. A failed dispatch is a CI failure visible in the project repo’s run history.The client_payload must be a real JSON object, not a stringified one. The next cairn has the bug story for how we learned this — short version: gh api -F stringifies; jq | gh api --input - doesn’t.
The simplicity of the seam is what makes the rest of the architecture viable. There is no temptation to “just add one more field” between publisher and consumer, because adding a field means a versioned schema bump, which means a coordinated migration across consumers. That cost keeps the seam clean.
Per-type anchors are the cadence insight
Each artifact type carries its own anchor in .conduit/state.json. That is the single design call that makes per-type cadence possible. Devblogs publish on push (narrative thrives on freshness); ADRs and sprint-reports publish on a weekly cron (decisions read as decisions only when the LLM gets a wide enough window to consolidate iteration chains). Without per-type anchors, both cadences would have to share one window, and one of them would be wrong.
That insight emerged from a real bug. The earlier publisher tracked a single last_published_sha and ran on every push to main. Each run handed the LLM one to three commits. The decision-log template’s “consolidate iterations into ONE ADR” rule could not fire, because the LLM only ever saw a snapshot. Iteration chains turned into separate ADRs. We landed an ADR proposing a model swap, then an hour later landed an ADR reverting it, both as standalone “decisions.” That was the cue that one cadence does not fit all artifact types.
The fix was to grow the state schema:
{
"next_adr": 12,
"anchors": {
"devblog": "<sha>",
"adr": "<sha>",
"sprint-report": "<sha>",
"release-notes": "<sha>"
}
}
Each artifact type carries its own anchor. Each workflow advances only the anchors it drafted. The devblog workflow runs on push and advances anchors.devblog. A separate decisions workflow runs on a Sunday cron and advances anchors.adr and anchors.sprint-report. The two workflows share a concurrency group so writes to state.json never race.
A cadence is a design parameter in this system, not a property of the publisher. Different artifacts have legitimately different ideal windows, and the architecture should let each have its own.
The migration from the single-anchor schema to the per-type-anchor schema was in-place and lazy. On first run after the upgrade, the publisher seeds every type’s anchor from the legacy last_published_sha, then drops the legacy field. No ops dance, no coordinated cutover.
The same migration mechanism, by the way, lets us seed anchors at any commit. When osprey-vantage migrated from its old single-publisher workflow, it had no committed anchor (the legacy publisher tracked the SHA in an ephemeral CI artifact). We seeded the anchors at the migration commit and the system handled it cleanly. Bounded first-run, no history reprocessing.
Why the publisher lives in the consumer repo
Three real options existed for where to put the publisher logic, and the choice was load-bearing enough to capture why one of them won.
Option 1: copy the publisher into every project repo. Each project owns its own scripts, its own templates, its own workflow YAML. This is what we started with — the original tools/conduit-publisher/ directory in osprey-strike, plus a similar setup in osprey-vantage. The first time we needed to fix a bug across all consumers, the cost showed up: N PRs, N reviews, drift between copies.
Option 2: publisher as a composite action in the consumer repo. Each project repo carries only state and configuration; the actual publisher logic lives in Constructured/conduit/.github/actions/publish/, and project workflows do uses: Constructured/conduit/.github/actions/publish@v1. One implementation, N consumers, asynchronous rollout via action versioning.
Option 3: conduit-side puller. Conduit hosts a cron workflow that, on a schedule, clones each registered project repo, installs its tools, runs the publisher itself, and self-dispatches into its own ingest. Project repos have no publisher infrastructure at all.
The trade-offs:
| Dimension | Option 1 (per-repo) | Option 2 (composite action) | Option 3 (puller) |
|---|---|---|---|
| Code duplication | High | None | None |
| Onboarding cost | Copy + adapt | ~15 lines of YAML | Add a config entry |
| Failure isolation | Per-project | Per-project | Centralized (single point of failure) |
| Refactor velocity | N PRs, slow | One PR, asynchronous rollout | One PR, immediate |
| Conduit’s role | Static site + ingest | Static site + ingest + action host | Build platform |
| Compute cost | Distributed across project orgs | Distributed | Centralized in Conduit |
| Provenance | Project-attributed | Project-attributed | Conduit-attributed |
| Project secrets | Per-project | Per-project | Centralized in Conduit |
Option 2 wins at our scale (two project consumers today) because it preserves project autonomy, isolated failure, and project-attributed provenance, while still putting the implementation in one place. Option 3 only starts to pay off above five-or-so consumers, when the per-project setup cost would dominate.
The architectural call was “do not centralize compute prematurely.” Centralizing the implementation (option 2) is a smaller change than centralizing the execution (option 3), and the migration from 2 to 3 is real if we ever hit the threshold.
Option 2 also gives us a versioning shape — patch fixes retag v1 in place; breaking changes ship as v2. The next cairn covers what that policy looks like in practice.
Truly silent on quiet weeks; loud on operational failures
Two failure modes sit very close to each other and need to look very different in the workflow log.
A quiet week. The LLM legitimately has nothing to say. No timbers entries qualify, or the decision-log template emits “No decisions to record from this batch.” In this case, the right behavior is silence: no commit, no notice, no zero-content placeholder. The anchor advances (because the window was covered) but no artifact ships.
An operational failure. The Gemini API rejects the request, timbers query exits non-zero, jq fails to parse output. In this case, the right behavior is loud failure: the workflow goes red, the failure is visible in CI, and somebody can act.
Distinguishing the two means the script tracks operational failures in a separate channel from artifact counts. resolve_scope returns exit code 1 for “legitimately quiet” and 2 for “operational failure.” A draft step that fails on a non-empty range increments an op_failures counter. The script writes op_failures to a file the workflow reads; a final Surface operational failures step fails the run when the count is non-zero — after dispatch and state-commit so partial-success anchors still land.
The result is that a broken Gemini key looks distinctly different from a quiet week in the workflow UI. Without that, an outage and silence are visually indistinguishable, which means an outage gets missed.
flowchart TD
Q[Run]
Q --> Z{Per-type<br/>resolve_scope}
Z -->|0 entries| QW[Skip type silently]
Z -->|N entries| D{Draft}
D -->|success| AC[Push artifact + bump anchor]
D -->|empty 'no decisions'| BA[Bump anchor, no artifact]
D -->|operational failure| OF[Increment op_failures, no anchor bump]
AC --> S[State write]
BA --> S
QW --> S
OF --> S
S --> CG{artifact_count > 0?}
CG -->|yes| DI[Dispatch + commit state]
CG -->|no| NS[Quiet window: no dispatch, no commit]
DI --> SF{op_failures > 0?}
NS --> SF
SF -->|no| OK[Workflow green]
SF -->|yes| RED[Workflow red]
The flowchart looks busier than the code is. Most paths handle exactly one concern, and the script is mostly bookkeeping between them.
The previous version of this code gated the state-commit step on artifact_count > 0. That meant a successful “no decisions” ADR draft advanced the anchor in memory but never committed it — so next week’s run reprocessed the same window forever. The fix was to drop the gate and let the inner git diff --quiet check handle the genuine no-op case. Worth flagging because the bug looked like “anchor bug” but was actually “wrong gate on a workflow step.”
That bug is the kind of thing reviewers exist for. (We caught it via an automated reviewer’s PR comment, two PRs into this rework. The next cairn talks about that loop.)
Project autonomy on what stays project-specific
Some things must live in each project repo because they are inherently project-specific. The state file (.conduit/state.json) carries the project’s anchors and ADR counter. The project metadata (.conduit/project.json) carries the human-readable name, color, icon, and repo URL. The two workflow YAMLs carry the triggers (push, cron) and the secrets the action consumes.
Some things must not. The bundle assembly script, the ADR splitter, the LLM templates, the dispatch logic, the cadence semantics — those belong with the consumer because changing them is a contract change, and a contract change should not require N project PRs.
The cleanest way to see the split is to look at what a new project actually has to write to onboard:
# .github/workflows/publish-conduit-devblog.yml (~50 lines, mostly trigger config)
on:
push: { branches: [main], paths-ignore: ['.conduit/state.json'] }
concurrency:
group: publish-conduit-$
cancel-in-progress: false
permissions: { contents: write }
jobs:
publish:
if: |
github.event.head_commit.author.name != 'argocd-image-updater' &&
!startsWith(github.event.head_commit.message, 'chore(conduit):')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with: { fetch-depth: 0 }
- uses: jdx/mise-action@v4
- uses: Constructured/conduit/.github/actions/publish@v1
with:
artifact_types: devblog
gemini_api_key: $
conduit_dispatch_token: $
That is most of what onboarding looks like. A second YAML for the decisions cadence (cron + workflow_dispatch with a backfill input). Two secrets. A 50-line state.json and a 5-field project.json. Done.
Templates — the LLM prompts that shape every artifact — also belong in the consumer rather than the project. They ship in the action and get installed into the runner’s global timbers template directory on every run, with project-side overrides as an opt-in escape hatch. The next cairn unpacks how that works in practice.
The architectural rule: Conduit owns the contract surface; projects own their state and configuration. Anything cross-project belongs in the action; anything project-specific belongs in the project repo.
Architectural edges
No architecture is finished when its diagram looks elegant. A few edges in this design are worth naming explicitly.
LLM prompt drift across providers. The action defaults to a Gemini model alias (flash → gemini-3-flash-preview). That is portable across most active Google API keys, but a future move to a different provider (Claude, OpenAI) would require generalizing the prompt-template path beyond timbers’ Google-flavored model resolution. Worth a re-think when we change providers, not before.
No reviewer-in-the-loop on artifacts. Conduit publishes the LLM’s output as-is. There is no human review step between draft and publish. For internal-audience artifacts on a small team, that has been fine. For an external-audience surface, it would not be — and changing it would touch the bundle envelope (a “needs review” flag) and the consumer (a draft staging area).
Single-org access setting. The action is hosted in a private repo. For other repos in the same GitHub organization to consume it, the consumer’s actions/permissions/access must be set to organization. One-time setting per consumer-action pair, but invisible in workflow YAML and easy to forget. The next cairn covers the operational details.
These are architectural shape questions. Operational deferred work — Q reviewer support across all repos, multi-cycle review convergence, release-notes drafting — lives in the next cairn instead.
Summary
- Two halves, narrow seam. A per-project publisher and a single-repo consumer, separated by a versioned bundle envelope and a
repository_dispatchevent. - Bundle envelope is the contract. Everything above and below it can change independently as long as the envelope shape stays stable.
- Per-type anchors enable per-type cadence. One state schema with four anchors lets devblog run per-push and decisions run weekly without colliding.
- Composite action over puller. Project autonomy, isolated failure, asynchronous rollout. Migration to a centralized puller stays available if scale changes.
- Quiet weeks silent, op failures loud. Two failure modes that look identical from the outside need to be operationally distinct in the workflow UI.
- Conduit owns the contract surface; projects own their state. Templates, scripts, dispatch logic live in the consumer;
state.json,project.json, workflow YAMLs live per-project.
- The bundle envelope is currently
v1. What change would justifyv2? What does the migration window look like across N consumers — is it acceptable for them to stay onv1indefinitely, or do we want a deprecation policy? - If we add a fifth artifact type (say, "weekly digest" or "incident report"), what would the surgery cost on the publisher and the consumer? Does the per-type anchor model scale cleanly, or does it want a different shape past four?
- The action lives in a private repo today. If we ever wanted external consumers (a partner repo, an OSS project of ours), what would change about the contract — and is that change worth doing speculatively, or only when the need shows up?
- The Work That Writes Itself — The first cairn in this trail. Establishes why Conduit exists and what problem it solves before this cairn dives into how it's shaped.
- GitHub Composite Actions — The mechanism that makes the publisher-in-consumer-repo pattern possible. The action's
action.ymlin the conduit repo is the canonical reference. - GitHub repository_dispatch API — The cross-repo event that delivers the bundle from publisher to consumer. The
client_payloadmust be a JSON object, not a stringified one — the most common implementation gotcha. - Timbers — The development ledger Conduit reads from. Without timbers there is no signal to aggregate; understanding timbers' query model clarifies how Conduit's
since_sha..HEADranges work. - Eleventy Data Cascade — How Conduit's per-project rollups (
byProject,feedByProject,crossProjectFeed) are computed. Useful background for anyone reading the consumer-side code. - PostgreSQL — Not used by Conduit, but a useful contrast: a static-site approach to artifact-driven knowledge gives you everything PostgreSQL gives you for this workload (filtering, sorting, search) without a runtime service. The decision to stay static is load-bearing.
- Events All the Way Down — The Strike cairn on event sourcing. Conduit isn't event-sourced, but the same instinct ("the event is the truth, the projection is the readable surface") shows up in the bundle-and-static-site shape.
Generated by Cairns · Agent-powered with Claude