How Conduit is Shaped
The architecture that lets one consumer site read N project repos without coupling either side to the other · ~17 min read ~– min read · Suggested by Bob technical
Conduit's architecture is a publisher-consumer split with a deliberately narrow contract between them. The publisher lives once, in the consumer repo. The state lives in each project repo. The cadence is per-artifact, not global. None of those choices were obvious; together they are what makes the system work at the second project as well as the first.
Two halves and a seam
Conduit is split into two halves: a publisher that lives in each project’s CI, and a consumer that is the static site. The seam between them is a repository_dispatch event carrying a structured bundle. The composite action is imported by the project repo but defined in the consumer repo — unusual, but it makes the consumer the owner of its own ingest contract. No shared databases, no message queues, no scheduled pollers; the project repo holds only its own state and configuration.
The bundle envelope is the contract
Everything in the system serializes through a single JSON envelope with bundle_version: 1, a repo/commit_sha/since_sha header, project metadata, and an artifacts[] array each carrying type, path, content, and provenance. As long as the envelope shape stays stable, anything above it (cadence rules, drafting, prompts) and below it (rendering, search) can change independently. The dispatch is a gh api POST to repository_dispatch with event_type: project-artifact-bundle. No broker, no retry queue — a failed dispatch is a CI failure visible in the project repo’s run history. The client_payload must be a real JSON object, not a stringified one — gh api -F stringifies; jq | gh api --input - doesn’t (the next cairn has the bug story).
The bundle envelope is the architectural line in the sand. Adding a field means a versioned schema bump and a coordinated migration across consumers; that cost is what keeps the seam clean.
Per-type windows are the cadence insight
Each artifact type carries its own window now, not a persisted anchor in the caller repo. The publisher used to track per-type anchors in .conduit/state.json; the current action is read-only against the project working tree and resolves a recent time window for each requested type instead. Daily pulse uses a 24-hour default. Devblog, ADR, sprint report, Product Updates, Leadership Brief, and Milestone use weekly windows unless the workflow overrides <TYPE>_WINDOW_HOURS. ADR numbering moved the other direction: Conduit derives the next number from its existing ADR files at publish time, so a caller no longer needs to carry or bump next_adr.
Why the publisher lives in the consumer repo
Three real options existed: copy the publisher into every project repo (Option 1, what we started with — fixing a bug cost N PRs and the copies drifted); host the publisher as a composite action in the consumer repo and have project workflows do uses: Constructured/conduit/.github/actions/publish@v3 after configuring AWS credentials through OIDC (Option 2); or have Conduit run a centralized puller that clones each registered repo on a schedule (Option 3). Option 2 wins at our scale because it preserves project autonomy, isolated failure, and project-attributed provenance while still putting the implementation in one place. Option 3 only starts to pay off above five-or-so consumers; the migration to it stays available if scale changes.
Truly silent on quiet weeks; loud on operational failures
Two failure modes sit close together and need to look very different in the workflow log. A quiet week is silent: no commit, no notice, no zero-content placeholder. An operational failure (Bedrock rejects, timbers exits non-zero, jq fails) is loud: workflow goes red, error body in the log. The script tracks operational failures separately — resolve_scope returns 1 for “legitimately quiet” and 2 for “operational”; an op_failures counter accumulates and a final step fails the run when it’s non-zero, after successful artifacts have already been bundled and dispatched. Without the split, an outage and silence look identical, and outages get missed.
An earlier anchor-based version gated the state-commit step on artifact_count > 0. A successful “no decisions” ADR draft advanced the anchor in memory but never committed it, so next week’s run reprocessed the same window forever. The current window-based action removes that write-back path entirely, but the lesson remains: quiet success and operational failure need separate signals.
Project autonomy on what stays project-specific
Some things must live in each project repo because they are inherently project-specific: .conduit/project.json (name, color, icon, repo URL) and the workflow YAMLs (triggers plus OIDC/IAM wiring). Some things must not: bundle assembly, the ADR splitter, LLM templates, dispatch logic, cadence semantics, anchors, and ADR counters. Templates ship in the action and install into the runner’s global ~/.config/timbers/templates/ directory on every run, with project-side overrides as an opt-in escape hatch. The architectural rule: Conduit owns the contract surface; projects own metadata and workflow configuration.
Architectural edges
No architecture is finished when its diagram looks elegant. Three edges worth naming: Provider coupling is lower, but not zero — the action now calls Bedrock through invoke-llm.sh, which isolates provider mechanics better than scattered timbers calls did. A future provider move should touch that helper and the action README first, not every workflow. No reviewer-in-the-loop on artifacts — Conduit publishes the LLM’s output as-is, fine for internal audiences but not for an external-facing surface. Audience channels expanded — the publisher now wires pulse, devblog, ADR, sprint report, Product Updates, Leadership Brief, and Milestone through one action, with release-notes rejected in favor of product-updates. Single-org access setting — the action lives in a private repo; cross-org consumption requires actions/permissions/access set to organization, invisible in YAML and easy to forget.
- The Work That Writes Itself — First cairn in this trail. Why Conduit exists at all.
- GitHub Composite Actions — The mechanism that makes the publisher-in-consumer-repo pattern possible.
- GitHub repository_dispatch API — The cross-repo event that delivers the bundle.
client_payloadmust be a JSON object, not a stringified one. - Timbers — The development ledger Conduit reads from.
- Eleventy Data Cascade — How Conduit's per-project rollups are computed.
A useful test for any architecture: how painful is the second consumer? The first one is always cheap because the system was designed to fit it. The second one is where the design tells the truth — every coupling glossed over the first time becomes a real cost.
Conduit’s architecture is shaped by that test. The publisher logic lives in one place. The contract between publisher and consumer is small enough to fit in a YAML file. State is per-project. Cadence is per-artifact. Onboarding the second project (osprey-vantage, after osprey-strike) cost about fifteen minutes of YAML and a state file, not a re-implementation.
Two halves and a seam
Conduit is split into two halves: a publisher that lives in each project’s CI, and a consumer that is the static site. The seam between them is a repository_dispatch event carrying a structured bundle.
graph TD
subgraph Project Repo
T[timbers ledger] --> W[Publish workflow]
P[.conduit/project.json] --> W
W -->|uses:| A[Composite Action<br/>Constructured/conduit/.github/actions/publish@v3]
end
A -->|repository_dispatch<br/>project-artifact-bundle| C
subgraph Conduit Repo
C[Receive workflow] --> F[src/projects/<slug>/<type>/<name>.md]
F --> B[Eleventy build]
B --> D[Cloudflare Pages]
end
The shape rewards a careful look:
- The composite action is imported by the project repo but defined in the consumer repo. That is unusual. The natural impulse is to put the publisher in the project repo (because the publisher reads the project’s data) or in a shared infrastructure repo (because it’s reusable). Putting it in the consumer repo makes the consumer the owner of its own ingest contract, which turns out to be exactly the right boundary.When the consumer owns the publisher, schema and behavior change together. When they live in different repos, every change becomes a coordination problem across N+1 PRs.
- The
repository_dispatchis the only thing crossing the seam. No shared databases, no message queues, no scheduled pollers. A bundle is either dispatched or not; if it is, the consumer ingests it. - The project repo holds only its own state and configuration. No publisher scripts, no LLM templates, no conduit-aware logic of its own.
The next sections take each of those pieces and explain why it ended up that shape.
The bundle envelope is the contract
Everything in the system serializes through a single JSON envelope. The publisher produces it, the dispatcher delivers it, the consumer reads it. Once you have looked at the envelope, you have most of the system in your head:
{
"bundle_version": 1,
"repo": "osprey-strike",
"commit_sha": "...",
"since_sha": "...",
"generated_at": "2026-05-04T15:00:00Z",
"project": { "name": "Osprey Strike", "color": "...", "icon": "📞", "repo_url": "..." },
"artifacts": [
{ "type": "devblog", "path": "devblog/2026-05-04.md", "content": "...", "provenance": { "range": "...", "model": "...", "template": "..." } },
{ "type": "adr", "path": "adr/0007-restrict-eco-deletion.md", "content": "...", "number": 7, "title": "...", "provenance": { ... } }
]
}
That envelope is the version-stable interface. bundle_version: 1 is the only schema declaration. As long as the envelope is shaped this way, any publisher can drop into Conduit’s ingest path. As long as Conduit accepts envelopes shaped this way, any project repo can publish into it.
The bundle envelope is the architectural line in the sand. Everything above it (cadence rules, drafting, prompts) and everything below it (rendering, search, the static site) can change independently as long as the envelope shape stays stable.
The dispatch path is small enough to fit in a paragraph: the publisher constructs the envelope, builds a gh api POST to the consumer’s repository_dispatch endpoint with event_type: project-artifact-bundle and the envelope as the client_payload, and a workflow on the consumer side picks it up. There is no broker, no sustained connection, no retry queue. A failed dispatch is a CI failure visible in the project repo’s run history.The client_payload must be a real JSON object, not a stringified one. The next cairn has the bug story for how we learned this — short version: gh api -F stringifies; jq | gh api --input - doesn’t.
The simplicity of the seam is what makes the rest of the architecture viable. There is no temptation to “just add one more field” between publisher and consumer, because adding a field means a versioned schema bump, which means a coordinated migration across consumers. That cost keeps the seam clean.
Per-type windows are the cadence insight
Each artifact type carries its own window. That is the single design call that makes per-type cadence possible. Daily pulse wants a 24-hour view. Devblog, ADRs, sprint reports, Product Updates, Leadership Briefs, and Milestones default to a weekly window unless the caller overrides <TYPE>_WINDOW_HOURS. Without per-type windows, every artifact would inherit one cadence, and at least one audience would get the wrong slice of history.
That insight emerged from a real bug. The earlier publisher tracked a single last_published_sha and ran on every push to main. Each run handed the LLM one to three commits. The decision-log template’s “consolidate iterations into ONE ADR” rule could not fire, because the LLM only ever saw a snapshot. Iteration chains turned into separate ADRs. We landed an ADR proposing a model swap, then an hour later landed an ADR reverting it, both as standalone “decisions.” That was the cue that one cadence does not fit all artifact types.
The first fix was to grow .conduit/state.json into per-type anchors. The current action has since moved again: it no longer writes project state at all. Scope is computed from time windows, the project working tree stays read-only, and since_sha_override exists for the rare backfill where an operator needs a precise range start.
# Defaults in build-bundle.sh
pulse # 24 hours
devblog, adr, sprint-report # 168 hours
product-updates, brief, milestone # 168 hours
ADR numbering followed the same simplification. Instead of carrying next_adr in every project repo, the action asks Conduit for the existing ADR files under that project’s directory and derives the next number there. The consumer already owns the published sequence, so making it the numbering authority removes another write-back to the caller repo.
A cadence is a design parameter in this system, not a property of the publisher. Different artifacts have legitimately different ideal windows, and the architecture should let each have its own.
The practical consequence is cleaner onboarding. A new project does not need a state migration, anchor seeding, or a branch-protection bypass for automated state commits. It needs metadata, workflows, credentials, and a dispatch token; the action owns the moving parts.
Why the publisher lives in the consumer repo
Three real options existed for where to put the publisher logic, and the choice was load-bearing enough to capture why one of them won.
Option 1: copy the publisher into every project repo. Each project owns its own scripts, its own templates, its own workflow YAML. This is what we started with — the original tools/conduit-publisher/ directory in osprey-strike, plus a similar setup in osprey-vantage. The first time we needed to fix a bug across all consumers, the cost showed up: N PRs, N reviews, drift between copies.
Option 2: publisher as a composite action in the consumer repo. Each project repo carries only metadata and workflow configuration; the actual publisher logic lives in Constructured/conduit/.github/actions/publish/, and project workflows do uses: Constructured/conduit/.github/actions/publish@v3. One implementation, N consumers, asynchronous rollout via action versioning.
Option 3: conduit-side puller. Conduit hosts a cron workflow that, on a schedule, clones each registered project repo, installs its tools, runs the publisher itself, and self-dispatches into its own ingest. Project repos have no publisher infrastructure at all.
The trade-offs:
| Dimension | Option 1 (per-repo) | Option 2 (composite action) | Option 3 (puller) |
|---|---|---|---|
| Code duplication | High | None | None |
| Onboarding cost | Copy + adapt | ~15 lines of YAML | Add a config entry |
| Failure isolation | Per-project | Per-project | Centralized (single point of failure) |
| Refactor velocity | N PRs, slow | One PR, asynchronous rollout | One PR, immediate |
| Conduit’s role | Static site + ingest | Static site + ingest + action host | Build platform |
| Compute cost | Distributed across project orgs | Distributed | Centralized in Conduit |
| Provenance | Project-attributed | Project-attributed | Conduit-attributed |
| Project secrets | Per-project | Per-project | Centralized in Conduit |
Option 2 wins at our scale (two project consumers today) because it preserves project autonomy, isolated failure, and project-attributed provenance, while still putting the implementation in one place. Option 3 only starts to pay off above five-or-so consumers, when the per-project setup cost would dominate.
The architectural call was “do not centralize compute prematurely.” Centralizing the implementation (option 2) is a smaller change than centralizing the execution (option 3), and the migration from 2 to 3 is real if we ever hit the threshold.
Option 2 also gives us a versioning shape — patch fixes retag v2 in place; breaking changes ship as v3. The next cairn covers what that policy looks like in practice.
Truly silent on quiet weeks; loud on operational failures
Two failure modes sit very close to each other and need to look very different in the workflow log.
A quiet week. The LLM legitimately has nothing to say. No timbers entries qualify, or the decision-log template emits “No decisions to record from this batch.” In this case, the right behavior is silence: no commit, no notice, no zero-content placeholder. The window was covered, but no artifact ships and no caller state changes.
An operational failure. Bedrock rejects the request, timbers query exits non-zero, jq fails to parse output. In this case, the right behavior is loud failure: the workflow goes red, the failure is visible in CI, and somebody can act.
Distinguishing the two means the script tracks operational failures in a separate channel from artifact counts. resolve_scope returns exit code 1 for “legitimately quiet” and 2 for “operational failure.” A draft step that fails on a non-empty range increments an op_failures counter. The script writes op_failures to a file the workflow reads; a final Surface operational failures step fails the run when the count is non-zero, after any successful artifacts have already been bundled and dispatched.
The result is that a broken AWS role or inaccessible Bedrock model looks distinctly different from a quiet week in the workflow UI. Without that, an outage and silence are visually indistinguishable, which means an outage gets missed.
flowchart TD
Q[Run]
Q --> Z{Per-type<br/>resolve_scope}
Z -->|0 entries| QW[Skip type silently]
Z -->|N entries| D{Draft}
D -->|success| AC[Push artifact]
D -->|empty 'no decisions'| BA[No artifact]
D -->|operational failure| OF[Increment op_failures]
AC --> S[State write]
BA --> S
QW --> S
OF --> S
S --> CG{artifact_count > 0?}
CG -->|yes| DI[Dispatch + commit state]
CG -->|no| NS[Quiet window: no dispatch, no commit]
DI --> SF{op_failures > 0?}
NS --> SF
SF -->|no| OK[Workflow green]
SF -->|yes| RED[Workflow red]
The flowchart looks busier than the code is. Most paths handle exactly one concern, and the script is mostly bookkeeping between them.
An earlier anchor-based version gated the state-commit step on artifact_count > 0. That meant a successful “no decisions” ADR draft advanced the anchor in memory but never committed it, so next week’s run reprocessed the same window forever. The current window-based action removes that write-back path entirely, but the lesson remains: quiet success and operational failure need separate signals.
That bug is the kind of thing reviewers exist for. (We caught it via an automated reviewer’s PR comment, two PRs into this rework. The next cairn talks about that loop.)
Project autonomy on what stays project-specific
Some things must live in each project repo because they are inherently project-specific. The project metadata (.conduit/project.json) carries the human-readable name, color, icon, and repo URL. The workflow YAMLs carry the triggers, id-token: write permission, AWS credential step, requested artifact types, and any explicit window overrides the action consumes.
Some things must not. The bundle assembly script, the ADR splitter, the LLM templates, the dispatch logic, cadence defaults, anchors, and ADR counters belong with the consumer because changing them is a contract change, and a contract change should not require N project PRs.
The cleanest way to see the split is to look at what a new project actually has to write to onboard:
# .github/workflows/publish-conduit-devblog.yml (~50 lines, mostly trigger config)
on:
push: { branches: [main] }
concurrency:
group: publish-conduit-$
cancel-in-progress: false
permissions: { contents: write, id-token: write }
jobs:
publish:
if: |
github.event.head_commit.author.name != 'argocd-image-updater' &&
!startsWith(github.event.head_commit.message, 'chore(conduit):')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with: { fetch-depth: 0 }
- uses: jdx/mise-action@v4
- uses: aws-actions/configure-aws-credentials@v6
with:
role-to-assume: $
role-session-name: publish-conduit
aws-region: us-east-1
- uses: Constructured/conduit/.github/actions/publish@v3
with:
artifact_types: devblog
conduit_dispatch_token: $
That is most of what onboarding looks like. Additional workflows can request other audience channels: pulse, adr, sprint-report, product-updates, brief, or milestone. One dispatch secret, one AWS role variable, and a small project.json. Done. No scripts, templates, .conduit/state.json, or per-repo publisher to maintain.
Templates — the LLM prompts that shape every artifact — also belong in the consumer rather than the project. They ship in the action and get installed into the runner’s global timbers template directory on every run, with project-side overrides as an opt-in escape hatch. The next cairn unpacks how that works in practice.
The architectural rule: Conduit owns the contract surface; projects own their metadata and workflow configuration. Anything cross-project belongs in the action; anything project-specific belongs in the project repo.
Architectural edges
No architecture is finished when its diagram looks elegant. A few edges in this design are worth naming explicitly.
Provider coupling is lower, but not zero. The action now defaults to a Bedrock cross-region inference profile (global.anthropic.claude-opus-4-7) and routes calls through invoke-llm.sh. That helper is the right seam for any future provider move: update one invocation shape and the action README, not every consumer workflow.
Audience channels are broader than the original trio. The action now accepts pulse, devblog, adr, sprint-report, product-updates, brief, and milestone. release-notes is intentionally rejected with a clear error because the customer-facing channel was renamed to Product Updates.
No reviewer-in-the-loop on artifacts. Conduit publishes the LLM’s output as-is. There is no human review step between draft and publish. For internal-audience artifacts on a small team, that has been fine. For an external-audience surface, it would not be — and changing it would touch the bundle envelope (a “needs review” flag) and the consumer (a draft staging area).
Single-org access setting. The action is hosted in a private repo. For other repos in the same GitHub organization to consume it, the consumer’s actions/permissions/access must be set to organization. One-time setting per consumer-action pair, but invisible in workflow YAML and easy to forget. The next cairn covers the operational details.
These are architectural shape questions. Operational deferred work — Q reviewer support across all repos, multi-cycle review convergence, and review gates for external channels — lives in the next cairn instead.
Summary
- Two halves, narrow seam. A per-project publisher and a single-repo consumer, separated by a versioned bundle envelope and a
repository_dispatchevent. - Bundle envelope is the contract. Everything above and below it can change independently as long as the envelope shape stays stable.
- Per-type windows enable per-type cadence. Daily and weekly channels can read different slices without project-side anchors or state commits.
- Composite action over puller. Project autonomy, isolated failure, asynchronous rollout. Migration to a centralized puller stays available if scale changes.
- Quiet weeks silent, op failures loud. Two failure modes that look identical from the outside need to be operationally distinct in the workflow UI.
- Conduit owns the contract surface; projects own metadata and workflow config. Templates, scripts, dispatch logic, cadence defaults, and numbering live in the consumer;
project.jsonand workflow YAMLs live per-project.
- The bundle envelope is currently
v1, even though the action major isv2. What envelope change would justifybundle_version: 2? What does that migration window look like across N consumers — is it acceptable for them to stay on the old envelope indefinitely, or do we want a deprecation policy? - If we add another artifact type (say, "weekly digest" or "incident report"), what would the surgery cost on the publisher and the consumer? Does the per-type window model scale cleanly, or does it want a richer scheduling shape?
- The action lives in a private repo today. If we ever wanted external consumers (a partner repo, an OSS project of ours), what would change about the contract — and is that change worth doing speculatively, or only when the need shows up?
- The Work That Writes Itself — The first cairn in this trail. Establishes why Conduit exists and what problem it solves before this cairn dives into how it's shaped.
- GitHub Composite Actions — The mechanism that makes the publisher-in-consumer-repo pattern possible. The action's
action.ymlin the conduit repo is the canonical reference. - GitHub repository_dispatch API — The cross-repo event that delivers the bundle from publisher to consumer. The
client_payloadmust be a JSON object, not a stringified one — the most common implementation gotcha. - Timbers — The development ledger Conduit reads from. Without timbers there is no signal to aggregate; understanding timbers' query model clarifies how Conduit's
since_sha..HEADranges work. - Eleventy Data Cascade — How Conduit's per-project rollups (
byProject,feedByProject,crossProjectFeed) are computed. Useful background for anyone reading the consumer-side code. - PostgreSQL — Not used by Conduit, but a useful contrast: a static-site approach to artifact-driven knowledge gives you everything PostgreSQL gives you for this workload (filtering, sorting, search) without a runtime service. The decision to stay static is load-bearing.
- Events All the Way Down — The Strike cairn on event sourcing. Conduit isn't event-sourced, but the same instinct ("the event is the truth, the projection is the readable surface") shows up in the bundle-and-static-site shape.
Generated by Cairns · Agent-powered with Claude