Osprey Strike API Architecture: The Developer Map

architecturedomaindevops

The shape in one pass

The Osprey Strike API is deliberately not a constellation of microservices. It is a Go API server with clear internal seams: GraphQL at the edge, CQRS inside the application layer, PostgreSQL as the durable substrate, and adapters around external systems. That shape is pragmatic. Emergency callout software needs traceability, recoverable workflows, and boring operations more than it needs service-count theater.

At the highest level, the server looks like this:

flowchart TD
    UI[Web UI / GraphQL clients] --> GQL[GraphQL API]
    GQL --> CMD[Watermill Command Bus]
    CMD --> AGG[Domain Aggregates]
    AGG --> STORE[(Event Store)]
    STORE --> OUTBOX[(Transactional Outbox)]
    OUTBOX --> EVT[Watermill Event Bus]
    EVT --> PROJ[Projection Handlers]
    EVT --> PM[Process Managers]
    EVT --> WS[WebSocket Publisher]
    PROJ --> VIEWS[(Read Models)]
    PM --> STATE[(Process State)]
    PM --> RENDER[Render Networks]
    PM --> TWILIO[Twilio]
    VIEWS --> GQL
    WS --> UI

There are several older cairns that explain pieces of this design: The Shape of the System covers the broader repo and runtime shape, Events All the Way Down explains event sourcing and CQRS, Projections Are Product Decisions explains read models, and The ECO Lifecycle walks the workflow from outage to completion. This cairn is the developer map that ties those ideas to the API server.

Where truth lives

The event store is the source of domain truth. ECOs and pager runs are event-sourced aggregates: a command asks for a change, the aggregate validates whether that change is allowed, and the system appends immutable events. Current state is rebuilt by replaying those events in order.

That matters because ECO work is audit-heavy. A row that says COMPLETED is not enough when someone later asks who created the callout, when the contractor was paged, whether the first contact declined, when the field task moved, or why the ECO reopened after apparent completion. Strike needs the sequence.

The architecture guide’s event table captures the shape: aggregate ID, aggregate type, event type, version, payload, metadata, schema version, and timestamp. The important field for correctness is the aggregate version. A unique (aggregate_id, version) constraint gives the write side optimistic concurrency control: two writers cannot both believe they wrote version 3 of the same aggregate.

sequenceDiagram
    participant Resolver as GraphQL Resolver
    participant Handler as Command Handler
    participant Aggregate as ECO Aggregate
    participant Store as Event Store

    Resolver->>Handler: CreateECOCommand
    Handler->>Store: Load events for ECO ID
    Store-->>Handler: Existing stream or empty stream
    Handler->>Aggregate: Replay, then Handle(command)
    Aggregate-->>Handler: ECOCreatedEvent
    Handler->>Store: Append event at expected version

How changes move

The command bus and event bus are not there for decoration. They make the flow explicit. A GraphQL mutation sends a command. A command handler loads the relevant aggregate, asks it to handle the command, and appends the resulting events. Event handlers then update projections, notify browsers, or kick off process managers.

The reliability hinge is the transactional outbox. Saving an event and publishing it are separate operations, and one can fail after the other succeeds. Strike avoids the classic lost-event problem by writing the event and an outbox row in the same database transaction. A publisher polls unpublished outbox rows, publishes them to Watermill, and marks them published afterward.

flowchart LR
    A[Append domain event] --> TX[(Same DB transaction)]
    B[Insert outbox row] --> TX
    TX --> P[Outbox publisher]
    P --> E[Event bus]
    E --> R[Projection updates]
    E --> W[WebSocket notifications]
    E --> M[Process managers]

The result is at-least-once delivery inside the process. Handlers still have to be idempotent because duplicate delivery is possible, but a committed event cannot disappear simply because the publisher crashed at the wrong moment.

Read models are product surfaces

The API does not ask users to read event streams. The query side is built from projections: eco_views, pager views, tenant views, and other read models shaped for GraphQL responses and subscriptions. This is where raw facts become operational language.

For ECOs, that distinction is not academic. Render has field-work statuses like blueprinted, allocated, releasable, released, jeopardy, and completed. Those are useful to the OSP supervisor and field technician. A NOC operator needs a smaller vocabulary: pending, assigned, in progress, blocked, complete. The projection layer is where that translation belongs.

The same event stream can support several different readers:

Reader	What they need
NOC operator	Current ECO status, latest activity, whether field work is moving
Support engineer	Render integration status, last poll, failed external call detail
Dispatcher	Pager outcome, attempts, accepted contact, timeout state
Billing or audit	Stable job identity, timeline, completion evidence
Background worker	Whether polling should continue or stop

That is why Projections Are Product Decisions treats read models as more than caches. In Strike, the projection often decides whether the system feels trustworthy during an incident.

Process managers make time visible

Some workflows cannot complete in one request. Pager dispatch waits for Twilio callbacks and deadlines. Render integration waits for external APIs and later polling observations. ECO completion has a grace period because field work can reopen after it appears done.

Strike’s process managers keep that coordination state in PostgreSQL tables. The pager process state, for example, records a pager run ID, ECO ID, status, start time, deadline, last event time, and version. A background worker can query overdue rows; a webhook handler can extend the deadline; optimistic locking handles races between the two.

That is less glamorous than adopting a saga framework, but it is easier to operate. When a callout is stuck, a developer can inspect ordinary rows and ask direct questions:

SELECT *
FROM pager_process_state
WHERE eco_id = '...';

Tenant context is not optional

Every ECO sits between two organizations. The NOC creates the callout; the OSP works it. Strike models both tenants explicitly because each side has different visibility, credentials, users, and operational responsibilities.

The tenant model has a single tenants table with type-discriminated NOC and OSP rows, a noc_osp_links relationship table, user memberships, and tenant-scoped resources. Render instances, pager lists, and job sequences belong to OSP tenants. ECO visibility depends on both the NOC tenant that created the record and the OSP tenant assigned to work it.

Authenticated requests therefore carry two pieces of context: who the user is, and which tenant they are acting inside. A system admin can manage across tenants, but ordinary work is tenant-scoped. This prevents a tempting but dangerous shortcut: treating a user identity as enough context to decide what an ECO means.

flowchart TD
    User[Authenticated user] --> Memberships[User tenant memberships]
    Header[X-Tenant-ID / active tenant] --> Auth[Auth context]
    Memberships --> Auth
    Auth --> Resolver[GraphQL resolver]
    Resolver --> Policy[Visibility and role checks]
    Policy --> ECO[ECO with NOC tenant + OSP tenant]

See Two Tenants, One ECO for the deeper argument. The developer-map version is shorter: do not write API code that ignores active tenant context just because the user is authenticated.

External adapters are boundaries

Render and Twilio are not domain models. They are external systems with their own vocabularies, failure modes, credentials, retries, and operational habits. The API server treats them as adapters around the domain rather than as the domain itself.

Render integration starts when an ECO is created. Strike creates an investigation task in the OSP tenant’s Render instance and uses the Render subsector name, formatted as ECO-{job_id}, to group all field tasks for that callout. Later polling queries tasks by that subsector, fingerprints changes to avoid noisy updates, and translates field status into NOC-facing state.

Twilio powers pager dispatch. The pager process manager starts the callout, records callbacks, handles timeouts, and emits pager-run events. The important thing is not that Twilio made a call. The important thing is that Strike records the callout attempt as part of the ECO’s operational history.

Secrets follow the same boundary discipline. Render client secrets are encrypted at rest, decrypted through the server’s encryptor interface, and cached through the client factory. The adapter owns OAuth2 mechanics; the domain does not.

How to trace one ECO

A developer debugging Strike should trace the system in the same order the work moves:

1. Start at the GraphQL operation. Confirm the resolver is building the right command with the right auth and tenant context.
2. Follow the command handler. Confirm it loads the aggregate, validates the expected version, and appends events.
3. Inspect the event store and outbox. Confirm the fact exists and was published.
4. Inspect projections. Confirm the read model received the event and shaped the expected UI state.
5. Inspect process state. For pager, Render, or polling questions, check the state table before assuming the aggregate is wrong.
6. Inspect adapter logs and external IDs. Render task IDs, subsector names, Twilio callback identifiers, and integration statuses tell you which boundary failed.

This order keeps debugging honest. If the event was never appended, it is a command-side problem. If the event exists but the UI is stale, it is probably projection or subscription. If the projection is right but external work is stuck, it is probably process-manager or adapter state. Mixing those layers too early wastes time.

What to take away

The API architecture is conservative because the work is not. Emergency callouts involve live outages, field crews, contractor handoffs, billing records, and later questions about what happened. The system needs to be easy to reason about when the happy path is already gone.

The core choices all serve that pressure:

1. Event sourcing preserves the operational record. ECO and pager history stays inspectable instead of being overwritten by the latest status.
2. CQRS separates correctness from usefulness. Aggregates validate changes; projections speak the language of operators, support, and workers.
3. The transactional outbox protects publication. Events saved in PostgreSQL do not disappear because an in-process publisher hiccuped.
4. Process managers keep long-running work recoverable. Deadlines, callbacks, retries, and grace periods live in ordinary tables that can be inspected.
5. Tenant context is part of every API decision. NOC and OSP responsibilities are different, and the server has to preserve that boundary.
6. External systems stay outside the domain. Render and Twilio are integrated carefully, but their vocabulary does not become the whole product vocabulary.

Discussion Prompts

• When debugging a stuck ECO, which layer would you inspect first: event store, projection, process state, or external adapter logs? What evidence would move you to the next layer?
• Where should Strike draw the line between domain events and operational metadata from adapters like Render polling?
• If another OSP field platform were added beside Render, which parts of this architecture should stay stable and which should become adapter-specific?

References

1. The Shape of the System — The broader Osprey Strike system map this API architecture fits inside.
2. Events All the Way Down — Event sourcing and CQRS in Strike, including the migration and escape hatch.
3. The ECO Lifecycle — The domain workflow that exercises the API architecture end to end.
4. Two Tenants, One ECO — Why tenant context has to distinguish the NOC that creates the ECO from the OSP that works it.
5. Projections Are Product Decisions — The deeper explanation of why read models are operational product surfaces.
6. Boundary Objects for Operational Software — How ECO identity, status, and timelines preserve meaning across organizational boundaries.
7. Watermill — The Go library Strike uses for command and event routing.
8. Transactional Outbox Pattern — Pattern reference for reliably publishing messages after database writes.