Render Is a Shared Request Budget

A rate limit sounds like a property of an API client. In Strike it is a property of a whole relationship: one OSP-owned Render connection, several callers, every active ECO attached to that connection, and one allowance they all spend.

That distinction changes the architecture conversation. The question is not merely whether the poller stays under two requests per second. It is whether every request path combined can remain inside 10,000 requests per day, recover sensibly from throttling, and still show operators useful field state.

This cairn reflects the request model documented on July 28, 2026, plus later fixes that routed dispatch fetches through the shared polling limiter, moved all connections and missing-instance fallbacks to 600 seconds, made UI detail reads replica-first, corrected restart initialization, stopped the poller retrying 429s in-request, and gave quota exhaustion its own cooldown separate from the breaker. Items identified as planned remain future work.

The Credential Is the Budget Boundary

Render enforces its limits per API credential. Each render_instances row has its own credentials, so each row gets a separate request budget.

The daily quota is tighter than it first appears. Ten thousand requests spread across a day is one request every 8.64 seconds. Two formulas describe the current capacity model:

minimum safe interval (seconds) = requests per cycle × 8.64
maximum ECOs per instance       = interval seconds ÷ 8.64

Polling costs one request per ECO per cycle. A five-second interval consumes 17,280 requests per day for one ECO. At 60 seconds, one ECO consumes 1,440.

Every connection now defaults to 600 seconds. Issue #1430 (landed as PR #1460) made that the create-time default, migrated existing rows, and raised the missing-instance fallback to the same value. A faster interval is still available as a deliberate per-connection override, but it inherits the arithmetic: twelve ECOs at 60 seconds consume 17,280 requests per day and exceed the quota.

A bounded demo is different from steady state. Twelve ECOs polled every minute for three hours cost about 2,160 requests. The hazard is forgetting to restore the longer interval after the demo.

Five Paths Spend the Same Allowance

Five paths can contact Render. Three have meaningful ongoing volume; the queued-ECO drain is normally quiet but intentionally bypasses the polling kill switch, and the admin probe is small but still spends the same credential allowance.

The polling package now has three shared slots, and the GraphQL cap of two brings the known live-fetch total to Render’s documented burst limit of five. That is a cleaner split than the old shape because dispatch-triggered fetches no longer form a third, uncounted burst path: they queue behind the same semaphore as the background poller. The queued-ECO drain remains outside that concurrency cap and outside the per-instance polling kill switch because suppressing it would strand an emergency, but its volume is bounded by emergencies actually waiting on Render. Nothing coordinates all packages through one credential-scoped limiter yet.

This is why capacity cannot be inferred from the worker-pool setting alone. A pool constrains one path. The credential sees all of them.

The Background Cycle Is Linear in ECO Count

The background loop is the steady-state request source:

Filtering is per instance because the credential is the quota unit. For every due instance, Strike fetches each pollable ECO by its exact label, with the page size capped at 2,000. Exact server-side filtering keeps one ECO’s result on one page and avoids silent truncation.

TASKED ECOs are polled to fill the cache but skip change detection and completion. Their tasks may already appear complete before paging has dispatched anyone, so treating those observations as workflow completion would move the domain too early.

Fingerprinting reduces internal event noise. It does not reduce API volume: Strike must fetch the tasks before it can know whether their fingerprint changed. ADR-007 now records that the fingerprint hashes an explicit task-state projection, so adding a field to RenderTask also requires an explicit decision about whether that field should drive render.tasks.changed and polling TTL extension.

Dispatch Buys Immediacy Outside the Cycle

The dispatch trigger exists because longer safe intervals create a visible delay. At a 300-second cadence, waiting for the background loop could leave a freshly dispatched ECO without task data for five minutes.

The per-instance kill switch applies here as well as in the timer loop. The trigger deliberately does not stamp the instance’s last-poll clock: it fetched one ECO, while that clock governs every ECO on the connection. Moving the clock would defer the whole background cycle and could starve sibling ECOs whenever dispatches keep arriving.

Routing this trigger through the shared limiter has landed. It preserves the useful fast refresh without letting dispatch bursts exceed the polling package’s concurrency allowance. The wait is capped at 30 seconds so shutdown and kill-switch changes are not trapped behind an unbounded queue; if the trigger gives up, the next background cycle re-reads the controls and catches the ECO.

UI Reads Are Replica-First

The list and detail screens now have different request behavior:

The old list path stopped selecting Render-reaching fields, and renderTaskCount was removed from the list surface. The detail path reads eco_render_tasks first and reaches Render only when the replica cannot answer: a never-polled ECO, an ECO outside the pollable set, or an unreadable replica row. A genuinely task-free ECO no longer falls through to Render on every ten-second acceptance-window refetch because successful polls write an eco_render_task_polls observation marker.

That removes the UI as a steady-volume Render caller. The cost is freshness, now made explicit with RenderTaskResult.asOf. stale has narrowed to the failure case: the replica could not answer and the live fallback also failed. A populated or observed-empty replica reports stale: false however old it is, so the as-of cue is the age signal.

Two caveats remain. A TASKED ECO is poll-bounded while open because TASKED cache refreshes do not publish TASKS_CHANGED; #1604 tracks publishing when that cache write changes something. The tasks panel’s refresh button also re-reads the replica rather than forcing a live Render read. A thresholded demand refresh is still deferred until it can draw on a shared per-credential budget.

Restart Replays the First Poll

render_tasks_cache is durable. The service state that schedules and compares polls is not: instanceLastPoll, initialized, initOnce, and fingerprints live in in-memory maps.

This happens once per process start: deploys, pod restarts, crash loops, and scale events. ArgoCD image updates make restarts routine.

At current scale the quota cost is negligible. At target scale it is severe. The older second cost has been removed: initialization used to stamp last_task_change_at, which could keep stale DISPATCHED ECOs alive across repeated deploys. It now seeds fingerprints without recording activity. The inactivity countdown is also anchored on COALESCE(last_task_change_at, render_integration_updated_at, created_at), so never-observed ECOs can age out. Per-project fetching later reduces restart cost from one request per ECO to the number of pages needed for each project.

Rate Limiting Fails on the Wrong Timescale

Failure handling is layered, and the poller side is now tuned for the daily quota rather than only for short outages:

Render uses a bare 429 for both per-second throttling and daily exhaustion, without a header that distinguishes “try again shortly” from “come back tomorrow.” The poller therefore treats its next cycle as the retry instead of spending extra in-request attempts against a credential that just refused work.

A single rejected ECO no longer proves the whole credential is exhausted. The cycle counts rate-limit failures per Render instance and stops issuing more requests to that instance after one full worker wave has been refused. Other instances on the same tick continue because each row has its own credential budget. With the current shared worker count, that wave is three requests, so a fully refused worker wave sets the instance aside and its remaining ECOs wait for the next cycle.

The breaker timescale has been fixed too (#1278, PR #1414). The breaker used to trip only on an unbroken run of failures, so a connection that refused some requests and served others never tripped, and its 30-second half-open probe would have spent a trial request every half minute against a quota that stays exhausted for hours. It now trips on either a run of more than five failures or a 50% failure ratio over at least five judged requests in its ten-second window. Rate-limit responses are kept out of the breaker entirely: a 429 means Render is working and declining, not that Render is broken. Quota exhaustion has its own per-credential gate instead. A 429 carrying a Retry-After of a minute or more arms a cooldown clamped between five minutes and six hours; because Render is not documented to send that header, a dense, unbroken run of refusals (at least five, spanning at least ten minutes) arms a flat one-hour cooldown instead. A request during a cooldown fails locally without reaching Render. What remains is expressing the concurrency cap per credential rather than per process, which is the limiter still open under #1268.

The Cache Has Two Different Kinds of Value

render_tasks_cache is a poller-maintained local replica of Render’s current state, plus history about when tasks appeared and disappeared. It is neither a read-through cache nor an event-sourced projection.

That makes “cache” a dangerous shorthand. A projection rebuild must not clear this table. Replay cannot reproduce external observations that were never domain events.

There is also a schema hazard. The ECO projection rebuilds eco_views rows by deleting and rewriting them. An ON DELETE CASCADE relationship from the task cache could silently erase the audit history during replay, so that foreign-key action must remain deliberate.

Refresh is push-based from polling. There is no TTL to invalidate. Cache freshness is bounded by the polling interval, which is why longer quota-safe intervals need an as-of timestamp and narrow demand-refresh policy.

Operator controls apply at two scopes:

The preferred emergency stop is the per-instance switch in the admin UI. For one or a few ECOs, set polling_stopped_at. The last resort is disabling polling globally and restarting the deployment.

The Honest Ceiling Is About Sixty-Nine ECOs

The current design is linear: one request per ECO per cycle. At ten minutes, one credential supports roughly 69 steady-state ECOs before the daily allowance is consumed. At five minutes it supports roughly 34, and at one minute it supports about six.

What remains open:

1. Fetch per Render project with pagination instead of once per ECO (#1270, deferred while Render lacks multi-value label_name filtering).
2. Publish TASKS_CHANGED for TASKED cache writes when the replica changes (#1604).
3. Coordinate every caller through one per-credential limiter so the cap can differ by environment (#1268).

A thresholded demand refresh for stale detail reads is recorded as deferred in the ADR-007 amendment and has no issue of its own.

The landed fixes make the current linear design safer. Per-project fetching changes its shape: cost follows project pages and task counts rather than ECO count. That is the change that lifts the 69-ECO ceiling at the current 600-second default.

That correlation question is now settled. ADR-010 makes the tenant-formatted ECO job ID, carried as an exact case-sensitive Render General label, the only definition of task membership; subsector is demoted to a literal ECO constant that satisfies Render’s required categorization field. The label stops Strike consuming a customer-owned taxonomy field and keeps exact server-side filtering, so a per-project design can treat the correlation field as fixed.

The operational rule until then is plain: treat every Render interval as a capacity decision, every live fetch as shared-budget spending, and every kill switch as part of the reliability model.

Discussion Prompts

1. What staleness window can the ECO list and detail screen tolerate before a demand refresh is worth spending shared quota?
2. Should the admin UI show estimated daily requests from the current interval and pollable ECO count?
3. At what connection size does per-project pagination become required rather than planned?

References

1. docs/architecture/render-integration-request-model.md in Osprey Strike — The source architecture review covering callers, quota math, cache behavior, restart effects, controls, and planned work.
2. docs/decisions/ADR-007-render-task-cache.md in Osprey Strike — Defines the cache as a poller-maintained replica with non-replayable audit history.
3. Osprey Strike Render polling package — Implements the background loop, per-instance scheduling, fingerprints, cache persistence, and completion checks.
4. Osprey Strike Render integration package — Implements task creation and the immediate post-dispatch fetch path.