Osprey Strike Twilio Deploy Runbook

operationsdeploymentsecurity

The pager deploy path is unusual inside Osprey Strike. The Go API, web app, mock server, and Kubernetes overlays follow the normal merge-to-build-to-ArgoCD route. The Twilio pager artifacts are different: parts of the running behavior live inside Twilio Studio and Twilio Serverless, while the public webhook front door lives in Cloudflare.

That split is why PR 1149 could merge code and still require a manual Twilio deployment, and why PR 1156 had to exist after the manual deploy exposed a Studio schema validation failure. A Git commit changed the source of truth, but Twilio did not receive a working Flow until someone rendered, upserted, fixed, and redeployed the external artifact.

Until issue 1131 automates Twilio Serverless and, ideally, the adjacent Worker path, this is an operational runbook. It is deliberately concrete.

What is manual

The manual Twilio pager deployment has three relevant repo surfaces.

Surface	Repo path	Hosted where	Deploy behavior
Studio Flow	infrastructure/twilio/flow-template.json	Twilio Studio	Render and upsert with infrastructure/twilio/twilio-flow.sh.
Serverless Function	infrastructure/twilio/serverless/functions/pager.protected.js	Twilio Serverless	Paste and deploy in Console, or deploy with Twilio CLI/Serverless Toolkit.
Webhook Worker	infrastructure/twilio/worker/ and infrastructure/opentofu/cf-worker/	Cloudflare Workers	Deploy through OpenTofu/Wrangler workflow, not ArgoCD.

The Kubernetes/API side is still normal GitOps. In the sandbox-mode API config, for example, PAGER_PROVIDER is set to twilio, TWILIO_FUNCTION_URL points at https://redops-dev-2971.twil.io/pager, TWILIO_FLOW_SID points at FW0815c9370169ed892180785acf5c886c, and TWILIO_FROM_NUMBER supplies the outbound caller ID. When that config changes and lands in the cluster, ArgoCD handles it.

The Studio Flow and Serverless Function are not in that pipeline. They must be pushed to Twilio itself.

The Cloudflare Worker is related but not the same deploy. The Flow’s HTTP widgets post pager status callbacks to a webhook host. The Worker validates Twilio’s signature at that public host, derives the backend environment from the hostname, and forwards to <env>-gw.constructured.com with Cloudflare Access service credentials. That means a Flow deploy must preserve the correct public Worker host, but it does not redeploy the Worker.

What to load

Treat the environment variables in two groups.

Credential variables identify and authenticate to the target Twilio account:

Variable	Meaning	Handling
TWILIO_ACCOUNT_SID	Twilio account SID for Studio API calls.	Sensitive enough to keep out of random logs; must match the token.
TWILIO_AUTH_TOKEN	Twilio auth token for API auth and signature validation.	Secret. Do not paste into shell history or PR comments.

Twilio also exposes test credentials in the Console. Those are useful for constrained console or API test-mode checks, but they are not a replacement for the live account credentials needed to publish Studio Flow or Serverless Function changes.

Deployment parameters describe the artifact you are about to render or publish:

Variable	Meaning	Handling
FLOW_NAME	Friendly name used by upsert to find an existing Studio Flow.	Read from the live Flow; do not invent it.
FLOW_HOST	Host used to build https://${FLOW_HOST}/webhooks.	Read from the live Flow definition; docs contain old examples.
FUNCTION_URL	Twilio Serverless Function URL used by the Flow’s run-function widget.	Confirm for the target account/environment.
TWILIO_FLOW_SID	Optional direct Flow SID for get, get-definition, and upsert.	Use it when identifying an existing Flow.

The practical load order for a human shell should be:

1. Let direnv load repo-local development defaults and .envrc.local if you use it.
2. Put stable Twilio credentials in ~/.jrm/twilio.env when using twilio-flow.sh; the script sources that file automatically.
3. Put per-deploy values inline on the command line: FLOW_NAME=... FLOW_HOST=... FUNCTION_URL=... ./twilio-flow.sh render.

Keep secrets out of visible command history. The safest pattern is for ~/.jrm/twilio.env to contain only credentials:

export TWILIO_ACCOUNT_SID="AC..."
export TWILIO_AUTH_TOKEN="..."

Then keep target parameters explicit and non-secret at invocation time. If you need to override credentials for one command, do not paste the token into a Slack-visible terminal transcript or a saved shell history entry. Use a temporary env file, a password-manager shell injection, or a subshell that disables history before exporting the secret.

What not to guess

Before rendering or publishing, identify the live Flow from the target Twilio account.

Start with the Flow SID when you have one from the deployed API config or the Twilio Console:

cd infrastructure/twilio

TWILIO_FLOW_SID="FW..." ./twilio-flow.sh get | jq -r '.friendly_name'

That output is the FLOW_NAME to use for render and upsert. Copy it exactly.

Then read the current webhook host from the live Flow definition:

TWILIO_FLOW_SID="FW..." ./twilio-flow.sh get-definition \
  | jq -r '.. | .url? // empty' \
  | grep -i webhooks \
  | sort -u

The matching host becomes FLOW_HOST unless you deliberately use BASE_URL to override the whole webhook base. Prefer FLOW_HOST for ordinary deploys because it makes the rendered base URL predictable:

FLOW_HOST=webhooks-main.constructured.com
BASE_URL=https://webhooks-main.constructured.com/webhooks

This read-before-write step matters because the docs still carry historical domain examples. TWILIO_CF.md describes redshifted.ai and redshifted.dev patterns. The current OpenTofu cf-worker configuration lists webhooks-demo.constructured.com, webhooks-main.constructured.com, and personal webhooks-<env>.constructured.com hosts. Other docs and examples mention UI hosts such as osprey-main.constructured.ai, backend gateway hosts such as main-gw.constructured.com, and older backend examples under redshifted.dev.

Those names are not interchangeable. Twilio calls the public Worker host. The Worker forwards to the backend gateway. The Strike UI host is neither of those.

How to deploy the Flow

Work from the Osprey Strike repo:

cd infrastructure/twilio

First, preview the rendered Flow. Use the exact friendly name and webhook host read from the live Flow:

FLOW_NAME="<friendly name from get>" \
FLOW_HOST="<webhook host from get-definition>" \
FUNCTION_URL="https://redops-dev-2971.twil.io/pager" \
  ./twilio-flow.sh render | jq .

Review the rendered JSON for three things:

1. All pager callback URLs use the expected Worker base, such as https://webhooks-main.constructured.com/webhooks/pager/accepted.
2. The call_next_agent_function widget points at the expected FUNCTION_URL.
3. call_agent.properties.timeout is an integer, not a Liquid string.

Then publish:

FLOW_NAME="<friendly name from get>" \
FLOW_HOST="<webhook host from get-definition>" \
FUNCTION_URL="https://redops-dev-2971.twil.io/pager" \
  ./twilio-flow.sh upsert

upsert renders the compact Flow definition and posts it to Twilio Studio with Status=published. If it finds the Flow by TWILIO_FLOW_SID or by FLOW_NAME, it updates the existing Flow. If it cannot find a matching friendly name and no SID is supplied, it creates a new Flow.

For a live target, confirm the target account and Flow SID with the operator who owns the Twilio account before this step. The repo currently shows redops-dev-2971 in sandbox-mode config, but the PR 1149 deployment note explicitly called the account/environment mapping muddled enough to require human confirmation.

How to deploy the Function

The Function source lives at:

infrastructure/twilio/serverless/functions/pager.protected.js

It is the controller for sequential pager calls. On first entry, Strike posts the contact list and callout context. On each no-answer, decline, failure, or loop transition, the Studio Flow re-enters the Function through the call_next_agent_function widget. The Function chooses the next contact, starts a new Studio Flow execution, loops the list up to maxAttempts, and dials escalationPhone once when configured.

There is not yet repo deploy tooling for this Function. Use one of two manual paths.

Console path:

1. Open the target Twilio account.
2. In the left navigation, open Functions & assets, then Services.
3. Open the service that owns the pager Function. In the captured development account, that service is redops-dev.
4. In the Function editor, expand Functions and open /pager.
5. Paste the current pager.protected.js source.
6. Click Save, then Deploy All to publish a new active Build.

CLI path:

twilio serverless:deploy --environment <exact-env>

Pin the environment explicitly. A typo can create or deploy to an unintended environment. The deployment comment on PR 1149 also called out TWILIO_SERVERLESS_API_CONCURRENCY=1; keep that setting for Function deploys until the automated path in issue 1131 makes this reproducible.

Function deploys create and activate a new Twilio Build. After deploying, confirm the active Function URL still matches the FUNCTION_URL used in the Flow render.

How to confirm it worked

For the Flow, confirm Twilio accepted and published the definition:

TWILIO_FLOW_SID="FW..." ./twilio-flow.sh get | jq '{sid, friendly_name, status, revision, valid}'

Then inspect the published definition:

TWILIO_FLOW_SID="FW..." ./twilio-flow.sh get-definition \
  | jq '.states[] | select(.name == "call_agent" or .name == "call_next_agent_function")'

Check that:

• call_agent.properties.timeout is an integer.
• call_next_agent_function.properties.url is the intended Function URL.
• The Function parameters include flowSid, fromNumber, message, contacts, index, ecoId, pagerRunId, ringTimeout, maxAttempts, escalationPhone, loop, and escalated.

For the Function, confirm the Twilio Console shows the new active Build and that /pager resolves under the expected Twilio domain. If using CLI, inspect the deploy output and the Console; do not rely only on local command success.

Then run a live smoke test against a NOC with an active, token-bearing Twilio instance:

1. Set System Configuration -> Voice callout knobs for a controlled test: short ring window if supported by the deployed Flow, maxAttempts=1, and a known escalation number.
2. Start an ECO pager run with test contacts.
3. Confirm the first contact is called from the expected Twilio number.
4. Decline or let the call fail and watch the next Function execution.
5. Confirm maxAttempts=1 reaches terminal exhaustion after one pass.
6. Confirm escalation dials once when configured.
7. Confirm permanent send/call failure skips to the next contact rather than wedging the run.

The token-bearing instance detail matters. PR 1149’s deployment note recorded that the callout policy only takes effect on the live database-instance path. If Strike falls back to env-level provider config, the policy values are not present even though Twilio may still place calls.

The 81022 failure mode

The sharpest edge from the PR 1149 deploy was a Studio validation error:

code 81022: string found, integer expected at #/states/1/properties/timeout

The failing definition set the make-outgoing-call-v2 widget timeout like this:

"timeout": "{{flow.data.ringTimeout | default: 60}}"

Twilio rejected it before Liquid could render because that widget field is typed as an integer. Other fields in the same widget accept strings, so nearby string values do not prove timeout can be templated. PR 1156 fixes the deploy blocker by restoring:

"timeout": 60

That means true per-instance ring timeout is not currently implemented through the Studio widget. The remaining ringTimeout plumbing in Go, Flow parameters, and Function event parsing is inert until the implementation changes. Two realistic follow-ups are a deploy-time integer substitution into the Flow template, or moving call placement into the Function where timeout can be set per API request.

N-loop and escalation are different. They live in pager.protected.js and are not blocked by the Studio widget timeout schema.

Which account

Do not treat redops-dev-2971 as a universal answer. It is a concrete clue in the repo, not the full account map.

The current sandbox-mode API config says:

Setting	Value
TWILIO_FUNCTION_URL	https://redops-dev-2971.twil.io/pager
TWILIO_FLOW_SID	FW0815c9370169ed892180785acf5c886c
TWILIO_FROM_NUMBER	+15015047082

That likely describes the sandbox target. It does not by itself prove which account is the live production target, and it does not tell you which friendly name upsert must use. The safe sequence is:

1. Confirm whether the deploy target is test, sandbox, demo, main, or live.
2. Confirm the Twilio account credentials belong to that target.
3. Confirm the Flow SID from the API config, Twilio Console, or operator handoff.
4. Read FLOW_NAME and FLOW_HOST from that Flow.
5. Render, review, then upsert.

For SMS specifically, the newer System Configuration flow and the earlier Twilio SMS Global Settings cairn use webhooks-<env>.constructured.com as the public Worker callback host. That same cairn now records where Twilio Messaging Service Opt-out Management lives for STOP, START, UNSTOP, and HELP behavior. Pager Flow callbacks should follow the live Flow and Worker configuration, not an old example copied from redshifted.ai or redshifted.dev documentation.

The operator checklist

Use this condensed sequence for a manual pager deploy:

1. Pull the current Osprey Strike source and inspect the Flow diff, Function diff, and PR notes.
2. Confirm the target Twilio account and environment with Noam or Bob for live-impacting deploys.
3. Load TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN from a safe local secret source.
4. Read FLOW_NAME from the target Flow with TWILIO_FLOW_SID=... ./twilio-flow.sh get.
5. Read FLOW_HOST from the target Flow with get-definition; do not guess from docs.
6. Render with explicit FLOW_NAME, FLOW_HOST, and FUNCTION_URL; review the generated URLs and integer-typed fields.
7. Upsert the Flow and confirm Twilio reports it published.
8. Deploy pager.protected.js to the matching Twilio Serverless environment.
9. Confirm the active Function URL and Flow definition agree.
10. Run a live smoke test through an active token-bearing instance.
11. Record the deployed Flow SID, Function URL, Twilio account/environment, commit SHA, and smoke-test result in the PR or deployment thread.

1. The pager Twilio artifacts are manual today. The Flow, Function, and Worker are repo-backed but not ArgoCD-deployed.
2. Credentials and parameters are different things. Keep `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` in a safe local secret source; keep `FLOW_NAME`, `FLOW_HOST`, and `FUNCTION_URL` explicit per deploy.
3. Read the live Flow. A wrong `FLOW_NAME` creates a duplicate Flow, and the webhook host is ambiguous across older docs.
4. Preview before publish. `render | jq .` should show the expected Worker URLs, Function URL, and integer `timeout` field before `upsert` publishes.
5. Studio type validation is real. Liquid in the integer `timeout` field fails with 81022; PR 1156 documents the fix.
6. Confirm the account map. `redops-dev-2971` and the sandbox Flow SID are evidence for the sandbox path, not blanket permission to deploy live.

• Should issue 1131 automate only the Serverless Function first, or should the Flow render/upsert path be included in the same workflow?
• Where should the authoritative test/demo/main Twilio account map live so deploys no longer depend on Slack memory?
• Should the Flow script require `TWILIO_FLOW_SID` for updates to prevent accidental duplicate Flow creation?

1. Osprey Strike PR 1149 - The configurable callout policy PR and deployment comment that recorded the manual Flow and Function deploy procedure.
2. Osprey Strike PR 1156 - The corrective PR for Twilio Studio error 81022 on the integer `timeout` field.
3. Osprey Strike issue 1131 - The tracked follow-up to automate Twilio Serverless Function deploys.
4. Twilio SMS Global Settings - The related runbook for deployment-global SMS callback configuration and Worker host derivation.
5. NOC-Scoped Does Not Mean NOC-Owned - The ownership model for Strike-managed telephony plumbing.