For Divvent’s own team. It is generated from the operations document in our repository, which is the one to edit. If you use Divvent, the help centre is for you.
Runbook
What an operator actually does, today, with the software that exists. Every step is a command or a console action; the ones a person has to do by hand in somebody else's dashboard are marked BY HAND and there is a reason beside each one.
Where a step is not built yet it says so rather than being left out — a runbook with a gap in it is a runbook somebody improvises around at 2am.
Written against the code on this branch. If a command here does not exist, that is a defect in this file and not in your memory: divvent with no arguments prints the real list.
The screens — what each portal and console screen is for, its one primary action, and the words used on it — are docs/SCREENS.md. This file names them as they are there: a customer asks on Changes, releases on Releases, and an operator decides on Requests and watches a platform update on Rollout.
---
Sections
- 1. Onboarding a tenant, end to end
- 2. A change request in managed mode
- 3. Rolling back
- 3a. The nightly dump, and the two windows
- 3b. A customer's restore, and when one needs an operator
- 3c. The restore drill
- 3d. Off the account: the nightly copy, and the control plane's export
- 3e. Moving a company's data to where it should live
- 4. Break-glass
- 4a. A portal account's second factors: the operator's reset
- 5. Trials, and seeding one from production
- 6. Destroying a tenant environment
- 6a. Is a deployment actually serving?
- 6b. Agent access, and the one thing not to do about egress
- 6c. The §2.1 round against staging (acceptance.yml)
- 6d. The dispatch App: a deploy that starts in seconds — BY HAND
- 6e. The command signing key: one per control plane, and rotating it
- 6f. Break-glass's second factor: what Access must require — BY HAND
- 6g. Re-capturing the AI recordings (recordings.yml)
- 6h. Tenant builds in CI: one leg per hash, as a user of its own
- 6i. Customer deploys on Cloudflare: the deployer Container
- 6j. Live deploy stages
- 6k. Proactive monitoring: incidents, and what to do in a provider outage
- 6l. Load runs
- 6m. Splitting a partitioned application's data
- 6n. The outside watch: nothing else watches the watcher
- 6o. Issue triage and GitHub Issues — BY HAND, once per environment
- 6p. A production error, mapped back to source
- 6q. The fleet update at scale: rings, lanes, and one build per model
- 6r. A control-plane database filling up
- 6s. An enforcement notice from our model provider
- 6t. Web push — BY HAND, once per environment
- 7. What is not built
- 8. Before any of this works
- 9. The first production release — BY HAND, once, in this order