Skip to content
Machine Behavior

Operations work done as software

The principle

SRE = operational knowledge + software engineering practices. The ops knowledge comes first, because nobody can automate work they do not understand. On top of it go automation in place of manual runbooks, tools in place of one-off scripts, SLOs in place of gut feel, and blameless postmortems. In Google's chapter on automation, the gains come in this order: consistency first, then a platform others can extend, then faster repair and faster action, and time saved last.

Source: the formula in the manifesto; Google's The Evolution of Automation at Google.

On this platform

Each page an operator would otherwise update by hand is built from a source in git, and each build checks its input before it writes anything.

OutputSourceGeneratorWhat the build checks
Service catalogservices/*.ymlscripts/build_inside.pyUnknown or missing fields, values outside the allowed set, docs links to missing pages, unknown dependencies
Status pageincidents/*.yml, the gate record, the deploy feedscripts/build_inside.pyImpact values, stages in order, services that exist in the catalog, ticket numbers
Docsscripts/docs/<space>/*.md and the knowledge vaultscripts/build_docs.py, scripts/import_vault_docs.pyUnknown doc links, malformed dates, unknown types, decision records that disagree, principle state
MapThe three sites and Substackscripts/crawl_map.pyKeeps the committed snapshot when a crawl comes back smaller (ADR-0004)
BoardGitHub Issuesscripts/board_snapshot.pyShows only issues with a type label
ChangelogClosed issues and commitsscripts/build_changelog.py
Top bar, breadcrumbs, footersite/nav.ymlscripts/site_chrome.pyOrphan pages, sitemap and llms.txt against the tree (Site navigation and chrome)
Grafana dashboardsgrafana/build.py in agent-observabilityThe same fileA rebuild in a clean directory gave identical JSON on 2026-10-09 (Dashboards as code)
Alert rulesprometheus/rules/*.yml in agent-observabilitypromtool21 test cases (Alerts and SLOs)

The YAML reader is scripts/mini_yaml.py, a strict subset in the standard library, so the builds need nothing installed beyond Python.

State

In place. For the sites, the catalog, the status page, the docs, the map and the dashboards, the source is in git and a generator writes the output. Two exceptions are named on other pages: the builds run on the author's machine and CI does not repeat them (Release engineering), and the observability stack is copied to its server by hand (Eliminate toil).

Services that name it

In a company of 10 to 200 people

  • Before automating a task, have the person who does it by hand write it down as a runbook. The runbook is the specification for the automation.
  • Prefer one generator per kind of page or config, with a schema check, over a wiki that people update by hand.
  • Fail the build on bad input. A catalog that accepts an unknown field keeps a typo for a year.
  • Judge automation by the consistency it gives first and by the hours it saves second.

Open work

Built from scripts/docs by build_docs.py.