The Six-Month Problem

You deployed an AI system. It passed every test before launch. Six months on it is confidently wrong about something it was right about on day one, a customer acts on the wrong answer, and you find out when the complaint arrives.

Air Canada already lived this

Its chatbot gave a traveller the wrong answer on a bereavement fare. The airline argued that the chatbot was a "separate legal entity" responsible for its own words. The tribunal rejected that and ordered the airline to pay C$812.02 (Moffatt v. Air Canada, 2024 BCCRT 149). The system was the airline's, and so was the error.

The numbers behind it

• Models fold. In a controlled test, one model gave up a correct answer under scripted pushback in 75 percent of eligible runs. The decision changed while the wording stayed calm. Three larger models held every answer, 0 of 72 each, so the test has to run per model.
• The system does not catch itself. In one operator's record, 11 incidents were caught by the human operator and 0 by the model's own self-audit (objections register).
• The goal is named in law, and finance runs the practice. EU law asks for post-market monitoring that lets a provider "evaluate the continuous compliance of AI systems" (AI Act Art 72(2)). Financial entities already test live production systems with independent parties, at least yearly (DORA Art 24(4), 24(6), 26(2)).

What finds the error first

A test of the deployed system, not the bare model, run on every change and on a schedule, graded by someone other than the system, with a record anyone can rerun. That is continuous conformity. Monitoring shows the error after a user has acted on it. A scheduled test finds it before.

The 36-requirement reference draft is public: Continuous Conformity for Deployed AI Systems, working draft 0.3. It is a reference text, not a standard, and no committee has seen it. Comment by clause number on the repository issues (title "CC-x.y: ...") or on r/MachineBehavior.