Keep your AI working as it should, every month
AI systems drift: models update, data changes, and a check that once worked can quietly stop. We monitor the AI you rely on, rerun its evaluation cases, check that controls still hold after every change, track what it costs, and report each month in plain English.
Service 12
No 17040048
Retained
From £1,200 a month
Monthly, rolling
Depends on how many systems are covered. Three-month minimum, then rolling with a month's notice. Change work is quoted per request or covered by a larger plan.
Who it is for
- Businesses running AI systems we built, who want them looked after properly.
- Teams with AI in production that nobody is formally watching.
- Companies that need evidence their AI still behaves as approved, for clients or insurers.
What the work covers
Why AI drifts
An AI system can change without anyone touching it. Model providers update the models behind their services, sometimes with little notice. Your data changes: new products, new prices, old documents nobody removed. And people change things on purpose, such as a prompt edit, a new integration or a code release, without rerunning the checks that approved the system in the first place. Each change can shift behaviour in ways nobody sees until a customer does. Monitoring exists to notice these shifts early, on your cases, rather than after a complaint.
Evaluation runs on schedule and after change
We rerun the test cases agreed when the system was built, or agree them with you if they were never written down. Each case has an expected outcome: the right answer, a refusal, a handover to a person. Runs happen on a schedule and after every change we know about. When a case fails, we look at why, whether the model changed, a source went stale or a rule stopped holding, and tell you what we found before customers run into it.
Change checks with Slipmark
A green check is only useful if it checked the right thing. In our own nightly pipeline of more than 20 agents, a gate reported green for months while inspecting the wrong branch. We built Slipmark, our open-source GitHub Action, so an approval is bound to a hash of exactly what was inspected, and it fails if what runs is different. Where your systems are built through code changes, we use it to confirm that what was approved is still what is deployed.
Cost and usage tracking
AI costs move with usage, model choice and how prompts are written, and they can rise quietly. We track what each system in scope costs to run each month, using the billing and usage data from your providers, and compare it with the month before. Unexpected jumps are flagged early, whether they come from a traffic spike, a loop in an agent, or a change that made every request longer. You see the cost alongside what the system did, so spend can be judged against use.
Alerts and the monthly report
Alerts are agreed with you during onboarding: what counts as a problem, who is told and how. We keep them few enough to be taken seriously. Each month you receive a short report in plain English: what ran, what changed, what failed, what we did about it and what we suggest next. Fixes are proposed and quoted per request unless your plan covers change work. The reports build into a record you can show clients or insurers that the AI still behaves as approved.
What we watch
These are illustrations of the kind of work, not client results.
Evaluation runs
Your agreed test cases rerun on a schedule and after every change, with failures flagged before customers see them. Example only.
Change checks
Gates that confirm what was approved is still what runs after a model, prompt or code change. Example only.
Cost and usage tracking
What each system costs to run each month, and early warning when usage jumps. Example only.
What you get
We use the monitoring your systems already have where possible, and Slipmark, our open-source verification gate, where code changes need checking.
- 01Monitoring of the systems in scope, with alerts agreed with you.
- 02Scheduled evaluation runs against your test cases.
- 03Change checks and drift detection after model, prompt or code updates.
- 04Cost and usage tracking.
- 05A short monthly report: what ran, what changed, what failed and what we did.
What changes
- Someone is formally watching the AI systems you rely on.
- Failures on your test cases are found by us, not first reported by a customer.
- Evidence, month by month, that your AI still behaves as approved.
- A clear view of what each system costs to run and when that changes.
How it runs
- From £1,200 a month, depending on how many systems are covered.
- Evaluation runs on schedule and after every change.
- A short monthly report in plain English.
- Business hours, not 24/7 incident response, unless agreed separately.
- Three-month minimum, then rolling with a month's notice.
What you supply
- Access to logs, configuration and billing for the systems in scope.
- A named contact for alerts.
- The test cases agreed at build, or time to agree them.
Limits
- Business hours, not 24/7 incident response, unless agreed separately.
- Monitoring finds problems; fixing them is change work, quoted per request unless covered by a larger plan.
- Systems we did not build are reviewed first so we know what we are monitoring.
How it works
- 01
Onboard
Systems, alerts and test cases agreed, and access set up.
- 02
Monitor
Checks and evaluation runs on schedule and after every change.
- 03
Report
A short monthly report in plain English.
- 04
Improve
Fixes and improvements proposed and quoted as needed.
Why monitoring matters
Evidence from our own work
A security gate in our own pipeline reported green for months while checking the wrong branch, and no monitor caught it. We built Slipmark so that cannot happen silently. This is our own engineering experience, not a client result.
How Slipmark worksWhere it fits
Monitoring usually follows a production build or agent pipeline, and a control review is a sensible first step for systems we did not build.
Questions before you start
How much does AI monitoring cost?
From £1,200 a month, depending on how many systems are covered, with a three-month minimum and then rolling with a month's notice.
Do you monitor AI systems you did not build?
Yes, after a short review so we understand what the system does and what good looks like.
Is this 24/7 support?
No. It covers business hours unless we agree otherwise in writing.
What happens when something fails?
You are alerted, we diagnose it and propose the fix. Fixes are quoted per request unless your plan covers change work.
What is the next step?
Tell us which AI systems you run in the contact form. We arrange a 20-minute call and send a monthly quote.
How do you know when a model provider has changed the model?
We do not rely on the provider telling us. Providers publish some changes, but scheduled evaluation runs on your own cases show whether behaviour has moved, whatever the cause. Where a provider lets us pin a model version, we record which one is in use and recheck when it changes.
What do you need access to?
Read access to the logs, configuration and billing for the systems in scope, plus a named contact for alerts. We agree the access during onboarding and ask for no more than the work needs. Where code changes are checked with Slipmark, it runs in your own GitHub workflow.
Can monitoring cover tools like ChatGPT that staff use directly?
Partly. We can track which tools are in use, what they cost and whether usage follows your policy, but evaluation runs suit systems built for a defined task. For broad staff use, an AI register and policy, often kept by a fractional AI lead, is usually the better control.
Tell us what needs to work better.
Describe the process and the tools involved. If it looks like a fit we will arrange a 20-minute call.