Five questions to ask any AI product at Web Summit

Every stand at a big conference shows the product on a good day: a clean question, a confident answer, a booking made. That is the part of an AI system that is easiest to build. The harder parts are what it does with a question it should not answer, a customer who is upset, or access it was never meant to use.

The failures we have written about this year were not failures of the impressive part. McHire's hiring chatbot was exposed by a forgotten test account and a missing access check. AI agents being tested by the labs used ordinary weak spots to reach systems they should not have. Customer-facing assistants made promises their owners then had to keep. None of that shows up in a ten-minute demo unless you ask.

Ask for the list of systems and data the product connects to, and what it can read versus change. A good answer is specific: this inbox, this calendar, read-only access to these records, nothing else. A vague answer, or “whatever you connect it to”, means you will be the one deciding its limits, usually after something goes wrong.

Follow up with: can we narrow that? An enquiry assistant does not need your whole customer database or the ability to issue refunds.

Every AI product draws a line between what it does on its own and what it hands to a human. Ask where that line is, and whether you can move it. Sending a reply, moving money, changing a record and making a decision about a person are different levels of risk and should not share one switch.

Then ask who the person is at that point and what they see. The Information Commissioner's Office has said that a person reviewing an agent's decision must have the authority and understanding to change it, not just approve it. An approval button that nobody can meaningfully say no to is not oversight.

This is the question that separates products. Ask the vendor to show you, live, what happens with a question outside its scope: a clinical question to a booking assistant, a legal question to a sales bot, an angry customer. The right behaviour is usually to say less and hand over, not to produce a fluent guess.

If the demo cannot be steered off its script, that tells you something too.

If a customer, your board or a regulator asked what the system did last Tuesday and why, could you answer? Ask to see the record: which conversation, which action, which rule allowed it, and where a person stepped in. The ICO's current thinking on agents says records should show which agent acted, on whose authority, and where a human intervened.

A log that only the vendor can read is not your record. Ask whether you can export it.

Every system that has run in the real world has had a bad day. A vendor who says theirs has not either has not run it for long or is not looking. Ask for one specific thing that went wrong and what they changed because of it. The quality of that answer is a better guide than any accuracy figure on a slide.

And ask what is not finished. An honest vendor will tell you what the product does not do yet.

We build AI products and we hold ourselves to these questions. ReplyOS®, our enquiry-to-revenue platform for UK dental and aesthetics practices, is exhibiting at booth A6-13 on Wednesday 11 November. It answers enquiries within rules the practice sets, hands over to a person when judgement is needed, and keeps a record of what was said and done. We will show you where it stops as well as what it does, and it is in alpha, so there are no published customer results yet.

If your team is putting agents into production, ask us about Slipmark, our open-source check that makes AI agents prove what they inspected, and our AI Control Review, a one-day look at what one agent can reach and where a person approves. Paul Redman, our founder, is in Lisbon all week: come to the booth on Wednesday, or book time in the Web Summit app on the other days.