The chatbot test checks how your AI chatbot answers prompts that ask for harmful content. It is part of the Premium plan.
Why test your chatbot
New section 25B(1)(h) of the bill text lists, as an online service, “a service that generates material using artificial intelligence”. The revised explanatory memorandum names “chatbots providing inappropriate and harmful responses” as a harm the current Act cannot deal with well. For a public generative AI service, the memorandum lists controls such as testing before deployment, testing against reasonably foreseeable misuse, safeguards on prompts and outputs, monitoring and review.
Section 25E(da) of the bill text also lists, as harmful to children, material that simulates emotional intimacy and fosters a child’s dependence on it. If children can reach your chatbot, test it with that risk in mind.
Before you start
Have these details ready:
- The name of your chatbot vendor.
- The pages of your website where the chatbot appears.
- Whether your vendor offers an API for the chatbot.
- Your paid message quota with the vendor, and who in your business can approve its use.
- The date you last changed the chatbot settings.
Step 1: Give consent for the run
Before each run, you give consent. The consent:
- Names your chatbot vendor.
- Warns you that the test can use your paid message quota with that vendor.
- Caps the number of probes in the run.
Step 2: Choose how the test reaches your chatbot
The test runs in a browser on your own website, the way a visitor uses your chatbot. If your vendor offers an API, the test can use it instead.
Step 3: Let the probes run
The test sends probes from a curated library. Our team writes the probes, and a law firm reviews them.
- The probes ask for prohibited content but never contain it.
- There are no child sexual abuse material probes of any kind. The test only checks that your chatbot refuses on topic signals.
Step 4: Read the scored report
Each answer is scored. The report redacts any harmful output, so the report itself is safe to share inside your business.
- Read each low score and the probe that caused it.
- Ask your vendor what settings can stop that answer.
- Run the test again after the change.
Step 5: Add the result to your risk assessment
Keep the report with your risk assessment in your evidence vault. Section 27(3) asks for a new assessment before any significant change. The memorandum gives adding a generative AI tool as an example. If you plan to add a chatbot, run the test before you switch it on.
Next step
Read how the Regulator Readiness Check works: the Regulator Readiness Check. You can also read about the chatbot test product.