Independent guidance on Australia's Digital Duty of Care. Smart Digital Care is an independent guidance provider and is not a government agency.

Step 7 of 8

The chatbot test

How the Premium chatbot test probes your AI chatbot safely and reports how it answers.

Last checked

A woman types a message on her laptop at a bright desk.

The chatbot test checks how your AI chatbot answers prompts that ask for harmful content. It is part of the Premium plan.

Why test your chatbot

New section 25B(1)(h) of the bill text lists, as an online service, “a service that generates material using artificial intelligence”. The revised explanatory memorandum names “chatbots providing inappropriate and harmful responses” as a harm the current Act cannot deal with well. For a public generative AI service, the memorandum lists controls such as testing before deployment, testing against reasonably foreseeable misuse, safeguards on prompts and outputs, monitoring and review.

Section 25E(da) of the bill text also lists, as harmful to children, material that simulates emotional intimacy and fosters a child’s dependence on it. If children can reach your chatbot, test it with that risk in mind.

Before you start

Have these details ready:

  • The name of your chatbot vendor.
  • The pages of your website where the chatbot appears.
  • Whether your vendor offers an API for the chatbot.
  • Your paid message quota with the vendor, and who in your business can approve its use.
  • The date you last changed the chatbot settings.

Before each run, you give consent. The consent:

  • Names your chatbot vendor.
  • Warns you that the test can use your paid message quota with that vendor.
  • Caps the number of probes in the run.

Step 2: Choose how the test reaches your chatbot

The test runs in a browser on your own website, the way a visitor uses your chatbot. If your vendor offers an API, the test can use it instead.

Step 3: Let the probes run

The test sends probes from a curated library. Our team writes the probes, and a law firm reviews them.

  • The probes ask for prohibited content but never contain it.
  • There are no child sexual abuse material probes of any kind. The test only checks that your chatbot refuses on topic signals.

Step 4: Read the scored report

Each answer is scored. The report redacts any harmful output, so the report itself is safe to share inside your business.

  1. Read each low score and the probe that caused it.
  2. Ask your vendor what settings can stop that answer.
  3. Run the test again after the change.

Step 5: Add the result to your risk assessment

Keep the report with your risk assessment in your evidence vault. Section 27(3) asks for a new assessment before any significant change. The memorandum gives adding a generative AI tool as an example. If you plan to add a chatbot, run the test before you switch it on.

Next step

Read how the Regulator Readiness Check works: the Regulator Readiness Check. You can also read about the chatbot test product.

Ready to take the first step?

Tell us about your online service. We will tell you where to start and when the products open.