Skip to main content

Analyze and Improve Performance

Once you've tested your assistant in DIRTBox, it's time to go deeper: analyze its performance, create automated tests, and review real-world conversations. The DIRTBox area includes powerful tools for this: the Tests, Transcripts, and Revisions tabs.

The Tests Tab​

In the Tests tab, you can create and run test cases to make sure your assistant behaves as expected.

  • Add test cases by specifying user and AI messages
  • Check that your assistant gives the right responses for different scenarios
  • Run all tests to catch regressions as you update your flows

Automated tests help you maintain quality as your assistant grows.

The Transcripts Tab​

The Transcripts tab shows real conversations your assistant has had with users after deployment.

  • Review actual user interactions
  • Spot misunderstandings, missed intents, or confusing responses
  • Use real data to guide improvements

The Revisions Tab​

The Revisions tab lets you document and fix issues by hand:

  • Add a revision: enter a question, the incorrect response, why it was wrong, and (optionally) a correct response
  • Use the preview button to test how the assistant would answer now
  • Adjust AI model settings for previewing answers

This workflow helps you systematically improve your assistant, just like with the Brain Vault.

For hands-on testing, see Test Your Assistant in DIRTbox. For debugging and simulation, see Simulate Conversations and Debug.