How to Optimize Conversations AI Prompts from a Trial Chat
Trial Chat Prompt Optimizer in Chat To Clients CRM helps you improve your Conversations AI agent directly from a trial conversation. It uses your trial chat and current prompt to generate test scenarios, evaluate performance, and suggest improvements. Review the results, apply the best variation, and save your agent when you're ready.
TABLE OF CONTENTS
- What is Trial Chat Prompt Optimizer?
- Key Benefits of Trial Chat Prompt Optimizer
- Requirements for Trial Chat Prompt Optimizer
- How Trial Chat Prompt Optimization Works
- How to Optimize Conversations AI Prompts from a Trial Chat
- Managing Optimization Runs
- Frequently Asked Questions
- Related Articles
What is Trial Chat Prompt Optimizer?
Trial Chat Prompt Optimizer allows you to start prompt optimization directly from the Test your agent panel in the Conversations AI builder. After sending a test message and receiving an agent reply, click Optimize Prompt to launch an optimization run. The optimizer uses the trial conversation and the prompt currently displayed in the builder, including unsaved edits, to test the agent and generate improved variations when needed.
Key Benefits of Trial Chat Prompt Optimizer
Trial Chat Prompt Optimizer simplifies agent testing by turning real trial conversations into opportunities for improvement, helping you refine responses before making changes live.
Optimize from Trial Chats: Start optimization directly from a completed test conversation.
Include Unsaved Changes: Test the current prompt without saving edits first.
Automate Testing: Generate scenarios, evaluate responses, and test improved variations.
Review Detailed Results: Compare accuracy, transcripts, evaluations, reasoning, and prompt differences.
Track Progress: Monitor runs in the builder or open Prompt Optimizer.
Control Updates: Review and apply improved prompts before saving your agent.
Requirements for Trial Chat Prompt Optimizer
The optimization option is available only when the required feature and permissions are enabled. Make sure the following conditions are met before starting:
The Conversations AI Prompt Optimizer Labs feature is enabled.
You have permission to manage the Conversations AI agent.
The trial chat contains at least one user message and one agent reply.
Note: The Optimize Prompt button appears only after the agent responds.
For more information, see Labs Features – Complete Overview.
How Trial Chat Prompt Optimization Works
The optimizer uses your trial conversation and current builder prompt to generate scenarios, test responses, and identify potential improvements. Testing runs on an evaluation copy of the agent, allowing you to review variations before updating your live configuration.
You can monitor progress in the builder or open the run in Prompt Optimizer. Completed variations include:
Accuracy: How well the variation performed in evaluated tests.
Scenarios: Situations used to test agent behavior.
Chat Transcripts: Conversations generated during testing.
Evaluations and Reasoning: Results and explanations for agent performance.
Prompt Differences: Changes between the original and improved prompt.
Important: Configured actions may execute during testing. Review connected workflows, calendars, and other resources before starting optimization.How to Optimize Conversations AI Prompts from a Trial Chat
You can start optimization directly from a trial chat, review the recommended improvements, and apply the selected prompt to your agent. Follow these steps to complete the process.
Step 1: Start a Trial Chat
Testing your agent with a realistic customer message helps identify responses that need improvement.
Navigate to AI Agents → Conversation AI.
Open the agent you want to optimize.
Go to the Agent builder and locate Test your agent.
Send a test message and wait for the agent to reply.
Review the response and click Optimize Prompt.
Step 2: Run the Optimization
The optimizer uses the trial conversation and the current prompt, including unsaved edits, to evaluate agent behavior and test possible improvements.
Review the optimization usage details.
Start the optimization run.
Monitor progress in the builder or open Prompt Optimizer for additional details.
Allow the optimizer to generate scenarios and evaluate prompt variations.
You can stop remaining optimization work without losing completed results.
Important: Testing uses an evaluation copy of the agent, but configured actions may still run. Use test resources when appropriate.Step 3: Review Optimization Results
Reviewing completed variations helps you select a prompt that addresses the original issue without introducing unwanted changes.
Open the optimization results.
Review the strongest completed variation.
Check accuracy, scenarios, chat transcripts, and evaluations.
Review the evaluation reasoning and prompt differences.
Confirm that the suggested changes match your requirements.
Note: A higher accuracy score does not guarantee perfect performance. Review the supporting results before applying changes.
Step 4: Apply and Save the Optimized Prompt
Applying the optimized variation updates the prompt in the builder, giving you an opportunity to review it before making changes live.
Select the preferred prompt variation.
Apply the prompt to the Conversations AI builder.
Review the updated instructions.
Click Save to make the changes live.
Test the agent again to confirm the desired behavior.
Managing Optimization Runs
Optimization runs remain accessible even when the original trial conversation is cleared. This helps you revisit results and continue reviewing prompt improvements.
Active Runs: Monitor progress directly in the builder.
Execution History: Access optimization runs after a trial chat reset or reload.
Stopped Runs: Review completed results even if remaining optimization work was stopped.
Unsaved Changes: Optimization includes the current builder prompt, but changes are not made live until you save the agent.
Frequently Asked Questions
Q: Why can't I see the Optimize Prompt button?
A: Verify that the Conversations AI Prompt Optimizer Labs feature is enabled, you have permission to manage the agent, and the trial chat contains at least one user message and agent reply.
Q: Can I optimize a prompt without saving my latest edits?
A: Yes. The optimizer uses the prompt currently displayed in the builder, including unsaved changes.
Q: What happens if I reset the trial chat?
A: The trial conversation may be cleared, but the optimization run remains accessible in Execution History.
Q: Can I stop an optimization run?
A: Yes. You can stop the remaining work without losing completed results.
Q: Can optimization testing trigger real actions?
A: Yes. Configured actions may execute even though testing uses an evaluation copy of the agent. Review connected resources before starting.
Q: Does applying an optimized prompt automatically make it live?
A: No. Applying the variation updates the builder. You must save the agent to make the changes live.