Testing your AI Analyst¶
Every AI Analyst page in Studio has a Test your analyst panel. Use it to ask questions while you build the AI Analyst, check the answers and the steps behind them, and adjust instructions or data access until the results are right. You don't need to deploy the AI Analyst first.
Availability
- Who can do this: members with the Edit capability in the active Workspace. Members who have Studio access without Edit (for example through Workspace Manage members) can test only AI Analysts that are deployed to them.
- Deployment: Actian SaaS and self-managed deployments.
How test runs work¶
- A test question runs a real conversation with the AI Analyst, the same way a question in Explorer does.
- The test uses the AI Analyst's saved configuration. When you send a question, any unsaved edits on the tab you have open are saved first. If a save fails, your question stays in the box and nothing runs: fix the problem on the tab that shows Save failed, then send again.
- If your plan or licence limits messages, test questions count toward that usage like questions asked in Explorer.
- Test runs don't affect the AI Analyst's usage figures. See Test runs in Monitoring.
Ask a test question¶
- In Studio, click AI Analysts and open the AI Analyst.
- If the panel is closed, click Open test panel on the right edge of the page.
- Type a question in Ask your AI analyst… and send it.
- Read the answer. To start a fresh conversation, click New chat.
The panel is open by default. Click Close test panel to hide it. Your browser remembers whether the panel is open.
The composer is the same one business users have in Explorer, so you can also run a saved prompt by typing / and its slug.
Rehearse a plan¶
Turn on Plan in the composer to have the AI Analyst draft a research plan for your review before it answers. The plan replaces the composer until you decide:
| Button | Result | Plan mode afterwards |
|---|---|---|
| Approve | The AI Analyst carries out the plan. | Turns off |
| Edit | You change the plan, then save it. | Stays on |
| Keep planning | The AI Analyst keeps working on the plan. | Stays on |
| Cancel | The plan is dropped. | Turns off |
Your browser remembers the Plan setting between sessions. For more about plans, see Plan Mode.
Read the steps behind an answer¶
Each answer shows your question, then the AI Analyst's steps (folded), then the answer. Steps include actions such as Search semantic layer, Run metric, Query a model, Run SemQL query, Create chart and Note assumptions. Expand a step to see what the AI Analyst did.
These step details also appear in Explorer, not only in Studio.
Query steps¶
When a query step succeeds, expanding it shows how the question became SQL:
- A Metric or Model card: what the AI Analyst asked for in semantic-layer terms.
- A SemQL card: the query in SemQL, the semantic query language. If the AI Analyst wrote SemQL directly, this is the first card and is titled Custom SemQL.
- A SQL card: the executable SQL sent to your data source.
Cards are joined by a compile arrow. Use Copy SemQL and Copy SQL to copy a query. A footer shows the number of rows, how long the query took, and truncated if the result was cut off.
Some successful steps (for example, in older conversations) don't have this flow. They show the SemQL, the Compiled SQL, and the Output with the row count.
Failed query steps¶
When a query fails, the step shows the SemQL (or, for a metric or model query, its input), the error category, the error message and, if available, a Suggestion. Compiled SQL isn't shown for a failed step.
| Category | What it means |
|---|---|
| Parse error | The query isn't valid SemQL. A query must be a single read-only statement: an optional WITH clause and SELECT statements, which can be combined with UNION, INTERSECT or EXCEPT. A second statement after ; is a parse error. |
| Security error | The query uses something SemQL blocks for safety, such as a disallowed function or nested aggregates such as SUM(SUM(x)), or it targets a data source that is private to another Workspace. A windowed aggregate such as SUM(COUNT(*)) OVER () is allowed. |
| Semantic error | The query doesn't fit the semantic layer. For example, a field doesn't match a known dimension or measure, a named filter doesn't exist on the model, a measure is used in WHERE or GROUP BY, or a measure is wrapped in another aggregate. |
| Unsupported | The query uses a construct SemQL doesn't support, such as a syntax that only some warehouses accept. When a portable alternative exists, the message names it. |
| Execution error | The data source rejected or failed the compiled SQL, the connection couldn't be opened, or the data source wasn't found. |
Parse, Security, Semantic and Unsupported errors are caught before anything is sent to your data source. Execution errors come from connecting to the data source or running the query there.
The AI Analyst can read these errors and try again with a corrected query. If the same error keeps coming back, it often points to something to fix in your setup:
- Semantic error: check that the AI Analyst has access to the right models, dimensions and measures on its Models tab, and that the model defines what the question needs.
- Execution error: check the data source connection and the model's underlying SQL.
- Security error about a private data source: the data source belongs to another Workspace. Only members of that Workspace can query it.
What's different from Explorer¶
The Test your analyst panel is for previewing. Compared with Explorer:
- There are no thumbs up or thumbs down buttons for feedback.
- Artifacts have no Save button. Save and share results from Explorer instead.
- The line under each answer that shows the model, duration and token count isn't shown. You can see it when you replay the conversation from Monitoring.
Test runs in Monitoring¶
Test conversations are recorded with the channel Studio test.
- They appear in Monitoring → Tasks by default. Turn on Hide test runs to filter them out. See Monitoring.
- They're always left out of the AI Analyst's Analytics and of the Tasks run and Active users (30d) counts on the Overview tab.
- They don't count toward semantic-layer usage statistics.
- They don't generate semantic-layer feedback. In real conversations, Semantic error and Execution error failures are recorded as signals that can lead to items in the Steward Inbox. Failures in test runs are not.
Troubleshooting¶
| Problem | Likely cause | What to do |
|---|---|---|
| You send a question and nothing happens. | A tab has unsaved edits that failed to save. | Open the tab that shows Save failed, fix the problem, and send the question again. |
| The AI Analyst says it can't find any data. | A new AI Analyst has no model access. | On the Models tab, choose the models it can use, then click Save changes. |
| The answer ignores your instructions. | Instructions are guidance, and the AI Analyst doesn't always follow them perfectly. | Reword the instructions and test again. |
| Business users can't find the AI Analyst. | Testing doesn't deploy it. | Deploy it to Explorer, Slack or Teams. |
| The test panel isn't visible. | It was closed. | Click Open test panel on the right edge of the page. |