Skip to content
SignalMD

For Retell AI agents

Test and monitor your Retell AI agent on its real calls

SignalMD works with Retell AI and nothing else. Connect an agent with a Retell API key. SignalMD reads its calls, finds the problems that repeat, and tells you which step to change and whether the change helped.

It connects over the Retell API, with no webhook changes. It writes to your live agent only when an admin on your team publishes a draft.

What you get for a Retell agent

  • Your production calls, read for you

    SignalMD imports up to 500 recent calls from the last 30 days, then keeps up with new calls on its own. Problems that repeat are put on the step in your flow where they happen.

  • A cause and a fix for each issue

    Each issue names a place, a cause and a change, with quotes from the calls behind it. When the calls do not show why, SignalMD says Cause not established. It does not guess.

  • Versions compared on real calls

    When you publish a new version in Retell, SignalMD compares production calls before and after. It tells you when a difference is real, and when there are not enough calls yet.

  • Tests made from real calls

    A call behind an issue becomes a test: a simulated caller with the same goal, run against a version of your agent with your own Retell key.

How to test a Retell AI agent

You can do this without any tool. It works, and it takes time.

  1. Start from real calls

    Pick 10 to 20 recent calls that went wrong and 10 that went well. For each, write down what the caller wanted and what should have happened.

  2. Turn each call into a test

    Give each test the caller's goal and the result you expect. Retell can run simulated calls against your agent, so you can run the set again after each change.

  3. Run the tests on the old and the new version

    A change counts only if it fixes the failing tests and the passing tests still pass. A change to one step can break another.

  4. Watch production after you publish

    Count the same problem before and after the change, over enough calls. One good day is not proof.

The hard parts are reading every call by hand, finding the problems you did not think to test, and knowing when a difference is bigger than the noise. SignalMD does those parts for you.

Questions about Retell AI

How do I test a Retell AI agent?

Turn real calls into tests, run them on the old and the new version, then compare production calls before and after you publish. SignalMD makes the tests from your calls and runs them with your Retell key.

How do I monitor Retell AI calls in production?

Connect the agent with a Retell API key. SignalMD reads new calls as they come in, groups the problems that repeat, and orders them by how much they hurt your callers. New calls show up within about 15 minutes, or as soon as a call ends if you add the webhook. Analysis takes a few minutes more.

Can I A/B test changes to a Retell agent?

Not as a live traffic split. Publish the change as a new version in Retell. SignalMD compares production calls before and after, and says when a difference is real and when there are not enough calls yet. It is a before and after comparison, not a controlled experiment. Before the change goes live, you can also run the same tests from real calls on both versions.

Does SignalMD change my Retell agent?

Only when you choose to. It reads your calls and your agent's settings through the Retell API and suggests changes. It writes to your live agent only when an admin publishes a drafted fix, and first checks that nobody changed the agent since. Test runs create test definitions in your own Retell workspace, and a change is tested on a copy of your agent.

Which voice platforms does SignalMD support?

Retell AI only, for now. SignalMD reads Retell calls, flows and settings, and places each issue in your Retell agent.

What does it cost?

Plans start at $49 a month, with one monthly balance of credits. There is no charge per metric. Test runs in Retell's simulator also bill to your own Retell account.

See what your Retell agent's calls say

Free. We review 2 weeks of your real calls and send you the 3 problems that cost you the most, in 2 working days.

We're letting teams in a few at a time. Join the waitlist and we'll email you when it's your turn.