The 100% Solution: How AI is Fixing Quality Assurance in 911 Dispatch

A Practical Guide for Communications Supervisors, QA Coordinators, and PSAP Directors

What You Will Find in This Guide

This guide is written for the person who owns quality assurance in a communications center. That is usually a supervisor or QA coordinator who already has a full shift schedule, a training program to run, and a stack of calls waiting to be reviewed.

Inside, you will find a real-world breakdown of how errors slip through manual review, a technical explanation of how automated scoring works against NENA and APCO standards, a three-step coaching framework you can use in your next one-on-one, talking points for introducing automated review to your floor, and a self-assessment checklist you can complete in about ten minutes.

You can use every framework and checklist in this guide whether or not your agency ever purchases software from us. That is intentional. We would rather earn your trust with something useful than talk at you about features.

Part 1: The Two Percent Problem

Ask any communications supervisor how they choose which calls to review, and you will hear a version of the same answer. They pull the calls that got attention. The pursuit. The working fire. The cardiac arrest. The call that generated a citizen complaint or a supervisor’s phone call at 2 a.m.

That is a reasonable approach when review time is scarce. It is also the root of the problem.

In a center handling 8,000 calls a month, a QA program that reviews five calls per telecommunicator per month is touching a very small slice of total volume. For most centers, the realistic review rate lands somewhere between one and three percent. Former 911 coordinators consistently describe the same bottleneck. The hardest part of quality assurance is not the evaluation itself. It is pulling CAD records, sorting through incidents, and deciding which handful of calls are worth the time. That selection process can consume hours before a single score is entered.

Three consequences follow from a two percent review rate.

1. You are reviewing outcomes, not behaviors. When you select calls because something went wrong, you are studying the exception. You learn very little about how your telecommunicators handle the routine work that fills eighty percent of a shift.
2. Your sample is biased toward your busiest people. High performers who work the busiest console positions generate the most reviewable calls. Quieter shifts, overnight positions, and part-time staff often go months without meaningful evaluation.
3. Your feedback arrives too late to be useful. In many centers, thirty to sixty days pass between the call and the review. By then, the telecommunicator does not remember the call, and the habit you are trying to correct has been repeated hundreds of times.

Part 2: The Anatomy of a Missed Call

Abstract numbers do not make the case. A specific call does.

The call nobody reviewed

A Tuesday afternoon noise complaint comes in on the non-emergency line. A resident reports loud music from a neighboring apartment. The telecommunicator takes the address, enters the complaint, and dispatches a low-priority unit. The call clears in twenty minutes with no report taken.

Nothing about this call would ever be selected for manual QA. There was no complaint, no injury, no supervisor involvement, and no reason for anyone to look twice.

Here is what actually happened inside that call.

The telecommunicator never verified a callback number, accepting the ANI display without confirming it. The location was entered as a street address without the apartment number, and the geo-validation warning was overridden. The responding unit was dispatched, but its arrival status was never updated in CAD. The unit went available forty minutes later, and the telecommunicator backfilled the times.

Four small deviations. On a noise complaint, none of them mattered.

The call everybody reviewed

Nine days later, the same telecommunicator takes a domestic disturbance at an apartment complex. The caller is whispering. She gives a street address. The telecommunicator, working from the same habit formed on dozens of routine calls, does not press for the apartment number and does not confirm a callback number. The line disconnects after ninety seconds.

The center now has an incomplete address, no verified callback number, and a disconnected caller on a domestic in progress. Officers spend seven additional minutes locating the correct unit.

That call gets reviewed. It gets reviewed by the supervisor, the director, the city attorney, and eventually the media.

The point

The failure did not happen on the domestic. It happened on the noise complaint, and on the twenty routine calls before it, where a habit formed unchecked because no one was looking.

Quality assurance that only examines high-acuity calls is diagnosing the symptom. The habits that determine performance on the worst call of a telecommunicator’s career are built on the most ordinary calls of their week.

Part 3: How AI Evaluates the Floor

Automated quality assurance solves the selection problem. Instead of choosing which calls to review, you review all of them and let the system tell you where to look.

What the system actually reads

Caliber Insight LS connects directly to your Caliber CAD system and ingests the standard XML data generated by each incident. It does not require telecommunicators to change how they take calls, and it does not add a single keystroke to their workflow. Within minutes of a call closing, that incident has been evaluated and scored.

Each incident is scored on a 100-point scale. Points are deducted for measurable deviations from your configured standards, which means every telecommunicator in your center is graded against the same rubric on every call.

The standards behind the scoring

Automated scoring is only credible if it is anchored to standards your staff already recognizes. Caliber Insight LS evaluates against established NENA and APCO call-handling standards, along with your agency’s own written policy.

• Measured Element 
• Standard Basis
• What the System Checks

Call answering time

• NENA call-processing guidance calls for 90 percent of 911 calls to be answered within 15 seconds, and 95 percent within 20 seconds
• Answer intervals per call, aggregated to show center-wide compliance percentages

Dispatch delay

• Priority-based dispatch intervals, commonly 60 seconds for Priority 1
• Time from call creation to unit dispatch, measured against the assigned priority

Unit progression

• APCO status-tracking practice
• Whether en route, on-scene, and clear statuses were entered in sequence and in real time 

Location validation

• Agency geo-validation policy
• Whether the address was validated or the warning was overridden

Documentation completeness

• Agency records policy
• Missing case numbers, missing callback numbers, incomplete caller information, thin narratives

For high-priority incidents and for calls that fall below your threshold, the system generates a narrative summary of what happened and where the deviation occurred, along with specific coaching recommendations tied to the standard that was missed.

You set the thresholds, not the vendor

A rural sheriff’s office running two consoles and a metro consolidated center running twenty-four do not share the same operational reality, and they should not share the same rubric.

Your agency configures the rules, the point values, the score thresholds, and the alerting. If your policy requires a 90-second dispatch on Priority 2 calls, that is the standard the system measures. When an incident falls below your threshold, the system pushes a detailed report to the right supervisor automatically. You are not alerted on calls that meet the standard, which means the calls that reach your desk are the ones that actually need a human being to look at them.

What this changes about your day

The practical difference is where your time goes. Manual QA spends roughly seventy percent of supervisor time on selection and data retrieval, and thirty percent on actual coaching. Automated review inverts that ratio. Selection time drops to near zero, and the review work that remains is the part that requires judgment, context, and a conversation with a person.

Part 4: From Catching Errors to Coaching Success

Reviewing 100 percent of your calls is only an improvement if it makes your telecommunicators better. If the only thing that changes is that you now catch more mistakes, you have built a more efficient way to write people up. That is not a QA program, and your staff will recognize it immediately.

Use this three-step framework for turning automated data into a coaching conversation.

Step 1: Read the data before you form an opinion

Open the incident summary and the score breakdown before you pull the audio and before you talk to the telecommunicator. Automated scoring gives you the objective facts of the call, including timestamps, status entries, and documentation gaps, without the influence of how the call turned out.

This matters more than it sounds. Manual QA is heavily affected by outcome bias. A call that ended badly gets scored harder than an identically handled call that ended well. Starting from the measured data keeps your evaluation anchored to behavior rather than result.

Practical rule: Do not open a coaching conversation with the phrase “this call went bad.” Open it with the specific interval or field the system flagged.

Step 2: Coach the pattern, not the incident

A single low score is noise. Three low scores on the same element across two weeks is a signal.

Because you now have data on every call rather than a handful, you can see trends that manual sampling would never surface. Common patterns worth watching include dispatch delays that appear only on medical calls, location validation overrides that cluster in one geographic area, and status-tracking gaps that appear only during shift change or during high-volume hours.

Bring the trend to the conversation, not the single worst example. A telecommunicator can dismiss one bad call. A pattern across fifteen calls is a conversation about how they work.

Practical rule: Wait for three data points before you schedule a formal coaching session, unless the deviation involves a safety issue that requires immediate correction.

Step 3: Anchor the plan to a written standard

End every coaching conversation with a specific, measurable target tied to a standard the telecommunicator can look up.

Vague direction produces vague improvement. “Be more thorough on your location entries” gives someone nothing to act on. “Our policy requires apartment or suite numbers on all multi-unit addresses, and the system flagged eleven overrides in the last three weeks. Let’s target zero overrides this month and check the data again on the first” gives them a standard, a number, and a date.

Then follow up on that date. Automated review makes the follow-up easy, because the data is already there.

A sample coaching structure

Use this sequence for a fifteen-minute one-on-one:

1. State the specific element and the standard it comes from, using two minutes.

2. Show the pattern in the data across the review period, using three minutes.

3. Listen to the telecommunicator’s explanation of what is driving it, using five minutes. This step is where you learn whether the cause is training, workload, console configuration, or policy confusion.

4. Agree on one target and one follow-up date, using three minutes.

5. Document the plan, using two minutes.

Coaching one element at a time produces better results than presenting a full scorecard. A telecommunicator handed a list of nine deficiencies will improve none of them.

Part 5: Getting Buy-In on the Floor

Any veteran supervisor reading about 100 percent automated review is already thinking about the conversation with their staff and, in many centers, with their union representative. That reaction is legitimate, and it deserves a direct answer rather than a marketing slogan.

Here is how to introduce automated quality assurance honestly.

Address the surveillance concern directly

Do not open with the technology. Open with what is changing about the review process and what is not.

What changes is the selection method. Instead of a supervisor choosing five calls, the system evaluates all of them and flags the ones that fall below standard. What does not change is that a human being makes every judgment that matters. The system does not discipline anyone, does not write anyone up, and does not make a determination about performance. It sorts data so a supervisor can spend time on coaching instead of searching.

Say that plainly and early.

Lead with the fairness argument

This is the strongest point you have, and it is the one telecommunicators care about most.

In a manual program, evaluation is uneven by design. Different supervisors grade differently. Some are known for being tough on narratives, others on times. Staff on high-volume consoles get reviewed more often than staff on quiet ones. Everyone on the floor knows this, and it is a persistent source of resentment.

Automated scoring applies one rubric to every call from every position on every shift. If your center adopts it, a telecommunicator’s score reflects their work rather than which supervisor happened to pull their calls that month.

Explain how it protects them

This point is often overlooked, and it lands hard with experienced staff.

When a call goes wrong, and the agency faces a complaint, a lawsuit, or a records request from a reporter, the telecommunicator’s defense is documentation. A scored record showing that the call was answered within standard, dispatched within standard, and documented completely is the strongest evidence your agency can produce that protocol was followed.

Manual QA cannot provide that, because the call in question was almost certainly never reviewed. Complete review means every telecommunicator has a documented performance record supporting them.

Give the floor a role in configuration

Bring one or two respected telecommunicators into the threshold-setting process before go-live. Let them argue about point values and flag rules that do not reflect operational reality.

Two things happen. First, you get better thresholds, because working telecommunicators know things about your CAD workflow that supervisors have forgotten. Second, the program arrives on the floor as something the staff helped build rather than something that was installed on them.

Set a no-discipline calibration period

Run the system for the first sixty to ninety days as a data-gathering and calibration period, with no disciplinary action tied to scores. Tell the floor this up front, in writing.

You need that window anyway to tune thresholds and identify false flags caused by CAD configuration rather than telecommunicator behavior. Announcing it also removes the immediate threat and gives staff time to see how the scoring actually works before their evaluations depend on it.

Anticipated questions and honest answers

“Is a computer grading my calls?” The system measures timestamps and required fields against your agency’s written standards. Judgment about call handling still belongs to your supervisor.

“What if the score is wrong?” Scores are reviewable and disputable, the same as any manual evaluation. Configuration errors are common in the first weeks, which is exactly why the calibration period exists.

“Will this be used against us in discipline?” Your agency decides that, and you should decide it before go-live and put it in writing. The most durable programs treat scores as a coaching input, with discipline reserved for the same conduct issues that would have triggered it before.

Part 6: The QA Vulnerability Checklist

Use this assessment to identify gaps in your current quality assurance program. Answer honestly, and note that any question you cannot answer with data is itself a finding.

Coverage

• What percentage of your monthly call volume receives a documented review? Calculate it rather than estimating it.
• Can you review 100 percent of your probationary telecommunicators’ calls during their probationary period?
• When was the last documented review for your overnight and weekend staff?
• Are part-time and per-diem telecommunicators reviewed at the same rate as full-time staff?

Consistency

• Do you have a written, standardized rubric, or does scoring vary by supervisor?
• If two supervisors scored the same call independently, how close would the scores be? Have you ever tested this?
• Are calls selected for review by a defined method, or by whichever incidents drew attention?

Timeliness

• How many days pass between a call being handled and the review being completed?
• How many days pass between the review and the coaching conversation?
• Can a telecommunicator receive feedback while they still remember the call?

Standards Alignment

• Can you produce a report showing your center’s compliance with the NENA 15-second call answering standard for last month?
• Can you produce dispatch interval compliance by priority level?
• Do your telecommunicators know which specific standards they are evaluated against?

Defensibility

• If a call from four months ago became the subject of litigation, could you produce a documented evaluation of it?
• Can you demonstrate that a performance issue was identified and coached before it escalated?
• Would your current QA documentation support or undermine your agency in an administrative hearing?

Staff Perception

• Do your telecommunicators believe the current process is objective and fair?
• Have you asked them, in a setting where an honest answer was possible?
• Does your QA program result in more coaching conversations or more disciplinary conversations?

Reading your results

If you answered “no” or “I do not know” to more than five of these questions, the gap is not your staff’s commitment to quality. It is that manual review cannot produce the coverage, consistency, and speed a modern communications center requires.

Where to Go From Here

You can act on most of this guide immediately. Write a standardized rubric. Test scoring consistency between supervisors. Measure your actual review percentage rather than assuming it. Shorten the interval between the call and the conversation.

Those changes will improve your program without any new technology.

What manual review cannot give you, at any level of effort, is complete coverage. There is no version of a manual program where a supervisor evaluates every call within minutes of closure, and no amount of process improvement changes that arithmetic.

That is the specific gap Caliber Insight LS was built to close. It reviews 100 percent of your CAD incidents automatically, scores them against NENA and APCO standards and your own policy, and routes the calls that fall below your threshold to the right supervisor with a narrative summary and coaching recommendations attached. Your telecommunicators do not change how they work, and your supervisors spend their review time coaching people instead of searching for records.

We built it with input from people who have sat in your chair, because we understand that the goal was never to catch more mistakes. The goal is a floor full of telecommunicators who handle the worst call of their career correctly, because someone was paying attention to the ordinary ones.

To see how automated quality assurance would work with your CAD data and your standards, contact Caliber Public Safety to schedule a demonstration of Caliber Insight LS.