Beyond Response Times, Part 2: Choosing the Measures That Actually Matter
In Part 1 of this series, I made the case that response time should remain an important part of EMS performance measurement, but it should no longer stand alone as the dominant definition of success.
That argument raises a more difficult question.
If response time is only one part of the story, what else should EMS leaders, communities, healthcare systems, and contract administrators measure?
At first glance, this seems like an easy problem to solve. Modern EMS organizations collect extraordinary amounts of information. Every patient encounter creates clinical data. Dispatch systems create operational data. Cardiac monitors capture physiologic information. Training platforms track education. Fleet systems track vehicle performance. Quality-improvement programs collect case-review information. Hospitals, billing systems, workforce platforms, and alternative-care programs all generate additional information.
The problem is no longer access to data.
The problem is deciding which data actually matter.
More measurement does not automatically produce better performance. In fact, organizations can create dashboards containing hundreds of indicators and still struggle to answer the most basic question: what should we do differently tomorrow because of what this data is telling us today?
That is where meaningful performance measurement begins.
The objective should not be to replace response time with dozens of new requirements. It should be to develop a balanced and manageable framework that measures the parts of EMS performance most closely connected to patient care, safety, operational reliability, and system value.
The Difference Between Available Data and Useful Data
Healthcare organizations often measure what is easiest to extract rather than what is most important to understand.
Response time became such a dominant EMS metric partly because it is administratively simple. A call is received. A unit is dispatched. The ambulance arrives. The interval can be calculated automatically.
Clinical quality is more complicated.
Determining whether an airway was managed appropriately requires more than knowing whether a tube was eventually placed. Evaluating stroke care requires understanding assessment findings, last-known-well time, blood glucose, destination selection, and hospital notification. Trauma quality involves assessment, hemorrhage control, physiologic monitoring, triage criteria, and destination decisions.
Those measures require context.
That complexity should not discourage us from measuring them.
It should encourage us to design better systems.
A meaningful EMS measure should tell leadership something important about patient care or system performance. It should be defined clearly enough to be measured consistently. It should reflect something the organization can reasonably influence. Most importantly, it should create an opportunity for action.
If a number changes and nobody knows what to do about it, the value of that measure is limited.
Measure What EMS Can Influence
One of the most important principles in performance measurement is controllability.
EMS organizations should be accountable for the parts of care they can reasonably influence.
That does not mean final patient outcomes are unimportant. Outcomes matter enormously. They tell us whether the healthcare system is ultimately achieving what it is supposed to achieve.
But an outcome can be important without being appropriate as a punitive contractual measure.
Consider cardiac arrest survival.
Survival to hospital discharge is one of the most meaningful outcomes an EMS system can monitor. At the same time, survival is affected by many factors that occur before and after the ambulance arrives.
Was the arrest witnessed?
Did someone recognize it quickly?
Was bystander CPR performed?
Was an AED available?
How quickly was the arrest communicated to dispatch?
What rhythm was present?
What underlying medical condition caused the arrest?
What care was provided after hospital arrival?
EMS influences part of that chain, but not all of it.
A strong performance framework therefore does not ignore survival. It examines survival while also measuring the parts of the chain EMS can directly improve.
Was CPR performed effectively?
Were interruptions minimized?
Was defibrillation appropriate?
Was ventilation managed correctly?
Was the airway managed safely?
Was post-resuscitation care appropriate?
Was the correct destination selected?
Those questions provide leadership with information that can actually drive improvement.
The same principle applies throughout EMS.
The best performance measures usually sit at the intersection of clinical importance and organizational influence.
Airway Management: A Real Example of Measurement Driving Change
Airway management provides one of the clearest examples of how meaningful measurement can lead directly to better clinical care.
Advanced airway management is a high-risk procedure. It is often performed in uncontrolled environments, under significant time pressure, and on patients who may already be critically hypoxic, hypotensive, or in cardiac arrest.
Simply measuring whether an airway was eventually secured does not tell us enough.
A better quality framework asks additional questions.
Was adequate oxygenation achieved before the attempt?
Was the airway secured on the first attempt?
Did the patient experience significant hypoxia or hypotension during the procedure?
Was waveform capnography used for confirmation and monitoring?
Were repeated attempts necessary?
Was the clinician using the most appropriate equipment and technique for the circumstances?
Those questions move the organization from procedural completion toward procedural quality.
That distinction became particularly important during a clinical improvement initiative in one large EMS system.
First-pass airway success was approximately 37 percent.
Rather than accepting that performance as an unavoidable consequence of field intubation, the organization treated it as a measurable opportunity for improvement.
Video laryngoscopy was introduced, but the device itself was only one part of the solution.
The implementation included education, hands-on competency development, field rollout, continued clinical monitoring, case review, and feedback to providers.
Within the first month, first-pass airway success increased from approximately 37 percent to 86 percent.
The significance of that result extends well beyond airway management.
Measurement identified the problem.
The data created urgency.
An intervention was selected.
Education and implementation followed.
Performance was measured again.
The organization could then determine whether the change had actually worked.
That is exactly what a quality metric should do.
The metric did not exist merely to produce a number on a dashboard.
It created a pathway toward better care.
Clinical Quality Should Reflect the Entire Patient Encounter
Airway performance is only one clinical domain that deserves attention.
Stroke care is another.
An EMS system can evaluate whether clinicians completed an appropriate stroke assessment, documented last-known-well time, checked blood glucose, recognized signs of large-vessel occlusion when appropriate, notified the receiving facility, and transported the patient to a facility capable of providing the level of care the patient needed.
Those actions are measurable.
More importantly, they are largely within the influence of the EMS system.
The same is true for STEMI care.
Did the patient receive a 12-lead ECG when clinically indicated?
Was it obtained promptly?
Was the STEMI recognized?
Was the receiving hospital notified?
Was the patient transported to an appropriate destination?
Trauma care presents similar opportunities.
Was serious hemorrhage identified and controlled?
Was pain assessed and treated appropriately?
Were critical physiologic findings documented?
Was the patient transported to the correct level of trauma care?
Was the destination decision consistent with established triage criteria?
These are the types of questions that begin to describe what actually happened after the ambulance arrived.
They also reveal why response time alone cannot adequately represent modern EMS performance.
Two agencies can meet the same response-time standard while producing very different clinical results.
A meaningful performance framework should be capable of showing that difference.
Patient Safety Deserves Its Own Place
Clinical quality and patient safety are related, but they are not identical.
An organization can follow a protocol correctly and still have underlying safety risks that deserve attention.
Medication errors are an obvious example.
An EMS system should be capable of identifying patterns involving incorrect dose, incorrect route, incorrect concentration, pediatric calculation errors, allergies, controlled-substance discrepancies, and adverse medication events.
When those events occur, the investigation should go beyond asking who made the mistake.
Was the medication packaged clearly?
Were multiple concentrations stored together?
Was the protocol unnecessarily complicated?
Did the electronic documentation system support the clinician?
Was the provider fatigued?
Was the labeling difficult to read in poor lighting?
Was the calculation process prone to human error?
Quality improvement becomes substantially more effective when the organization looks at both individual actions and the system surrounding those actions.
The same philosophy should apply to equipment failures, patient drops, ambulance crashes, airway complications, high-risk refusals, communication failures, and other adverse events.
A healthy safety culture does not attempt to produce a dashboard showing zero reported problems.
In some organizations, an increase in reported events may actually represent progress because clinicians have become more willing to identify near misses and safety concerns.
The goal should not be to make the numbers look perfect.
The goal should be to make the system safer.
Documentation Is Part of Clinical Quality
Documentation is often treated as an administrative problem.
That is a mistake.
The patient care report is the foundation of continuity, quality review, medical oversight, reimbursement, research, regulatory compliance, and legal accountability.
It is also the foundation of almost every clinical performance dashboard.
If the source documentation is incomplete, the analytics built from it are immediately weakened.
An organization cannot accurately measure first-pass airway success if airway attempts are documented inconsistently.
It cannot evaluate clinical reassessment if repeat vital signs are routinely missing.
It cannot reliably evaluate stroke performance if last-known-well time or assessment findings are absent.
It cannot build sophisticated clinical analytics on unreliable clinical records.
That is why documentation quality belongs in a modern performance framework.
The objective should not be to reward longer narratives or create unnecessary administrative work.
The objective should be to ensure that the clinical record contains enough accurate information to understand what happened and support the next stage of care.
When documentation quality declines, leadership should ask why.
Was education inadequate?
Is the electronic patient care record poorly designed?
Are required fields confusing?
Are clinicians completing charts long after the call?
Is operational pressure discouraging complete documentation?
Again, the metric becomes valuable only when it leads to a meaningful question and a possible intervention.
Where the Patient Goes Matters
EMS quality does not end when the ambulance reaches the hospital.
Destination decisions can have major consequences for patients.
A trauma patient may need a trauma center.
A patient with a large-vessel-occlusion stroke may require a comprehensive stroke center.
A patient with STEMI may need immediate access to cardiac catheterization.
An ambulance that arrives quickly but repeatedly transports patients to facilities that cannot provide the required level of care should not be considered high-performing based on response time alone.
Destination decision-making is one of the clearest examples of EMS functioning as part of the broader healthcare system rather than simply as a transportation service.
It also represents a domain where EMS organizations can often exercise meaningful influence through protocols, education, decision support, and medical oversight.
That makes destination performance highly relevant to both clinical quality and system accountability.
Operational Performance Still Matters
Moving beyond response time should never be interpreted as dismissing operational performance.
EMS must remain reliable.
Communities need ambulances available when emergencies occur.
Dispatch systems must function efficiently.
Units need to be deployed appropriately.
Vehicles must remain operational.
Hospitals cannot routinely hold ambulances for excessive periods without consequences for system readiness.
Operational measures such as call-processing time, turnout time, response time, unit availability, mutual-aid use, hospital turnaround, fleet reliability, and out-of-service time remain important.
The difference is that these measures should exist alongside clinical measures.
Operational reliability tells us whether the system reached the patient.
Clinical quality tells us what happened after it arrived.
A high-performing EMS system must be able to demonstrate both.
Workforce Measures Can Predict Clinical Problems
Clinical performance cannot be separated from the workforce delivering the care.
Vacancy rates, turnover, excessive overtime, fatigue, injuries, field-training completion, experience levels, and orientation quality can eventually appear as clinical problems.
A system operating with chronic vacancies and exhausted clinicians may still meet operational standards for a period of time.
That does not mean the model is sustainable.
Workforce measures therefore deserve consideration within a broader system-performance framework.
They should be interpreted carefully.
An agency may struggle with recruitment because of regional labor conditions, reimbursement limitations, housing costs, licensing requirements, or competition from other healthcare employers.
Those factors may be outside the direct control of local leadership.
For that reason, workforce metrics may be more appropriate as system-health indicators than punitive contract measures.
They still matter because they help leaders understand what may be happening beneath the surface.
Education Should Measure Competency, Not Attendance
EMS organizations devote significant resources to education.
Yet educational performance is still often measured by attendance.
A clinician attended the class.
A clinician completed the online module.
A clinician signed the roster.
Those measures demonstrate participation.
They do not necessarily demonstrate competence.
For high-risk clinical procedures, competency matters far more than attendance.
Airway management, pediatric resuscitation, cardioversion, pacing, medication administration, neonatal emergencies, and other low-frequency skills require the ability to perform under pressure, not simply exposure to information.
A stronger quality framework connects education to field performance.
If airway performance declines, education should respond.
If stroke recognition varies significantly between clinicians, education should respond.
If medication errors cluster around a particular process, the organization should examine both training and system design.
Then performance should be measured again.
That creates a continuous relationship between education and clinical practice.
Training is no longer an isolated department activity.
It becomes part of the quality system.
Alternative Care Requires Different Measures of Success
The evolution of EMS makes this discussion increasingly important.
Community paramedicine, mobile integrated healthcare, nurse navigation, telehealth, treat-in-place programs, and alternative destination models challenge traditional assumptions about what successful EMS activity looks like.
Historically, the ambulance transported the patient to an emergency department.
The completed transport became an easily measured unit of activity.
But what happens when the best outcome is avoiding an unnecessary emergency department visit?
A nurse-navigation program may connect a patient with urgent care, behavioral health, primary care, telehealth, or another appropriate resource.
A community paramedicine program may prevent a repeat 911 call.
A treat-in-place pathway may safely resolve a problem without ambulance transport.
If an EMS performance system measures only ambulance movement, these programs may appear less productive precisely because they are doing something more sophisticated.
That is a fundamental measurement problem.
Alternative-care programs need measures that reflect their intended value.
Was the patient safely navigated to another level of care?
Did the patient require a repeat 911 response?
Was escalation to emergency care appropriate when necessary?
Were adverse events identified?
Did the patient successfully access the referral?
Was the patient satisfied with the experience?
Did the program reduce unnecessary utilization without compromising safety?
As EMS changes, performance measurement must change with it.
Otherwise, new models will continue to be judged by standards designed for an older version of the profession.
Not Every Measure Belongs in a Contract
This is one of the most important distinctions EMS leaders and contract administrators need to make.
A clinically meaningful measure does not automatically belong in a contractual penalty structure.
Some measures are mature enough, standardized enough, and sufficiently within organizational control to support formal accountability.
Others are better suited to required reporting.
Still others should be used primarily for continuous quality improvement.
These categories should not be confused.
A contractual performance measure should have clear definitions, reliable data, meaningful organizational influence, and enough importance to justify consequences when performance repeatedly falls below expectations.
A reporting measure may be highly valuable even when the organization does not control every contributing factor.
Cardiac arrest survival is a good example.
It should be monitored.
It should be discussed.
It should influence system design.
But variation in survival should not automatically be interpreted as a contractual failure by the ambulance provider.
Quality-improvement measures serve a different purpose.
Perhaps an agency identifies that preoxygenation before advanced airway placement is inconsistent.
That can become a focused performance objective.
Leadership establishes a baseline.
Education and operational changes are implemented.
Performance is monitored.
The measure remains prominent until meaningful improvement occurs.
That approach allows the performance system to respond to actual needs rather than permanently expanding the contract every time a new improvement opportunity is identified.
Financial Penalties Require Caution
Metrics change behavior.
That fact should never be underestimated.
When an organization knows that a particular number determines contractual success, leadership and frontline behavior will naturally move toward optimizing that number.
Sometimes that is exactly what we want.
Sometimes it creates unintended consequences.
Response-time standards demonstrate this clearly.
Aggressive requirements can drive deployment strategies, staffing patterns, posting decisions, emergency driving behavior, and resource allocation.
Clinical measures can do the same thing.
If airway performance is measured only by successful intubation, clinicians may feel pressure to attempt intubation when another airway strategy would be safer.
If destination compliance is designed poorly, clinicians may bypass hospitals unnecessarily.
If pain-management performance is defined only by medication administration, providers may feel pressured to medicate even when medication is not clinically necessary.
If documentation scores are based on boxes being checked, clinicians may learn to complete required fields without improving the underlying quality of the clinical record.
Before implementing a metric, leaders should therefore ask two questions.
What behavior are we trying to encourage?
And what behavior might we accidentally encourage?
That second question is often where unintended consequences begin.
Definitions Matter More Than They Seem
Every percentage depends on a denominator.
Suppose an organization reports an 86 percent first-pass airway success rate.
Eighty-six percent of what?
All advanced airway attempts?
Only endotracheal intubations?
Adult patients only?
Cardiac arrest and non-arrest patients together?
Were pediatric cases included?
Were rapid-sequence procedures included?
Were unsuccessful attempts documented consistently?
Were supraglottic airways treated differently?
Without standardized definitions, two agencies can report the same percentage while measuring fundamentally different populations.
That makes meaningful benchmarking impossible.
A mature performance system requires clear numerator and denominator definitions, appropriate exclusions, consistent documentation, and standardized interpretation.
The less glamorous parts of performance measurement often determine whether the final dashboard can actually be trusted.
How Many Measures Are Enough?
There is no perfect number.
But there is a point where measurement becomes counterproductive.
An airway committee may appropriately monitor dozens of variables related to advanced airway management.
A clinical department may follow a much larger portfolio of quality indicators.
An executive team does not need to review every one of those variables every month.
Different levels of the organization require different levels of detail.
A well-designed executive scorecard should identify the areas requiring leadership attention.
It should not recreate the entire clinical database.
Ten meaningful indicators that influence decisions may be more valuable than one hundred indicators that are routinely presented but rarely discussed.
The purpose of an executive dashboard is not to prove that the organization has data.
It is to help leaders recognize what requires action.
The Real Goal Is Improvement
Measurement is not quality improvement.
It is only the beginning.
An organization can collect thousands of data points, build sophisticated dashboards, and publish impressive reports without changing patient care.
The value appears when the information creates action.
Performance is measured.
Variation is identified.
Leadership investigates.
Clinicians provide context.
A potential cause is identified.
An intervention is developed.
Education or operational changes occur.
Performance is measured again.
Feedback is provided.
The process is adjusted.
Then the cycle continues.
That is quality improvement.
A performance measure should not simply tell an executive team that compliance is 82 percent.
It should help leaders understand why performance is 82 percent, whether that level is acceptable, which factors are influencing it, and what the organization can do next.
When data creates those conversations, measurement becomes useful.
When it does not, the dashboard becomes decoration.
A Better Definition of EMS Performance
A modern EMS performance framework should provide a balanced picture of the system.
It should measure operational reliability because timely response matters.
It should measure clinical quality because effective treatment matters.
It should measure patient safety because reducing preventable harm matters.
It should measure documentation because reliable clinical records and reliable data matter.
It should measure destination decision-making because EMS is part of a broader healthcare system.
It should consider workforce stability and competency because people ultimately deliver the care.
It should measure quality improvement because identifying problems without demonstrating improvement is not enough.
And increasingly, it should measure system value because EMS is evolving beyond emergency transportation.
That does not require hundreds of contractual indicators.
It requires disciplined selection.
Before adding any measure, leaders should ask:
Does this matter?
Can EMS influence it?
Can we measure it consistently?
Can we act on the result?
Could the metric create unintended behavior?
Does it belong in the contract, in required reporting, or in a quality-improvement program?
And finally:
Will knowing this number help us make the system better?
If the answer to that last question is no, the metric may not deserve a place on the dashboard.
The Conversation Continues
Part 1 of this series argued that EMS performance must measure more than speed.
Part 2 asks us to be equally careful about what comes next.
Moving beyond response time does not mean measuring everything.
It means selecting measures that provide a clearer picture of whether patients are receiving timely, safe, effective, appropriate, and well-documented care from a reliable healthcare system.
The next challenge is implementation.
Part 3 will examine how these concepts can be translated into a practical EMS performance scorecard.
How should measures be weighted?
How should baselines and thresholds be established?
Which measures should trigger corrective action?
When should financial incentives or penalties be used?
How should clinical quality and operational reliability be balanced?
How should organizations report performance without creating an administrative system so complex that it becomes impossible to manage?
Those questions will determine whether moving beyond response time becomes a meaningful evolution in EMS accountability or simply another reporting requirement.
The objective is not to eliminate the clock.
The objective is to make sure we are also measuring what happens after the ambulance arrives.
Astera Global Solutions
Astera Global Solutions works with EMS and healthcare organizations at the intersection of clinical quality improvement, education, operational performance, leadership, and system development.
For organizations evaluating EMS performance measurement, clinical quality programs, contract governance, or system improvement, the goal should not simply be to create another dashboard.
It should be to identify measures that are meaningful, measurable, actionable, and capable of creating better systems and better patient care.
For more information or to discuss EMS performance measurement and clinical quality improvement, contact info@asteraglobalsolutions.net or visit asteraglobalsolutions.net.
References
Kupas, D. F., Zavadsky, M., Burton, B., et al. (2024). Joint Position Statement on EMS Performance Measures Beyond Response Times. Prehospital Emergency Care.
National EMS Quality Alliance. National EMS Quality Alliance Measure Set.
National EMS Quality Alliance. National EMS Quality Alliance Measure Development Process.
National EMS Quality Alliance. National EMS Quality Alliance Airway and Trauma Quality Measures.
National Highway Traffic Safety Administration, Office of EMS. EMS Agenda 2050: A People-Centered Vision for the Future of Emergency Medical Services.
Comments