Dr. Sanghavi has managed to, yet again, create quite a storm within EMS circles. For those of you that don’t recognize her name, she is the same researcher that brought us last November’s  much criticized “Outcomes After Out-of-Hospital Cardiac Arrest Treated by Basic vs Advanced Life Support”within JAMA Internal Medicine.  This time, she has published “Outcomes of Basic Versus Advanced Life Support for Out-of-Hospital Medical Emergencies” within the Annals of Internal Medicine.  And it appears she is getting the same not-so-positive reception from within the industry.

But that’s ok, because what she’s doing has amazing consequences for EMS.  The great thing about empirical research, especially of the magnitude that she is publishing at, is that single studies, with a single population, only carry so much weight.  This is even more profound within healthcare, and particularly so within an industry as under-studied as EMS.  Dr. Sanghavi is putting EMS research on the map, and for that I would like to buy her a drink (or a coffee).

While I’m very happy that EMS research is being published outside of the industry, I stand relatively neutral on the results of this study.  That said, I have done a preliminary read of the new article and its supplements.  These are my thoughts:

Basic Summary:

• The authors are looking at four categories of patients (trauma, stroke, heart attack, and respiratory failure).
  • They are evaluating survival at five different points in time (discharge, and then at 30/90/365/730 days) as well as the neurological function of the patient during their initial hospital stay.
  • Within each of these categories they use two different statistical analyses (propensity score weighting for within county analysis and an instrumental variable approach for between county analysis) on both the unadjusted and adjusted samples.
  • All told, this means that there were over 60 different regression models run to produce these results.
• Basic Life Support was shown to have statistically significantly higher outcomes in the following situations:
  • Trauma, unadjusted, propensity score, survival (dc/30/90)
  • Trauma, unadjusted, propensity score, neurological performance
  • Trauma, adjusted, propensity score, survival (dc/30/90/365/730)
  • Trauma, adjusted, propensity score, neurological performance
  • Trauma, instrumental variable, survival (30/90)
  • Stroke, unadjusted, propensity score, survival (dc/30/90)
  • Stroke, unadjusted, propensity score, neurological performance
  • Stroke, adjusted, propensity score, survival (dc/30/90/365/730)
  • Stroke, adjusted, propensity score, neurological performance
  • Stroke, instrumental variable, survival (30/90/365)
  • AMI, unadjusted, propensity score, neurological performance
  • AMI, adjusted, propensity score, survival (dc)
  • AMI, adjusted, propensity score, neurological performance
  • AMI, instrumental variable, survival (30/90/365/730)
  • Resp Failure, unadjusted, propensity score, survival (dc/30)
  • Resp Failure, unadjusted, propensity score, neurological performance
  • Resp Failure, adjusted, propensity score, survival (dc/30/90/365)
  • Resp Failure, adjusted, propensity score, neurological performance
• Advanced Life Support was shown to have statistically significantly higher outcomes in the following situations:
  • AMI, unadjusted, propensity score, survival (30/90)
  • AMI, adjusted, propensity score, survival (90/365/730)
• There was no statistically significant difference in the following situations:
  • Trauma, instrumental variable, survival (365/730)
  • Trauma, instrumental variable, neurological outcome
  • Stroke, instrumental variable, survival (730)
  • Stroke, instrumental variable, neurological outcome
  • AMI, unadjusted, propensity score, survival (dc)
  • AMI, adjusted, propensity score,  survival (30)
  • AMI, instrumental variable, neurological outcome
  • Resp Failure, unadjusted, propensity score, survival (90)
  • Resp Failure, adjusted, propensity score, survival (730)
  • Resp Failure, instrumental variable, survival (dc/30/90/365/730)
  • Resp Failure, instrumental variable, neurological outcome
• The authors made the following assumptions and conclusions based on their results:
  • Similar or better health outcomes associated with prehospital BLS than ALS in all cases of analyses for trauma, stroke, and respiratory failure.
  • Suggest that decreased outcomes in ALS patients could be result of delays in hospital management or iatrogenic injury (aka EMS caused patient harm).

Sample Construction:

• Patients living in rural counties (11% of initial sample) as well as Connecticut, Delaware, Hawaii, and Washington D.C (3% of initial sample) were excluded from the study.  A random selection of 20% of the remaining sample was used in the final analysis.  Given that the authors had close to 8 million EMS transports, their decision to use only 20% of the sample is not surprising.
• Outcomes of patients that were transported more than once were restricted to those at least X days since the last ride with the same diagnosis (X depends on which outcome level was being evaluated: 30, 90, 365, or 730 days).  I’m not certain how this would impact the types of patient observations being included within this sample, but I am absolutely positive it changes the demographics.
• Medicare patients were used because Medicare claims data is easily bought from CMSand it can be matched across various healthcare organizations (as these authors did), these two things led to a comprehensive data set that doesn’t exist anywhere else within EMS.
  • That said, because the data is being collected for insurance purposes, some variables that the EMS industry would deem valuable such as the interventions performed or the amount of time on scene do not exist.  This is the logic behind the creation of other EMS data registries such as CARES, ROC, and NEMSIS.
  • Medicare also lacks external validity within EMS, since not all patients are eligible for Medicare. This is likely the most pronounced for patients with traumatic injuries in urban environments.
• Patients who died at the scene were excluded.  Based on the requirements for the analysis this is the most appropriate decision for this study.  Per the authors EMS bills at the BLS level if a patient dies on scene, regardless of the actual training of the providers (but I question that based on my interpretation of the CMS Medicare Benefit Policy Manual, if anyone knows the answer please reach out).  A sensitivity analysis was done to see if the exclusion of this set of observations would impact the results, the authors claim that “accounting for deaths in the field would not change the direction of their results.”  Although by making this claim, I wonder if it would have changed the statistical significance.
• Diagnosis codes were assigned via ICD-9 coding done within the hospital of treatment since the authors stipulate that EMS diagnosis coding is “less accurate”.  They also use this logic to support their decision to exclude those patients that died on scene.  Although in this case they expand their argument against EMS  diagnosis coding by saying it “is unlikely to be accurate in general, but even more so in cases where there was little time to observe the patient.” As a result the logic EMS providers and dispatchers were following is lost in the analysis, even if their logic was incorrect, and therefore potentially the real source of any differences in outcomes between ALS and BLS.

Outcome Measures:

• Survival was measured to discharge, as well as 1 month, 3 months, 1 year, and 2 years after the incident.  This means that the authors would need to control for any differences in care the patients receive in the months and years after their initial 911 call.
  • To do this they used propensity score weighting to “match” on observable characteristics of the geographic region, the hospital, and the patient.  By controlling for observed characteristics, the authors are assuming that unobserved characteristics like all of the healthcare services the patient will receive between arrival to the emergency department and death will also be controlled for.  This is a very large assumption, but one that is necessary to do this type of analysis.
  • The authors also used an instrumental variable approach, which is less likely to be impacted by unobserved characteristics and able to account for confounding.  Instrumental variables are notoriously tricky, and this case is no different.  This study used the county-wide probability of other diagnoses (unrelated to the four being studied) receiving ALS care as the instrumental variable, which was calculated such that it was “dependent solely on shared resources and policy.”  However, by definition, the authors chose four potentially serious conditions for their study.  This means that the likelihood of ALS for other diagnoses would potentially be lower than the likelihood of a patient in the study sample receiving ALS (unless the county is 100% ALS).
• As with the JAMA article, poor neurological functioning was designated by the two most severe Cerebral Performance Categories, either a 4 (coma or vegetative state) or a 5 (brain death).  A score of 3 (severe cerebral disability) is not included in this indicator.  Some have argued against the exclusion of a 3, however the authors never explain whether changing the cut off would change the results.

Conclusions:

• The authors suggest that too much time is being spent on scene when ALS providers are present.  Nothing in their data can support this, although there is peer-reviewed literature that supports the idea that ALS crews spend more time on scene than BLS crews.
  • I can believe that major trauma might have had better outcomes under the “scoop and run” mentality, but only because of past research like the study showing that penetrating trauma victims fared better in Philadelphia when transported by Police rather than waiting for EMS.  That said, I’m not sure how much a gun shot victim on the streets of Philadelphia can tell us about a grandmother who fell in a nursing home, and vice-versa (unless we were comparing system A to system B, which isn’t what is done in this study).
  • The authors cite the AHA Stroke guidelines when arguing that time is potentially lost because care is being provided on-scene when it should be provided en-route.   One of the authors is a physician, I’d be curious to see him try to get a line in 90 year old veins on the bumpy streets of New Orleans. I know medics that can, but it’s never their optimal decision.
  • The authors cite a 2005 Pennsylvania study that suggests the median paramedic only performs one intubation annually.  If you read the actual study, the next paragraph explains that intubation frequency is correlated with patient contacts, such that urban EMS paramedics average 2.3 intubations per year.  The authors of this study also show that intubation frequency is not associated with transport time.  I will agree that intubating patients is uncommon for many in EMS, but I would argue that the sample chosen in the Sanghavi study is going to be on heavier end of intubation experience.  Urban paramedics are going to have more experience with these types of calls, and many of the urban EMS systems likely have rapid response vehicles with experienced paramedics that would go on the more severe calls (like those requiring an intubation).  Exactly the types of calls being studied in this analysis.
• The authors go on to suggest that prehospital providers are unknowingly harming their patients (aka: iatrogenic injury) during the course of care.  The specific example used involves IV fluids for penetrating traumas.  Regardless of the validity of that assertion, I question how many Medicare patients experience penetrating traumas on an annual basis.  In this study those would have likely been included in the category of “high severity” which only accounted for 16% of the total trauma sample size.
  • I commend the authors for having the gall to make such an accusation, however there is almost no patient safety literature in EMS.  Do errors occur?  Absolutely.  Do we need more research on it?  Absolutely.  But to suggest that errors occur at such a level to warrant causal linkage to patient outcomes of this magnitude is a far fetched supposition that has yet to be backed by scientific proof.
• The authors admit that propensity score weighting and the instrumental variable approach, when used in conjunction, allow for the “robustness” of inferences to be tested.  That being said, of the 20 instrumental variable regression models presented, over half were not statistically significant.  I suspect this influenced the use of the word “similar” next to the word “better” when they made their conclusions.

I stand relatively neutral on the results of this study. Using what the author’s considered to be the best available data, they performed a very clean analysis.  Methodologically speaking they were on par for the health services research industry.  And they are above par for the EMS industry, since no one within EMS has done a study of this magnitude (although I have a feeling that’s about to change).

I am even willing to believe that in many situations ALS may not provide addition benefits, but to suggest that it is potentially harmful is an exaggeration of the strength of the study’s research design. To a lay person, like a Washington Post reporter, the authors reach some very convincing conclusions that follow with the logic they present within their study.  But a lay person would assume that the study design being used was stronger than it actually is.  I applaud the authors decision to study something so complex, but I criticize their willingness to draw such exponential conclusions from a design as ill-fitting as the one they used.
For now, the most important lesson to be learned by the EMS industry, is that researchers outside of EMS are starting to realize that there is a way to study this industry.  Meaning that more research will be done, and more papers published in journals that are not specific to EMS.  If the EMS industry wants to have a seat at the table in order to avoid many of the potential misconceptions that an outside might make about the industry, it needs to make sure that it is supportive of the research process, and open to the idea of new questions being asked.

This article, and the one that came before it, are opening a window of opportunity for EMS. An opportunity we should not let go to waste.

Catherine Counts