A flawed approach to studying EHR cost-effectiveness

  • Steven Shaha, Ph.D., DBA
  • 03/08/2012

A study published this week in Health Affairs purports to call into question the ability of Electronic Health Records to reduce imaging and lab tests and therefore save money.  The study has been reported in the New York Times and elsewhere but generally with little insight into the substance of the authors’ argument.  

A closer look reveals that, far from proving that EHRs cannot deliver their promised cost savings, the study provides compelling evidence that EHRs armed with advanced Clinical Decision Support (CDS) are what is most needed to cut costs while improving quality. 

The size of datasets analyzed by the researchers is rock solid, as are the results quoted. But the conclusions in the study raise as many questions as they answer. Most notably, the final sentence in the authors’ abstract reads: “We conclude that use of these health information technologies, whatever their other benefits, remains unproven as an effective cost-control strategy with respect to reducing the ordering of unnecessary tests.”  

At least three criticisms come immediately to mind.

First, there appears to have been no evaluation of the appropriateness of the additional tests conducted.  Did the initial imaging test or lab result raise important clinical questions, and the diagnosticians involved were wise to seek further information?  The conclusions speak of “unnecessary tests,” yet the data are limited to frequencies of “additional tests,” with no insight into clinical appropriateness or diagnostic precision.    

Second, the real question of cost is never directly addressed, only assumed or surmised.  The authors did not compute the total cost-of-care for patients who received additional tests.  The additional tests may in fact have dramatically REDUCED the longer term costs of care versus making wrong diagnoses based on partial data.  Without any true or comparative cost-of-care data, the reader lacks a full picture upon which to evaluate the impact of the additional testing. 

Finally, these data evaluate the impact of accessibility of test-related information, not the underlying benefits of EHRs or CDS.  The study did not explore the impact of CDS within an EHR, even CDS as simple as alerts and order sets. Yet ambulatory EHRs that are equipped with advanced CDS enable users to set alerts to obviate additional tests deemed clinically unnecessary, or duplicate order error.  The advanced central nervous system in these solutions lets them search their data, analyze it and inform the clinician. In the case of duplicate tests or tests flagged as clinically unnecessary, a CDS alert would pose this simple question: “Are you sure this duplicate test is needed?” or “Are you sure you want to order this additional test against the EBM reviewed by our own clinical leadership team?”

Armed with that degree of insight into the patient’s health status, clinicians can make clinically sound, cost-effective decisions about further tests.  It’s the kind of decision support that differentiates today’s most advanced EHRs from the pedestrian variety – and exactly what’s needed to transform our healthcare system today.

One final thought. Technology is both inevitable and a better future than paper records.  I would like to suggest to the editors of Health Affairs that they run an issue dedicated specifically to studying the impacts of EHR on practice-related decisions and outcomes, including reductions in cost-per-case, improved clinical outcomes, and enhanced safety. 

Those would be studies worthy of headlines.   


Tags: , ,

About the author

Steven H. Shaha, Ph.D., DBA, is a Professor at the Center for Policy & Public Administration, and the Principal Outcomes Consultant for Allscripts. Dr. Shaha received his first doctorate in Research Methods and Applied Statistics from UCLA and has taught and lectured at universities including Harvard, University of Utah, UCLA, Princeton, Cambridge and others. An internationally recognized thought leader, lecturer, consultant and outcomes researcher, Dr. Shaha has provided advisory and consulting work to healthcare organizations including the National Institutes for Health (NIH), and to over 50 non-healthcare corporations including RAND Corp, AT&T, Coca-Cola, Disney, IBM, Johnson & Johnson, Kodak, and Time Warner. Dr. Shaha has presented over 200 professional papers, has over 100 peer-reviewed publications in print, over 35 technical notes and two books. He served on the 15-member team that authored and piloted the Malcolm Baldrige National Quality Award for Health Care, and he contributed to the Baldrige for Education.


Todd Stein says:

03/08/2012 at 1:20 pm

Thanks Steve. I noticed that Dr. Mostashari, National Health IT Coordinator, shares your view that the study was not really about EHRs, much less meaningful use: http://www.healthit.gov/buzz-blog/meaningful-use/study-facts/#comment-134904


Your email address will not be published. Required fields are marked *