Summative Usability Evaluations

A Summative Usability evaluation involves recruiting targeted users as test participants and asking these users to complete a set of pre-defined tasks. An expert test facilitator conducts the testing via an established test protocol while the test sessions are recorded and later analyzed.

EHR vendors are required to conduct and report on a Summative Usability Evaluation of their system as part of the 2014 Meaningful Use Certification
A Summative evaluation is conducted towards the end of the project to validate the design against specific goals.

Summative testing requirements for ONC Certification

Summative usability testing is an industry standard usability methodology (see http://bit.ly/13WYqTU ) During the testing a trained user experience professional works with the EHR vendor to create a series of representative tasks and then measures the amount of time it takes to complete the task (time on task), how many and what type of errors occur (error rates), and satisfaction with the task interaction (user satisfaction). The results —as measured by things like task times, assisted and un-assisted completion rates and standardized satisfaction scores --can be benchmarked against current industry standards.

Summative testing for ONC certification needs to test the following 9 areas:

  • Section 170.315(a)(1) Computerized provider order entry – medications
  • Section 170.315(a)(2) Computerized provider order entry – laboratory
  • Section 170.315(a)(3) Computerized provider order entry – diagnostic imaging
  • Section 170.315(a)(4) Drug-drug, drug-allergy interaction checks
  • Section 170.315(a)(5) Demographics
  • Section 170.315(a)(9) Clinical decision support (remove 1/1/25)
  • Section 170.315(a)(14) Implantable device list
  • Section 170.315(b)(2) Clinical information reconciliation and incorporation
  • Section 170.315(b)(3) Electronic prescribing
  • Section 170.315(b)(11) Decision support interventions