Randomised controlled trial to assess the effect of a Just-in-Time training on procedural performance: A proof-of-concept study to address procedural skill decay
Abstract
Background: A subset of high-risk procedures present significant safety threats due to their (1) infrequent occurrence, (2) execution under time constraints and (3) immediate necessity for patient survival. A Just-in-Time (JIT) intervention could provide real-time bedside guidance to improve high-risk procedural performance and address procedural deficits associated with skill decay.
Objective: To evaluate the impact of a novel JIT intervention on transvenous pacemaker (TVP) placement during a simulated patient event.
Methods: This was a prospective, randomised controlled study to determine the effect of a JIT intervention on performance of TVP placement. Subjects included board-certified emergency medicine physicians from two hospitals. The JIT intervention consisted of a portable, bedside computer-based procedural adjunct. The primary outcome was performance during a simulated patient encounter requiring TVP placement, as assessed by trained raters using a technical skills checklist. Secondary outcomes included global performance ratings, time to TVP placement, number of critical omissions and System Usability Scale scores (intervention only).
Results: Groups were similar at baseline across all outcomes. Compared with the control group, the intervention group demonstrated statistically significant improvement in the technical checklist score (11.45 vs 23.44, p < 0.001, Cohen’s d effect size 4.64), the global rating scale (2.27 vs 4.54, p < 0.001, Cohen’s d effect size 3.76), and a statistically significant reduction in critical omissions (2.23 vs 0.68, p < 0.001, Cohen’s d effect size -1.86). The difference in time to procedural completion was not statistically significant between conditions (11.15 min vs 12.80 min, p = 0.12, Cohen’s d effect size 0.65). System Usability Scale scores demonstrated excellent usability.
Conclusion: A JIT intervention improved procedure perfromance, suggesting a role for JIT interventions in rarely performed procedures.
Elizabeth D. Rosenman
Assistant Professor, Emergency Medicine & Director of Simulation
James A. Grand
Associate Professor, Psychology
A scientist at heart, an organizational psychologist by training, and a lucky dad and husband all the time.
