Page 317 - International Perspectives on Effective Teaching and Learning in Digital Education
P. 317
Assessment Tools for Non-Technical Skills
Discussion
We conducted a systematic literature review to examine validation evidence
for the tools used to assess NTS in multidisciplinary, simulation-based clin-
ical education. Based on this review, we identified seven assessment tools
that are suitable for evaluating NTS in simulated environments, particularly
for multidisciplinary healthcare teams, as these skills are crucial for their ef-
fectiveness. Similar to technical skills, mastering NTS involves more than just
acquiring knowledge or reaching a certain competency level – it requires the
ongoing and accurate application of these skills in clinical practice (Garbee
et al., 1).
The tools examined in this study were designed to identify the key elements
of NTS that are crucial for safe and efficient clinical care, and to evaluate these
skills through simulation-based scenarios. The results of our literature review
demonstrate that the seven tools we identified are mainly built around four
NTS domains: communication, situation awareness, teamwork and leader-
ship, which is also consistent with the findings of Gawronski et al. (). In
their literature review, Garbee et al. (1) identified two additional domains
frequently categorised as NTS tools: decision making and task management,
which are often included within ‘leadership’, ‘teamwork’ or ‘situation aware-
ness’ (Gawronski et al., ). In addition, Garbee et al. (1) show that the
six NTS domains in the ten tools studied have overlaps and convergences
that are consistent with recognised frameworks for teamwork in different in-
dustries. They also emphasise that the inclusion of additional tools to assess
multidisciplinary teams in systematic reviews would not lead to the emer-
gence of new related NTS domains.
The results of our review of the psychometric properties of various tools,
including validity and reliability, suggest that the TEAM tool is a suitable op-
tion for assessing NTS in multidisciplinary teams in simulation-based clinical
education. The TEAM tool was employed in four of the studies analysed. The
tool comprises three domains (leadership, teamwork, and task management)
and 11 items. The TEAM was studied in medical emergency and obstetric/
gynaecologic settings (Gawronski et al., ). Originally designed to evalu-
ate teamwork performance in both real and simulated emergency scenarios
(Cooper et al., 1), the tool has undergone extensive psychometric test-
ing, including measures such as Cronbach’s α, ICC, CVI, and Cohen’s κ (von
Wendt & Niemi-Murola, 18). Its face validity and content validity have been
reviewed by an international panel of resuscitation experts, demonstrating
strong internal consistency and moderate-to-high inter-rater reliability (Coo-
per & Cant, 14; Gawronski et al., ). However, although the tool is reliable
317
