Essentially what we are looking for is advice on what statistical analysis to run and how to manage the data etc...I have other articles that are similar that detail there statical analysis if need be.
In the domain of paramedicine students typically have their simulation scenarios evaluated by using a Global Rating Scale (GRS) that has been validated by Tavares and colleagues (Tavers et al., 2013). The GRS is a seven-dimension scale that is used to assess paramedic student’s competence as it pertains to: situational awareness, history gathering, patient assessment, decision making, resource utilization, communication, and procedural skills (Tavers et al., 2013). Each of the seven themes listed above are measured on a scale that ranges from 1 (unsafe) to 7 (exceptional) (Tavers et al., 2014). It has been established that the GRS is valid and possesses high inter-rater reliability when it is used to evaluate paramedics in training, additionally, there is evidence to suggest that values achieved on a GRS in simulation are transferable to values that are attained in a real clinical context (Tavers et al., 2014).
Although the GRS has been proven to be the gold standard when it comes to paramedic student evaluation, to the best of knowledge, no one has explored the effects that differential rater function over time (DRIFT) has on the outcomes of the grades obtained on the GRS. The concept of DRIFT has been demonstrated in other areas of education and is typically a result in increasing leniency due to rater fatigue (McLaughlin et al., 2009). Fairness in assessment is crucial to education, additionally, in a domain such as paramedicine it is important that standardization occurs in evaluation as public health and safety could be compromised if this is not the case (Yeates et al., 2019).
As it is crucial for student success and public safety to ensure that evolutions of paramedic performance are accurate, the primary purpose of this study is to explore if rater fatigue contributes to DRIFT during multiple GRS evaluations for the GRS raters.
The research was approved by the Collège Boréal Research Ethics Board. Following ethical approval consent was acquired from participants post-practical examinations. This study examined rater evaluations during a practical paramedic student evaluation using the GRS that compromised ______stations lasting ___minutes.
The raters were all currently employed paramedics who were in possession of the Advanced Emergency Medical Care Assistant (AEMCA) qualification and legally allowed to work as a paramedic in the province of Ontario, Canada. Additionally, the raters were staff at Collège Boréal (Collège Boréal, Sudbury, Ontario, Canada). Each of the raters involved in the study were familiar with the GRS evaluation tool and had previously been involved in their evaluations before. The examinees were first- and second-year paramedic students that were enrolled in the college and were participating in the curriculum required practical assessments that take place three times per semester.
The student examinees were randomly assigned to start at a different station and moved through the GRS circuit in the same order. Each rater scored each paramedic student examinee at the same station and was given a single rest period during the practical examination that was staggered throughout.
In order to estimate the difficulty for each station we used the Prehospital Canadian Triage and Acuity Scale (CTAS) as outlined in the Paramedic Guide Version 2.0 that is published by the Emergency Health Services Branch from the Ministry of Health and Long-Term Care in Ontario, Canada. The CTAS scale is a 1-5 scale with level one being resuscitation, level two being emergent, level 3 urgent, level 4, less urgent, and level 5-non urgent. We presumed that the more urgent the situation the more complex the simulated scenario.