ORCID Profile
0000-0002-5377-809X
Current Organisations
Monash Health
,
Monash University
Does something not look right? The information on this page has been harvested from data sources that may not be up to date. We continue to work with information providers to improve coverage and quality. To report an issue, use the Feedback Form.
Publisher: CSIRO Publishing
Date: 2011
DOI: 10.1071/AH10928
Abstract: The process of development, and implementation, of a multi-source feedback tool for consultant anaesthetists is described. Rater groups included the anaesthetist-in-charge, anaesthetic assistants, anaesthetic trainees and, for some, the nurse-in-charge of the floor. Multiple items were developed to rate consultant behaviour, especially non-technical aspects of behaviour, and used across some or all of the rater groups. Response rates for items and by rater groups are discussed. The collated information formed part of the annual staff appraisal process conducted by the director. The information is useful for the director and the process is well-accepted by the staff.
Publisher: BMJ
Date: 2014
Publisher: Springer Science and Business Media LLC
Date: 23-09-2016
DOI: 10.1007/S12630-016-0740-9
Abstract: Workplace-based assessment is integral to programmatic assessment in a competency-based curriculum. In 2013, one such assessment, a mini-Clinical Evaluation Exercise (mini-CEX) with a novel "entrustability scale", became compulsory for over 1,200 Australia and New Zealand College of Anaesthetists (ANZCA) trainees. We explored trainees' and supervisors' understanding of the mini-CEX, their experience with the assessment, and their perceptions of its influence on learning and supervision. We conducted semi-structured telephone interviews with anesthesia supervisors and trainees and performed an inductive thematic analysis of the verbatim transcripts. Eighteen supervisors and 17 trainees participated (n = 35). Interrelated themes concerned the perceived purpose of the mini-CEX, its value in trainee learning and supervision, and the process of performing the assessment. While few participants saw the mini-CEX primarily as an administrative burden, most focused on its potential for facilitating trainee improvement and reported positive impacts on the quantity and quality of feedback, trainee learning, and supervision. Finding time to schedule assessments and deliver timely feedback proved to be difficult in busy clinical workplaces. Views on case selection were ided and driven by contrasting goals - i.e., receiving useful feedback on challenging cases or receiving a high score by choosing lenient assessors or easy cases. Whether in idual mini-CEXs were summative or formative was subject to intense debate, while the intended summative use of multiple mini-CEXs in programmatic assessment was poorly understood. Greater clarity of purpose and consistency of time commitment are necessary to embed the mini-CEX in the culture of the workplace, to realize the full potential for trainee learning, and to reach decisions on trainee progression.
Publisher: Springer Science and Business Media LLC
Date: 03-09-2020
DOI: 10.1007/S10459-019-09913-5
Abstract: Medical educators are tasked with decisions on trainee progression and credentialing for independent clinical practice, which requires robust evidence from workplace-based assessment. It is unclear how the current promotion of workplace-based assessment as a pedagogical approach to promote learning has impacted this use of assessments for decision-making meeting both these purposes may present unforeseen challenges. In this study we explored how supervisors make decisions on trainee progress in practice. We conducted semi-structured interviews with 19 supervisors of postgraduate anesthesia training across Australia and New Zealand and undertook thematic analysis of the transcripts. Supervisors looked beyond the formal assessment portfolio when making performance decisions. They instead used assessment 'shadow systems' based on their own observation and confidential judgements from trusted colleagues. Supervisors' decision making involved expert judgement of the perceived salient aspects of performance and the standard to be attained while making allowances for the opportunities and constraints of the local learning environment. Supervisors found making progress decisions an emotional burden. When faced with difficult decisions, they found ways to share the responsibility and balance the potential consequences for the trainee with the need to protect their patients. Viewed through the lens of community of practice theory, the development of assessment 'shadow systems' indicates a lack of alignment between local workplace assessment practices and the prescribed programmatic assessment approach to high-stakes progress decisions. Avenues for improvement include cooperative development of formal assessment processes to better meet local needs or incorporating the information in 'shadow systems' into formal assessment processes.
Publisher: Elsevier BV
Date: 11-2021
Publisher: Springer Science and Business Media LLC
Date: 14-11-2019
DOI: 10.1007/S12630-018-1251-7
Abstract: Competency-based anesthesia training programs require robust assessment of trainee performance and commonly combine different types of workplace-based assessment (WBA) covering multiple facets of practice. This study measured the reliability of WBAs in a large existing database and explored how they could be combined to optimize reliability for assessment decisions. We used generalizability theory to measure the composite reliability of four different types of WBAs used by the Australian and New Zealand College of Anaesthetists: mini-Clinical Evaluation Exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussion (CbD), and multi-source feedback (MSF). We then modified the number and weighting of WBA combinations to optimize reliability with fewer assessments. We analyzed 67,405 assessments from 1,837 trainees and 4,145 assessors. We assumed acceptable reliability for interim (intermediate stakes) and final (high stakes) decisions of 0.7 and 0.8, respectively. Depending on the combination of WBA types, 12 assessments allowed the 0.7 threshold to be reached where one assessment of any type has the same weighting, while 20 were required for reliability to reach 0.8. If the weighting of the assessments is optimized, acceptable reliability for interim and final decisions is possible with nine (e.g., two DOPS, three CbD, two mini-CEX, two MSF) and 15 (e.g., two DOPS, eight CbD, three mini-CEX, two MSF) assessments respectively. Reliability is an important factor to consider when designing assessments, and measuring composite reliability can allow the selection of a WBA portfolio with adequate reliability to provide evidence for defensible decisions on trainee progression.
Publisher: BMJ
Date: 2014
Publisher: SAGE Publications
Date: 2015
DOI: 10.1177/0310057X1504300117
Abstract: A questionnaire on substance abuse was distributed electronically to the heads of 185 Australian and New Zealand College of Anaesthetists accredited anaesthesia departments in Australia and New Zealand. The response rate was 57%. From January 2004 to December 2013, 61 cases of substance abuse were identified, giving an estimated incidence of 1.2 cases per 1000 anaesthetist years. Of 44 detailed reports completed, the majority were aged between 30 and 49 years, were male and of specialist grade. However, when corrected for gender and grade, the estimated overall incidence was higher in females and twice as high for trainees compared with specialists. When compared with prior surveys, the pattern of substance abuse in Australia and New Zealand appears to have changed significantly, with a notable increase in propofol and alcohol abuse and a decrease in reported cases of opioid abuse. Common presenting features of abuse included intoxication and witnessed abuse. Seventy percent of cases had more than one comorbid condition, most frequently either mental health or family problems. Only 32% of abusers had made a long-term recovery within the specialty. Death was the eventual outcome in 18% overall, with a particularly high mortality associated with propofol abuse (45%). Trainee suicide from all causes was reported at three times the rate of specialists. The findings indicate that substance abuse remains a significant problem in Australia and New Zealand and is associated with a significant mortality rate.
Publisher: SAGE Publications
Date: 07-2019
Abstract: In this qualitative study, we report how Supervisors of Training, educational supervisors overseeing the learning of anaesthesia trainees, experience their role in practice. Using purposive s ling, we interviewed Supervisors of Training from across Australia and New Zealand. The interviews began by asking ‘what do you see as your role as a Supervisor of Training?’ then explored the response in detail. Following the technique of thematic analysis, inductive analysis occurred as data were collected until we generated a thematic structure sufficient to address our research question after 19 interviews. In the first three of the four identified themes, Supervisors of Training perceived themselves as the fulcrum of the learning environment, ‘the something in between’. These three themes were: guiding and assessing trainees identifying, supporting, and adjudicating trainee underperformance and mediating trainees’ relationship with the hospital. Participants perceived themselves as a broker between trainees, their colleagues, their hospital, the Australian and New Zealand College of Anaesthetists and the community to varying degrees at different times. Negotiating these competing responsibilities required Supervisors of Training to manage multiple different relationships and entailed significant emotional work. Our fourth theme, scarcity, described the imbalance between these demands and the time and resources available. The complexity of the Supervisor of Training role and the tensions between these competing demands is underappreciated. Our findings would support strategies to mitigate the administrative load and share the decision-making burden of the role and to enhance the capability of Supervisors of Training by requiring formal training for the role.
Publisher: Elsevier BV
Date: 09-2011
DOI: 10.1093/BJA/AER152
Abstract: The aim of this study was to explore anaesthetists' perceptions and experience of the implementation of multi-source feedback (MSF) for their performance review. Twelve semi-structured thematic interviews were conducted with staff anaesthetists and their director to explore the perceptions of, and responses to, the MSF system, the feedback given, and their work context. An inductive thematic cross-comparative analysis of the interview data was conducted. The themes that emerged clustered around the anaesthetists' understanding of MSF, the facilitation process, and the value anaesthetists placed on feedback from the different professional groups available as actual and potential sources of feedback data. The anaesthetists interviewed in this study accepted performance review and the role of MSF in it. Anaesthetists thought anonymity an important feature of MSF, and viewed the process as an opportunity for self-improvement. To the extent that MSF was viewed by the anaesthetists as surveillance by management, this was seen as being largely benign. Rather a view of the acceptance of self-responsibility for using the information gathered to improve their own performance was held by the majority of those interviewed. While broad s ling of sources within and outside anaesthesia was desired, most value was placed on feedback from both nurses and trainees within anaesthesia. The value of feedback from surgeons was contentious, and this may reflect underlying issues with this important relationship that are yet to be explored.
Publisher: Wiley
Date: 04-05-2023
DOI: 10.1111/MEDU.15118
Abstract: Specialty trainees often struggle to understand how well they are performing, and feedback is commonly seen as a solution to this problem. However, medical education tends to approach feedback as acontextual rather than located in a specialty‐specific cultural world. This study therefore compares how specialty trainees in surgery and intensive care medicine (ICM) make meaning about the quality of their performance and the role of feedback conversations in this process. We conducted a qualitative interview study in the constructivist grounded theory tradition. We interviewed 17 trainees from across Australia in 2020, eight from ICM and nine from surgery, and iterated between data collection and analytic discussions. We employed open, focused, axial and theoretical coding. There were significant ergences between specialties. Surgical trainees had more opportunity to work directly with supervisors, and there was a strong link between patient outcome and quality of care, with a focus on performance information about operative skills. ICM was a highly uncertain practice environment, where patient outcome could not be relied upon as a source of performance information valued performance information was diffuse and included tacit emotional support. These different ‘specialty feedback cultures’ strongly influenced how trainees orchestrated opportunities for feedback, made meaning of their performance in their day‐to‐day patient care tasks and ‘patched together’ experiences and inputs into an evolving sense of overall progress. We identified two types of meaning‐making about performance: first, trainees' understanding of an immediate performance in a patient‐care task and, second, a ‘patched together’ sense of overall progress from incomplete performance information. This study suggests approaches to feedback should attend to both, but also take account of the cultural worlds of specialty practice, with their attendant complexities. In particular, feedback conversations could better acknowledge the variable quality of performance information and specialty specific levels of uncertainty.
Publisher: Elsevier BV
Date: 05-2017
DOI: 10.1093/BJA/AEX037
Abstract: The learning environment describes the context and culture in which trainees learn. In order to establish the feasibility and reliability of measuring the anaesthetic learning environment in in idual departments we implemented a previously developed instrument in hospitals across New South Wales. We distributed the instrument to trainees from 25 anaesthesia departments and supplied summarized results to in idual departments. Exploratory and confirmatory factor analyses were performed to assess internal structure validity and generalizability theory was used to calculate reliability. The number of trainees required for acceptable precision in results was determined using the standard error of measurement. We received 172 responses (59% response rate). Suitable internal structure validity was confirmed. Measured reliability was acceptable (G-coefficient 0.69) with nine trainees per department. Eight trainees were required for a 95% confidence interval of plus or minus 0.25 in the mean total score. Eight trainees as assessors also allow a 95% confidence interval of approximately plus or minus 0.3 in the subscale mean scores. Results for in idual departments varied, with scores below the expected level recorded on in idual subscales, particularly the 'teaching' subscale. Our results confirm that, using this instrument, in idual departments can obtain acceptable precision in results with achievable trainee numbers. Additionally, with the exception of departments with few trainees, implementation proved feasible across a training region. Repeated use would allow departments or accrediting bodies to monitor their in idual learning environment and the impact of changes such as the introduction of new curricular elements, or local initiatives to improve trainee experience.
Publisher: Ovid Technologies (Wolters Kluwer Health)
Date: 27-04-2022
DOI: 10.1097/ACM.0000000000004501
Abstract: In competency-based medical education, workplace-based assessment provides trainees with an opportunity for guidance and supervisors the opportunity to judge the trainees’ clinical practice. Learning from assessment is enhanced when trainees reveal their thinking and are open to critique, which requires trust in the assessor. If supervisors knew more about how trainees come to trust them in workplace-based assessment, they could better engender trainee trust and improve trainees’ learning experience. From August 2018 to September 2019, semistructured interviews were conducted with 17 postgraduate anesthesia trainees across Australia and New Zealand. The transcripts were analyzed using constructivist grounded theory methods sensitized by a sociocultural view of learning informed by Wenger’s communities of practice theory. Participants described a continuum from a necessary initial trust to an experience-informed dynamic trust . Trainees assumed initial trust in supervisors based on accreditation, reputation, and a perceived obligation of trustworthiness inherent in the supervisor’s role. With experience and time, trainees’ trust evolved based on supervisor actions. Deeper levels of trainee trust arose in response to perceived supervisor investment and allowed trainees to devote more emotional and cognitive resources to patient care and learning rather than impression management. Across the continuum from initial trust to experience-informed trust, trainees made rapid trust judgments that were not preceded by conscious deliberation instead, they represented a learned “feel for the game.” While other factors are involved, our results indicate that the trainee behavior observed in workplace-based assessment is a product of supervisor invitation. Supervisor trustworthiness and investment in trainee development invite trainees to work and present in authentic ways in workplace-based assessment. This authentic engagement, where learners “show themselves” to supervisors and take risks, creates assessment for learning.
Publisher: Wiley
Date: 18-02-2021
DOI: 10.1111/MEDU.14464
Abstract: ‘Culture’ is a word frequently invoked within medical education literature to explain challenges faced by learners in practice. While social settings and practices are widely acknowledged as critical influences on medical education, there is vast variability in how the term ‘culture’ is employed. This may lead to confusion, resulting in assumptions and oversights. This critical literature review aims to characterise how the term ‘culture’ is explicitly and implicitly conceptualised in medical education research. Four leading English language journals in the medical education field were searched in a twelve‐month period for research papers or reviews that mentioned culture in title or abstract in a substantive way. A content analysis was undertaken of extracted definitions. In addition, metaphor analysis was used to identify conceptual metaphors, which were subsequently clustered thematically. Our search yielded 26 papers, 8 of which contained definitions, mostly from the organisational literature. We interpreted nine conceptual metaphors related to how the term culture was employed (terroir, ider, dominant force, toxic force, obstacle, microclimate, object, brand and holdall) in four categories (unchanging, powerful, can adapt around, can be used). This critical review reveals that medical education as a field: 1) draws most explicitly from the organisational literature 2) invokes culture in multiple means but in ways that privilege either acontextual human agency or all‐powerful social forces and 3) regards culture as a negative or neutral force but rarely a positive one. There is a notable absence around conceptualisations of 'culture' that allow educator, student and administrator agency but at the same time acknowledge the deep forces that various social settings and practices exert. Other literatures investigating learning cultures and cultural reflexivity focus on this nexus and may provide possible means to advance considerations of culture within medical education research.
Publisher: SAGE Publications
Date: 11-2015
DOI: 10.1177/0310057X1504300616
Abstract: The ‘Roles in Practice’ framework was introduced into the revised ANZCA curriculum in 2013. We conducted an online survey of Supervisors of Training in Australia and New Zealand to assess understanding of this framework, and teaching and perceived value of the non-scholar intrinsic roles within the framework. One hundred and forty-three survey responses were received (response rate 60.8%). The majority of respondents (52.1%) reported only a fair understanding of the framework. Formal teaching of all the roles was consistently reported as infrequent, with most teaching occurring through the informal curriculum. The Communicator, Collaborator and Professional Roles were rated as better taught and more important to teach than the roles of Health Advocate and Manager. The Communicator Role was perceived as being the role for which the development of resources would be most valuable. Respondents overwhelmingly nominated small group teaching as the preferred medium for resource development of all intrinsic roles. Our survey indicates that there is a need to increase both the understanding of the Roles in Practice framework and the teaching resources available in the ANZCA Supervisor of Training community.
Publisher: University of Otago Library
Date: 16-12-2022
Abstract: Introduction: Workplace-based assessments (WBAs) are used in competency-based training programs to facilitate learning and inform decisions on trainees’ progression. Competency in assessing WBAs is generally assumed how assessors develop these skills remains largely unknown. This study explored how novice assessors acquire WBA skills to generate insights to inform future faculty development. Methods: We conducted seven semi-structured interviews with provisional fellows, who are final-year trainees of the Australian and New Zealand College of Anaesthetists. Provisional fellows assess junior trainees as part of their preparation for consultant practice. We used constructivist grounded theory methods to analyse their experience of learning to perform WBAs. Results: Participants primarily learned WBA skills through experiential learning and self-reflection on their practice. They developed practical skills to accommodate WBA into clinical practice, including finding time to facilitate WBAs and selecting appropriate cases. Simultaneously, participants developed expertise in the critical aspects of the assessment, particularly learning to determine the standard required of trainees and improving their feedback skills. This learning engendered an increased appreciation of the purpose of WBA, helped participants develop their clinical expertise and prompted them to envision themselves in the consultant role. Participants felt underprepared for their assessor role and appreciated access to the “safety net” ongoing supervision provided. Conclusion: This study provides valuable insights into what is learned in becoming a WBA assessor and how it is learned. These insights could inform future faculty development initiatives to help improve graduating consultants’ development in this important area of consultant practice.
Publisher: SAGE Publications
Date: 11-2009
DOI: 10.1177/0310057X0903700605
Abstract: Changes in work practices have led to a decline in the opportunities for anaesthetic trainees to learn technical procedures in supervised practice. Efforts to mitigate medical error and other changes have coincided with the development of alternative training methods so that it is increasingly difficult to justify the traditional model of teaching technical procedures. The range of simulators available for training in technical procedures in anaesthesia continues to expand. While simulation has been widely adopted in anaesthesia for crisis management training, there is little documented evidence of its use for technical skills training. The theoretical basis for the use of simulation to aid the acquisition of psychomotor skills and the development of expertise is now well established. In addition, practical frameworks that allow this theory to be applied in a systematic fashion have been developed and successfully used in other specialties. Using the available simulation equipment and educational tools, trainees can be prepared to begin supervised practice having demonstrated adequate procedural knowledge and expertise in simulation. With the use of simulated patients there is also the opportunity to integrate non-technical skills as well where appropriate. This review summarises the justification for the use of simulation in technical skills training in anaesthesia and the educational theory that supports its use, and outlines one of the available frameworks that can be used to aid its application.
Publisher: Elsevier BV
Date: 05-2023
Publisher: Springer Science and Business Media LLC
Date: 12-2005
DOI: 10.1007/BF03021606
Publisher: Elsevier BV
Date: 06-2022
DOI: 10.1016/J.BJA.2022.03.013
Abstract: In postgraduate specialist training, workplace assessments are expected to provide the information required for decisions on trainee progression. Research suggests that meeting this expectation can be difficult in practice, which has led to the development of informal processes, or 'shadow systems' of assessment. Rather than rejecting these informal approaches to workplace assessment, we propose borrowing from sociology the concept of 'desire paths' to legitimise and strengthen these well-trodden approaches. We asked what information about trainees is currently used or desired by those charged with making decisions on trainee progression, and how is it obtained? We undertook a qualitative study with thematic analysis of semi-structured interviews of supervisors of training across Australia and New Zealand. From 21 interviews, we identified four interrelated themes, the first being the local context of training sites. The other three themes represent dilemmas in the desire for authentic and representative information about the trainee: 1) how the process of gathering and documenting information can filter, transform, or limit the original message 2) deciding when possible trainee deviation from performance norms warrants a closer look and 3) how transparent vs covert information gathering affects the information supervisors will provide, and how control over assessment is distributed between trainee and supervisor. From these themes, we propose a set of design principles for future workplace assessment. Understanding the reasons desire paths exist can inform future assessment redesign, and may address the current disjunct between the formal workplace assessment system and what happens in practice.
Publisher: SAGE Publications
Date: 11-2017
DOI: 10.1177/0310057X1704500615
Abstract: Burnout has a high prevalence among healthcare workers and is increasingly recognised as an environmental problem rather than reflecting a personal inability to cope with work stress. We distributed an electronic survey, which included the Maslach Burnout Inventory Health Services Survey and a previously validated learning environment instrument, to 281 Victorian anaesthetic trainees. The response rate was 50%. We found significantly raised rates of burnout in two of three subscales. Ninety-one respondents (67%) displayed evidence of burnout in at least one domain, with 67 (49%) reporting high emotional exhaustion and 57 (42%) reporting high depersonalisation. The clinical learning environment tool demonstrated a significant negative correlation with burnout (r=-0.56, P .001). Burnout was significantly more common than when previously measured in Victoria in 2008 (62% versus 38%). Trainees rated examination preparation the most stressful aspect of the training program. There is a high prevalence of burnout among Victorian anaesthetic trainees. We have shown a significant correlation exists between the clinical learning environment measure and the presence of burnout. This correlation supports the development of interventions to improve the clinical learning environment, as a means to improve trainee wellbeing and address the high prevalence of burnout.
Publisher: Wiley
Date: 20-09-2021
DOI: 10.1111/MEDU.14631
Abstract: For trainees to participate meaningfully in workplace‐based assessment (WBA), they must have trust in their assessor. However, the trainee's dependent position complicates such trust. Understanding how power and trust influence WBAs may help us make them more effective learning opportunities. We conducted semi‐structured interviews with 17 postgraduate anaesthesia trainees across Australia and New Zealand. Sensitised by notions of power, we used constructivist grounded theory methodology to examine trainees' experiences with trusting their supervisors in WBAs. In our trainee accounts, we found that supervisors held significant power to mediate access to learning opportunities and influence trainee progress in training. All episodes where supervisors could observe trainees, from simply working together to formal WBAs, were seen to generate assessment information with potential consequences. In response, trainees actively acquiesced to a deferential role, which helped them access desirable expertise and minimise the risk of reputational harm. Trainees granted trust based on how they anticipated a supervisor would use the power inherent in their role. Trainees learned to ration exposure of their authentic practice to supervisors in proportion to their trust in them. Trainees were more trusting and open to learning when supervisors used their power for the trainee's benefit and avoided WBAs with supervisors they perceived as less trustworthy. If assessment for learning is to flourish, then the trainee–supervisor power dynamic must evolve. Enhancing supervisor behaviour through reflection and professional development to better reward trainee trust would invite more trainee participation in assessment for learning. Modifying the assessment system design to nudge the power balance towards the trainee may also help. Modifications could include designated formative and summative assessments or empowering trainees to select which assessments count towards progress decisions. Attending to power and trust in WBA may stimulate progress towards the previously aspirational goal of assessment for learning in the workplace.
Publisher: Wiley
Date: 21-02-2023
DOI: 10.1111/MEDU.15028
Abstract: Sociocultural learning theory can help us address difficulties in achieving assessment for learning by expanding what we think of as learning and when we think learning occurs.
Publisher: Wiley
Date: 18-04-2014
DOI: 10.1111/ANAE.12625
Abstract: Assessment tools must be investigated for reliability, validity and feasibility before being implemented. In 2013, the Australian and New Zealand College of Anaesthetists introduced workplace-based assessments, including a direct observation of a procedural skills assessment tool. The objective of this study was to evaluate the psychometric properties of this assessment tool for ultrasound-guided regional anaesthesia. Six experts assessed 30 video-recorded trainee performances of ultrasound-guided regional anaesthesia. Inter-rater reliability, assessed using absolute agreement intraclass correlation coefficients, varied from 0.10 to 0.49 for the nine in idual nine-point scale items, and was 0.25 for a 'total score' of all items. Internal consistency was measured by correlation between 'total score' and 'overall performance' scale item (r = 0.68, p < 0.001). Construct validity was demonstrated by the 'total score' correlating with trainee experience (r = 0.51, p = 0.004). The mean time taken to complete assessments was 6 min 35 s.
Publisher: Elsevier BV
Date: 02-2017
DOI: 10.1093/BJA/AEW412
Abstract: Workplace-based assessments should provide a reliable measure of trainee performance, but have met with mixed success. We proposed that using an entrustability scale, where supervisors scored trainees on the level of supervision required for the case would improve the utility of compulsory mini-clinical evaluation exercise (CEX) assessments in a large anaesthesia training program. We analysed mini-CEX scores from all Australian and New Zealand College of Anaesthetists trainees submitted to an online database over a 12-month period. Supervisors' scores were adjusted for the expected supervision requirement for the case for trainees at different stages of training. We used generalisability theory to determine score reliability. 7808 assessments were available for analysis. Supervision requirements decreased significantly (P 0.7) with a feasible number of assessments. Adjusting scores against the expected supervision requirement considerably improved reliability, with G > 0.8 achieved with only nine assessments. Three per cent of trainees generated average mini-CEX scores below the expected standard. Using an entrustment scoring system, where supervisors score trainees on the level of supervision required, mini-CEX scores demonstrated moderate reliability within a feasible number of assessments, and evidence of validity. When scores were adjusted against an expected standard, underperforming trainees could be identified, and reliability much improved. Taken together with other evidence on trainee ability, the mini-CEX is of sufficient reliability for inclusion in high stakes decisions on trainee progression towards independent specialist practice.
Publisher: SAGE Publications
Date: 03-2015
DOI: 10.1177/0310057X1504300209
Abstract: The learning environment describes the way that trainees perceive the culture of their workplace. We audited the learning environment for trainees throughout Australia and New Zealand in the early stages of curriculum reform. A questionnaire was developed and sent electronically to a large random s le of Australian and New Zealand College of Anaesthetists trainees, with a 26% final response rate. This new instrument demonstrated good psychometric properties, with Cronbach's α ranging from 0.81 to 0.91 for each domain. The median score was equivalent to 78%, with the majority of trainees giving scores in the medium range. Introductory respondents scored their learning environment more highly than all other levels of respondents ( P=0.001 for almost all comparisons). We present a simple questionnaire instrument that can be used to determine characteristics of the anaesthesia learning environment. The instrument can be used to help assess curricular change over time, alignment of the formal and informal curricula and strengths and weaknesses of in idual departments.
Location: Australia
No related grants have been discovered for Damian Castanelli.