ORCID Profile
0000-0003-0711-8792
Current Organisations
Royal Australasian College of Physicians
,
NSW Rural Doctors Network
Does something not look right? The information on this page has been harvested from data sources that may not be up to date. We continue to work with information providers to improve coverage and quality. To report an issue, use the Feedback Form.
Publisher: Wiley
Date: 08-07-2013
DOI: 10.1111/TCT.12029
Abstract: Medical education curricula are required to teach ethics, professionalism and patient safety, but there is no clear evidence as to when these topics should be introduced. The Personal and Professional Development (PPD) theme incorporates these topics, and is integrated throughout our postgraduate medical programme, but we were particularly interested in knowing when and how to introduce them to year-1 students. We describe an intensive PPD programme in the context of broader issues associated with the appropriate timing of PPD curricula in a medical programme. The PPD Intensive was held over 2 days (267 students in 2008 and 299 in 2009). Attendance was mandatory. Teaching included a mixture of didactic, small group and interactive sessions. Students completed a formative case-based assessment and evaluation questionnaire. In response to questions about a patient narrative, many students focused on the health provider perspective, even when asked specifically to put themselves in the patient's position when answering questions about the case. Students showed a superficial understanding of patient safety factors, even those who were previous health professionals. The Intensive was evaluated highly, but from the perspective of faculty staff there were issues relating to timing and lack of clinical context to give real meaning and purpose to the basic concepts underpinning the learning areas. The Intensive was designed to demonstrate to students the importance of PPD. A counter position is that students lack the clinical context required for effective learning of PPD topic areas. We discuss these conflicting beliefs and the changes made to the Intensive programme in 2010. As a result of the lessons, we changed the 2010 Intensive programme to strongly emphasise professionalism patient safety was moved to the later years.
Publisher: Wiley
Date: 12-2013
DOI: 10.1111/ANS.12388
Abstract: Conventional wisdom suggests that a patient with a positive sentinel node requires a completion axillary clearance to obtain full staging and durable regional control. However, this dictum has been challenged by the recent American College of Surgeons Oncology Group Z0011 Trial demonstrating that women with node-positive breast cancer who underwent sentinel node biopsy only, and were treated with breast conserving surgery and radiation, had equivalent locoregional recurrence and survival rates to those who had a completion axillary clearance. The aim of our study was to determine what the clinical impact of the Z0011 findings might be if patients were managed according to the Z0011 criteria in an Australian teaching hospital setting. We performed a retrospective review, using prospectively collected data, of all female patients with breast cancer assessed at the Westmead Breast Cancer Institute in 2010 and identified the subgroup who would potentially have fulfilled all Z0011 criteria. The characteristics and management of this group were compared with node-positive and to mastectomy patient subgroups. A total of 280 patients with invasive breast cancer were identified. Twenty-six patients satisfied all Z0011 criteria, representing 9.3% of all patients and 21.5% of node-positive patients. Twenty-two (84.6%) patients had a subsequent axillary clearance, with six (27.3%) having additional positive nodes. The Z0011 study is relevant to 9.3% of all breast cancer patients and 21.5% of node-positive breast cancer patients treated in a major Australian teaching hospital.
Publisher: Springer Science and Business Media LLC
Date: 12-2013
Publisher: Springer Science and Business Media LLC
Date: 18-01-2013
Publisher: AMPCo
Date: 02-2015
DOI: 10.5694/MJA14.00772
Abstract: To report the level of participation of medical schools in the Australian Medical Schools Assessment Collaboration (AMSAC) and to measure differences in student performance related to medical school characteristics and implementation methods. Retrospective analysis of data using the Rasch statistical model to correct for missing data and variability in item difficulty. Linear model analysis of variance was used to assess differences in student performance. 6401 preclinical students from 13 medical schools that participated in AMSAC from 2011 to 2013. Rasch estimates of preclinical basic and clinical science knowledge. Representation of Australian medical schools and students in AMSAC more than doubled between 2009 and 2013. In 2013 it included 12 of 19 medical schools and 68% of medical students. Graduate-entry students scored higher than students entering straight from school. Students at large schools scored higher than students at small schools. Although the significance level was high (P < 0.001), the main effect sizes were small (4.5% and 2.3%, respectively). The time allowed per multiple choice question was not significantly associated with student performance. The effect on performance of multiple assessments compared with the test items as part of a single end-of-year examination was negligible. The variables investigated explain only 12% of the total variation in student performance. An increasing number of medical schools are participating in AMSAC to monitor student performance in preclinical sciences against an external benchmark. Medical school characteristics account for only a small part of overall variation in student performance. Student performance was not affected by the different methods of administering test items.
Publisher: Wiley
Date: 15-06-2010
DOI: 10.1111/J.1365-2923.2010.03689.X
Abstract: There are significant levels of variation in candidate multiple mini-interview (MMI) scores caused by interviewer-related factors. Multi-facet Rasch modelling (MFRM) has the capability to both identify these sources of error and partially adjust for them within a measurement model that may be fairer to the candidate. Using facets software, a variance components analysis estimated sources of measurement error that were comparable with those produced by generalisability theory. Fair average scores for the effects of the stringency/leniency of interviewers and question difficulty were calculated and adjusted rankings of candidates were modelled. The decisions of 207 interviewers had an acceptable fit to the MFRM model. For one candidate assessed by one interviewer on one MMI question, 19.1% of the variance reflected candidate ability, 8.9% reflected interviewer stringency/leniency, 5.1% reflected interviewer question-specific stringency/leniency and 2.6% reflected question difficulty. If adjustments were made to candidates' raw scores for interviewer stringency/leniency and question difficulty, 11.5% of candidates would see a significant change in their ranking for selection into the programme. Greater interviewer leniency was associated with the number of candidates interviewed. Interviewers differ in their degree of stringency/leniency and this appears to be a stable characteristic. The MFRM provides a recommendable way of giving a candidate score which adjusts for the stringency/leniency of whichever interviewers the candidate sees and the difficulty of the questions the candidate is asked.
Publisher: Wiley
Date: 04-2009
DOI: 10.1111/J.1365-2923.2009.03292.X
Abstract: The multiple mini-interview (MMI) was initially designed to test non-cognitive characteristics related to professionalism in entry-level students. However, it may be testing cognitive reasoning skills. Candidates to medical and dental schools come from erse backgrounds and it is important for the validity and fairness of the MMI that these background factors do not impact on their scores. A suite of advanced psychometric techniques drawn from item response theory (IRT) was used to validate an MMI question bank in order to establish the conceptual equivalence of the questions. Bias against candidate subgroups of equal ability was investigated using differential item functioning (DIF) analysis. All 39 questions had a good fit to the IRT model. Of the 195 checklist items, none were found to have significant DIF after visual inspection of expected score curves, consideration of the number of applicants per category, and evaluation of the magnitude of the DIF parameter estimates. The question bank contains items that have been studied carefully in terms of model fit and DIF. Questions appear to measure a cognitive unidimensional construct, 'entry-level reasoning skills in professionalism', as suggested by goodness-of-fit statistics. The lack of items exhibiting DIF is encouraging in a contemporary high-stakes admission setting where candidates of erse personal, cultural and academic backgrounds are assessed by common means. This IRT approach has potential to provide assessment designers with a quality control procedure that extends to the level of checklist items.
Publisher: Wiley
Date: 04-2008
DOI: 10.1111/J.1365-2923.2008.03018.X
Abstract: We wished to determine which factors are important in ensuring interviewers are able to make reliable and valid decisions about the non-cognitive characteristics of candidates when selecting candidates for entry into a graduate-entry medical programme using the multiple mini-interview (MMI). Data came from a high-stakes admissions procedure. Content validity was assured by using a framework based on international criteria for s ling the behaviours expected of entry-level students. A variance components analysis was used to estimate the reliability and sources of measurement error. Further modelling was used to estimate the optimal configurations for future MMI iterations. This study refers to 485 candidates, 155 interviewers and 21 questions taken from a pre- prepared bank. For a single MMI question and 1 assessor, 22% of the variance between scores reflected candidate-to-candidate variation. The reliability for an 8-question MMI was 0.7 to achieve 0.8 would require 14 questions. Typical inter-question correlations ranged from 0.08 to 0.38. A disattenuated correlation with the Graduate Australian Medical School Admissions Test (GAMSAT) subsection 'Reasoning in Humanities and Social Sciences' was 0.26. The MMI is a moderately reliable method of assessment. The largest source of error relates to aspects of interviewer subjectivity, suggesting interviewer training would be beneficial. Candidate performance on 1 question does not correlate strongly with performance on another question, demonstrating the importance of context specificity. The MMI needs to be sufficiently long for precise comparison for ranking purposes. We supported the validity of the MMI by showing a small positive correlation with GAMSAT section scores.
Publisher: Wiley
Date: 04-2009
DOI: 10.1111/J.1365-2923.2009.03291.X
Abstract: Multiple mini-interviews (MMIs) are increasingly used in high-stakes medical school selection. Yet there is little published research about participants' experiences and understandings of the process. We report the findings from an international qualitative study on candidate and interviewer experiences of the MMI for entry into a graduate-entry medical school. Qualitative data from six interviewer focus groups and 442 candidate and 75 interviewer surveys were analysed using framework analysis. Multiple researchers (n = 3) analysed a proportion of the data and developed a thematic framework capturing content-related (i.e. what was said) themes that emerged from the data. This thematic framework was then used to code the complete dataset. Several key themes were identified, including participants' perspectives on having: (i) a one-to-one interview (ii) multiple assessment opportunities (iii) a standardised, scenario-based interview (iv) a mini-interview, and on (v) the attributes currently measured by the MMI, and (vi) other attributes that should be assessed. We gained a deeper understanding of participants' experiences of a high-stakes, decision-making process for selection into a graduate-entry medical school. We discuss our findings in the light of the existing literature and make recommendations to address the issue of differing participant expectations and understandings of the MMI, and to improve the credibility and acceptability of the process.
Publisher: Wiley
Date: 10-07-2013
DOI: 10.1111/MEDU.12224
Abstract: Multiple mini-interviews (MMIs) have been used by The University of Sydney graduate medical and dental programmes since 2006. In 2011, interviews with international candidates were conducted using Skype (iMMI), whereas interviews with local candidates were conducted in person. We determined whether the MMI scores derived from both methods were comparable. We describe the feasibility, acceptability and cost-effectiveness of the iMMI. We compared 2011 international student internet-based iMMI results with data from 2009 international student MMIs and 2011 local student MMIs. Analyses of variance (anovas) were used to investigate equivalence of the two formats by exploring whether the medium of interviewing resulted in significantly different mean scores and variance for the in-person MMI and the iMMI. Acceptability of the process was informed by feedback surveys from interviewers and candidates, and cost savings were estimated. No significant difference was found between the 2011 iMMI scores for international candidates and MMI scores in 2009 (p > 0.05). There was no significant difference between the MMI scores for local and international candidates in 2011 (p > 0.05) the MMI scores for international candidates had greater variation (p < 0.01). Using generalisability theory, the reliability of the nine-question iMMI was 0.76 and for the MMI was 0.70. Delivery of the iMMI occurred smoothly and candidates and interviewers gave positive feedback on its format and delivery. Cost savings have been estimated to be over AU$50 000, representing an 84% saving. We believe this is the first study reporting an internet-based MMI for a high stakes interview. We have shown that interviewers were able to make valid and reliable decisions about candidates through the iMMI in a process that was acceptable to participants, producing comparable results to the in-person MMI with a saving of resources. The slightly wider variance in iMMI scores warrants further investigation.
No related grants have been discovered for Imogene Rothnie.