AJHPE 210

Experiences of graduating students from a medical programme five years after curricular transformation: A descriptive study

P McInerney, PhD; L P Green-Thompson, MB BCh, FCA (SA), MMed; D M Manning, PhD

Centre for Health Science Education, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Corresponding author: P McInerney (patricia.mcinerney@wits.ac.za)


Background. The University of the Witwatersrand introduced a new curriculum in 2003 where students could gain admission to the medical programme at two levels: directly as school leavers or following a degree as graduate entrants at the third year of study. From this point both groups of students continue in a combined class in a single curriculum.

Objective. To determine the experiences of the fifth cohort of graduating students from a medical programme following curricular transformation.

Method. A quantitative descriptive study was undertaken using a semi-structured questionnaire with both open- and closed-ended questions. There were 201 students in the graduating class, all of whom were invited to complete the questionnaire.

Results. A 74% response rate was obtained, of which 66% were school leaver entrants and 34% were graduates. Among the best experiences there were 59 comments relating directly to the programme. The worst experiences included perceptions of the lack of standardisation in clinical exams and feelings of inadequacy in relation to pharmacology and microbiology. Just under three-quarters of the participants felt ‘adequately prepared’ for the clinical years; 82% of the participants stated that they would make changes to the programme.

Conclusion. The placement of this evaluation at the conclusion of formal assessments may have contributed to the depth of responses and openness of respondents in the completion of the questionnaire. We highly recommend the value of obtaining data on students’ experiences and opinions of a programme at the point of exit from the programme.

AJHPE 2013;5(1):34-36. DOI:10.7196/AJHPE.210


The South African medical education curricular landscape has changed over the last two decades.1 Many of the innovations have attempted to embrace the ideals of the ‘five-star’ doctor first espoused by Boelen2 in 1993 and reinforced by local guidelines from the Health Professions Council of South Africa (HPCSA).

The focus of these changes has been the development of student-centred learning and a biopsychosocial approach to healthcare which is best learnt and delivered at the primary healthcare level. The development of reflective lifelong learners is another feature of these innovations.

The University of the Witwatersrand introduced a new curriculum in 2003.

Students were able to gain admission to the medical programme at two levels: directly as school leavers or following a degree as graduate entrants at the third year of study. From this point both groups of students continue in a combined class in a single curriculum. It had all the hallmarks of an innovative modern international curriculum. The transformed curriculum produced the first graduates in 2006.

Students spent two years in a hybrid problem-based learning programme arranged in organ-system blocks. The problem-based learning process is supported by content-based lectures and learning topics. The teaching of applied anatomy and physiology, pathology, pharmacology and clinical skills is integrated together with themes related to the patient-doctor relationship, the community-doctor relationship and personal and professional development, including bioethics and evidence-based medicine. Early clinical exposure was ensured through one health practice day a week at the teaching hospitals on the academic circuit.

The final two years of the programme were made up of six weekly rotations through the clinical disciplines, which include an Integrated Primary Care (IPC) block in the final year. Each of these rotations is concluded with a summative assessment which is immediately followed by remediation – if required. Student learning during this period is guided by a list of case competencies graded for the level of competence to be attained. The primary method of instruction remains teaching at the bedside. The clinical teachers are encouraged to consider the following during these teaching sessions: epidemiology, patient communication, quality of care and its evidence base, follow-up care, the multidisciplinary team, appropriate levels of healthcare delivery and the impact of resource constraints in patient management. In 2010, the fifth cohort of students graduated from the revised curriculum. This group was chosen to reflect on their experiences of the curriculum in terms of their achievement of outcomes in the domains of knowledge, skills and professional behaviour. The objective of the study was to determine the experiences of the fifth cohort of graduating students from a medical programme following curricular transformation.

Curriculum evaluation is integral to curriculum development and implementation.3 Students are important stakeholders in the curriculum. Graduates of medical programmes have been followed up at varying intervals after their graduation4 , 5 to determine how well they were prepared for practice. Watmough et al. 6 , 7 interviewed graduates from both old and new curricula to compare perceptions of preparedness for practice. Data collected after graduation are valuable in that it reflects the experiences of practitioners, compared with data collected at the point of graduation which has not been influenced by post-graduation experiences.8

Methods

A quantitative descriptive study was undertaken using a semi-structured questionnaire with both open- and closed-ended questions. The questionnaire established the respondents’ route of entry to the programme, explored students’ best and worst experiences of the curriculum, their perceptions of assessment throughout the curriculum and their perceived level of preparation for the final two years by the earlier years of the programme. The respondents were asked what changes they would make to the programme, if they felt changes were needed. They were able to make any general comments. Permission to undertake this study was obtained from the Human Ethics Research Committee in the Faculty of Health Sciences, University of the Witwatersrand.

Following their last final examination session, students were invited to participate in the study through the voluntary completion of the questionnaire. Subject information sheets and questionnaires were given to students and a box was provided for students to return their completed questionnaires anonymously. Completion of the questionnaire was taken as consent. Responses to the close-ended questions were captured in an Excel spreadsheet and the open-ended questions were analysed using a qualitative approach, where similar responses were grouped together and categorised.

Results

Of the 201 students in the graduating group, 149 completed the questionnaire, giving a 74% response rate. Of these, 66% were school leaver entrants and 34% were graduates, a similar representation to that of graduates in the class. In an open-ended question participants were asked to describe their best and worst experiences of the programme. Among the best experiences there were 59 comments that related directly to the programme, with statements such as the ‘integration of the subject matter’, ‘being taught to think out of the box’, ‘learning to communicate with patients’ and ‘the mix of a diversity of people in the programme who have different skills and viewpoints’. There were 31 comments that related directly to clinical practice, with participants commenting on the experience of becoming part of the healthcare system and around relating to patients and their diagnoses compared with learning facts from a textbook. Among the best experiences were comments related to the teaching that participants had received – ‘being taught by the best in the field’. Likewise, several of the worst experiences also related directly to the programme. Among the 50 comments in this category were the perceptions of the lack of standardisation in clinical exams and the feelings of inadequacy in relation to pharmacology and microbiology. Consistent with the lack of standardisation in the clinical exams were the comments related to inconsistencies in teaching and learning methods at the different learning sites. The pressure of examinations every six weeks were also related to as ‘worst experiences’.

There were two open-ended questions about assessment, the first of which asked respondents to comment about their overall experience of assessment in the programme. Of the 149 respondents, 40 (26.8%) made a global comment that the assessments were ‘good’, ‘okay’ or ‘fair’, and three did not answer this question. Many of these respondents were included among those who elaborated on the assessments. These additional responses could be categorised into clinical assessments (61 comments); theoretical assessments (22 comments); and general comments (10 comments). The majority (54.1%) of the comments about clinical assessment referred to the subjectivity of the examiners and 18% to the varying standards between the different clinical sites. Despite the frequency of examinations being listed in an earlier question as a ‘worst experience’, only 6.6% of the comments referred to the frequency of examinations as being a negative factor in the assessments.

The second of the questions asked participants to comment on the relationship between learning and assessment. While 46 (30.8%) did not answer this question, 99 (66.4%) gave responses that could be categorised into clinical assessments, theoretical assessments and general comments. In the category clinical assessments there were approximately 2.5 times the number of negative comments compared with positive comments. The most commonly mentioned negative comments were that clinical assessments tend to include aspects not taught in the ward, because they were not seen; that learning for clinical work and assessment are two different aspects; assessors’ expectations being too high for the period of exposure in the discipline; and a tendency for the ‘most interesting patient’ to be used in exams. The positive comments were if ‘people were willing to teach, it made a difference to the assessment’ and that learning and assessment correlated most of the time. There was a total of 22 comments in the category of theoretical assessment, with the most frequent comment (10; 45.4%) being that MCQs are not related to the learning objectives, with the most common clinical conditions often not being asked. Four participants felt that the frequency of exams made the assessment exam driven, resulting in ‘cramming’ for exams. In the general category were comments such as ‘assessment is not a true reflection of one’s knowledge’ and ‘projects were often very time consuming and the amount learned was not proportional to the time put in’.

Just under three-quarters (107; 72%) of the participants felt ‘adequately prepared’ for the clinical years. Twenty (13%) felt that they were well prepared and 22 (15%) felt that they were not prepared. None of the respondents who felt well prepared for the final years reported a need for change in the early clinical exposure. In contrast, of those who felt adequately prepared or unprepared for the final two years 27 (21%) felt that more clinical exposure would have improved their levels of preparedness. A participant who felt s/he was adequately prepared for the clinical years stated ‘I wish I had applied myself better in GEMP I and II [third and fourth year] in order to make GEMP III and IV [fifth and sixth year] easier’, and another wrote ‘you’ll never be prepared clinically to enter the wards, this comes with time and exposure’. ‘Even though there was not extensive clinical exposure, the theoretical exposure gave me some confidence when going to the hospitals’, demonstrates how participants linked the theoretical and clinical components of the programme in their responses.

The greater majority (122; 82%) of the participants stated that they would make changes to the programme. Changes suggested by 111 (91.3%) of this group fell into four categories, i.e. clinical, theory, teacher and other. The majority of the suggestions were in the categories clinical (68%) and theory (71%). The most frequent suggestion in the clinical category was to begin practical work earlier. In the theory category two suggestions received an equal number of responses – the request for specific courses in microbiology and pharmacology and for some rotations such as internal medicine to be increased in length. Teacher-related comments were few and each seemed to address a different aspect. In the ‘Other’ category a range of comments were made from a request for the administration of the programme to be improved to a request for a ‘mixer’ at the beginning of the third year so that the entire class is given an opportunity to get to know each other.

Discussion

The high response rate of 74% as well as the range and depth of the responses to the open-ended questions is an indication that the graduating students appreciated the opportunity to reflect on their experiences in the programme and express their opinions on what they found to be most and least valuable. One of the intentions of the new curriculum was to encourage students to become reflective practitioners, as suggested by Schön, who encouraged the integration of theory and practice.9 We believe that the demonstration of these insights is evidence of appropriate professional attitudes, one of the attributes of the five-star doctor. Many of the positive comments related specifically to integration of theory and practice which supported problem solving. There are a number of comments that emphasised the confidence which students felt in their clinical competence, shown by their feeling of being adequately prepared for the clinical years.

The negative comments which focused on the subjectivity and lack of standardisation in assessment of some of the clinical disciplines raise concern, as standardisation is a necessary condition for reliability and validity of assessment.10 The comments on written examinations which are not aligned with objectives, and the selection of clinical cases with diagnoses which are not necessarily important or common, are further indications that issues of content and construct validity may require further attention. Additional written assignments are often experienced by students as consuming a greater amount of time relative to the learning benefits derived. This sentiment is shared by other students in studies of reflective portfolios.11

The students’ suggestions for earlier clinical practice were interesting, as the new curriculum actually does introduce clinical experience a year earlier than prior to 2003. It is however likely that the time allocated in the weekly ‘health practice days’ may not be used as effectively as possible and further changes should be considered. Teaching and learning of both microbiology and pharmacology remain areas of concern, confirming the findings of a recent major study in which Smuts established a gap in these areas of knowledge for recent medical graduates.12 Smuts compared the performance of interns who were the last graduates of the old curriculum with the first graduating class of the new curriculum and found that both groups lacked confidence in their ability to prescribe. Similar findings have been reported in other studies,7 , 13 , 14 indicating that this is widely recognised as an area of concern in medical degree programmes.

One of the limitations of this study was the wide range of responses obtained in the open-ended questions. This made analysis and categorisation of results difficult. While this limited the ability to measure the frequency of a particular experience, it has highlighted areas for structured questionnaires in future research.

The findings of the study are valuable as a contribution to the evaluation of the programme from the point of view of the participants’ experiences in the acquisition of knowledge and skills as well as the development of professional attitudes and behaviours, i.e. levels 1 - 3 of Kirkpatrick’s evaluation model.15 The placement of this evaluation at the conclusion of formal assessments may have contributed to the depth of responses and openness of respondents in the completion of the questionnaire. The strength of the findings of this study are twofold. Firstly, respondents had just completed a final assessment in their programme and yet were prepared to complete the questionnaire as evidenced by just under three-quarters of the class returning the questionnaire. Furthermore, respondents did not confine themselves to the space provided for each question. Several wrote in the margins or made use of space at the bottom of the page, demonstrating the students’ commitment to providing constructive feedback on the programme. Their responses demonstrated considered thought to educational concepts. As a result, important areas have been identified for curricular modification and further faculty development. We highly recommend the value of obtaining data on students’ experiences and opinions of a programme at the point of exit from the programme.


References
    1. Seggie JL. MB ChB curriculum modernisation in South Africa – growing doctors for Africa. Africa Journal of Health Professions Education 2010; 2(1):8-14.

    1. Seggie JL. MB ChB curriculum modernisation in South Africa – growing doctors for Africa. Africa Journal of Health Professions Education 2010; 2(1):8-14.

    2. Boelen C. Frontline doctors of tomorrow. World Health 1994;47:4-5.

    2. Boelen C. Frontline doctors of tomorrow. World Health 1994;47:4-5.

    3. Goldie J. AMEE Education Guide No.29: Evaluating educational programmes. Medical Teacher 2006;28(3):210-224. [http://dx.doi.org/10.1080/01421590500271282]

    3. Goldie J. AMEE Education Guide No.29: Evaluating educational programmes. Medical Teacher 2006;28(3):210-224. [http://dx.doi.org/10.1080/01421590500271282]

    4. Schmidt HG, Vermeulen L, van der Molen HT. Longterm effects of problem-based learning: A comparison of competencies acquired by graduates of a problem-based and a conventional medical school. Med Educ 2006;40:562-567. [http://dx.doi.org/10.1111/j.1365-2929.2006.02483.x]

    4. Schmidt HG, Vermeulen L, van der Molen HT. Longterm effects of problem-based learning: A comparison of competencies acquired by graduates of a problem-based and a conventional medical school. Med Educ 2006;40:562-567. [http://dx.doi.org/10.1111/j.1365-2929.2006.02483.x]

    5. Prince KJAH, van Eijs PWLJ, Boshuizen HPA, van der Vleuten CPM, Scherpbier AJJA. General competencies of problem-based learning (PBL) and non-PBL graduates. Med Educ 2005;39:394-401. [http://dx.doi.org/10.1111/j.1365-2929.2005.02107.x]

    5. Prince KJAH, van Eijs PWLJ, Boshuizen HPA, van der Vleuten CPM, Scherpbier AJJA. General competencies of problem-based learning (PBL) and non-PBL graduates. Med Educ 2005;39:394-401. [http://dx.doi.org/10.1111/j.1365-2929.2005.02107.x]

    6. Watmough S, O’Sullivan H, Taylor D. Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews. BMC Med Educ 2010;10:65 [http://dx.doi.org/10.1186/l472-6920-9-64]

    6. Watmough S, O’Sullivan H, Taylor D. Graduates from a traditional medical curriculum evaluate the effectiveness of their medical curriculum through interviews. BMC Med Educ 2010;10:65 [http://dx.doi.org/10.1186/l472-6920-9-64]

    7. Watmough S, O’Sullivan H, Taylor D. Graduates from a reformed undergraduate medical curriculum based on Tomorrow’s Doctors evaluate the effectiveness of their curriculum 6 years after graduation through interviews. BMC Med Educ 2010;10:65. [http://dx.doi.org/10.1186/1472-6920-10-65]

    7. Watmough S, O’Sullivan H, Taylor D. Graduates from a reformed undergraduate medical curriculum based on Tomorrow’s Doctors evaluate the effectiveness of their curriculum 6 years after graduation through interviews. BMC Med Educ 2010;10:65. [http://dx.doi.org/10.1186/1472-6920-10-65]

    8. Green-Thompson L P, McInerney P, Manning DM, Mapukata-Sondzaba N, Chipamaunga S, Maswanganyi T. Reflections of students graduating from a transforming medical curriculum in South Africa: A qualitative study. BMC Med Educ 2012;12:49. [http://dx.doi.org/10.1186/1472-6920-12-49]

    8. Green-Thompson L P, McInerney P, Manning DM, Mapukata-Sondzaba N, Chipamaunga S, Maswanganyi T. Reflections of students graduating from a transforming medical curriculum in South Africa: A qualitative study. BMC Med Educ 2012;12:49. [http://dx.doi.org/10.1186/1472-6920-12-49]

    9. Schön DA. Educating the Reflective Practitioner. San Francisco, California: Jossey-Bass Inc, 1987.

    9. Schön DA. Educating the Reflective Practitioner. San Francisco, California: Jossey-Bass Inc, 1987.

    10. Downing SM, Haladyna TM. Validity threats: Overcoming interference with proposed interpretations of assessment data. Med Educ 2004;38:327-333. [http://dx.doi.org/10.1046/j.1365-2923.2004.01777.x]

    10. Downing SM, Haladyna TM. Validity threats: Overcoming interference with proposed interpretations of assessment data. Med Educ 2004;38:327-333. [http://dx.doi.org/10.1046/j.1365-2923.2004.01777.x]

    11. Rees CE, Shepherd M, Chamberlain S. The utility of reflective portfolios as a method of assessing first year medical students’ personal and professional development. Reflective Practice: International and Multidisciplinary Perspectives 2011;6(19):3-14. [http://dx.doi.org/10.1080/1462394042000326770]

    11. Rees CE, Shepherd M, Chamberlain S. The utility of reflective portfolios as a method of assessing first year medical students’ personal and professional development. Reflective Practice: International and Multidisciplinary Perspectives 2011;6(19):3-14. [http://dx.doi.org/10.1080/1462394042000326770]

    12. Smuts KB. Effects of Curriculum Change on Medical Graduates’ Preparedness for Internship. Johannesburg: University of the Witwatersrand, 2011.

    12. Smuts KB. Effects of Curriculum Change on Medical Graduates’ Preparedness for Internship. Johannesburg: University of the Witwatersrand, 2011.

    13. Woodward CA, Ferrier BM. The content of the medical curriculum at McMaster University: Graduates’ evaluation of their preparation for postgraduate training. Med Educ 1983;17:54-60. [http://dx.doi.org/10.1111/j.1365-2923.1983.tb01094.x]

    13. Woodward CA, Ferrier BM. The content of the medical curriculum at McMaster University: Graduates’ evaluation of their preparation for postgraduate training. Med Educ 1983;17:54-60. [http://dx.doi.org/10.1111/j.1365-2923.1983.tb01094.x]

    14. Han WH, Maxwell SRJ. Are medical students adequately trained to prescribe at the point of graduation? Views of first year foundation doctors. Scott Med J 2006;51(4):27-32. [http://dx.doi.org/10.1258/rsmsmj.51.4.27]

    14. Han WH, Maxwell SRJ. Are medical students adequately trained to prescribe at the point of graduation? Views of first year foundation doctors. Scott Med J 2006;51(4):27-32. [http://dx.doi.org/10.1258/rsmsmj.51.4.27]

    15. Morrison J. ABC of learning and teaching in medicine: Evaluation. BMJ 2003;326:385-387. [http://dx.doi.org/10.1136/bmj.326.7385.385]

    15. Morrison J. ABC of learning and teaching in medicine: Evaluation. BMJ 2003;326:385-387. [http://dx.doi.org/10.1136/bmj.326.7385.385]

Article Views

Abstract views: 2538
Full text views: 5220

Comments on this article

*Read our policy for posting comments here