The pandemic has been a boon to evidence based medical education as it has finally compelled every stakeholder to engage in an unprecedented manner that makes for online archival of their learning activities, which serve as valuable material for audit and assessment.
It often happens that most program directors don't pay much attention to these knowing well that they are simply fabricated as a template with no department head seriously remembering any of their past thousand or so students when they revert for the letter of recommendation. This problem can be resolved if the department heads had easy access to some of the original work of their students to refresh their memory?
We have been working on an evidence based online accessible portfolio for each of the 150 UG (MBBS, BDS) and 10 PG (MD General Medicine) students graduating in medicine from our department every year (our department is among others in the undergraduate medical course that also involves surgery, gynecology, pathology, anatomy etc) and we have made a beginning by encouraging the students of each batch to maintain an open access E log book that documents their real patient centered learning experiences in verbal reports and non verbal presentations or procedural videos to evaluate and archive their competencies in patient care as well as theory driven conceptualizations and peer review competencies evidenced in their monthly summative assessments also accessible in their E logged online learning portfolios that are displayed for each batch entry year wise here :
MBBS
2015: https://
2017: https://
2018: https://generalmedicinedepartment.blogspot.com/2021/10/2018-mbbs-batch-formative-and-summative.html
2019: https://
MD General Medicine :
2019: https://generalmedicinedepartment.blogspot.com/2021/07/department-of-general-medicine_84.html?m=1
2020: https://generalmedicinedepartment.blogspot.com/2021/07/department-of-general-medicine_5.html?m=1
BDS:
2018: https://medicinedepartment.blogspot.com/2021/09/2018-bds-batch-internal-assessment-from.html?m=1
Elective students :
Now if any of the students from the above logged batches (with 150 in each batch) asks for a recommendation it will proceed in an evidence based manner as follows:
Dear Program Director,
I am pleased to write a letter of reference for Ms/Mr. _________ in her/his application for residency.
I currently look after the Department of General Medicine at xyz Institute of Medical Sciences and other than a graduate residency training and undergraduate program, we also host a global elective learning program supported nominally by the BMJ group detailed here: https://casereports.bmj.
Our department has known ______ since her second year General Medicine rotations, and she has interacted with us ever since in her group of 30 and most of her verbal and non verbal interactions can be accessed here :
http://medicinedepartment.
She also attended various academic sessions and actively participated in daily rounds. One such 11:00 AM to 1:00 PM regular ward round with students like her is archived here: https://youtu.be/ls-
Our department can strongly attest to most of our students abilities as we’ve had the opportunity to oversee them in both the In-patient and Out-patient setting and work with them during internship.
We wish him/her well in a new learning journey perhaps in your own institutional program.
FAQs and conversational learning on above:
The sample above was shared globally and following are the questions and conversations that ensued that are presented here after deidentifying :
Input 1
[8/3/2021, 9:15 PM] US med school program director : Interesting concept. Clearly you put a lot of effort in clinical teaching, as well as developing self directed learning habits in medical students. Then packaging that material into objective "evidence based" LORs.
[8/3/2021, 9:16 PM] US med school program director: The main comments are around usability and comparison. The latter point is less important because it does not matter if the student is top 1% of a poorly trained cohort or bottom 10% in a fantastic cohort.. so, summative assessment rubric is what I am more interested in.
[8/3/2021, 9:18 PM] US med school program director : In addition to the evidence base the videos and logs also showcase the learning environment from which the student is coming from as well as the students' engagement and performance. Soft things that are difficult to capture or convincingly convey in LOR text.
[8/3/2021, 9:18 PM] US med school program director: Thanks for sharing all these exciting things that you do.. :)
Input 1a:
1. What is the role of formative assessment versus summative assessment in the overall grading of a student?
"In our department we value the role of formative assessment as it shapes both the candidate's learning as in many ways ours is assessment for learning rather than assessment of learning.
Having said that we probably fall back on the default setting aka formal university evaluation for most of our official needs where the value of summative is pegged at maximum although as far as the practical exams are concerned it is the formative assessment that gives the head internal examiner (who generally coordinates the summative practical evaluation on the day of the exams), a formative bias that can lift a sinking student on the basis of regular past performances or downsize the game of a lucky one day player who may otherwise take away all the laurels from other better regular students based just on her single day performance.
2. Which variant of Bloom’s/Anderson’s Learning Taxonomy is followed in defining the Learning Outcomes/Objectives of your students? Can I see a sample of these?
Thanks for this very stimulating question. Would like to point here to one of my past lectures on this topic archived here: https://sites.pitt.edu/~
where we have made a case for approaching learning from Bloom level 3 and integrating other levels in the candidate's problem based learning journey.
3. Evaluation matrix used? Can I see a sample?
Yes I realized that the evidence presented here in the LOR for assessment of the candidate is akin to spreading out all the ingredients for cooking and asking the program director to DIY cook and savor it but may not be feasible for anyone to find the time to do that DIY and we may have to fall back on some evaluation matrix as you hinted.
We do plan to add more dimensions to the students online learning portfolio as you suggested and hope it becomes a life long learning portfolio for them and they continue to use it to publish and share their work open access even as they become teachers.
As earlier mentioned, we probably fall back on the default setting aka formal university evaluation for most of our needs and we may believe the formal curriculum isn't currently what we desire for our students or patients and our formative assessment is an attempt to improve on the status quo.
4. Whether all evaluation matrices used are shared at the beginning of each course?
As this is an evolving strategy for the subsequent batches we do share, while for the earliest batches we may not have been able to make it as clear.
5. What is the extent of objective and subjective indicators of quality of performance of students in practical tasks?
Very good question.
Will provide a few samples in student links below :
Subjective student reported :
Subjective assessment of body language (level of confidence etc) through video links: https://youtu.be/t21gVeCrTKE
Objective peer review assessment by the first student whose work is linked above:
We address all the above issues further in our recent book chapter available full text here : http://medicinedepartment.
Input 2:
The effort to make assessments mire objective is commendable. I can speak primarily to the US experience. My only comment would be that program directors in the US do not have the time to click through all the links and view the assessments. It would be helpful to have a “summary” of the students achievements-i.e. the highlights of the student’s strengths as assessed by the faculty. Specific links to cases/oral presentations can then be used to present evidence of these strengths.
As written currently, the letter is providing links to X or Y and asking the US program director to make an assessment by themselves without providing any information on what the med school faculty/director’s thought about the student’s skills
Sometimes, cultural barriers can affect the assessments when a US prigram directos views the student’s performance and tries to make an independent assessment
For instance, in India, our presentations are relatively deferential to the faculty and we do not raise our voices above a certain decibel when in front of senior doctors. It is quite the opposite in the US (especially in places like NYC-where a lot of Indian graduates train). They may interpret “deferntial” as “not confident” and further interpret “not confident” as “inadequate knowledge.”
Input 3:
I like the links etc that would help the program directors in writing a comprehensive letter of recommendation . Great job in getting all in one place .
When I see the letter on this side I want to see why this candidate wants to come to my institution and what would she be bringing that will add value and how is she different from the other candidates that are applying .
Knowing what she has presented/ read is good but I feel that the actual letter that goes out should not have any links attached .
It could end with “ feel free to call me with any further questions.”
Let me run this by our residents directors and other who interview candidates .
Input 4:
Looks authentic and convincing, detailing the curricular achievements of the person getting the recommendation.
I like the first part where the recommender is detailing his qualifications and his involvement, which gives the reader a sense of reassurance and honesty from the writer.
A good letter overall
