Sunday, December 5, 2021

Formative (internal) assessment criteria for guaging students impact on a patient centered learning ecosystem

Introduction background (and problem statement) : 


Formal healthcare education systems in Indian Medical education curriculum, tailor their assessments into two groups namely theory and practical. 

Although currently, "theory" appears to be an assessment framework based on "student conceptualizations," it's design has evolved in many universities to allow students to simply memorize factual content and crack it by copy pasting the same in their answers (reference : all university data bases archiving student answer papers and the one linked here may be an outlier: https://medicinedepartment.blogspot.com/2021/11/selected-answers-to-2017-internal.html?m=0). 

Practical and viva exams (interviews), on the other hand are meant to test a student's ability to capture and analyze patient data toward clinical decision making along with demonstrable procedural competencies. While much has been written around summative practical assessments (reference : https://medicinedepartment.blogspot.com/2021/03/final-university-mbbs-medicine.html?m=0), very little exists around how formative practice assessments have to be conducted in our local learning ecosystems (problem statement).

Although, many review articles exist, developed by local eminent educationists and policy makers (reference : https://www.ijabmr.org/article.asp?issn=2229-516X;year=2021;volume=11;issue=4;spage=206;epage=213;aulast=Saiyad;type=3) around overall assessment they are largely focused on summative quantitative assessment rather than formative qualitative assessment and to quote from the same article, "quantitative measurements provide the idea about the overall achievement of the students but give no idea about the factors affecting the performance often resembling a cross-sectional study, which didnot allow the teachers and students to learn contextually (due to lack of longitudinal information continuity)."

Method :

Getting back to how our current theory papers are structured in Indian Medical education system, it can at best claim to address the first level of Blooms taxonomy that is remembering and understanding (A candid lecture on blooms taxonomy here : https://sites.pitt.edu/~super1/lecture/lec54091/002.htm). Also, most colleges find it easier to administer repeated monthly theory assessment papers that they call FA1, FA2, FA3 FAn etc (where FA stands for formative assessment) and in some colleges this is internal assessment so it becomes IA1, IA2, IA3, IAn etc. 
 
We would have preferred not to mix qualitative formative assessments with quantitative summative assessments but due to the majority usage of this mixed method model (which is in reality a more frequently repeated quantitative summative assessment model masquarading as formative),  we too are compelled to develop and share our method of a compromise where we try to accommodate the summative theory paper quantitation as a springboard to begin their assessment process with their prowess in tackling the first level of blooms taxonomy. 

So the theory quantitation of 60 marks is divided into scores of 

10-20 

20-30

30-40

40-50

if a candidate achieves 

10-20 s/he would in the conventional summative be declared a failure but because our formative, internal assessment is not assessment OF learning but assessment FOR learning, we try to find out if the same person has made an impact on our learning ecosystem in other ways with her inputs in the wards around her patient that reflects his her's :

Approach to disease localization (BT level 1 -3)

enthusiasm to resolving initial diagnostic uncertainty (BT level 1-4)

toward therapeutic decision making and tenacity to evaluating patient requirements and outcome (BT level 1-5)

Developing and testing innovative diagnostic and therapeutic solutions ( BT level 1-6)

On patient care outcomes :

Was empathic trust built with patient and relatives?

Were the patient requirements identified adequately and a proper problem list made toward assessment? 

Was standard of care provided with provision for care continuity ?

So once we have the theory quantitation of each candidate's answer paper and find that a candidate securing 10-20 out of 60 hasn't been able to also participate in the patient centered practical learning ecosystem and is unable to touch upon any of the above listed impact criteria we may flag the student in the red zone of 10-20 and monitor his her progress closely to improve his her competence to a higher zone. 

20-30 would be still an orange zone outlier 

30--40 would be average and more than that would be a positive outlier  

Results :

So the internal/formative assessment results could be displayed in a mixed method manner with a quantitative-qualitative zone that is numbered and color coded. 

Monday, November 22, 2021

Rough draft of internal assessment 2017 batch September 2021 performance

 DEPARTMENT OF GENERAL MEDICINE 

8TH SEM 1ST INTERNAL MARKS – SEPTEMBER - 2021




The students who achieved marks toward higher end of the spectrum demonstrated reasonable competency with regard to scholarship (theory summative answers and formative log book) as well as leadership (pro activity in making an impact on our patient care workflow by their useful inputs both synchronously recorded in their presentation videos as well as asynchronously in their texted inputs recorded in their E logs.  

Possible explanations for those who received low marks  are :

Didn't attempt all questions in theory paper 

Inspite of attempting didn't demonstrate substantial scholarship competency in the topic. 

In day to day assessment were unable to share their daily learnings with peers, were largely uncommunicative and overall provided poor demonstration of scholarship and leadership competency necessary for practice of medicine. Hopefully they will improve after this feedback. 

R. No.

Name

Max Marks

(60)

1



2





66


20



108


20



144


20




147


20



172


20



174


20



176


20



181


20



186


20


Project : Outcomes of bicarbonate correction in severe high anion metabolic acidosis due to renal failure along with one case report form and CDSS

Project introduction :


"Replacement of sodium bicarbonate to patients with sodium bicarbonate loss due to diarrhea or renal proximal tubular acidosis is useful, but there is no definite evidence that sodium bicarbonate administration to patients with acute metabolic acidosis, including diabetic ketoacidosis, lactic acidosis, septic shock, intraoperative metabolic acidosis, or cardiac arrest, is beneficial regarding clinical outcomes or mortality rate. Patients with advanced chronic kidney disease usually show metabolic acidosis due to increased unmeasured anions and hyperchloremia. It has been suggested that metabolic acidosis might have a negative impact on progression of kidney dysfunction and that sodium bicarbonate administration might attenuate this effect, but further evaluation is required to validate such a renoprotective strategy."

Above quoted from :


Case report form for one patient :


Conversational learning around the same patient :

[11/21, 7:42 PM] Moderator : 

Log Book assessment: PG Med  


Context : An elderly patient with severe anemia, hypotension, severe metabolic acidosis, encephalopathy and recent fracture femur 

More :


Impact : 


On learning ecosystem :


Cognitive competency levels touched :


Approach to clinical localization of disease pathology was done reasonably well and differential diagnostic hypothesis formulated 


On patient care outcomes :


Empathic trust built with patient and relatives 


Standard of care provided with provision for care continuity  


Challenges met/not met : managing diagnostic  arranging of repeating hemoglobin levels, repeat imaging to assess hematoma, continuing dialysis, reviewing the literature around giving bicarbonate in high anion gap metabolic acidosis 


[11/21, 7:43 PM] Moderator : 👆How is this patient now? 🤔


[11/21, 7:44 PM] PG Med: Sir he expired at 10 am today ,



[11/21, 7:44 PM]  PG Med: 

Quoting from above link :

Dialysate (HCO3–) prescriptions differ widely throughout the world. About half of HD patients in the United States are dialyzed in ≥37 mEq/L dialysate (HCO3–) [34], while 83% of HD patients in Japan are dialyzed with dialysate (HCO3–) of <30 mEq/L [35]. Metabolic alkalosis due to the high dialysate (HCO3–) has been associated with the occurrence of cardiac arrhythmia, intradialytic hypocalcemia, hypokalemia, hypercapnia, hypoxia, intradialytic hypotension, and vascular calcification [36, 37]. In patients with chronic obstructive pulmonary disease (COPD) or other causes of ventilator impairment, higher dialysate (HCO3–) can cause CO2 accumulation and potentiate hypoxia. The DOPPS report also demonstrated a dramatic increase in mortality due to severe infection in patients receiving higher (HCO3–) dialysate [34]. Increasing dialysate (HCO3–; in the range of 28 and 40 mEq/L) induces intra-dialytic and post-dialysis alkalosis but has no effect on pre-dialysis acidosis [38, 39]. To prevent pre-dialysis acidosis, oral NaHCO3 may thus be considered.





[11/21, 7:45 PM] PG Med: The bicarb correction had also caused respiratory acidosis sir


[11/21, 7:46 PM] PG Med: His sensorium also deteriorated and when we wanted to intubate, he went into cardiac arrest

[11/21, 7:49 PM] Moderator : So what was the dialysate prescription that had been given to our patient on Day 1? 

Interestingly his hypotension too was intradialytic and he had come with a normal BP on admission?

[11/21, 7:52 PM] PG Med: Yes sir presentation bp was 150/90mmhg, first intradialytic bp drop required single ionotropes for 6 hours post dialysis


[11/21, 7:52 PM] Moderator : Share his first dialysate prescription

[11/21, 7:58 PM] PG Med: Yesterday he had an episode of bradycardia in the afternoon when his pco2 wasn't raised sir,revived with atropine , so I think bradycardia is a seperate episode sir

[11/21, 8:00 PM] PG Med: Sir casesheet is dispatched
Intradialysis first day for 2hour dialysis 200 meq was given in the can sir,the rate of delivery is variable per hour, so around 75 to 50 meq was given 
Second day 300meq in 4 hours , which was again 50 to 75 new/hour sir


[11/21, 8:03 PM] Moderator : How do our dialysate bicarbonate concentrations compare with the ones shared in the above linked paper


[11/21, 8:04 PM] Moderator : Did our high dialysate concentration induce metabolic alkalosis? 

Also do our dialysis doctors vary the dialysate concentration in a case by case basis?


[11/21, 8:08 PM] PG Med: Vary case by case sir

[11/21, 8:08 PM] PG Med: We undercorrect it sir, only half of the requirement is given to prevent metabolic alkalosis


[11/21, 8:09 PM] PG Med: But for him as acidosis was refractory, contemplating whether we overcorrected it

[11/21, 8:11 PM] Moderator : So how do we decide how much to give?


[11/21, 8:12 PM] Moderator : His terminal acidosis was respiratory?


[11/21, 8:12 PM] PG Med: We calculate the deficit sir
And give half of it in the can and not directly if the patient can wait till dialysis,
Rate of delivery per hour ,I did not check sir
If he's having symptoms of overload we simultaneously remove more ultrafiltrate sir


[11/21, 8:12 PM] PG Med: Yes sir breaths became shallow


[11/21, 8:13 PM] Moderator : Could his respiratory depression have been due to metabolic acidosis which precipitated respiratory acidosis terminally?


[11/21, 8:15 PM] PG Med: Been trying to find that mechanism sir, the extra co2 did it come from the bicarb or did hypoventilation cause it


[11/21, 8:16 PM] Moderator : Hmm generally when we see extra Pco2 we always think respiratory while if we would have thought extra bicarb that would be reflected as metabolic alkalosis? 🤔


[11/21, 8:18 PM] PG Med: Yes sir but if his respiratory centre is compromised will alkalosis happen


[11/21, 8:18 PM] Moderator : Did he have alkalosis or acidosis?


[11/21, 8:19 PM] PG Med: Respiratory acidosis sir

[11/21, 8:19 PM] Moderator : So why think about alkalosis?

Project "resolving diagnostic uncertainty in fever" Case report form link and Patient centered conversational learning

Past projects around the above context in the links below :


UG student thesis 2003:

UG student thesis 2005:


PG student thesis : 2012-13

Review of literature links for the above thesis :


Current project link :


Current case report form link and beginning of November 19, 2021 discussion link below :


Conversational decision support system CDSS for the above case copied below :


[11/21, 9:22 AM] : intern notes 


39M AMC BED 1

S


- Continuous fever spikes present above 102 F associated with chills 
C/o burning micturation subsides and increased frequency of urination,low backache increased on bending forward

O- TEMP-101F
Bp-110/70mmhg
Pr-90bpm
Cvs-s1,s2heard
Rs- NVBS
CNS-NAD

A


- Diagnosis is ? Acute pyelonephritis and urge incontinence .
Dm-2 
?clinical malaria

Urine c/s - no growth 
Sent repeat blood and urine c/s
Smear for mp - negative.
Repeat cue - no RBC casts and pus cells -8-10 .
Xray kub was done in view of ??emphysematous pyelonephritis .


P

1.Tab.CIPROFLOXACIN 500 MG /BD
2.Tab.PAN 40MG OD
3.TAB.PCM 650mg QID
4.Tab.AMLONG 5mg PO OD
5.Tab.URISPAS PO BD
6.INJ.HUMAN INSULIN
7.Syp.CITRALKA 10ML IN ONE GLASS WATER POBD
8.GRBS MONITORING
9.INJ.FALCIGO 120MG IV STAT
10.BP/PR/TEMP MONITORING

[11/21, 9:22 AM] moderator : USG abdomen revealed any perinephric abscess?

[11/21, 9:22 AM] PG Med: No sir it did not reveal any abscess, there is no loin tenderness on palpation

[11/21, 9:22 AM] :  moderator
Start him on iv Meropenem today asap

11/21, 9:22 AM] PG Med: Ok sir

[11/21, 9:23 AM] Moderator : Share his blood sugar trends since admission


[11/21, 7:02 PM] Moderator : Why did we add falcigo when the localization was pointing towards UTI? 🤔

[11/21, 7:17 PM] PG Med: Sir because of the spike every 12 hours ,

[11/21, 7:19 PM] Moderator : Why didn't we think it was UTI

[11/21, 7:21 PM]  PG Med: Yes sir his fever had chills each episode,true for uti, 
We thought of malaria as well sir , inspite of antibiotics frequency or temp didn't come down

[11/21, 7:21 PM] Moderator : So purely from our medical cognition research insight perspective one can conclude that in the face of diagnostic uncertainty that keeps continuously giving irritating alarms, doctors often tend to cover all the differential diagnostic possibilities and reduce the chances of their future regret?

[11/21, 7:39 PM] PG Med: Sir may I ask, why did we choose meropenem for this patient?

[11/21, 7:40 PM] moderator : 

The answer is here 👉([11/21, 7:02 PM] Moderator : Why did we add falcigo when the localization was pointing towards UTI? 🤔)

I didn't even notice that he had been started on empirical falcigo from yesterday evening

Meropenem was the antibiotic escalation for the strong differential of UTI in the moderator's mind although rest of the team thought falcigo was good enough. 

Medical cognition Inferential queries:

Could we have held back on antibiotic escalation if the PG Med had voiced her query regarding meropenem earlier instead of immediately agreeing to switch over? 

What really helped the patient? Falcigo or meropenem?   




 
PG Med Yes he has a spike of 102 now

11/22, 7:37 PM] Moderator : Subjectively?


[11/22, 7:39 PM] PG Med: At the time of spike ,he is having chills sir ,but not as disabling as the rest of the times as per him, 
Rest of the time he's comparatively more active


[11/22, 7:39 PM] moderator : This sounds like malaria 🤔😬


[11/22, 7:40 PM] moderator : Are we still continuing the Artemesin?


[11/22, 7:40 PM] PG Med: Yes sir , but he already has taken 3 doses of falcigo so far


[11/22, 7:40 PM] PG Med: The 48th hour dose is due tomorrow morning at 8 sir


[11/22, 7:40 PM] Moderator : What is the continuation strategy for falcigo in toto?


[11/22, 7:42 PM]  PG Med: Should have voiced about giving falcigo and notified the initial dip so that meropenem wouldn't have been started sir, 
His loin pain is more musculoskeletal than pyelonephritis as he's having it on bending .


[11/22, 7:47 PM] PG Med: 2.4mg/kg at 0,12,24 and 48 hrs sir


[11/22, 7:49 PM] Moderator : Hmm that would have saved some meropenem. 
How about reviewing the repeat urine culture and if negative for a second time we can stop meropenem?


[11/22, 8:09 PM] Moderator : Yesterday in the 2016 group someone said the urine routine was showing 8-10 pus cells although the prior RBCs had receded. What could be the reason for that? He also gave a history of hematuria with his loin pain in the initial phases?


[11/22, 8:43 PM] PG Med: Sir the transient haematuria can be due to uti itself, due to inflammation, his symptoms of burning micturition subsided when the repeat cue (complete urinary examination) came back negative for rbc as well


[11/22, 8:47 PM] Moderator : But he had 8-10 pus cells. 

How many pus cells did he have when he had 3-4 RBCs?


[11/22, 10:21 PM] PG Med: On the day of admission he had 3-4 pus cells with 3-4 rbc's
Repeated cue on Saturday has 8 to 10 pus cells ,with 1-2 rbc sir

November 19, 2021

Friday 

Group discussion: 

Context : PUO in a male diabetic with loin pain and lower urinary tract symptoms of urgency and burning 


Presenters : Dr Vaishnavi, Dr Nikita, Dr Pooja (intern) and Dr (Soujanya 2017)


Impact : 

On learning ecosystem :

Cognitive competency levels touched :

Approach to clinical localization of disease pathology was done reasonably well and differential diagnostic hypothesis formulated in the face of high diagnostic and therapeutic uncertainty 

On patient care outcomes :

Empathic trust built with patient and relatives 

Standard of care provided with provision for care continuity  

Challenges met/not met : 

Fever monitoring and post admission illness timeline was well supervised. 

Imaging and labs for further  clinical localization of the PUO was driven by the discussion 

Multiple therapeutic interventions were made for the differentials in the face of diagnostic uncertainty further driven by ongoing patient suffering and an attempt to gather the "medical cognition" learning points made here : 

Sunday, November 21, 2021

Thesis 2012-2013 Resolving diagnostic uncertainty in fever patients using fever and symptom pattern recognition by Dr Himanshu Jain, Bhopal

 TITLE:   Minimizing Antibiotic Usage for Poorly localizable fevers of short duration in Adults using a Check List and Temperature Monitoring.

PLACE OF STUDY: The study was conducted in Medicine, Peoples Hospital, PCMS & RC, Bhopal, M.P.

STUDY DESIGN – It was Prospective Observational Study.

STUDY PERIOD – over one year

SAMPLE SIZE - 500 patients

SAMPLE POPULATION -Patients of age 14-50 yrs attending Medicine Department in PCMS & RC, Bhanpur, Bhopal.


INCLUSION CRITERIA:

  1. Written informed consent from each patient or legal guardian prior to enrollment.

  2. Patients Age 14 to 50 years

  3. Recent-onset fever (within one to five days) when the diagnostic uncertainty is high.

  4. Symptoms commonly suggestive of self-limiting flu like illnesses such as myalgia, arthralgia with commonly known symptoms indicating involvement of upper respiratory tract such as cough, rhinorrhoea, frontal headache etc. 


EXCLUSION CRITERIA:

  1. Obvious clinical localization for fever such as pneumonia, meningitis, urinary tract infection etc.

  2. Clinical diagnosis suggesting important clinical localizations such as spontaneous bacterial peritonitis in cirrhotic patients with ascitis.

  3. Elderly, underlying immunosuppression or any associated conditions predisposing to severe infections such as diabetes.


CONSENT: 

Written and informed consent in local language was obtained prior to enrollment in study.

ETHICAL APPROVAL: 

Institutional Ethical clearance for this study was obtained from Institutional Ethics Committee at Peoples Hospital, Barkatullah University, Bhopal.

METHOD OF STUDY:

Five hundred clinically matched patients over one year presenting with recent-onset fever to the Peoples Hospital PCMS & RC, Bhopal, was prospectively monitored for fever, duration of illness and satisfaction levels. 

In fever patients presenting to PCMS&RC we had used a check list to allocate suitable patients to just antipyretics and/or antibiotics and temperature charting for 2 days (extended to 4 days if fever persists but lesser number of fever episodes per day and temperature spikes <1030F) and study their health care outcomes. After monitoring for two days if the fever shows a progression only then antibiotics been instituted. (depending on the judgment of the treating physician). 

Two groups were studied:

Group A – 

This group comprise of patients encountered by investigators (not blinded to the patient’s inclusion status in the study), they waited two days before taking a decision on antibiotics after selecting patients for the above on the basis of inclusion criteria and continue to encourage monitoring of their fever patterns over two days. 

Group B – 

This group comprise of patients encountered by Physicians delivering their usual care for fever (as per their prevailing clinical judgment). They remained blinded to the patient’s status in the study.

Patient’s recruitment was done through OPD attender from OPD and was alternately allocated in each group. Patients been recruited such that to cover all seasons in a year. OPD Patients were advised Temperature charting 8 hourly and also during fever episodes for 2 days (duration been extended for 4 days if fever persists but with lesser number of fever episodes per day and temperature spikes <1030F). Patients who were admitted in hospital, their temperature been monitored by patient itself or by his/her relative or by nursing staff. 

Patient reviewed after 2 days by personal appearance or Phone call for follow up about temperature charting, fever pattern, duration of illness/time to recovery, events inbetween two days if any, check list score and satisfaction level on VAS. A Check List being provided for deciding whether to use antibiotic or not during the course of illness or to explain whether Antibiotic need was there or not. VAS (Visual Analogue Scale) being used before and after the illness to access the satisfaction level of Patient’s. Patient was advised to report immediately for any urgent need.

STUDY TOOLS –

  1. Mercury Thermometer.

  2. Check List.

  3. VAS (Visual Analogue Scale)

             

                          CHECK LIST

                         (To decide whether to use Antibiotics or Not)


- Cough with sputum production/ haemoptysis

 / Breathlessness                                                        0 1


-  Chest pain, palpitation                                                0 1


- Tachypnoea, Hypotension                                           0 1


- Epigastric or right hypochondrial or 

  right iliac fossa (abdominal) pain / jaundice                0 1                               


-Vomiting and/or constipation or diarrhoea                   0 1

-  Burning micturition/pain during micturition and 

   Low back pain/ Urinary dysuria                                   0 1


- Yellow discolouration of sclera                                    0 1


-  Confusion, altered sensorium                                       0 1


- Black coloured stool (malena) and/or blood                 0 1

In vomiting


 - Immunosuppressive conditions like                            0 1

 Diabetes/CRF (kidney failure)/Malignancy/HIV

 /Any chronic illness                                                      


  • High grade  fever (>1030F)/ continuous fever          0 1


  • Neck pain/Neck stiffness                                          0 1






VISUAL ANALOUGE SCALE (VAS) – 


A Visual Analogue Scale (VAS) is a measurement instrument that tries to measure a characteristic or attitude that is believed to range across a continuum of values and cannot easily be directly measured. 


READING AT THE TIME OF PRESENTATION-
 
1     2 3     4 5 6     7 8 9 10                                                                                                                                             
                                                             

The symbols indicate patient’s level of satisfaction/well-being with their current health.


READING AFTER RECOVERY


1     2 3     4 5 6     7 8 9 10

                                                                                                                                                                                                                                                 

The symbols indicate patient’s level of satisfaction/well-being with their current health




                      

 STATISTICAL ANALYSIS:

Statistical analysis were done using SPSS software version 20. Average/median duration of fever will be ascertained. Dispersion in terms of range and standard deviation will be calculated. Antibiotic/ antipyretics consumptions will be associated/correlated by duration of fever by using Student T-test, and patient satisfaction through Mann-Whitney Test and appropriate statistical test.






                               OBSERVATION

The present study was conducted in Department of Medicine, Peoples Hospital PCMS & RC, Bhopal - a tertiary care centre from June 2012- May 2013. Ethical approval for this study was obtained from Institutional Ethics Committee at Peoples Hospital, Barkatullah University, Bhopal.

.  A total of 500 subjects (246 males, 254 females) were recruited in the study. 

SEX

FREQUENCY

PERCENT

MALE

246

49.2

FEMALE

254

51.8

TOTAL

500

100.0











Group 


                  GROUP

Gender

Total

MALE

FEMALE


NO ANTIBIOTIC

118

132

250


ANTIBOITIC

128

122

250

                    Total

246

254

500












T-Test –



Group Statistics


Group

N

Mean

Std. Deviation

Std. Error Mean

DURATION OF ILLNESS

NO ANTIBIOTIC

250

2.57

1.008

.064

ANTIBOITIC

250

2.58

1.070

.068

NO. OF FEVER SPIKES

NO ANTIBIOTIC

250

2.13

1.607

.102

ANTIBOITIC

250

2.13

1.513

.096