Perceptions and Attitudes About AssessmentModerator: James Nixon, MDDiscussant: Kimberly Ephgrave, MDPoint-of-Care Assessment of Medical TraineeCompetence for Independent Clinical WorkTara J. T. Kennedy, Glenn Regehr, G. Ross Baker, and Lorelei LingardAbstractBackgroundClinical supervisors make frequentassessments of medical trainees’competence so they can provideappropriate opportunities for trainees toexperience clinical independence. Thisstudy explored context-specificassessments of trainees’ competencefor independent clinical work.MethodIn Phase One, 88 teaching teammembers from internal and emergencymedicine were observed during clinicalactivities (216 hours), and 65 participantscompleted brief interviews. In PhaseTwo, 36 in-depth interviews wereconducted using video vignettes. Datacollection and analysis employedgrounded theory methodology.ResultsSupervisors’ assessments of traineetrustworthiness for independent clinicalwork involved consideration of fourdimensions: knowledge/skill, discernmentof limitations, truthfulness, andconscientiousness. Supervisors’ relianceon language cues as a source oftrustworthiness data was revealed.ConclusionsThis study provides an initial explorationof context-specific competenceassessments, which affect both patientsafety and education, and provides anovel framework for study of the linksbetween language use and competence.Acad Med. 2008;83(10 Suppl):S89–S92.Throughout North America, the activeparticipation of medical trainees inpatient care is considered to be critical totheir learning process.1,2Maintaining anappropriate balance between independencefor learning and supervision for safetyis an ongoing process for the clinicalsupervisors of medical trainees. In anearlier stage of our research program, wedescribed how supervising physiciansconsider a number of factors whenmaking decisions about how muchclinical oversight to provide to trainees.3Identified triggers for the provision ofmore intensive oversight included acuteor severe clinical situations, issues raisedby nurses or family members, andconcerns about a trainee’s competenceto handle a specific clinical situation.3The present study was intended as anin-depth exploration of supervisingphysicians’ assessments of this case-specific competence for independentclinical work.Although the process of formal evaluationof medical trainee competence has beenthe subject of much empirical study,4the process of assessment of trainees’competence to provide independent carefor a given patient or in a specific clinicalcontext has not been described. This“point-of-care” competence assessment(i.e., occurring at the time and in thesetting of clinical care) arguably hasmuch more practical impact on patientcare and trainee education than does anyformal evaluation process, because itguides decisions about the nature of theday-to-day monitoring of trainees’clinical activities provided by supervisingphysicians. As part of an ongoing study ofclinical supervision practices, this studyaimed to explore supervising physicians’assessments of trainees’ competence toprovide independent clinical care, andthe process employed to make theseassessments.MethodThe study was designed using groundedtheory methodology.5,6The study tookplace in three teaching hospitals affiliatedwith an urban Canadian medical school.Institutional review board approval wasobtained. Study settings included theemergency medicine (EM) departmentand the general internal medicine (GIM)inpatient teaching wards. These areaswere chosen because of their heavyinvolvement in clinical teaching andbecause they employ different clinicalsupervisory structures (in EM, traineesreport to the attending physician,whereas in GIM, senior trainees supervisejunior trainees in a “hierarchical”supervisory structure).Study participants were clinical teachingteam members in GIM and EM, includingattending physicians (AP), junior andsenior residents (JR and SR), and medicalstudents (MS). Participants at differentlevels of experience and of both genderswere recruited through purposefulsampling.5Saturation of the data (thepoint at which further sampling ceases toyield any new analytic concepts)7was thefinal determinant of sample size.The study design involved two phases.Phase One involved nonparticipantobservations and brief, on-site interviews.Phase Two employed in-depth interviewsusing video vignette prompts.In Phase One,3nonparticipant observation8of 12 teaching teams was performed (seventeams in GIM and five in EM; total of 88team members observed). Each team wasobserved for six 3-hour periods during thecourse of one month (total 216 hours ofobservation), and a brief (15 minutes)on-site interview was conducted near theend of the month of observations with65 participants. Details of Phase OneCorrespondence: Tara J. T. Kennedy, MD, PhD, StanCassidy Centre for Rehabilitation, 800 Priestman St.,Fredericton, NB, E3B 0C7, Canada; e-mail:([email protected]).Academic Medicine, Vol. 83, No. 10 / October 2008 Supplement S89methodology have been previouslypublished.3Phase Two, conducted in the year afterPhase One, was designed to refine andexpand the emerging understanding ofpoint-of-care competence assessmentthrough in-depth interviews using videoprompts. A series of 10 videotapedvignettes was developed (five set in GIMand five in EM), each crafted to present adilemma relevant to decisions aboutsupervision. The vignettes were based onevents which occurred during Phase Oneobservations (with details altered torender the original participantsunidentifiable). For example, onevignette portrayed a resident who hadordered an erroneous investigationwithout checking with the AP.The 36 Phase Two participants includedAPs (n ⫽ 19), residents (n ⫽ 13), andMSs (n ⫽ 4). Although the traineeinterviews provided some contextual andconfirmatory data which were relevant,the present report is drawn primarilyfrom Phase One data and the APinterviews from Phase Two. During theinterviews, participants viewed therelevant videos (in the same sequence)and were asked to discuss their opinionof what they would do in response to thedilemma presented in each vignette. Togain insight into both tacit and explicitinfluences on supervision decisions,participants were asked to discuss therationale for their responses and toarticulate other possible responses to thevignette dilemma and their reasons forrejecting these (the discourse-basedinterview method).9Interviews were