Faculty Development to Improve Mini-CEX Ratings

This study has been completed.
Sponsor:
Information provided by:
Mayo Clinic
ClinicalTrials.gov Identifier:
NCT00667940
First received: April 23, 2008
Last updated: April 25, 2008
Last verified: April 2008
  Purpose

We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.


Condition Intervention
Faculty Development
Behavioral: Rater training workshop

Study Type: Interventional
Study Design: Allocation: Randomized
Endpoint Classification: Efficacy Study
Intervention Model: Parallel Assignment
Masking: Open Label
Primary Purpose: Treatment
Official Title: Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial

Further study details as provided by Mayo Clinic:

Primary Outcome Measures:
  • Interrater reliability (videotaped resident-patient encounters) [ Time Frame: April-June 2006 ] [ Designated as safety issue: No ]

Secondary Outcome Measures:
  • Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores [ Time Frame: April-June 2006 ] [ Designated as safety issue: No ]
  • Interrater reliability (actual resident-patient encounters) [ Time Frame: March 2005 (historical) to June 2007 ] [ Designated as safety issue: No ]

Enrollment: 52
Study Start Date: March 2006
Study Completion Date: June 2007
Primary Completion Date: June 2006 (Final data collection date for primary outcome measure)
Arms Assigned Interventions
Experimental: Workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
Behavioral: Rater training workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
No Intervention: Delayed workshop
No specific intervention until after follow-up assessment, then receives same workshop.

Detailed Description:

Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.

Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.

Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.

Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.

Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.

  Eligibility

Genders Eligible for Study:   Both
Accepts Healthy Volunteers:   Yes
Criteria

Inclusion Criteria:

  • Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota

Exclusion Criteria:

  • None
  Contacts and Locations
Please refer to this study by its ClinicalTrials.gov identifier: NCT00667940

Locations
United States, Minnesota
Mayo Clinic
Rochester, Minnesota, United States, 55905
Sponsors and Collaborators
Mayo Clinic
Investigators
Principal Investigator: David A Cook, MD, MHPE Mayo Clinic
  More Information

Additional Information:
No publications provided by Mayo Clinic

Additional publications automatically indexed to this study by ClinicalTrials.gov Identifier (NCT Number):
Responsible Party: David A. Cook, M.D., Mayo Clinic
ClinicalTrials.gov Identifier: NCT00667940     History of Changes
Other Study ID Numbers: 06-002730
Study First Received: April 23, 2008
Last Updated: April 25, 2008
Health Authority: United States: Institutional Review Board

Keywords provided by Mayo Clinic:
Medical education
faculty development
rater training
clinical competence
assessment

ClinicalTrials.gov processed this record on April 15, 2014