Developing a Practice-Based Learning and Improvement (PBLI) Quality Improvement (QI) Systems Impact Assessment Questionnaire

This study has been completed.
Sponsor:
Information provided by (Responsible Party):
Department of Veterans Affairs
ClinicalTrials.gov Identifier:
NCT00754377
First received: September 16, 2008
Last updated: April 16, 2014
Last verified: April 2014
  Purpose

The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed practice-based learning and improvement (PBLI) -- a competency that is typically omitted from medical curriculum. The goal is to have residents competent to investigate and evaluate their own patient care practices, integrate scientific evidence and be able to improve their practices. Available assessment tools do not adequately address all of the components of PBLI and few assessment tools attempt to capture the residents' ability to develop and implement clinically-based CQI projects that involve the practice setting. Curriculums without such foci miss the importance of system perspectives and opportunities for interprofessional team development. Our aim is to evaluate preliminary data on the curriculum we developed to address the gaps, to develop an assessment tool, and to provide methods for assessing the sustainability of system projects.

The key component of the curriculum is the integration of system quality improvement projects. PBLI curriculum was offered on alternate rotations. Preliminary data is available from 6 PBLI QI Systems Curriculum blocks (n=50) and 5 comparison blocks (n=42). Data includes closed- and open-ended questions designed to assess resident PBLI application skills, the notes and presentation slides for the residents' presentation.


Condition Intervention
Education
Other: PBLI curriculum comparison
Other: Quantify the systems practical components

Study Type: Interventional
Study Design: Allocation: Non-Randomized
Endpoint Classification: Efficacy Study
Intervention Model: Parallel Assignment
Masking: Open Label
Primary Purpose: Health Services Research
Official Title: Developing a PBLI QI Systems Impact Assessment Questionnaire

Further study details as provided by Department of Veterans Affairs:

Primary Outcome Measures:
  • What themes and issues emerge in responses to the open-ended items and presentation slides that can be coded to help create an assessment tool for issues related to applying the six ACGME points regarding PBLI and QI principles? [ Time Frame: 4 months ] [ Designated as safety issue: No ]

Secondary Outcome Measures:
  • To develop a PBLI QI assessment tool and manual based on closed-ended knowledge items, open-ended application items, and notes from residents teams' project development and presentation slides for their QI projects at the internal medicine's morbidity an [ Time Frame: 4 Months ] [ Designated as safety issue: No ]
  • How does current data compare to the baseline data collected by resident teams for the projects? [ Time Frame: 4 Months ] [ Designated as safety issue: No ]

Enrollment: 92
Study Start Date: January 2008
Study Completion Date: December 2011
Primary Completion Date: July 2008 (Final data collection date for primary outcome measure)
Arms Assigned Interventions
Active Comparator: Arm 1
To evaluate preliminary data on a PBLI curriculum grounded on QI system projects.
Other: PBLI curriculum comparison
The design is a pre-post comparison of PBLI curriculum participants versus non-participants. Data will come from the closed-ended items on the questionnaire given before the rotation and at the end of the rotation. PBLI curriculum was offered on alternate rotations (residents on alternate month were involved with a different curriculum). Preliminary data is available from 11 blocks, 6 PBLI QI Systems Curriculum blocks (n=50) and 5 comparison blocks (n=42) during the previous academic year. Closed-ended items assessed beliefs about different aspects of CQI project development and implementation (6 items). In addition, there were 5 short definition items to address knowledge. Finally, content analysis methods will be used to evaluate responses to the open-ended item which asked respondents to develop a project to improve patient care.
No Intervention: Arm 2
To develop a PBLI QI assessment tool and manual notes from opened questions and residents teams' project development and presentation slides for their QI projects at the internal medicine's morbidity and mortality conference (IM MMC).
Active Comparator: Arm 3
Provide preliminary data regarding the sustainability of the residents' systems projects and the impact of those projects on patient care and the organization.
Other: Quantify the systems practical components
A pre-post design will be used. The methods to address this research question are a combination of qualitative and quantitative. It is possible to quantify the systems practical components linked to the projects and then to determine which components are still sustained, that is, still in use by the organization, by checking with relevant stakeholders. It will also be possible to compare the sustained components with the components of residents' projects that weren't initiated and/or sustained to identify critical themes or aspects of the projects that can be integrated into the curriculum and the assessment tool. In this process, it will be important to differentiate whether a non-sustained component of the projects related to lack of necessary development by residents versus natural evolution of moving beyond the quality improvement project such that the original component is no longer visible but an important step in the evolution.

Detailed Description:

Physicians in training operate in complex healthcare delivery systems but many have not been equipped with the knowledge or skills to analyze clinical environments and continually improve patient care.[1] Instead, their training emphasizes the clinical management of individual patients. The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed one of two novel core competencies that are typically omitted from formal medical curriculum: practice-based learning and improvement (PBLI).[2,3] The ACGME's PBLI competency involves six points. The overall objective is to have residents competent to investigate and evaluate their own patient care practices, evaluate and integrate scientific evidence into their clinics and be able to improve their practices. However, the ACGME was not prescriptive about how to successfully implement and evaluate PBLI, and an established curriculum and validated assessment tool did not exist.[4] Efforts to date to develop an assessment tool provide a foundation, but none adequately addresses all of the components of PBLI as described by the ACGME. [3, 5-10] In particular, few assessment tools attempt to capture the residents' ability to develop and implement clinically-based CQI projects that involve the practice setting and assess impact on the practice setting and/or organization. Curriculums without such foci miss the importance of system perspectives, opportunities for interprofessional team development, and meaningful impacts on patient care at a broader organizational level.

After several iterations, a PBLI QI curriculum that addressed the gaps identified in many other curriculums was instituted. [11] The key component was the integration of system quality improvement projects that could evolve over several resident outpatient rotation blocks. Data from 11 blocks of the new curriculum is available for analysis. Data includes closed- and open-ended questions designed to assess resident PBLI application skills and the notes and presentation slides for the residents' presentation at the internal medicine's morbidity and mortality conference (IM MMC) at the end of each block. Funding would make it possible to more quickly evaluate the existing preliminary data. Our overall aim is to evaluate the preliminary data on the new curriculum and develop an assessment tool that addresses the six ACGME points and provides methods for assessing the sustainability of system projects. The next step will be to apply for a grant to more fully evaluate the curriculum and assessment tool.

  Eligibility

Ages Eligible for Study:   25 Years to 75 Years
Genders Eligible for Study:   Both
Accepts Healthy Volunteers:   Yes
Criteria

Inclusion Criteria:

  • None, data being used has already been collected from a previous study which the inclusion criteria included all Internal Medicine and Medicine-Pediatrics residents completing a 4 week ambulatory block from 2005-2006 were required to participate in the PBLI curriculum to satisfy an ACGME's core competency.

Exclusion Criteria:

  • None, the residents from 2004 that did not complete a 4 week ambulatory block and residents participating in ambulatory block 7 and 13 were excluded from the study previously IRB approved and exempted study. Blocks 7 and 13 are not structured to permit teaching.
  Contacts and Locations
Choosing to participate in a study is an important personal decision. Talk with your doctor and family members or friends about deciding to join a study. To learn more about this study, you or your doctor may contact the study research staff using the Contacts provided below. For general information, see Learn About Clinical Studies.

Please refer to this study by its ClinicalTrials.gov identifier: NCT00754377

Locations
United States, Ohio
VA Medical Center, Cleveland
Cleveland, Ohio, United States, 44106-3800
Sponsors and Collaborators
Investigators
Principal Investigator: David C Aron, MD MS Louis Stokes VA Medical Center