Motivational interviewing recordings were reviewed during group supervision. To objectively evaluate MI skills, supervisors who were VA-certified in MI used the Motivational Interviewing Treatment Integrity (MITI) coding tool to review and code both the live- and simulated-patient recordings.24 The MITI coding involves quantitative and qualitative analysis using standardized coding items.
Quantitative items included percentage of open-ended questions (Proficiency: 50%; competency: 70%); percentage of reflections considered complex reflections, or reflective statements adding substantial meaning or emphasis and conveying a deeper or more complex picture of what patients say (Proficiency: 40%; competency: 50%); reflection-to-question ratio (Proficiency: 1:1; competency: 2:1); and percentage of MI-adherent provider statements (Proficiency: 90%; competency: 100%).
Qualitative coding items were a global rating of therapist “empathy,” which evaluated the extent to which the trainee understood or made an effort to grasp the patient’s perspective, and “MI spirit.” This coding intended to capture the overall competence of the trainee in emphasizing collaboration, patient autonomy, and evocation of the patient experience (Proficiency score: 5; competency score: 6).
The PC-MHI teaching staff met midyear and end of year as a team to complete trainee evaluations focused on the 3 areas of learning objectives: patient-centered PC-MHI, collaborative team-based practice, and population health and program improvement. Patient-centered PC-MHI involves direct observation and supervision of trainee clinical contacts with veterans, including assessment and treatment planning, clinical documentation, and review of live- and simulated-patient MI recordings. Collaborative team-based practice involves review of trainee participation in day-to-day teamwork, huddles, team meetings, and IPE activities. Population health and program improvement involves review of trainee work on a panel measurement-based care management or program improvement project. In each learning objectives area, trainees were rated on a 3-point scale: needs improvement (1); satisfactory (2); achieved (3). Core knowledge about PC-MHI evidence base, structure, and clinical topics was assessed with case-based written examination at midyear and end of year.
Surveys and qualitative interviews were used to assess trainee perceptions about the IPE program. A midyear and end-of-year survey assessed trainee satisfaction and perceived efficacy of the IPE training program in meeting core learning objectives. A midyear survey designed by pharmacy residents as part of their program improvement project evaluated attitudes around interprofessional learning and team practice. Trainees met individually with the PC-MHI IPE director at midyear and end of year to gather qualitative feedback on the IPE program.
Outcomes
All trainees advanced to the expected level of supervision for clinical contacts (drop-in or indirect clinical supervision). Over the year, there was significant improvement in trainees’ MI skills as measured by MITI coding of at least 2 live-patient or 2 simulated-patient recordings (Figures 1A and 1B). By end of year, most trainees had reached proficiency or competency in several MITI coded items: percentage of open-ended questions (4/12 proficient, 8/12 competent), percentage of complex reflections (2/12 less than proficient, 3/12 proficient, 7/12 competent), MI adherence (1/12 proficient, 11/12 competent), global empathy rating (2/12 proficient, 10/12 competent), and global MI spirit rating (12/12 competent). Average reflection-to-question ratio for the trainee group increased from 0.63 to 0.96 from midyear to end of year, but only 3 of 12 trainees reached the proficiency level of a 1:1 ratio, and no trainee reached the competency level of a 2:1 ratio.
According to the PC-MHI team’s midyear evaluation, most trainees were already making satisfactory progress in the 3 domains of learning objectives for the training program. At end of year, 13 of 14 trainees were assessed as competent in all 3 domains (Figures 2A and B). All trainees passed the midyear and end-of-year written examinations with overall high scores (average score, 82%) demonstrating acquisition of core PC-MHI clinical knowledge.
Trainee evaluations of the IPE program were overall highly favorable at both midyear and at end of year. Trainees rated the program effective or extremely effective in developing their skills in patient-centered care, interprofessional communication, and collaborative team-based practice. They also rated the program highly effective in preparing them to use team-based practice skills in other settings. On a midyear survey, trainees moderately to strongly agreed with several positive beliefs and attitudes about team-based care.
In qualitative interviews at program completion, trainees across disciplines rated the MI training with group supervision as one of their most valuable interprofessional learning experiences. Other highly valued training experiences were PACT PC-MHI panel management projects, team-based clinical case reviews, and cross-disciplinary supervision.
Discussion
This article describes the successful development and implementation of a VA-based IPE program in PACT PC-MHI. Interprofessional clinical training and educational experiences were highly valued, and trainees identified positive attitudes and improved skills related to team-based care. These findings support previous findings that IPE is associated with high satisfaction and positive attitudes toward team-based collaborative practice.12-17 Program implementation presented several challenges: nonsynchronous academic calendars and rotation schedules, cross-disciplinary supervision regulations, variations in clinical and supervisory requirements for accreditation standards, the traditional health care hierarchy, and measurement of the impact of IPE experiences.11,25,26