Capitalizing on the PCOA: How Colleges of Pharmacy Are Utilizing the PCOA
04/1/20
Categories: Research PrideCapitalizing on the PCOA: How Colleges of Pharmacy Are Utilizing the PCOA
Laura Nice, PharmD, PGY2 Oncology Pharmacy Resident, University of Louisville Hospital
Tyler Perry, PharmD, PGY1 Pharmacy Practice Resident, Summa Health System Akron Campus
A team of authors from the University of Oklahoma College of Pharmacy published their research to determine factors for predicting student performance on the Pharmacy Curriculum Outcomes Assessment (PCOA) exam.1 Prior to the 2016 accreditation standard requiring the PCOA, the University of Oklahoma College of Pharmacy utilized six local summative knowledge examinations to evaluate student progress. These exams were administered during the final exam of each semester of the didactic coursework, each worth 10% of the overall course grade. After the accreditation standard update, the assessments were condensed into a one-time, 100-item “PCOA prep” test. This test was administered in the third didactic year the week before fall finals and worth 10% of the course grade.
The group evaluated PCOA results for students in their third-professional year in the class of 2017 and 2018. Students were provided with PCOA Prep objectives four weeks before the PCOA Prep and encouraged to study. After the PCOA Prep, students were given a report card to identify areas of strength and weakness. The PCOA was then administered in May of the third professional year, with bonus points awarded within the subsequent fourth professional year fall seminar course based on PCOA score. Students were invited to complete a 12-item survey about demographics, study habits, perceived utility of the PCOA Prep, and confidence in PCOA areas.
Of the n=179 students in the study, n=168 (94%) of students completed the survey. Rates of some amount of studying was significantly higher for the PCOA Prep (82%) vs. PCOA (73%) (p<0.001). When asked about the usefulness of the PCOA Prep, 62% of students agreed it was of average or above average usefulness. Higher student confidence levels correlated with increased average scaled score for the PCOA, with the most impact seen with confidence in basic biomedical sciences (Table 1).
Table 1. Pharmacy students’ self-reported confidence. (N= 168; Adapted from Medina et al.1)
| PCOA Content Area | No to Little Confidence | Average Confidence | Confident to Very Confident | |
| Area 1: Basic biomedical sciences | % Response | 22% | 56% | 22% |
| Mean (SD) PCOA area scaled score | 345.2 (60.4) | 376.7 (65.1) | 433.3 (81.2) | |
| Area 2: Pharmaceutical sciences | % Response | 11% | 62% | 27% |
| Mean (SD) PCOA area scaled score | 367.7 (56.8) | 378.9 (54.7) | 409.2 (59.2) | |
| Area 3: Social, behavioral, and administrative sciences | % Response | 11% | 55% | 35% |
| Mean (SD) PCOA area scaled score | 365.0 (45.2) | 385.6 (55.5) | 396.9 (54.8) | |
| Area 4: Clinical sciences | % Response | 8% | 49% | 42% |
| Mean (SD) PCOA area scaled score | 351.6 (37.1) | 378.7 (52.0) | 395.7 (47.4) | |
Out of 19 demographic variables assessed, 13 were found to correlate with PCOA total scaled scores. PCOA Prep examination scores (r=0.66), cumulative didactic GPA (r=0.57), and PCAT composite scores (r=0.51) had the highest positive correlation. Interestingly, a negative relationship was found for being female and scoring lower in area 1 (basic biomedical sciences), as well as being non-white and scoring lower in area 2 (pharmaceutical sciences), area 3 (social, behavioral, and administrative sciences), and area 4 (clinical sciences). Also, students who studied more had a higher area 3 score than those who did not.
The authors described the uncertainty of what PCOA scores mean for students and colleges/schools of pharmacy, and whether said colleges/programs should help students prepare for the PCOA. Additionally, the question remains if colleges and schools should use the PCOA as a measure of their program’s curriculum. While additional research is needed to answer these questions, this study showed that the PCOA Prep predicted success on the PCOA and allowed for earlier intervention for at-risk students.
The second AJHP publication, entitled “National Trends in the Adoption of Pharmacy Curriculum Outcomes Assessment for Student Assessment and Remediation,” from the AACP assessment special interest group utilized a survey of the PCOA from 2008-2009 to 2015-2016.2 The purpose of this study was to determine and describe the current uses of the PCOA by US colleges of pharmacy. The administration year of the exam, the stake assigned to student performance, the determination and measurement of student minimum score for remediation or progression in the program, and the strategies for remediation were examined using the 38-item survey. The targeted demographic was all professional students in a Doctor of Pharmacy program in the United States. A result of 92 of the 171 surveys sent were collected from 135 pharmacy programs.
The majority (n=69,75%), of the pharmacy programs used a low-stakes approach to this examination when administered to students in their third-professional year. Alternatively, only 17% (n=16) and 2% (n=2) of programs assigned medium and high stakes, respectively. Also, less than 25% of programs had students who demonstrated deficient or poor performance on the examination and required remediation. About 25% (n=20) of third-year students required remediation and the survey found that about 45% (n=9) of schools identified at-risk students utilizing a committee. Lastly, 55% (n=11) of programs indicated that no reassessment was mandated as part of their remediation process.
This publication concluded that the number of programs utilizing the PCOA in the 2015-2016 academic year significantly increased from the 2008-2009 academic year. It also found great variability in the details of program’s assessment process (i.e. stakes, remediation, remediation length). Lastly, most of the studied programs utilized low stakes for the PCOA however, 55% (n=46) of programs were considered changing their assessment stakes.
Some of the very same troubles that we face with the PCOA are also being questioned by other institutions. From student satisfaction with the exam to remediation for underperforming students, other schools are attempting to improve this process for their students. Our key takeaway from these two articles, among others, is that while the PCOA is an excellent tool for assessing student performance and licensure exam correlation, best practices have yet to emerge regarding administering this exam.
References:
- Medina MS, Neely S, and Draugalis JR. Predicting Pharmacy Curriculum Outcomes Assessment Performances Using Admissions, Curricular, Demographics, and Preparation Data. AJPE Dec 2019; 83(10):7526.
- Gortney J, Rudolph MJ, Augustine JM, et al. National Trends in the Adoption of Pharmacy Curriculum Outcomes Assessment for Student Assessment and Remediation. AJPE Aug 2019; 83(6):6796.
