In this guest post, , Chair of the American Board of Internal Medicine (ABIM) Board of Directors and Internal Medicine Chairman at UT Southwestern Medical Center in Dallas, and , Senior Vice President of Evaluation, Research, and Development at the ABIM explain what determines pass rates for the MOC exam.
Two previous commentaries have addressed MOC exam pass rates: the increasing failure rate for first-time Maintenance of Certification (MOC) examinees and speculation about why physicians may be failing MOC exams.
Recently, based on a slow decline in pass rates over the past 5 years for the American Board of Internal Medicine Maintenance of Certification exam, some social media observers have made erroneous assertions about how the exams are constructed and scored.
Without a full understanding of the exam process, they have nevertheless claimed that the exam is being made more difficult, that the standard for passing the exam changes every time an exam is given, or that the exam is "graded on a curve." These assertions are all incorrect.
ABIM exams are constructed to assess the competencies required for certification and maintenance of certification. ABIM the exam so that the content proportions and difficulty level are the same across administrations, thus ensuring fairness. Someone taking the exam in the spring would experience the same level of difficulty if they took the exam in the fall. Our tests for psychometric validity assure this consistency.
The minimum passing score (i.e., "passing standard") that ABIM sets is based on the exam's content, and subject matter experts determine how much of that content a physician needs to get correct to be deemed certified. A content-based is used to set the passing score.
The method combines the judgments of many physician experts and is well-researched and commonly used in credentialing exams. As long as a physician gets enough content correct (demonstrates appropriate mastery of the content), he/she will pass the exam; it doesn't matter how well others who take the exam at the same time perform.
Considering all MOC exams, a physician must answer, on average, about 65 percent of items correctly to achieve a passing score. The passing standard remains the same over exam administrations and is reviewed periodically. Significant changes in the practice of the discipline may require that the passing standard be reset.
ABIM does not "grade on a curve," and the "pass rate" (i.e., the percent of physicians who pass the exam) is not set ahead of time. That is, no predetermined percentage of physicians will pass or fail the examination. Once a passing standard for a discipline area is set, differ from year to year solely because the characteristics of physicians taking the exam change each time it is given. A number of examinee-specific factors -- such as examinee motivation, training, preparation, and knowledge -- impact pass rates.
Other things to consider:
- For MOC exams, where individuals are being assessed anywhere from 6 to more than 20 years after completing formal internal medicine residency or fellowship training, the numbers and complexity of factors that contribute to their performance on the exam is large.
- Pass rates are higher among first-time test takers -- so the number of repeat examinees in a test-taking pool can affect the aggregate pass rate each time the exam is given.
- Regardless of what the pass rate is for those taking an ABIM exam for their first time, the ultimate pass rate (i.e., pass rates computed for examinees who have had at least three opportunities to test) for ABIM examinations is quite consistent and high, at around 95%.
- There has always been and always will be fluctuation in the pass rate. Pass rates in 2013 are similar to pass rates in 2006. In 2006 the IM MOC first-taker pass rate was 79%, and in 2013 it was 78%.
ABIM understands that taking the exam can be a high-stress event, and that failing can result in a number of significant consequences that may accompany losing certification. Our Board has recently taken steps to address the feedback we have heard about the MOC exam:
- To help physicians maintain their certification, our Board recently decided to give physicians who attempt and fail their first MOC examination an additional year to pass before their certification status changes (assuming all other MOC requirements are being met). This change will go into effect in 2015. Diplomates who have a certification that expires in 2014 and have attempted but failed the exam previously will be given an extra year to pass the examination.
- Beginning next year, we are reducing the first MOC exam retake fee from $775 to $400.
- We have also heard concerns that the information provided in ABIM's exam score reports should be more detailed and understandable so that physicians can learn from their experiences. In response to this concern, we are developing exam feedback that will be more informative to test takers. We anticipate that the additional information will help candidates better understand their personal areas of strength and areas where they may need improvement. These exam enhancements will also be rolled out next year.
We know there are myriad pressures on physicians right now. This is, in many ways, an unprecedented time. Therefore, now, more than ever, we take seriously our mission "to enhance the quality of healthcare by certifying internists and subspecialists who demonstrate the knowledge, skills, and attitudes essential for excellent patient care." The MOC program remains an integral part of how ABIM does this work.
Here are some previous installments of MOC Watch:
MOC Watch: If You Credit It, They Will Come
MOC Watch: CRNA Version Raises Hackles
MOC Watch: ACC Responds to Member Complaints
MOC Watch: Alternative Credit Pathways for FPs
MOC Watch: Society Leaders Give ABIM an Earful
MOC Watch: More Docs Fail Exams
MOC Watch: Pediatricians Weigh In
MOC Watch: Love the Rules? Hate Them? No In-Between
There's no sitting on the fence when it comes to revised maintenance of certification (MOC) rules. This blog gathers MOC-related news and opinion to foster discussion on what has happened since January 2014, when the American Board of Internal Medicine adopted the requirements -- and what is likely to happen in coming months.
Have a tip or want to counter a point? Email s.yurkiewicz@medpagetoday.com.