This is the third in a 番茄社区 series on the controversial but growing trend among healthcare institutions to screen physicians of a certain age with cognitive and physical dexterity tests. Part one reported the growing interest among hospitals around the country in testing their senior physicians; part two reported on the reactions in the physician community. Here, we take a closer look at the MicroCog cognitive screening tool most often used in these programs.
Internist Paul Speckart, MD, age 78, joked that he finds himself back where he was a half-century ago in medical school -- studying for a test. This one, however, is not about his knowledge of renal physiology or foot structure. It's about his memory and mental processing ability.
He practices in San Diego's Scripps Health system, which plans to require all physicians to undergo cognitive screening after they turn 70.
"I've learned to spell the word 'world' backwards, because I've heard that's one of the questions on the test," Speckart said with a grin.
Jokes aside, he knows it's a serious matter. And large systems like Scripps need to find ways to assess senior providers since so many now practice into their late 70s and 80s, instead of retiring at 65. The reality is that mental and physical agility deteriorates, and that can't be ignored when patient safety is at stake, he said.
"It's an attempt to get a better handle on physicians who have declined, and who need to have their practice restricted," Speckart said. "If we look around and we're honest, we know there are some of us who shouldn't be practicing any longer."
But among many of his senior colleagues, there is growing angst about the validity and evidence behind the key tool most often used to assess these older doctors. Some vocalize their suspicions to him that there is little evidence a computer test can accurately tell when a physician is starting to become cognitively impaired.
The exam in question is the , an hour-long computerized test with five question domains: attention and mental control, memory, reasoning and calculation, spatial processing, and reaction time. Many health care systems across the country are now using it to screen late-career physicians.
Developed in 1993 by the Risk Management Foundation of Harvard Medical Institutions, the MicroCog is now marketed by as an appropriate way to detect mild to moderate cognitive impairment for anyone from age 18 to 89.
Clinicians sit alone in a room in front of a computer without paper or a pen, and no mobile devices to assist. A proctor, perhaps a neuropsychologist, sits nearby.
"MicroCog offers research-based insights into the cognitive abilities of physicians and other professionals in high-pressure career fields," said Pearson spokesman Scott Overland, in a statement. "This assessment's recommended use is as a part of a general neuropsychological examination, as one measure of an individual's abilities."
But the MicroCog is controversial. Several physicians who face these new screening policies conveyed their dread that they'll score poorly even though they're not impaired.
"If a doctor is showing signs of dementia whether old or young, then they should have a cognitive test," said Albert Ray, MD, a family practitioner and partner emeritus with Southern California Permanente Medical Group. "But just as a screening tool to pick on older doctors without cause just because of age is discriminatory. I don't believe it is evidence-based."
Pearson's Overland wouldn't allow 番茄社区 to speak with company scientists who could address the evidence.
Many physicians interviewed stressed that any age-related slowness or forgetfulness is overwhelmingly offset by their experience and knowledge of their patients, which makes them better doctors than their younger colleagues. Several spoke on condition that they not be named lest their colleagues stop referring them patients.
False positives/false negatives
The MicroCog is said to have a said , PhD, one of the neuropsychologists who interprets clients' MicroCog results for , the Center for Personalized Education for Professionals in Denver.
Turned around, those figures indicate a 17% false negative rate and a 4% rate for false positives in trying to identify clinicians who are truly impaired.
Studying for the test
Many clinicians anticipating the MicroCog are, like Speckart, preparing for it the best they can.
There's a section that requires participants to do math calculations in their heads. Thus, some are rehearsing their grade school multiplication and division. And they're practicing how to count backwards from 100 in increments of seven.
Semi-retired Gordon Banks, MD, 74, was required to take the MicroCog for the first time last year by Legacy Health's hospitals in Oregon where he works as a neurohospitalist one week a month. In one particularly challenging segment, the computer briefly flashed seven numbers on the screen, one at a time, and he had to recall them.
Midway through it, Banks figured out a strategy. He passed with a grade of "above normal" for his age.
"The trick was to group the digits in groups of three and try to recall them that way," he said. Next year, when he retakes the test for recertification, he'll be more prepared.
Banks said he was originally nervous, but then realized it would provide useful information. "Do I want to practice if I'm getting demented? I'd be crazy to think that."
Banks poohed-poohed concerns of those who say the screening test is age discrimination, and put it bluntly. "The reason we're singled out is because of age, because that's who gets demented. The prevalence for dementia at age 45 is 0.01%. The prevalence at 75 is 10%."
Not a deal-breaker
For most systems using the MicroCog, a poor score doesn't necessarily mean a clinician won't be re-credentialed. It merely provokes a more comprehensive review, with a neuropsychologist keying in on areas that the MicroCog found lacking. In some cases, a treatable medical condition may be discovered as the underlying cause.
"The MicroCog is a screen, and only a screen. It's not a full assessment and that's how it should be used," said Korinek, who did her on the test in 2005 and talked with 番茄社区 at length about the process.
"It is not a test that takes a physician's job away," she continued. "It's designed to test people who are highly functioning, and who have intellectual abilities that are a little bit higher than normal, and achieve higher in their careers."
Korinek also asserted that the MicroCog is "very sophisticated, much more than most screens ... and treats physicians better than almost any other type of screen out there."
CPEP has been using the MicroCog in a different way than as a screening tool -- primarily to evaluate physicians after state licensing agencies or other groups have referred them as part of a disciplinary action. It is only beginning to screen older physicians who have not had another reason for review.
According to Korinek, the MicroCog is versatile in that its components test various aspects of cognitive functioning. A key portion tests memory, a mental skill known to decline with age. Clinicians are asked to read a short story, then promptly answer multiple choice questions about it. About 20 minutes later, the clinician is again prompted to answer more multiple-choice questions about the story.
In the spatial recall portion of the test, the clinician is briefly shown a pattern of darkened squares within a 3 x 3 tic-tac-toe grid, then reproduce the pattern on a keypad. Another test displays clock faces with hour and minute hands, but no markers, and the clinician has to tell the time.
The math portion requires the provider to add, subtract, multiply, and divide, and enter answers on a numeric keypad left to right, which Korinek notes in her dissertation is different than how arithmetic problems are solved on paper, from right to left.
The MicroCog also weighs processing speed, also known to decline with age, against the test taker's accuracy.
MicroCog pushback
The idea that late-career physicians now must undergo such scrutiny -- which some see as an extra-humiliating form of age discrimination -- as part of their two-year re-credentialing has resulted in pushback.
Some physicians say it has little to no bearing on their ability to treat patients, asks no medical questions, and fails to account for their experience and wisdom. If they need to make a math calculation these days, they just pull out their phones.
As a result, several institutions are now reviewing their use of the MicroCog or have stopped using it as a screening tool for older doctors.
The issue has heated up in Utah, where the large Intermountain Healthcare system had been using the MicroCog since 2012 to screen all physicians for cognitive impairment every two years after their 72nd birthday. Last September, the legislature made that testing illegal, but changed its mind with a new law that took effect in April.
But the new law includes an important catch.
The Utah Medical Association doesn't want any health system to use the MicroCog to screen clinicians, at least not without more evidence that it is effective. Besides, if the test is such a good one for health providers, why not require it of every professional who treats patients?
So the UMA had a inserted in the new law that allows age-based screening only if the methods "are relevant to physician practice and to the physician's ability to perform the tasks specifically required in the physician's practice environment."
Does the MicroCog meet that requirement?
"No, I wouldn't say that it does," said Michelle McOmber, the UMA's chief executive officer. The group is working with the American Medical Association to determine whether the evidence supports MicroCog's use to score older physicians' thinking skills. But she's skeptical.
That clause has prompted Intermountain to put the program under review, according to spokesman Daron Cowley, who declined to say whether the screening program will go forward. "We have no specifics to share at this time."
Another issue is that when screenings only begin for physicians late in their careers, there's no baseline for comparison with how they performed 10 or 20 years ago. "That's one of our huge problems with the MicroCog," McOmber said.
Stanford University also was using the MicroCog to screen its clinical faculty after they reached age 74.5. But after many senior physicians protested, the policy was revised. It's now a rigorous peer review process, but without the MicroCog.
"There is not enough evidence in the literature that a cognitive screen of late career practitioners improves the quality of patient care," said Ann Weinacker, MD, Stanford's senior vice chair of medicine for clinical operations and associate chief medical officer. And that includes the MicroCog, although the MicroCog "is the best screening test for cognitive impairment in physicians."
She added that the MicroCog has not been studied in the context of improving the quality of patient care, while many studies have shown that rigorous peer review alone is effective.
Presidents' pro/con debate
Utah and Palo Alto aren't the only places where the issue is now a hot topic.
After news that Scripps would require a screening, including a component with the MicroCog, the San Diego County Medical Society's Bioethics Commission decided to hold a "special discussion" entitled "Health Screening of the Senior Physician: Pros and Cons," with two society presidents squaring off at a dinner forum on May 6. (番茄社区 was not allowed to attend.)
Jim Hay, MD, a family physician in Encinitas, California, for 44 years and former president of the California Medical Association, was slated to take the "con" view.
Speaking to 番茄社区 before the event, he said that he understands that physicians his age aren't immune from cognitive decline. He just wants to see data that the MicroCog and other screening tools can find truly cognitively impaired physicians without falsely identifying physicians who aren't, or missing those who are.
"What I don't see data for is that [the] screening makes a difference," Hay said. "Until you can show me ... you're picking up enough (cognitively impaired) people to make it worth putting all the others at risk, I'm not convinced."
Hay gave two examples of caregivers he knew who would have been missed by current age-based screening programs, one of whom was a former surgeon and chief of a hospital's medical staff. "He became demented in his early 60s, and died at 69." Another member of his team began losing her ability to think clearly in her 50s and died in her 60s.
"If you really want to detect people who have a cognitive disorder and you're starting at 70 -- or like at Stanford, at 75, you're missing a lot of them anyway," he said.
David Bazzo, MD, takes the "pro" side. He's the society's current president who also happens to run , the PACE Aging Physician Assessment program used by Scripps and other health organizations.
"High stakes competency assessments in the U.S. have relied on the MicroCog because there are age-based and education-based norms that are already established that can be used for comparison with regard to cognitive function," Bazzo said in an interview several months ago.
"And in the eyes of some of our consulting neuropsychologists who do this for a living, it's a good test. Is it the best test? Probably not. But is it the best one that exists to date? I think many programs would agree that it is."
Variable implementation
It's not just the test's accuracy that's of concern. Many are voicing questions about what to do with the results and how to roll out the assessment programs:
- The MicroCog does not have a set "fail" or "pass" score on what constitutes mild or moderate cognitive impairment. Rather, individual neuropsychology programs administering the screen make recommendations based on patterns, but medical executive committees or medical groups that pay for them can determine what to do with the results or how to weight various components as part of their peer review function. Does a very low score on one MicroCog module but not on others warrant a higher level of review?
- How quickly must a peer review organization take action when a late career clinician gets a low MicroCog score? What might be their legal liability if something goes wrong?
- Could the mere fact of cognitive screening lower an organization's medical malpractice premiums?
- Anticipating the new requirements, many physicians in San Diego said they plan to prophylactically take the test at their own expense, so they alone can get the results. If they score poorly, they could retire without embarrassment. But if they don't believe the results, can they move to another setting that doesn't require cognitive screening?
- Who should pay the cost of testing, which can get expensive? Bazzo of PAPA estimated the cost of screening at $1,100 to $2,200. But when it triggers a full-blown review, that can push the cost from $12,000-$20,000.
- What is a hospital or medical group's legal and regulatory obligation to report to professional licensing agencies, or even to the motor vehicle department, the names of clinicians who score poorly?
- Korinek pointed out that "while the MicroCog is a well-normed test for the general population, there are no current norms for physicians." Is it appropriate to judge a physician's score without knowing what's normal for physicians?
- Should having English as a second language be a factor in interpreting results?
In her dissertation, Korinek tried to provide some data. She compared MicroCog scores for 68 physicians she recruited as controls against 264 physicians under competency review at CPEP. But she recruited only 13 in the control group ages 60 or older, a decade before the age when most late career testing programs kick in.
Late-career physicians' MicroCog scores are compared with scores of others that same age, so an 80-year-old is said to be normal, above, or below average for 80-year-olds who took the test. But is it right to have passing scores for 80-year-olds that are lower than for doctors who are 70?
"That's a fair question to ask, and I don't have an answer," said Korinek. "My personal opinion is that this is an area that needs to be developed and discussed." An important factor is that as people age their neuropsychological scores have a much wider variation than those for younger cohorts. "There are physicians who are doing absolutely fine and even great, and there are physicians who are not," she said.
In San Diego, Hay asks for evidence that most doctors with cognitive problems aren't self-withdrawing from practice before they cause problems, "to make it worth protecting the public by screening everybody."
"What you're going to end up writing here is that there are a lot of unanswered questions," he said.