Rating the Raters

Let’s say you develop some heart problem and get sent to me for an evaluation. You show up to the office, check in, get ushered into an exam room, then you wait and wait and wait. When I finally come into the room I seem terse, impatient and rushed, and you end up visiting with me for less than 5 minutes. I provide you with a diagnosis, order a couple of tests and dash off a prescription.

What’s your impression of me? What’s your opinion of me as a doctor, or of my medical skills? If you were asked to rate the quality of care you receive what kind of score would you give me?

Now let’s add some more information to the mix. Would it change your mind to learn that I was late seeing you because I was in the emergency department providing lifesaving care to Sister Margaret Mary, an elderly nun who is the sole caregiver for a dozen homeless orphans? And I cut my visit short because I had to get back to the cath lab to open her critically blocked heart artery so that she can survive to continue her motherly care and what seemed like impatience was actually my anxiety about the welfare of the poor orphaned children.

Would your opinion of me be any different?

And what if you see me for a follow-up visit on a day when I am not busy saving nuns or orphans, have all the time in the world for you, and yet I indifferently rush through my appointment just so I can get back to my office to surf the internet and update my Facebook profile? Are you now back to your original opinion of me?

My real question for you is this: Given the limited interaction you’ve experienced, are you in a good position to pass judgement on my skills as a doctor?

According to many websites on the Internet, you are.

Go to your favorite search engine and type in “rate your doctor” and you’ll find a dozen sites that allow you to grade your doctor on knowledge, timeliness, professionalism, competency, and even compassion. The rise of these sites mirrors other service industries—hotels, restaurants, home repairpersons, etc.—that have had to begin to factor open, anonymous, internet-based feedback into their management strategies. One area where this has become amazingly popular is education, where students can now seek revenge on boring instructors who were not insightful enough to reward the student’s languid brilliance with anything more than a C-.

In order to come to some understanding of this trend I spent a little time browsing my own entry on various doctor rating websites and have come to the conclusion that they tend to be inaccurate, dated, and sparsely applied.

I learned from RateMDs.com that my name is actually Dr. Van and that I practice in Lincoln (my office used to be headquartered there). No one has provided any feedback on Dr. Van, although one person gave a glowing report to my brother, Dr. Van De Graaff, a dermatologist in Omaha (whom I easily found): “Staff was super nice as was he, and he didn't waste my time. He isn't out for the money as he even recommended the least expensive option which worked great. I would totally recommend him.” This anonymous rater, who thought my big brother was “super nice,” obviously didn’t grow up with him.

Vitals.com also believes I practice in Lincoln but at least they got my last name right. In addition to the usual demographic information they provide about me they also go to the trouble to rate my medical school (3 stars) and my residency training program (4 stars).

DrScore doesn’t even know who I am, but did identify another sibling of mine as a head and neck surgeon in Lincoln (unfortunately, he moved to Idaho over two years ago). He hasn’t had anyone rate him yet, although I was tempted to grade him on his competence as a little brother.
DoctorScorecard and Healthcare Reviews advertise themselves as rating sites but are pretty shoddy. To be fair, though, they seemed to target an international audience by providing ratings for physicians in India and the U.K. Needless to say I couldn’t find my name on these sites.

The very popular Angie’s List has come to doctor rating a little late in the game. This is the only site that requires you to pay a subscription to search for your doctor (the other sites will give you basic information for free but you’ll have to pay a small fee to learn if your doctor has malpractice suits or is wanted by the police). I can’t give you any feedback about Angie’s List since I’m too much of a cheapskate to sign up for service.

The most popular site for doctor rating appears to be HealthGrades (another site, PhysicianReports, links straight into HealthGrades). I was able to find my name along with current addresses for our CHI Health offices, although I had to search under “Lincoln” to locate myself. Miraculously I learned that one patient went to the trouble to locate me on this site and provide a rating (I got 5 stars in all categories, which makes me suspicious of my 8-year-old daughter and her precocious internet skills).

Thinking that perhaps my patients are less internet-savvy (or perhaps more indifferent to the care I provide) I browsed HealthGrades to see if other doctors in Omaha have more patient feedback that I have. Of the 84 cardiologists listed, only 36 showed any rating at all, with most of those having only a solitary patient who went to the trouble to find and grade them. Only one cardiologist had ratings from 3 patients.

Our experience in Omaha doesn’t differ significantly from the trend nationwide.

A study in last month’s Journal of General Internal Medicine showed that Boston-area practitioners were as rarely reviewed as my colleagues and I are. Additionally, almost all the reviews given where overwhelmingly glowing—not much help if you want to weed out the doctors you want to avoid. A commentary in the LA Times summed up the overall impact of these sites:

Patients just don't appear too interested in providing feedback on their doctors, the authors noted, despite the fact that consumers generally love and use ratings systems. In contrast to the scarcity of doctor reviews, a search of restaurants in Boston's Beacon Hill area "turned up 38 narrative reviews for a single Lebanese restaurant," the authors pointed out.

The sites have the potential to empower patients. And, to be sure, consumers today are encouraged to be smarter and more discriminating shoppers of healthcare services and products in order to reduce costs. But this is one Internet function that doesn't seem to do patients much good or doctors much harm.

From a statistical perspective it makes sense that these ratings sites have yet to achieve prime-time utility. Just one or two reviews on a doctor are not enough to be able to provide a representative opinion on the overall care of the provider. A single, heavily biased review (whether good or bad) would impart an inordinate influence on the overall score. In order for such a system to really provide useful feedback it would have to include reviews from hundreds of patients (such as is the case with consumer websites such as Amazon), a scenario we’re not likely to see anytime soon.

My verdict is that the idea as a whole is sound but that the execution thus far is lacking. I feel comfortable that my value as a service provider could be accurately scored and quantified if it were based on the feedback of hundreds of my patients—if I’m a lousy doctor the truth will undoubtedly come out in the overall tally. Just one perfect score (like I currently have on HealthGrades) doesn’t give my prospective patients any useful information about what they might expect when they come to see me. Someday this system might be an excellent way for patients to have more control over what sort of provider interaction they can expect, but as it stands they still have to rely on old-fashioned word-of mouth and hope for the best.

Now, if you’ll excuse me, I’ve got to get back to caring for nuns and orphans.