Medical Studies – How They Work

Many of you wrote in appreciation of last month's newsletter about the scientist in each of us. The idea is not original to me. Child psychologists have been all over this topic for at least a decade, as you can see at http://wkar.org/post/my-messy-child-scientist

In this newsletter, I want to narrow my focus to the science of medical studies. I hope this will help you when you read the latest health news or research your own health issues. I'll start at the beginning with the simple (but crucial) observational study and build toward what is today the gold standard of medical studies – the randomized double-blind placebo-controlled study.

The Observational Study

In 16^th century Switzerland lived a brilliant doctor named Paracelsus. Like most doctors of his age, he wasn't shy to give you an astrological reason for your illness, and he also dabbled in alchemy. Still he had a way with sick people. A powerful governor had a beloved daughter no local doctor could help. Having heard of Paracelsus' skills, he sent for the learned doctor. Sure enough, Paracelsus showed up and healed the daughter. Pleased, the governor gave him a leading role in the canton, from which position Paracelsus managed to so alienate all the other physicians he was inevitably sent packing.

Not a perfect human being, Paracelsus was nonetheless the pioneer of pharmaceutics, medicines not found in nature but concocted by man. Paracelsus discovered that certain compounds of mercury possessed diuretic powers far beyond those possessed by herbs. When lungs begin to fill with fluid backed up behind a failing heart, mercurial diuretics could and did save lives.

Now, how did Paracelsus know that his mercurial diuretic could save lives? He started with a theory, of course, but he didn't know for certain until he tested it. He found when he gave his medication to people with heart failure, the fluid left their lungs and they could relax, move around and give effusive thanks. This was an early observational study.

The Placebo-Controlled Study

Fast forward now to His Majesty's Ship Salisbury in 1747. Sailors on lengthy sea voyages had long faced the threat of fatigue, severe pain, bruising, loose teeth, and even death. Magellan lost 208 out of his 230 crew on his round-the-world voyage; mostly to this illness they called scurvy. A few innovators had found that citrus fruit prevented and treated this illness, but others thought that any acidic drink would be effective.

James Lind, the Salisbury physician, decided to put those ideas to the test. Twelve sailors on his ship were in a similar stage of the illness, so he administered oranges and lemons to some, acidic drinks (such as vinegar or cider) to the others. Only the citrus was effective. As far as historians know, Lind's experiment was the first placebo-controlled study, the first time that an active drug was compared with an inactive one.

The Blinded Study

Fast forward again to Duke Medical School in 1968, when the dismal parade of incurable illnesses was interrupted by something more diverting. A letter appeared in the New England Journal of Medicine, reporting an apparent reaction to the MSG used in Chinese cooking, a flushing sweating condition called "Chinese Restaurant Syndrome." The NEJM was supposedly an impeccable source of medical information. Suddenly millions of people discovered that MSG caused unpleasant symptoms.

However, some skeptics noted that the original report was the most primitive of research studies, the observational study. There followed several placebo-controlled studies in which patients were given either a capsule containing MSG or a placebo. None of the studies found consistent evidence that individuals were sensitive to MSG. When they didn't now what they were swallowing, when they didn't taste it, they did not react.

Lind's sailors could see whether were getting an orange or an acidic drink, so while Lind's study was placebo controlled, it wasn't blinded.

The MSG and food allergy studies were blinded. This prevented the patients' expectations of what might happen from interfering with what actually did happen.

The Randomized Placebo-Controlled Double-Blind Study

When only the patient doesn't know what's in the capsule, we call that a single blind study, or simply a blinded study.

However, when the doctor knows which patient had received which substance, their own assessment of the drug's effectiveness or lack of it often colors their report of how well or poorly the patient fared. Even worse, the investigators can choose to give their pet pharmaceutical to people more likely to recover and the placebo to the seriously ill patients.

So came into being what we now consider the gold standard, the placebo-controlled randomized double blind study. Neither patient nor doctor knows what is in the capsule, so the no one's expectations can color the outcome. People are chosen by lot to receive placebo or active drug. This way investigators can't give their pet medicine to people more likely to recover. The pills come in numbered bottles with identical appearance, and only when the study is over do researchers analyze the results.[1]

Nature Can Still Surprise Us!

Sometimes medical studies turn up results nobody ever expected. Take the case of sildenafil, a blood vessel dilator, intended to treat heart disease. People with heart disease took either sildenafil or a placebo, and the researchers discovered identical symptom relief from both. Thus, millions of heart disease sufferers were spared the hazard of an ineffective drug. When the study ended, however, many men refused to return their left-over pills. When the Pfizer discovered why, they promptly changed course, did another study, and released sildenafil under the brand name Viagra.

Can You Give a Surgical Placebo?

Your knee hurts, the anti-inflammatory agents don't help, nor does the physical therapy. Your orthopedist obtains an MRI, which shows beat-up cartilage. She tells you she needs to trim up that cartilage.

She anesthetizes you, enters your knee through a small incision, clips away, and more often than not, you feel better. If your orthopedic surgeon works in an academic center, she collects many cases, yours among them, and publishes a paper about them. Her paper reports a long-term improvement rate somewhere between 50 and 75 percent. This you now know is an observational study.

Around 2002 a bright orthopedic surgeon decided to put this surgery to the test. Accordingly patients were chosen at random and assigned to one of two groups. Half the patients got a sham surgery where an incision was made when the patient was unconscious but the knee joint was not entered. The other half had the actual surgery with the knee joint was entered and the cartilage trimmed.

Low and behold, both groups did just as well after surgery and we learned that the surgery was no better than a placebo procedure. You can read the details at http://www.nejm.org/doi/pdf/10.1056/NEJMoa013259

Pop Quiz

Sharpen your pencils and take this five question quiz.

1. How would you describe the study on knee surgery?

Placebo-controlled?

Double-blind?

Single blind?

Randomized?

Answer: The authors described this as a "placebo-controlled study," the placebo being the "sham surgery." The surgeon and his team knew what they were doing, so it was not technically not double-blind. The evaluators, those who graded the patients' recovery were not told who had what, so they were blinded. Finally, patients who agreed to the study were chosen by lot, so this qualifies as randomized.

2. When I began in medicine, many patients were certain that penicillin helped their cold and sore throat regardless of cause. "My doctor always gives me penicillin and I always improve right away." I could tell them that viruses cause most sore throats and penicillin has no effect on them. They knew otherwise. When antibiotics are given for conditions not caused by bacteria, why do people frequently feel better?

Answer: Even though antibiotics were are not effective against a virus, the very act of being cared for and given something causes changes in the brain that promote healing. This is called the placebo effect.

3. In a placebo-controlled study of treatment for colds and flu, chicken soup would make an excellent placebo.

Answer: false. Chicken soup has been shown to have certain beneficial effects in people with colds and flu. It would skew the results because it is an active placebo.

4. In the early 1940s, meningitis killed just about everyone who developed it. Doctors observe that penicillin treatment reverses that outcome. How would you describe that kind of study?

Answer: an observational study

5. No randomized placebo-controlled double-blind study was ever done to test the effectiveness of penicillin against meningitis in those early years. Why was such a study never done?

Answer: When a drug brings about obvious positive results, it is unethical to ithhold that remedy in order to conduct a placebo-controlled trial. Sometimes an observational study is all we need.

Next month

For all the care that went into developing medical studies, you would think medical research gives reliable results. As has been so ably pointed out by Stanford and Tufts professor John P. A. Ioannidis, something in the range of 40 percent of published studies give results that are flat wrong.

Next month we'll explain what goes wrong and give you the tools to evaluate medical studies on your own.

Endnotes

[1] Today, researchers will keep a running tally of the results, never disclosed to the physicians evaluating and treating the patients. If the tally shows the treatment to be far superior to the placebo, or shows that the treatment is harming people, the study is terminated forthwith.