Investors in stocks learned a valuable lesson the hard way when their portfolios came tumbling down as overrated equities crashed during the recent bear market reversal.
Hindsight placed the blame, to a large extent, on the fancy illegal manipulations utilized by several top rated companies designed to demonstrate profit where none actually existed. Obviously, only the insiders were aware of what was going on. However, let us assume that financial reports submitted by these companies were legitimate, in the sense that everything was published, including the manipulations, thereby avoiding any lying. If you were an investor, do you honestly believe that you would have been able to detect the manipulations? Let’s face it, would you have even gone to the trouble of trying to do so? The truth is that most individuals tend to rely on the conclusions presented by the investigators and their financial advisers. If you have been reading my articles, you would probably realize by now that this approach to the truth can be fraught with danger.
Unfortunately, failure to properly assess the validity of a research study may well lead the unwary into making incorrect treatment decisions. This was the conclusion of a study by Schwartz and Hupert called “Medical students’ application of published evidence: randomized trial†which appeared in the March 8, 2003 BMJ. Suppose we now examine this study in order to determine its own validity.
The study was done at the University of Illinois. All 164 participants were fourth year graduating medical students who were asked to examine 10 standardized patients. However, one of the ten patients was utilized to evaluate the manner in which each student approached the problem in order to make clinical decisions. The test patient was introduced as having been in an automobile accident, and was now suffering from an apparent neck injury. Students were asked to examine, and take a history of the patient with regard to the neck injury. They were then queried as to the need for an x-ray of the cervical spine by rating their certainty in the correctness of their decision on a visual analogue scale. Then students were told that another student on their team had given them an abstract from an article in a well-respected journal, which covered the topic. The object of the article was to provide a three-question method for determining the need for an x-ray. However, before providing the students with a copy of the abstract, four different versions were created varying in levels of validity and levels of importance of results. These different versions were then randomly distributed among the students.
Once the students had viewed their version of the abstract, they were asked to repeat their decision and rating of certainty followed by a brief statement about why they had made a change or did not, without seeing the patient again.
The results indicated that when the importance level was low the students tended to shift their decisions incorrectly more frequently. However, regardless of the validity of methods or of the interaction between the validity and importance, students tended not to be influenced one way or the other as to whether their treatment decisions were correct. In other words, medical students can take new evidence gained from a research study and apply this evidence correctly. However, they often fail to take into account how well a study is designed. They don’t tend to appraise studies. They just interpret the results. It was also found that only a few students acquired all necessary clinical data from the patient.
If we appraise this study, we find reasons to object to the methodology utilized here as well.
First, medical students vary widely as to interest. The case presented was that of injury due to an accident. Students oriented towards emergency medicine, surgery, orthopedics, neurology, etc. would be more likely to evaluate the new evidence appropriately than those interested in becoming psychiatrists, gynecologists, allergists, etc. They would also be more likely to acquire necessary data from the patient. It would be interesting to see the difference in results if this test were run utilizing a group of residents in emergency medicine.
Second, it seems rather strange that the new evidence was presented in abstract form. How can someone evaluate the design of a study properly if they do not have access to the complete manuscript? Furthermore, good clinicians rarely base their judgment concerning proper procedure on a single article, let alone a single abstract.
Finally, how can you reach a valid conclusion from a study that includes a mixed bag of only 164 students from a single institution? Perhaps students at another medical school would have been better trained in evaluating research studies, or vice-versa? There are too many variables involved in determining the ability of a neophyte to interpret or apply the intricacies of a research study with regard to its design and results as compared to those of other similar studies.
Does this mean that medical students are poorly equipped to practice medicine because they are not properly trained to keep up with, and evaluate the literature as a tool for enhancing their own capability in the future? As someone who has been involved in providing continuing medical education for physicians for more than forty years, I feel justified in concluding that most doctors do not have the time to spend keeping up with the literature. Even when they do make the effort, very few are capable of properly interpreting the design or the results obtained by a research study. Anyone who has not been directly involved in preparing and instituting protocols for research projects is not likely to be capable of readily evaluating the design and technical intricacies of studies. This is the reason why most continuing medical education today has become a requirement of relicensure in all fifty states of this country.
Physicians rarely read journals. The important material is first thoroughly digested and evaluated by specialists in medical education, then spoon-fed at meetings, on audio and videotapes, on the Internet, or in any way that's practical. Unfortunately, the presenter of this information may also be the originator of the research or sponsored by the company that developed the product, and therefore somewhat biased. It pays to toot your own horn.
He’d made his decision,He was certain - no doubt.
Please Make Comments and Share It Below
About Marvin Ackerman, MD
Dr. Ackerman provides helpful insights and answers to the important medical issues that affect you. He is also recognized as a prolific lecturer, researcher and inventor, and an author of numerous medical articles for journals, magazines, and the Internet.
More about Dr. Ackerman.
Cartoons and Poems following each article are created and copyrighted by Dr. Ackerman and cannot be copied or reproduced without his permission. Copyright © 2009 by Marvin Ackerman, M.D.
Shots Disclaimer
Editorial content of Shots belongs to and reflects the thoughts of the author only. Do not consider medical editorial reviews, news items and other general information found on Shots as a prescription, medical advice or an endorsement for any treatment or procedure. Always seek any medical advice from your doctor. Medical editorial reviews and other news items that you read about in Shots may or may not be appropriate for your particular health problem or concern. Always refer these matters to your physician for clarification and determination. Any information provided in Shots may be controversial, totally unrelated to your own situation, even harmful if taken merely at face value without appropriate evaluation of your specific condition, and therefore must be considered simply to be an editorial review, a news review or a general medical information review and not as relating to your specific condition or as information for diagnosis, evaluation or treatment of your specific condition. Unauthorized reproduction, and linking of Shots in whole or in part to any other website, webpage, print and other electronic media, i.e. TV, Videos etc. is strictly prohibited and is punishable by law.<
