BMJ investigation: Serious Concern about regulation, evaluation systems

boardswalker

A worrying study article in the BMJ highlights serious concern about top-selling diabetes drug rosiglitazone (Avandia), its safety, as well as the whole system by which medications are regulated, evaluated and promoted worldwide.

The article is here:
http://www.bmj.com/content/341/bmj.c4848.full

Some comments worth noting
- Dr. Deborah Cohen, investigations editor of the BMJ, obtained data under the Freedom of Information Act that note "a paucity of evidence during the European approval process and outline concerns from some panel members about the long term risks and benefits of rosiglitazone."
- Concern is also expressed regarding the lack of publicly available trial results for independent scientific scrutiny, a lack of transparency in the European system, and the ability of the European regulator to assess individual patient data.

trowel

Scandalous. EBM my @rse.

opinion guy

trowel wrote: »

Scandalous. EBM my @rse.

The problem with EBM is medical degrees don't include teaching on the scientific method. how can one understand evidence based medicine when one doesn't understand the basic tenet of science ?

tallaght01

opinion guy wrote: »

The problem with EBM is medical degrees don't include teaching on the scientific method. how can one understand evidence based medicine when one doesn't understand the basic tenet of science ?

Interestingly that's what me and my boss teach med student. But whatever.

opinion guy

tallaght01 wrote: »

Interestingly that's what me and my boss teach med student. But whatever.

Fair enough.
Please do explain how you teach the scientific method ? (genuinely curious)

tallaght01

opinion guy wrote: »

Fair enough.
Please do explain how you teach the scientific method ? (genuinely curious)

We do it prospectively and retrospectively.

We bring in papers, and analyse them. Always use the format of "you're in A+E at 3am, you get a patient with condition x....you're gonna treat him with drug Y......Here's the evidence for it. Do you think you should use it?" Or you can do it prospectively.....eg you want to show that helmets reduce mortality from head injuries in kids A+E patients, how would you design the trial? And ut them in groups to design the study.

In fairness to you, though, a lot of the shortened courses don't teach it now. That's why me and my boss do these tutes for the senior students on their placements. Plus one local uni has stopped teaching it altogether, so one of our local paediatricians and a statistician now run a weekly compulsory journal club for med students on their paeds attachment.

I also run a fortnightly journal club, but that's for staff (and students if they want to come).

Just to add, any doctor that graduated under the old system will have had a lot of stats lectures. Plus they're art of most membership exams, so senior docs have got a good bit of EBM knowledge, by and large.

opinion guy

tallaght01 wrote: »

We do it prospectively and retrospectively.

We bring in papers, and analyse them. Always use the format of "you're in A+E at 3am, you get a patient with condition x....you're gonna treat him with drug Y......Here's the evidence for it. Do you think you should use it?" Or you can do it prospectively.....eg you want to show that helmets reduce mortality from head injuries in kids A+E patients, how would you design the trial? And ut them in groups to design the study.

In fairness to you, though, a lot of the shortened courses don't teach it now. That's why me and my boss do these tutes for the senior students on their placements. Plus one local uni has stopped teaching it altogether, so one of our local paediatricians and a statistician now run a weekly compulsory journal club for med students on their paeds attachment.

I also run a fortnightly journal club, but that's for staff (and students if they want to come).

Just to add, any doctor that graduated under the old system will have had a lot of stats lectures. Plus they're art of most membership exams, so senior docs have got a good bit of EBM knowledge, by and large.

See this is EXACTLY what I'm talking about. None of this is the scientific method. Doctors think this is science. It isn't.
The scientific method involves makign a hypotheises, making predictions from the hypothesis, testing the predictions by experiment, refining the hypothesis. Reading some articles and discussing them is not the scientific method. Doctors don't make scientific predictions (as distinct from prognostic predictions) and in the vast majority of cases they don't perform experiments. Statistics are only as good as the basic science that underlies the assumptions upon which the statistics are built (a point which eludes almost any doctor I've ever spoken to - even the smart ones).

http://en.wikipedia.org/wiki/Scientific_method

In fairness to you - this is one of those things where doctors will read this and agree with you, scientists will read this and agree with me.

Ultimately, there is a reason a medical degree is not a B.Sc and is actually a Bachelors in the Art of medicine and surgery

tallaght01

opinion guy wrote: »

See this is EXACTLY what I'm talking about. None of this is the scientific method. Doctors think this is science. It isn't.
The scientific method involves makign a hypotheises, making predictions from the hypothesis, testing the predictions by experiment, refining the hypothesis. Reading some articles and discussing them is not the scientific method. Doctors don't make scientific predictions (as distinct from prognostic predictions) and in the vast majority of cases they don't perform experiments. Statistics are only as good as the basic science that underlies the assumptions upon which the statistics are built (a point which eludes almost any doctor I've ever spoken to - even the smart ones).

http://en.wikipedia.org/wiki/Scientific_method

In fairness to you - this is one of those things where doctors will read this and agree with you, scientists will read this and agree with me.

Ultimately, there is a reason a medical degree is not a B.Sc and is actually a Bachelors in the Art of medicine and surgery

I'm in the lucky position of previously being a lab scientist before med school, but I'm still perplexed.

I think we all know that your opinions can be a bit out there, but if you think medicine is comparable to an arts degree, then you're way off.

Looking at data retrospectively is about stats, study design, biases, controls etc.

Looking prospectively is all about formulating a hypothesis (ie helmets save lives) sand testing that hypothesis and the null hypothesis etc.

My hypothesis is that you know very little about medicine, but talk about it a lot.

I did lots of experiments in med school do, and I imagine most here did. Though I appreciate they're no longer abig part of the short courses, but I think they're still used.

ORLY?

opinion guy wrote: »

and in the vast majority of cases they don't perform experiments.

What is a randomised control trial? An experiment. A hypothesis is made, hopefully with some reasonable evidence behind it, and then tested and the results analyzed.

As for not being taught about EBM, as far as I am aware all the Irish schools have a fair amount of medical statistics in them, which are all about the scientific method really. Half the time we're talking about hypothesis testing. We had med stats lectures most weeks for the first 2 years, several workshops this year and next and assignments and research projects throughout the whole course that make up a reasonable amount of the total grade.

opinion guy

tallaght01 wrote: »

I think we all know that your opinions can be a bit out there

Errr what does this mean ? How are my opinions "out there"?

boardswalker

Actually, the concerns here have much less to do with theory than with practice. It has more to do with ethics and integrity than the theoretical scientific method.

On July 12/13 the FDA conducted a safety review of Avandia.

Prior to the review they released a 700 page report. This is widely available.

In that report they acknowledged that they had not adequately overseen the initial trial. If they had they would not have approved the trial.

On reviewing the data, they found "72 mishandled events".
This included 4 cases who developed heart problems and were then pulled from the trial so that the results would not be included in the trial.

When the FDA met previously with the manufacturer to review safety the manufacturers argued the drug was safe and failed to disclose that they had recently received results which showed a 46% increase in heart problems.

The result of the July 10 review was that of the 33 people on the committee, 12 voted to withdraw the drug; 10 said that its use should be restricted; and the remainder wanted it to stay on the market, but perhaps with more warnings. However in the immediate weeks after the review it emerged that one of the panellists who backed avandia had been receiving fees from the manufacturer.

So there is, at best, dodgy evidence for this drug but it remains on the market.

Kurtosis

opinion guy wrote: »

Ultimately, there is a reason a medical degree is not a B.Sc and is actually a Bachelors in the Art of medicine and surgery

I think the designation of B.A. vs B.Sc. is actually more just a historical convention rather than reflecting the content of the degree.

tallaght01

penguin88 wrote: »

I think the designation of B.A. vs B.Sc. is actually more just a historical convention rather than reflecting the content of the degree.

My degree just says bachelor of med, bachelor of surgery. SO I don't even get an arts degree!!!

Vorsprung

I have never seen anyone with a B.Sc medical degree. And it is purely historical. In Trinity lots of people graduate with BAs having done science.

opinion guy

Vorsprung wrote: »

I have never seen anyone with a B.Sc medical degree. And it is purely historical. In Trinity lots of people graduate with BAs having done science.

You can get a B Sc in Medical Science in some med schools. To get this you usually have to have high grades and do an extra reseach project and write a fairly hefty thesis. Very few do it. Its an extra degree alongside the bachelors in medicine and surgery

Vorsprung

opinion guy wrote: »

You can get a B Sc in Medical Science in some med schools. To get this you usually have to have high grades and do an extra reseach project and write a fairly hefty thesis. Very few do it. Its an extra degree alongside the bachelors in medicine and surgery

A Med Sci degree doesn't let you practice medicine, which was the point of the eArlier comments.

That was initailly set up so that people who did the preclinical years but dropped out during the clinical years (because they didn't enjoy the clinical stuff, or other reasons) - the colleges didn't want them leaving empty handed after 4 years work.

opinion guy

Vorsprung wrote: »

A Med Sci degree doesn't let you practice medicine, which was the point of the eArlier comments.

That was initailly set up so that people who did the preclinical years but dropped out during the clinical years (because they didn't enjoy the clinical stuff, or other reasons) - the colleges didn't want them leaving empty handed after 4 years work.

I didn't say it did let you practice med. The point was there is such a thing as a B Med Sci that had different requirements to medicine. May have been for drop outs at one point but these dyas its for the high grades - but maybe that varies college to college

tallaght01

More than half the year in most med schools in the UK were doing an intercalated BMed Sci.

I've never heard of anyone being refused permission to do it, though I'm sure it may have happened. In some med schools it's compulsory.

Nonoperational

I have a BSc and am now a first year med student and I must say I don't quite get when your point is opinion guy.

Just on a tangent, in the brief look I had at the Avandia issues, it looks like pioglitazone is a very similar drug apparently without the risks associated with rosiglitazone but I very rarely see Actos prescribed. Is there a reason for this or is it just the GPs in my area have a habit of prescribing Avandia?

opinion guy

gpf101 wrote: »

I have a BSc and am now a first year med student and I must say I don't quite get when your point is opinion guy.

Just on a tangent, in the brief look I had at the Avandia issues, it looks like pioglitazone is a very similar drug apparently without the risks associated with rosiglitazone but I very rarely see Actos prescribed. Is there a reason for this or is it just the GPs in my area have a habit of prescribing Avandia?

When you were doing a BSc you studied theory, went to the lab did the experiment, saw that the results fit the thoery or didnt. You learned the scientific method by practice. In medicine you just memorise a bunch of stuff. The science degree thought you a process of thought, the medical degree teaches you to regurgitate facts. There's a big difference.

Kurtosis

opinion guy wrote: »

When you were doing a BSc you studied theory, went to the lab did the experiment, saw that the results fit the thoery or didnt. You learned the scientific method by practice. In medicine you just memorise a bunch of stuff. The science degree thought you a process of thought, the medical degree teaches you to regurgitate facts. There's a big difference.

Let me guess, you have a BSc in science but have absolutely no experience of a medical degree whatsoever?

Nonoperational

opinion guy wrote: »

When you were doing a BSc you studied theory, went to the lab did the experiment, saw that the results fit the thoery or didnt. You learned the scientific method by practice. In medicine you just memorise a bunch of stuff. The science degree thought you a process of thought, the medical degree teaches you to regurgitate facts. There's a big difference.

I can see what you are angling at but I don't agree it is that simplistic. Obviously for a medical degree you have to have the basics correct. There are very specific conditions and treatments you have to know. You have to treat people with well established techniques and medications. These must be learned off. You obviously cannot use your patients as guinea pigs as part of the scientific method. We are however thought to look at things scientifically and I would hope doctors would always question and apply common sense and an enquiring mind to all their tasks. There is hell of a lot more to being a good doctor than spiting out things you learned off in med school.

If you are using the trials GSK used to get the initial approval for Avandia as an argument there looks like there could be very serious concers over the methodology and how specific results were/were not included that goes far beyond any philosophical debate about application of the scientific method.

El_Dangeroso

I think a huge problem is that most doctors do not understand statistics to a sufficient level to correctly evaluate RCT's. I work with an excellent biostatistician and am lucky that I can bring things to his desk to have a look at. 90% of the papers I have brought to him have had a serious flaw in the statistics section. If I spent three days solid on them I wouldn't have spotted the error and I've done three courses on biostats to date. Once he pointed them out it's obvious. These are studies that have professional stats people on board.

Then there are inherent issues of what Bayesian statistics can and cannot do but that's a whole nuther can of worms.

boardswalker

Temple_Grandin wrote: »

I think a huge problem is that most doctor's do not understand statistics to a sufficient level to correctly evaluate RCT's.

It is also very hard to analyse data that has been suppressed.

boardswalker

Update -

European Medical Regulators are meeting this week to decide if Avandia should be pulled from the market. Commentators think that the BMJ article has put more pressure on the Regulators to pull the drug.

In another development, two FDA officials alleged last week that the July FDA Panel review was biased as several members of a previous Avandia safety review were recalled for the July review. Apparently, it is normal practice not to recall members of previous reviews as it is considered less likely that they will vote to pull a drug they previously voted safe. The FDA officials argue that this "unprecedented" recall shows the July panel was rigged.

Meanwhile, Actos a rival drug to Avandia is now being subjected to a safety review because of links to bladder cancer.

boardswalker

In two separate announcements today

European regulators said that Avandia should be withdrawn from the market and the FDA placed major restrictions on the drug.

Statements from the regulators showed they agreed on the risks associated with the drug, but reached different conclusions on action.