Show me that it works. Show me how it works.

Genghiz Cohen

An interesting debate I had with Girlfriend over an Eddy Rockets lunch.

Is it enough to say, I don't understand how but X seems to work 90% of the time, therefore X works.
Or should you have to figure out how X works before you can make such a claim. (Then, obviously, grab a compass).

The idea sprung from Homoeopathy. If someone could show that under controlled circumstances (No other treatments in that time and such), 1000 people took a homoeopathic remedy and they were without a doubt cured during the time they were part of the study, is that enough to say that this particular treatment works and start getting doctors to prescribe it?

I say no, until you understand how the remedy works, you cannot say that it was the cause of the healings. Girlfriend disagrees.

What say ye, wise sages of A&A?

Gregor Samsa

Ask yourself this;

You have the condition that the study you describe is investigating the homeopathic treatment of.

You hear that 1000 out of 1000 of the people in the study were cured without doubt during the time they were part of the study.

Would you take the treatment as it's presented, or wait until someone figured out exactly how it worked?

I know I'd take the treatment. It it worked for me too, I'd say I was cured by it, even if I (or anyone else) doesn't fully understand the mechanism by which it cured me. If it was later discovered that it was actually something else entirely that cured all 1001 of us, then I'd be happy to change my position - but I can only go with the evidence to hand, and a 100% success rate in properly controlled conditions would be evidence enough for me to be getting on with.

robindch

Genghiz Cohen wrote: »

If someone could show that under controlled circumstances (No other treatments in that time and such), 1000 people took a homoeopathic remedy and they were without a doubt cured during the time they were part of the study, is that enough to say that this particular treatment works and start getting doctors to prescribe it?

Been done a thousand times with homeopathy. People take it, people don't take it, makes no difference.

In the few studies where homeopathy was "found" to be effective, it turned out that the homeopaths were only (a) asking people whether the treatment was effective and (b) only asking those people who returned for multiple appointments.

Obviously, if you're not happy with the treatment, or you've been cured or you're dead, then you're not going to show up to give your opinion. Which is obviously going to be at least "happy" enough to go back.

/sheesh

Genghiz Cohen

phutyle wrote: »

Ask yourself this;

You have the condition that the study you describe is investigating the homeopathic treatment of.

You hear that 1000 out of 1000 of the people in the study were cured without doubt during the time they were part of the study.

Would you take the treatment as it's presented, or wait until someone figured out exactly how it worked?

I know I'd take the treatment. It it worked for me too, I'd say I was cured by it, even if I (or anyone else) doesn't fully understand the mechanism by which it cured me.

Well this is true but not exactly what I am getting at.

Something about the treatment works but until you know for sure what's happening, can you say it is the treatment as a whole?
Reminds me of a weight loss device. It claimed to use infra-red rays to 'melt the fat' then all you had to do was spend a half an hour on an exercise bike to burn off the liquid fat. I'm sure that treatment worked for some, but did it work because of the infra-red ray along with exercise or is there another explanation?

Genghiz Cohen

robindch wrote: »

Been done a thousand times with homeopathy. People take it, people don't take it, makes no difference.

In the few studies where homeopathy was "found" to be effective, it turned out that the homeopaths were only (a) asking people whether the treatment was effective and (b) only asking those people who returned for multiple appointments.

Obviously, if you're not happy with the treatment, or you've been cured or you're dead, then you're not going to show up to give your opinion. Which is obviously going to be at least "happy" enough to go back.

/sheesh

Wut?

Gregor Samsa

Genghiz Cohen wrote: »

Something about the treatment works but until you know for sure what's happening, can you say it is the treatment as a whole?
Reminds me of a weight loss device. It claimed to use infra-red rays to 'melt the fat' then all you had to do was spend a half an hour on an exercise bike to burn off the liquid fat. I'm sure that treatment worked for some, but did it work because of the infra-red ray along with exercise or is there another explanation?

It's just that your OP said under "controlled circumstances". If that weight loss treatment was tested properly under controlled circumstances, they'd be checking that the fat actually melted after the application of the infra red rays (with a scan or biopsy, or whatever) - not just checking that people lost weight at the end.

Maybe I'm assuming that the "controlled circumstances" are more stringent than you intended.

bnt

Well, Gravity works, but we don't know exactly how it works. We don't have much of a choice when it comes to gravity: it's there, whether we like it or not. :pac:

When it comes to medicine, we do have a choice. I am actually taking a trial medicine at the moment whose exact method of working is not fully understood. It has successfully passed Phase III double-blind trials and was approved by the US FDA and other international bodies this year. It's derived from a natural source, and researchers have figured out what it's doing and why that's a good thing, but can't yet say exactly how it does that. It didn't stop me from signing up for the trial, and it's not going to bother the patients who'll benefit from it. So yes, we don't wait until something is fully understood before exploiting it.

Genghiz Cohen

phutyle wrote: »

It's just that your OP said under "controlled circumstances". If that weight loss treatment was tested properly under controlled circumstances, they'd be checking that the fat actually melted after the application of the infra red rays (with a scan or biopsy, or whatever) - not just checking that people lost weight at the end.

Maybe I'm assuming that the "controlled circumstances" are more stringent that you intended.

My idea would be, for the weight loss one would be like a controlled diet controlled exercise and a set amount of infra-reds a day. The subject would be weighed but not told what their current weight was and such.

Nothing militaristic

Latatian

The question isn't "Do 1000 people get better when they're given this treatment", it's "If I give these people the treatment- will they get better? Will it be because of the treatment?"

Say for example I wanted to show that eating chicken nuggets makes people's arthritis better, I could do a study that started in winter and then goes on until summer. Voila, most of the people felt better in summer, therefore chicken nuggets work!

Heck, you could do something like this by accident- say you got people with asthma and asked people who had recently had an unusually severe attack to participate. Most people don't have severe attacks all the time, so it would look like most people got better (they all had awful attacks at the beginning, but by 3 weeks most of them hadn't had an attack). Looks like a really exciting result, but it doesn't mean anything.

Or I could do the same study for a bug that's going around and show that people get better in 3 days on chicken nuggets: if I don't know how long that bug normally lasts for, I don't know whether my treatment has any effect.

That's why they control trials. Get your group, give half of 'em the treatment and the other half something that won't do them any good or harm, or nothing at all. See what the difference is.




Then there's placebo of course. Everyone knows flat 7up works wonders for any kind of illness, up to and including a broken leg. Doesn't have any effect on the disease itself of course, but it's what I was always given so everything magically gets better when I take it. Other people only can take fizzy 7up, or chicken soup, or diet fizzy drinks, because that was the cure-all they were given as kids.


As to how it works- it should be reasonably plausible that it should do something of course (not something like wearing yellow underwear to cure meningitis) and it should be shown to have a real effect, but the specifics of how it works isn't as important. There are plenty of things which we know work, but we don't know exactly what they do yet.

Genghiz Cohen

Latatian wrote: »

The question isn't "Do 1000 people get better when they're given this treatment", it's "If I give these people the treatment- will they get better? Will it be because of the treatment?"

[...]

Then there's placebo of course. Everyone knows flat 7up works wonders for any kind of illness, up to and including a broken leg. Doesn't have any effect on the disease itself of course, but it's what I was always given so everything magically gets better when I take it. Other people only can take fizzy 7up, or chicken soup, or diet fizzy drinks, because that was the cure-all they were given as kids.

Not exactly the kind of responses I'm after.
Look at bnt, that's what I want. Tell me why I'm wrong/right to wait until the mechanism for a treatment is understood before actually declaring it to be a treatment.

Knasher

Well it firstly depends on what we are talking about, I'd imagine that almost 100% of people who take homeopathic cold "medicine" do get better. Their rate of recovery just isn't statistically significantly faster than the people not on homeopathic crap.

Assuming we were talking about something that was statistically significant (and I'm aware you were, I just wanted to point that out) then a 90% improvement would be enough for me to start trials, prove that it is repeatable and after all that start using it as a viable cure.

At no point, except in the colloquial sense, would I say it *conclusively* works without understanding how it works.

magicbastarder

Genghiz Cohen wrote: »

Is it enough to say, I don't understand how but X seems to work 90% of the time, therefore X works.
Or should you have to figure out how X works before you can make such a claim. (Then, obviously, grab a compass).

yes and no. i know this is not specifically about homeopathy, but if you tell someone 'try process X', and they do, and it has effect Y, you can say that 'trying process X results in Y', but not that 'process X itself results in Y' - unless you can *completely* remove any side effect the process itself causes.

swampgas

Genghiz Cohen wrote: »

Not exactly the kind of responses I'm after.
Look at bnt, that's what I want. Tell me why I'm wrong/right to wait until the mechanism for a treatment is understood before actually declaring it to be a treatment.

Well, the theory that explains why something works doesn't usually appear fully formed, it usually evolves as more and more research is done. Medical research is difficult, human biology is complex.

Take Grave's Disease for example - a well known disease, with well understood treatments, but it is one of those auto-immune diseases which is a long way from being fully understood.

I would settle for best-practice treatments based on a empirical data, even if a solid theoretical basis is not available.

strobe

Genghiz Cohen wrote: »

Not exactly the kind of responses I'm after.
Look at bnt, that's what I want. Tell me why I'm wrong/right to wait until the mechanism for a treatment is understood before actually declaring it to be a treatment.

Because that gap between discovering something works and finding out exactly how it works could be pretty large and lots of people could die or suffer needlessly while they are waiting for people to figure it out.

mikhail

Genghiz Cohen wrote: »

An interesting debate I had with Girlfriend over an Eddy Rockets lunch.

Is it enough to say, I don't understand how but X seems to work 90% of the time, therefore X works.
Or should you have to figure out how X works before you can make such a claim. (Then, obviously, grab a compass).

The idea sprung from Homoeopathy. If someone could show that under controlled circumstances (No other treatments in that time and such), 1000 people took a homoeopathic remedy and they were without a doubt cured during the time they were part of the study, is that enough to say that this particular treatment works and start getting doctors to prescribe it?

I say no, until you understand how the remedy works, you cannot say that it was the cause of the healings. Girlfriend disagrees.

What say ye, wise sages of A&A?

This is done all the time. No one understands the mechanism by which general anaesthetics work. They just do.

Knasher

magicbastarder wrote: »

yes and no. i know this is not specifically about homeopathy, but if you tell someone 'try process X', and they do, and it has effect Y, you can say that 'trying process X results in Y', but not that 'process X itself results in Y' - unless you can *completely* remove any side effect the process itself causes.

Damn you, I had just come up with an elaborate hypothetical example involving acupuncture to illustrate exactly that in a long and drawn out way but it seems kinda pointless to state it now, but here you go.

Lets say I have a disease X with no known cure and one day I notice that I'm cured shortly after using acupuncture. Being rather popular I tell a load of people with the same illness and they give it a go and are all cured as well. The scientific community take notice and they preform clinical trials and such and conclude that acupuncture is 100% effective in curing X. (notice the wording btw)

Now lets say that after 10 years of it being widely used as a cure for X some researcher finally finds out why. Turns out that the substance that the needles are cleaned with leaves some residue behind and this is reacting in some way with disease X. The acupuncture is completely incidental, just acting as one viable (but less effective) method of application. Anybody who concluded that acupuncture actually works would have been wrong in retrospect and would have carved "fancy that" into the side of their cock for no good reason. That's why the how really does matter.

smk89

Genghiz Cohen wrote: »

Not exactly the kind of responses I'm after.
Look at bnt, that's what I want. Tell me why I'm wrong/right to wait until the mechanism for a treatment is understood before actually declaring it to be a treatment.

If we have to wait till the mode of action is found for any drug then using it before then (even if it has strong evidence to help the patient) would be a legal minefield. If an adverse reaction occurs it would constitute malpractice.
Digoxin has been used in heart failure and arrythmias since roman times (at least foxgloves have) but the mode of action wasn't known. Even willow tree bark given for pain and only in 200 years have we discovered it contains aspirin.
Modern drugs like l-dopa for parkinsons disease and even tylenol or anti-malarials have strange effects that can't be explained. Essentially until we can simulate a human body in a computer and monitor every reaction it's unlikely we will understand why.
Even when we know how it works drugs have effects on many systems and it cant be classed as a treatment until all are understood.
In the mean time many will die or suffer due to the wait. This goes against the do no harm principle. Though it is also true that many will be harmed by the drugs many more will benefit from them.

Genghiz Cohen

smk89 wrote: »

Though it is also true that many will be harmed by the drugs many more will benefit from them.

I'm not sure it's fair to say that until you know everything about the drug or you have widespread use of the drug.

raah!

If we had to wait to know the precise nature of causation before pronouncing judgements on the future correlation of presently correlated events, then we would never be able to make judgements on anything ever. All we can ever say is that "in the past this has followed that, in the future it will too", that is what all science is.

As was pointed out, when newton was formulating his laws of gravitation, he didn't see the force vectors or gravity waves, all he saw was things being attracted towards each other lots of times. He made a leap and introduced a model where a "force" attracts one mass to another. He has absolutely no rational or experiential/experimental justification for introducing this concept of "force" as something which was real.

Classic Hume.

smk89

Genghiz Cohen wrote: »

I'm not sure it's fair to say that until you know everything about the drug or you have widespread use of the drug.

If a drug passes 3 stages of clinical trial before even going on the market as well as being monitored for the 4th trial while being used it is fairly certain that it will at least statistically speaking be more good than bad. Otherwise it wouldn't be released.
Of course many risks arn't found till after the drug is put on sale as shown in champix the anti smoking drug which after marketed for 1 year before showing large suicide risk and cardiovascular risk after 5 years.
But likewise I don't know everything about computers but that doesn't stop them from using them.

DublinWriter

Genghiz Cohen wrote: »

An interesting debate I had with Girlfriend over an Eddy Rockets lunch.

Is it enough to say, I don't understand how but X seems to work 90% of the time, therefore X works.
Or should you have to figure out how X works before you can make such a claim. (Then, obviously, grab a compass).

The idea sprung from Homoeopathy.

Pushing the nonsense of homoeopathy aside for a minute, this is how many modern medicines are discovered.

For example - Rogaine/Regaine. This originally was under clinical trial as an blood-pressure reducing medicine in the 1980's when many of the subjects started reporting hair re-growth.

Same with Aspartame, the artificial sweetener used in all diet drinks. Again, started life as an anti-high blood pressure medicine. One of the chemists tasted it and noticed it tasted sweet. CEO of the pharma company R&Ding it at the time was one Donald Rumsfeld who used his Republican connection with Ronald Regan to have it fast tracked through the FDA, despite evidence of it being a carcinogenic to mice to large doses.

DublinWriter

Genghiz Cohen wrote: »

I'm not sure it's fair to say that until you know everything about the drug or you have widespread use of the drug.

Damn sure, thalidomide is a great example of this.

recedite

Genghiz Cohen wrote: »

What say ye, wise sages of A&A?

I say this girlfriend better be very sexy if you have to spend the evening listening to her rabbiting on about the wonders of homeopathy

robindch

DublinWriter wrote: »

thalidomide is a great example of this.

Drug approval procedures prior to thalidomide were patchy at most, and to the best of my knowledge, most of the damage was done by doctors handing out the drug without any testing at all having taken place. Basically, "Here, try this. Tell me if it works".

Thalidomide has since been approved for certain cancers and , afair, leprosy.

Coriolanus

You'd be surprised how much modern medicine is ran on the same principle. Off the top of my head, lithium is one of them.

bnt

Getting back to the OP's girlfriend's point: if we compare aspirin to homeopathy, the first and primary difference is that aspirin was found to work in both informal and formal trials. Every attempt at a formal trial of homeopathy has failed to demonstrate any meaningful effect.

"It Works" is the first hurdle, and homeopathy fell at that one. Talk of "how it works", such as the "memory of water" is moot. That doesn't rule out any future developments, of course, but if it doesn't seem to do anything, no researcher is going to spend time and money on it. I know a lady who's in to all that stuff, and can't help noticing that the word "Dr." on a label is the most important thing she looks for.

Genghiz Cohen

recedite wrote: »

I say this girlfriend better be very sexy if you have to spend the evening listening to her rabbiting on about the wonders of homeopathy

While yes, she is hot, she and I hold very similar beliefs.
Homoeopathy was picked because neither of us believe it works. I know it may have been a bad choice because it's in no way likely that it will ever be shown to work, so yeah...

recedite

Genghiz Cohen wrote: »

While yes, she is hot, she and I hold very similar beliefs.

See what he wrote there Girlfriend, I hope you're happy now

Yeah Genghiz, you probably should have said "a new treatment" instead of mentioning homeopathy... it was bound to throw people off course a bit, but anyway, good luck.

sbsquarepants

I think the gravity example is a perfect one.
Throughout history, the pattern has to been to discover "that" something happens and only after further research do we understand the "how" or the "why".
Even the most ancient and scientificaly ignorant civilisations recognised the tides ebbed and flowed in time with the phases of the moon, it was centuries before the reason why became apparent, and even now it's only in a vague sense we "understand" We know it's the moons gravity, but we've no idea what that is, just how it works.
Is that reason enough to say we don't know what controls the tides? Of course not!

aidan24326

Genghiz Cohen wrote: »

I say no, until you understand how the remedy works, you cannot say that it was the cause of the healings. Girlfriend disagrees.

Girlfriend doesn't understand that correlation doesn't imply causation until you understand the underlying mechanisms. And with homeopathy that's irrelevant anyway since it's been shown countless times to be no more effective than placebo.

The reason homeopathy became popular in the first place was, oddly enough, because it did pretty much nothing ie some people would get better (placebo effect) and the rest would neither get better nor worse. And because mainstream medicine at the time was outright quackery by today's standards, a doctor was often more likely to make you worse than make you better. So back then you were nearly better off with homeopathy.

Genghiz Cohen

aidan24326 wrote: »

Girlfriend doesn't understand that correlation doesn't imply causation until you understand the underlying mechanisms.

she does, but thinks that if there is something about the treatment that works then that's good enough.

Sarky

Once you understand how a treatment works, you can work on making it work better.
Penicillium fungi may have antibiotic properties, but it's far more effective to isolate the particular compound that has those properties and work out how to mass produce it than rubbing a load of fungus on a wound...

Then there's the question of bacteria evolving resistance to something. If you don't know how an antibiotic works, how are you going to change it to remain effective against a new generation of microbes?

It's handy to have something that works, but it's much more useful to have something that works AND you understand how it works.

JimiTime

Genghiz Cohen wrote: »

An interesting debate I had with Girlfriend over an Eddy Rockets lunch.

Is it enough to say, I don't understand how but X seems to work 90% of the time, therefore X works.
Or should you have to figure out how X works before you can make such a claim. (Then, obviously, grab a compass).

The idea sprung from Homoeopathy. If someone could show that under controlled circumstances (No other treatments in that time and such), 1000 people took a homoeopathic remedy and they were without a doubt cured during the time they were part of the study, is that enough to say that this particular treatment works and start getting doctors to prescribe it?

I say no, until you understand how the remedy works, you cannot say that it was the cause of the healings. Girlfriend disagrees.

What say ye, wise sages of A&A?

i know you said 'what say yee A&A', but I think the following is relevant.

Only yesterday I was looking at how to make penicillin at home. In the course of my reading, I came accross articles talking about how penicillin was being used by Native Americans many moons ago to prevent infection in wounds. Now, they were simply using ALL of the mold rather than extracting the active ingredient, but it worked for them. A similar story in relation to Aspirin, and the white willow bark etc.

pH

Just remembered about this thread, one of the best books on this subject is available for free in pdf form - or available from amazon as a real book or for kindle.

http://www.testingtreatments.org/new-edition/

Peregrinus

Obviously it's a good thing, for a number of reasons, to understand how something works. And sometimes understanding how something works may increase your confidence that it does indeed work, and work reliably.

But it is often possible to observe that it works, and works with a useful degree of reliability, and to act on that information in real life, without understanding how it works. I doubt if one mobile phone user in a thousand, for example, could explain to you how it does all the things it does, but that doesn't prevent him from knowing that it does them.

ShooterSF

Peregrinus wrote: »

Obviously it's a good thing, for a number of reasons, to understand how something works. And sometimes understanding how something works may increase your confidence that it does indeed work, and work reliably.

But it is often possible to observe that it works, and works with a useful degree of reliability, and to act on that information in real life, without understanding how it works. I doubt if one mobile phone user in a thousand, for example, could explain to you how it does all the things it does, but that doesn't prevent him from knowing that it does them.

That's a bit different though. The info is out there if someone wants to find it. There is no mystery in the mobile phone to mankind. That's a lot different than say if homoeopathy worked for example.

Peregrinus

ShooterSF wrote: »

That's a bit different though. The info is out there if someone wants to find it. There is no mystery in the mobile phone to mankind. That's a lot different than say if homoeopathy worked for example.

OK, that suggests a modified theory; we can only know that something works if we are confident that someone - not necessarily us - knows how it works.

That's probably closer to the mark, but it does give rise to two questions in my mind.

First, what if I'm confident that someone knows how, e.g, homeopathy works, but my confidence is misplaced?

Secondly, there could still be a class of phenomona where, in truth, nobody really knows how they work; all we have is competing more-or-less plausible theories which have not been either refuted or confirmed. And yet they still work. The placebo effect might be an example.

A further refinement of the rule would be something like; we can only know that something works if we are confident that someone knows how it works or, at least, that how it works is in principle knowable.

The problem with this, though, is that my confidence that, e.g, the placebo effect is capable of being thoroughly understood is itself an act of faith, or possibly an act of hope.

Latatian

Here's a question: was opium (/digitalis etc) not a working drug before we had an idea of how it works?

Sarky

Still did interesting things to your brain. It's good to know what interesting things it does though. Makes recovery from addiction a bit easier. And allowed the mass production of other, sometimes synthetic, opiates. All I know is that morphine is a bloody miracle if you've just had surgery, let me tell you.

Peregrinus

Latatian wrote: »

Here's a question: was opium (/digitalis etc) not a working drug before we had an idea of how it works?

I suspect most, or at least most common, working drugs were working drugs before we came to understand how they worked. We observed that they worked, and made use of that knowledge, and then - sometimes much later - we investigated why.

Liamario

The 1000 people got better while taking the treatment, does not necessarily mean that 1000 people got better as a result of the treatment.
Therefore, it would be foolish to assume the treatment made the 1000 people better.

On saying all that, it would be reasonable for someone to take the treatment in the hope that it works for them.