Home Critic's Corner

Discuss a review. Provide constructive feedback. Talk to the industry.

[ Follow Ups ] Thread: [ Display  All  Email ] [ Critic's Corner ]
[ Alert Moderator ]

Re: John Atkinson's Rebuttal of Ben Goldacre's "Bad Science"

Your "Ben" says:

--

I give you the editor of Stereophile, a respected hi-fi magazine of 33 years standing. He's talking about blinded tests on amplifiers: "It seems," he says, "that with such blind listening tests, all perceived subjective differences ... fall away ... when you have taken part in a number of these blind tests and experienced how two amplifiers you know from personal experience to sound extremely different can still fail to be identified under blind conditions ..." Now I'm getting worried. Here comes the money shot. "... then perhaps an alternative hypothesis is called for: that the very procedure of a blind listening test can conceal small but real subjective differences." Ouch. "Having taken part in quite a number of such blind tests, I have become convinced of the truth in this hypothesis." What voodoo is this? If there is a difference to be heard, then you will hear it.

--

See how easy it is? if there is a difference to be heard, then you will hear it, simple! Yet when we have reports of sonic differerences from publications that do blind tests that are intentionally designed to emulate normal listening conditions (at least normal for the listening sessions of the magazine reviewers I have in mind, UHF, and some english mags) the cries goes out ... Invalid!, Not double blind!, Insufficient controls! Much the same for amateur tests when the results are unpalatable (to hardcore objectivists).

So we turn to the 'rigorous' types of testing. Here Stereophile *has* demonstrated some of what "Ben" considers "extraordinary claims". They have also shown that, utilizing rigorous methodologies, doing such is no trivial task.

Practicality asside, you've pointed out such tests are designed to give a Yes/No to the question ... is there a audible difference? But I would maintain that such is of little interest to audiophiles, we're interested in what you call "real subjective differences", i.e. we want a description of the attributes of the sound.

You see we've long since passed the "is there a difference" question. The simple fact is, that as a tool for what we're interested in, rigorous blind testing methodologies offer little. The hardcore objectivists can scream "No No, don't go there!, there are no audible difference, you're all being conned" as much as they like but serious audiophiles merely consider them a nuisance ... the topic of why we're generally less than proficient at ignoring them is an interesting one, likewise the field of science that could help explain such, but perhaps another day.

--

Now back to "Ben", it is clear to any reasonable observer that he presented a entirely simplistic view ... make the test 'blind' and presto!, all problems solved. JA, basing his comments upon his direct and relevant experience in the matter of testing HiFi Audio components utilizing controlled blind conditions, pointed out how silly such a view is.

Fine, but in Mr. Goldacre's (i.e. "Ben") case the offence was worst than mere silliness specifically because he cloaking himself in the mantle of scientic respectibility. I think JA's articulated the offence clearly, "But Mr. Goldacre appears to be making the naïve assumption that the mere fact that a test is blind inherently—his word was intrinsically—confers legitimacy on the test and its results. That assumption, I suggest, is 'bad science'—even voodoo."

This post is made possible by the generous support of people like you and our sponsors:
  Atma-Sphere Music Systems, Inc.  

Follow Ups Full Thread
Follow Ups
You can not post to an archived thread.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2025, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]