James Randi on audio equipment

My understanding is nobody has made it past the first (easier!) round anyway, so I'm not sure why it's an issue.

 

There is no deviation from "scientific standards." If I claim psychic powers and by sheer luck correctly guess the outcome of 16 of 20 coin flips, asking me replicate that effort in a second battery of ten sets of 20 coin flips doesn't change anything, except the possibility that I just got lucky.

 

You'll note that I said "defined for this purpose". This was specifically to include the "all everythings sound the same" morons who like to argue in audio discussions, even thought they really don't like audio.

 

Keep putting the boot in like this and the thread will get locked again.

Not that I care.

 

I like this example.

 

Fine with me. But if someone's going to claim that some tweak or cable that -- despite the pseudo-scientific jargon in the full-page ad -- adheres to no known acoustic or electrical principal improves the reference system's sound "as if I'd upgraded every component several times over," then, yes, I'd like to see the writer test his or her own observations with some blind testing.

I'm willing to believe that ebony hockey pucks, porcelain cable lifters, and exotic and temperamental circuit designs that measure poorly can actually make a system sound better, or at least different. As opponents of single- and double-blind testing complain, We can't measure everything.

But we can measure whether we (or the reviewer) can hear such changes, regardless of its rationale. That people would refuse to entertain such a test, and even badmouth the basic research principles that allow them to enjoy the fruits of technology in the first place, says more about our hobby than I wish to know.

 

If there is no deviation from scientific standards then they must be accepting 95% probability as a positive. I'm pretty sure your example 16 out of 20 meets that standard. If they are worried about someone getting lucky they made a bad bet with a million dollars. But then I have been saying that all along haven't I?

 

"Scientific standard" would include additional testing to eliminate blind luck. Or are you willing to believe that everyone who wins the lottery is a psychic?

 

Bingo.

 

Originally Posted by Scott Wheeler
I was thinking about going to the Munch Box today and get a hickory burger. It is my favorite burger in the world. can I have that opinion without having done bias controlled taste tests?

So you are OK with bias effects affecting opinions about food but not about cables? Why do you discriminate when it comes to your concern about bias effects? indeed we can put any critic to the test under blind conditions if we want to be they a food critic, wine critic, audio reviewer etc. and for those people who wish to do this knock yourself out. But the fact is the vast majority of subjective opinions about anything, food, audio, video, wine, perfume etc are formed under sighted conditions. Either you can live with the inherent uncertainty of that reality or not. I can. i take opinions for what they are worth. If anyone cant live with it I feel for them. What a burden it must be to have to verify every subjecrtive impression with bias controled tests. I don't believe anybody does that. So why pick and choose? OTOH should any audio reviewer, food critic or anything of the like claim that their sighted evaluations are scientifically valid i think it is fair to demand they prove it with bias controled tests.

 

I agree with the gist of your argument here. However if you are going to invest in a piece of audio gear costing thousands of dollars because it is state of the art and you wish to happy with with it long term, it might call for some rigorous listening tests and comparisons beforehand. I can see the value of blind testing in a situation like this just to satisfy yourself about the consistency of your own listening impressions before you spend BIG bucks. Then again, some wine is very expensive, too. But too much testing is impractical for wine and food because it gets consumed in the process!

 

No, science certainly allows for follow up trials but it does not demand it.
Although if there were so few trials like 20 and the results were borderline like 16 correct it would be prudent to repeat the test. It would likely depend on what is being tested and how difficult it is to conduct the tests. Not all bias controlled tests are about audibility.

I am not willing to believe everyone or anyone who wins the lottery is psychic because there is no reason to. I know of no lottery wins that go beyond the predictions dictated by statistics. OTOH if a psychic steps to the plate and predicts the winning numbers under controlled conditions to prevent cheating I would have to concede the point. I don't think it will ever happen though.

 

I think you make a fair point here. But shouldn't this apply to speakers? Yet how many people who demand proof of opinions about cables ever do bias controlled comparisons with the one thing they think really does make a difference, the speakers? Ironically I actually did insist on doing blind comparisons between two speaker systems i was considering for purchase many years ago. Thankfully th guy at that high end dealership was a jovial person and the speakers were not that heavy. Yeah, I think it is prudent to be very careful when the price is meaningful. But in the end we listen to our systems under sighted conditions. May as well incorperate the bias effects in the audition period since we will be living with the bias effects after the purchase.

 

Ron, I think the issue with that reasoning is that audio phenomena are not always able to be tied back to known scientific principle. Sometimes we know things sound better that way and later on we get more scientific grounding. That is what happened with jitter.

 

I like to borrow a piece of new gear with my credit card on deposit and test it over the period of a weekend. It's very hard to make sonic mistakes that way.

 

Lee,

If a dealer will agree to an arrangement like that with return priveleges, that is ideal. I've been turned down on that kind of thing in the past. I once tried to arrange an in-home audition of Spica Angelus speakers back in the 80s in northern California - no dice. Too bad, because I probably would have bought those speakers. They had awesome soundstage depth and layering capabilities. Instead, I hung onto my Dahlquist DQ-10s awhile longer and moved on to consider something else later after Spica went out of business. They were the perfect speakers for my family too at that time when my daughter was an aggressive little toddler. They just couldn't be knocked over. Their shape made them super-stable.

Well maybe store policies have changed since then, what with the high-end industry living through more difficult times.

 

What does that have to do with anything? The point being made here is very simple. A manufacturer of some product makes a claim about the effects of said product. There is no science to back up the claim, but that does not stop the manufacturer from making the claim. Now, it is only reasonable to assume that the manufacturer (at the very least) can prove the claim via a comparative test. The DBT, by removing any grounds for bias, will show, in its simplest form, whether there is a perceptible difference (be this audio, taste, smell, whatever). We don't need to know whether there is some scientific principle invovled.

 

Because if I think you're wrong about your favorite burger, I'm out $5.00. So what? But a pair of $1500 cables or $12,000 monoblocks raise the bar on my skepticism.

Incidentally, lots of food and beverages are blind-tested. That's how almost every recipe for almost every chain restaurant and shelved grocery item is established; with big money on the line, commercial interests do consumer research.

Often you can participate in a double-blind taste-test (and pick up $20 for your trouble) at the nearest mall. It's pretty tedious, though, and because it's double-blind the tester can't tell you what cola/frozen pizza/hot chocolate you actually preferred. It's just A, B, C, or D.

 

IIRC, he said the same thing about the WAVAC amplifiers, the Tetra 505 Limited Loudspeakers, and a set of Antique Sound Labs SETs...

 

What does this have to do with being able to reliably distinguish an item under test in a DBT?

 

I don't know about that, but you certainly confused me!

I know, I've been re-listening to my dEUS discs thanks to one of the threads in the music forum.

Well, I'm not losing any sleep over this thread, and I'm quite happy with my SS system.

 

Another reason for me to hang out at funeral parlors.

 

OK so you discriminate based on price. I think that is fair. hv you ever demanded bias controled tests to varify cheap tweaks like green markers for CDs? Not much more than a burger really. OTOH did you do bias controled tests in your auditions of speakers? They do tend to be big ticket items and our impressions of speakes have been demonstrated to be very affected by sighted bias? I mean if it is just about the price ticket I would hink you would be more worried about bias controled reviews of speakers than cheap tweaks. is that the case?

I'm not sure i would hold up the recipes of chain restaurants or brand named goods at the grocery store as any kind standard of excellence or as convincing support for the use of DBTing of food. I think you will find the producers of food products with the highest reputations for quality do their testing sighted. I actually have some experience in this regard as the former owner of a high end chocolate store. Blind taste tests are frought with their own particular baggage and can be very tricky to do with any kind of meanngful results.