James Randi on audio equipment

Apples and oranges. Wouldn't you think DBT could aid system matching?
WVK

 

Did you miss the whole discussion earlier WVK? Randi doesn't have the money in a 3rd party escrow so he is highly incented not to pay it.

 

Harmonic Technology CyberLight would be an example.

http://www.stereophile.com/cables/805harm/index3.html

"Following the measurements, I spent some time auditioning the CyberLight cables both in my system and in Michael Fremer's. Even with me knowing how they measured, the cables surprised me with the general acceptability of their sound. There was a coherence to the stereo image, a nice three-dimensionality to the sonic objects within the soundstage, and a vivid overall presentation. Against those, there was a "hummy" quality to the sound of bass guitar, with the tonal emphasis shifted away from the fundamental to the harmonics, and closely miked voices, such as Willie Nelson's on "Stardust," took on a bit of a bark. Dynamics seemed exaggerated, with climaxes sounding louder than I was expecting. In the long term, I found the CyberLight's presentation rather relentless.

I think that what the listener perceives with this cable is that at low levels, the sound is fattened and made more coherent-sounding by the dominant second-harmonic distortion. In addition, the presence of background noise cannot be dismissed, as there is some evidence that introducing small amounts of random noise results in a sound that is preferred by listeners. At higher signal levels, transients are accompanied by bursts of higher harmonics. However, these subside as quickly as they appeared. The overall effect is to render the system sound as being more vivid, I believe. However, the inevitable intermodulation products that are generated by the cable's bent transfer function leads, I conjecture, to the relentless quality I noted in my own auditioning.

If this review were of a conventional product, I would dismiss it as being broken. Ultimately, no matter what someone might think of its sound—and Michael Fremer is one the most skilled listeners I know of—I really don't see how the CyberLight P2A and Wave cables can be recommended. I am puzzled that Harmonic Technology, which makes good-sounding, reasonably priced conventional cables, would risk their reputation with something as technically flawed as the CyberLight.—John Atkinson"

I just feel like repeating that Stereophile (and others) take a lot of flack but don't get much credit. It seems hypocritical to hold Stereophile to the loftiest standards of fairness and balance that those toughest critics of the magazine don't abide to themselves when judging the magazine.

end of editorial-

 

And yet there is no audiophile or even manufacturer of these fine tweaks, anywhere in the world, that could use the good press of shaming the great and famous Randi by forcing him to welch on his bet. You'd end up on the cover of every audiophile magazine and, if you are a manufacturer, your sales would go up a thousandfold. Yeah, there's nobody in the world who could use that kind of publicity. What a joke it is to think that nobody would take the opportunity to win this bet.

FYI, the money isn't Randi's. He put up ten grand years ago, but this million is from a private individual who was tired of watching BS artists. Randi is the third party, it isn't his money.

I guess since there's no valid argument that the JREF protocols are unfair, now everybody just wants to say "well, he'd never really pay it anyway". Prove it! Shame the skeptics!

 

One wonders what hi-end audio would look like today if J. Gordon Holt, "in whose ears we trust", had retained control of Stereophile.

Read this from 1982.

P.

 

Very interesting. Two things jump out at me though.
http://www.randi.org/research/index.html
"At JREF, we offer a one-million-dollar prize to anyone who can show, under proper observing conditions, evidence of any paranormal, supernatural, or occult power or event."
If any of the items being challenged by Randi in audio work it is not due to any paranormal event. No paranormal claims ar being made by the manufacturers.
The other thing that jmps out at me is there is no description of any proposed tests. There in lies the rub. What are the chances of the parties agreeing upon a fair test when a million dollars is on the line. Did i miss something? Did Randi already propose a test with details of it's design and names of the parties that would proctor the test? The devil is in the details.

 

James Randi has specifically allowed a variety of audiophile tweaks to be eligible for the contest. See one such affirmation at http://www.randi.org/jr/082704gluton.html

The protocol of the specific test is negotiated between the applicant and the JREF. No two claims are alike so the details vary. These protocol negotiations are publicly documented at the JREF forum and they have shown themselves to be very reasonable. There are a variety of groups around the country that can administer the first round of testing (reducing the burden of travel expenses), and if you pass, you can be tested by the JREF for the final round and collect your million dollars.

 

Originally Posted by LeeS
Quote: Did you miss the whole discussion earlier WVK? Randi doesn't have the money in a 3rd party escrow so he is highly incented not to pay it.

Excellent point, Michael.

 

"James Randi has specifically allowed a variety of audiophile tweaks to be eligible for the contest. See one such affirmation at http://www.randi.org/jr/082704gluton.html"

Could you post the quote? i didn't find anything about this particular challenge with your link.

"The protocol of the specific test is negotiated between the applicant and the JREF. No two claims are alike so the details vary. These protocol negotiations are publicly documented at the JREF forum and they have shown themselves to be very reasonable."


"Reasonable" is a bit subjective although I have no doubt they are quite reasonable with claims of paranormal since there will always be a real world cause and effect involved. OTOH with claims that do not involve paranormal or do not violate the known laws of physics there is a chance that they will lose. Can you cite any examples of their dealings with these sorts of challenges? After all, isn't the claim of the Shakti stones that they block EMI? Seems to me if you find a component that is audibly snsitive to EMI (hardly impossible) and the stones really help block EMI (not entirely impossible either) there is a substantial risk that one could easily set up a test where an audible diffence could be detected. Randi seems to be basing his challenge based on opinions of friends that frankly have some dubious opinions.



" There are a variety of groups around the country that can administer the first round of testing (reducing the burden of travel expenses), and if you pass, you can be tested by the JREF for the final round and collect your million dollars."


So we have no idea how reasonable the JREF will be should one pass the first round. It really is a bit of a shell game. it is easy to appear reasonable when you know you are going to win. When there is a doubt the true nature of the participants will come out.
The problem still remains, the stakes are too high. I strongly suspect the underlying goal of the JREF is more to not pay the bounty than it is to find the truth.

 

"I assured Mike that such a device certainly comes under the JREF Challenge, then I sent this message to the Bedini people:"

Such statements go right along with his explicit challenge to the Shatki and Electret Foil people. This flies in the face of your claim that tweaks are ineligible because they aren't supernatural.

There are a variety of protocol negotiations over at the forum. Here is one for the golden sound chip: http://forums.randi.org/showthread.php?t=36074 To read in in the full context, you'll need to read the discussion at http://forums.randi.org/showthread.php?t=36079 (edit: and here http://forums.randi.org/showthread.php?t=38604)

Perhaps you can tell us what you find unreasonable about the kinds of protocol controls that the JREF requires.

Go for it!

If anybody can ever even pass the first round, I suppose we might find out. The purpose of the final round is to make sure that the proper controls and conditions are being observered, and to make sure that the results are held to a higher statistical standard than the first round (where there's a chance within thousands that you might have gotten lucky and just guessed it, and only a fool would give away a million bucks on such easy odds...eventually somebody will luck out). This would typically mean a higher number of trials to ensure that luck isn't the reason that the first round was passed.

It would be very easy for you to spend a few minutes writing a proposal with a protocol, and you'll be on your way to proving it and shaming the skeptical community. Share your experience with us! Be warned that they'll ask if you've at least done a simple blind test on your own first, something that very few applicants are even willing to do, for some reason.

 

Based on what I've seen in this thread, it seems the negotiations for a challenge are made in public. If they collapse, the facts will be known and we can decide for ourselves the legitmacy of each side's presentation. I'm in favor of blind evaluation but I can imagine a circumstance where JREF might insist on a restrictive protocol that I wouldn't feel was valid.

So I don't understand what someone accepting the challenge has to lose. Worst case is that negotiations break down and each side claims victory. With a million bucks and a lot of prestige to be had, what is the reasonable argument against beginning the negotiating process? It seems a no-lose situation for a reviewer secure in his or her listening skills or a manufacturer confident in the sonic benefit of a product.

IMO, the only legitimate restrictions JREF could impose on the protocol are those that insure that the comparisons are truly double blind and no fraud takes place. The person accepting the challenge should be free to chose, within reason, the ancillary equipment, listening location, length of trials, etc. that he or she feels best allows for differentation between components.

P.

 

"I assured Mike that such a device certainly comes under the JREF Challenge, then I sent this message to the Bedini people:"

Thank you.

"Such statements go right along with his explicit challenge to the Shatki and Electret Foil people. This flies in the face of your claim that tweaks are ineligible because they aren't supernatural."

It's not my claim. i quoted your link to the JREF. If they are making an exception then fine but tif they are then it's new ground and the risks are now real that someone might be able to prove something. so all behavior based on prior challenges are old news.


"There are a variety of protocol negotiations over at the forum. Here is one for the golden sound chip: http://forums.randi.org/showthread.php?t=36074 To read in in the full context, you'll need to read the discussion at http://forums.randi.org/showthread.php?t=36079 (edit: and here http://forums.randi.org/showthread.php?t=38604)"


Well that sure broke down with little effort. It wasn't even a matter of reason as much a matter of patience really.

"Perhaps you can tell us what you find unreasonable about the kinds of protocol controls that the JREF requires."


I don't know that they are really so much unreasonable as simply poorly designed. At least that one proposed test.



Quote:
After all, isn't the claim of the Shakti stones that they block EMI? Seems to me if you find a component that is audibly snsitive to EMI (hardly impossible) and the stones really help block EMI (not entirely impossible either) there is a substantial risk that one could easily set up a test where an audible diffence could be detected. Randi seems to be basing his challenge based on opinions of friends that frankly have some dubious opinions.




"Go for it!"


Why? i don't believe that the JREF is going to pay me a million dollars to prove something like EMI can affect the sound of playback.


Quote:
So we have no idea how reasonable the JREF will be should one pass the first round. It really is a bit of a shell game. it is easy to appear reasonable when you know you are going to win. When there is a doubt the true nature of the participants will come out.



"If anybody can ever even pass the first round, I suppose we might find out."


Yes, that is when the true colors will show.


" The purpose of the final round is to make sure that the proper controls and conditions are being observered, and to make sure that the results are held to a higher statistical standard than the first round"


Really? You mean they don't use the standard accepted by science of 95%?


" (where there's a chance within thousands that you might have gotten lucky and just guessed it, and only a fool would give away a million bucks on such easy odds...eventually somebody will luck out). This would typically mean a higher number of trials to ensure that luck isn't the reason that the first round was passed."


I see. That would make the challenge a sham unfortunately.



Quote:
The problem still remains, the stakes are too high. I strongly suspect the underlying goal of the JREF is more to not pay the bounty than it is to find the truth.



"It would be very easy for you to spend a few minutes writing a proposal with a protocol,"

No it wouldn't. I don't know nearly enough about EMI.

" and you'll be on your way to proving it and shaming the skeptical community."


I consider myself to be a skeptic. Unfortuantely far to many skeptics shame themselves with their attitudes.


" Share your experience with us!"


I have no experience with Shakti stones. My expereince with a wide array of Belt products yielded no discrenable difeences IME.


" Be warned that they'll ask if you've at least done a simple blind test on your own first, something that very few applicants are even willing to do, for some reason."


Thanks for the warning.

 

I followed the whole discussion in realtime, as it occurred. They were far more patient than I ever would have been. The applicant flaked out, plain and simple.

But can you find anything unreasonable in the protocol details and controls that are unreasonable, and make their kind of tests impossible to win?

I also don't buy any argument that the testing of such tweaks automatically requires substantially different protocols than 'supernatural' claims. Take dowsing for an example. Find the water enough times in a blind test to statistically prove that you can douse. This is no different than blindly picking a 'treated' disc or interconnect. Check out their other protocols for claims such as dousing, they don't treat tweaks and treatments any differently.


Look, I'm not going to keep going back and forth with you, point by point. The bottom line is that you say they will not allow anyone to win and that their goal is not the truth, it's about making it impossible to beat them.

Neither I or anybody else can prove a negative, so it is impossible to prove that you are wrong.

You, however, can very easily prove that you are right. Choose a qualifying tweak that you can hear, when applied, and submit yourself as an applicant for the challenge. Show us how they are unfair. If you can't hear any of the qualifying tweaks, find a friend that does!

 

James Randi is clearly not in a good position to be an independent observer if he is the man trying to debunk high end audio. It's just common sense that he will be biased against any results in favor of high end audio.

 

Two quick observations:

1). There seems to be an almost religious belief among some of the posters in the infallability of DBT for audio components.

2). In previous discussions with the proponents of DBTesting on other venues, I've asked such proponents for scientific proof that DBT do in fact show up small scale audio changes such as the ones under discussion. Logically, for a test to be "scientific" (as DBT proponents claim) then the proponents of the test should be able to prove so. In these discussions all I received back was waffle on how DBTesting is used in medicle trials, etc, not with audio.So to turnt this around to the proponents of DBTesting: no proof = not scientific.

 

From what I gather, none of these publicly staged double blind tests have ever made the news as having shown that there is statistically significant a difference between pieces of equipment regarding perception by the listeners of sound quality, whether we are talking about amplifiers, wire, or digital players. They have always involved multiple participants picked at random from the general population. As others have said, what the double-blind tests really show, is which participants out of the group have the ability to discriminate the differences under test. Some of these tests have shown maybe one or two participants out of ten on average can reliably discriminate. But this is considered insignificant according to statistical theory if the focus of the test is considered to be the equipment rather than the listeners.

Maybe different uses of this test have been made - I don't really know. But I would say that if you just took one person as the listener, who could hear a difference repeatedly without error, then you would have proved something, but this is a different approach to anything I've heard of. Does anyone else know of an example like this?

The reason I raise the question is because I can hear obvious differences in mastering between issues and reissues of various recordings I have. Most forum members presumably have had that same experience many times over, else why hang out here? Yet I have played different masterings on my home system for comparison to friends, and often they don't notice the differences. I know I am not imagining it. You know you're not. We frequently have MP3 posts of sample recording here and you can even hear differences through the compression. What does that tell you?

I think Bob Carver once said that all amplifiers will tend to sound the same under certain conditions of drive and load. But he also said that once you push beyond those restrictions, as you inevitably will in a real world listening environment with particular source material, volume levels, and complex impedances presented by speaker loads, you will get audible performance differences. I realize that all these limits can be pushed and tested under the double-blind test if it is set up properly, but I've never heard of this being pursued rigorously and to the end. Patience runs out and they always seem to be done under duress with agendas and grinding axes. In principle I agree that one should be able to repeat one's perceptions accurately without peeking at the labels, so to speak.

 

No, I stick a component in my system and If it sounds better than what I already have then, I consider it an upgrade. There is no need to go to the trouble of blind listening tests. I have a better test. Take two components and play them in your system for several weeks and the component that you end going back to or prefer is the better component for your system.

 

There seems to be an almost religious belief among many audiophiles that blind testing is inherently flawed.

When do these products ever claim slight or subtle changes? Is your claim that tweak enhancements are so subtle that one cannot detect them with ones own hearing during a blind test, while a vodka taster can tell a vodka that has been charcoal filtered five times versus the one that is filtered six times?

Then this is a perfect opportunity for audiophiles to silence the skeptical community! Show the skeptics for the sham that they are! It would be so easy to prove that these tests are not objective or independent! Why hasn't anybody done it? Again, there is a huge PR and revenue (likely a lawsuit to boot!) opportunity here for many!

Again, once the protocol is locked in there is really no wiggle room for either party. You perform and win, or fail and lose. Heck, Kramer (enginer/producer/performer for Bongwater, King Missile, Galaxie 500, John Spenser, etc.) used to manage the application process, including the negotation of the protocols. If you say that Randi is being unfair, you need to point out what in the JREF's proposed protocols are unfair.

 

I agree with you. The test should be correctly set up to measure what needs to be measured. We can usually detect differences between recordings because we are used to the sound of our equipment, our room's acoustic and the albums we listen to. This, as far as I know, is another one of the variables that is never maintained when these tests are done. If the listener is not acquainted with the equipment, the room's acoustic and the record being played and is also in a stressful situation whereby he/she feels under pressure to demonstrate their capacity to tell between recordings all of these are the monkey wrench anybody in their sane mind would think of in the face of accepting such a challenge, especially if it is from someone known to hold - and vigorously defend - the opposite view. (Just my 2 cents)

 

This is most amusing. I just found out that Kramer, the producer, engineer, musician, and skeptic who has been beating up on those poor old audiophiles as an employee of James Randi for the last few years, has returned to the music business and is focusing on CD mastering among other things.

From what I know of Kramer, I'm sure he is just as skeptical as ever.

 

Your argument is illogical. No one's done a double-blind taste test on the difference between untreated sewage and fresh milk that I know of, either. Does it therefore follow that DBT would be unable to determine a taste difference between the poison and the beverage?

That DBT is standard practice across commercial fields and institutional disciplines says a lot more about DBT than its conspicuous absence in the realm of audiophile reviews.

 

There has been an enormous amount of DBT work done in areas such as MPEG audio encoding and speech codecs for digital cellular telephony. In these cases, there are defined reference signals and researchers compete (and sometimes collaborate after initial trials) to develop encoders that reduce the data volume/rate while being as imperceptible from the reference as possible. Not all of the results of these trials make it into the public literature. For the MPEG trials, I know the ITU-R B.S. 1116 recommendation was the basis for the tests. If you Google 'itu-r b.s. 1116 mpeg' you'll find some results form the BBC and NHK on MPEG audio codecs.

 

It depends on what the test is out to show. Let's take the example of Verance again. The developers claimed it was 'inaudible'. For certain journalists, like England's Barry Fox, that was a red rag to a bull. Listening tests were demanded, and some were held at a major recording studio in New York. I know of two staff engineers who could identify the watermark every time in the tests. Clearly, Verance is NOT inaudible. Do you need to sample millions of the population to confirm this? No, since your critical listeners have already debunked the theory.

Now, on the other hand, someone might claim that their product gives an X% improvement, and anyone can hear it. Here you would clearly need to draw on a larger cross-section of society to prove/disprove the claim if your expert listeners 'confirm' the claim.

In audibility tests, one usually starts with expert listeners since if they can't hear the improvement/effect/whatever, the chances are that casual listeners will not hear it either. This simply makes testing more economical (in both time and money).

See above!

People hear differently. They also see differently. One person focusses more on one thing, another on something else. The DBT reveals who can hear what.

The DBT is geat for removing many biases: cost, size, build-quality, etc. that often influence a consumer's perception of a product. If, in a DBT, you picked Bud every time, then it really would be the King of Beers for you!

 

What is so illogical for wanting a test, said by some to be scientific, to in fact be based in science? As i said, I've asked other proponents of DBTesting about this, and their responces haven't answered the question either.

 

"I followed the whole discussion in realtime, as it occurred. They were far more patient than I ever would have been."

Then perhaps you have even less patience but when they started arguing over whether they ere going to do the priliminary test in June or in August it was obviously a breakdown in patience. If I were involved and the folks at JREF started getting pissy about my schedule I would walk away. The guy has other things going on in his life as do I.


" The applicant flaked out, plain and simple."

Another opinion you get to have. But like I said, if I were being badgered about scheduling I guess I would be tagged a flake as well. Good to know. I think stay away is the obvious way to deal with the JREF and it's fans.



"But can you find anything unreasonable in the protocol details and controls that are unreasonable, and make their kind of tests impossible to win?"


I already answered this question but I will happily annswer it again. I don't see the test that was proposed as unreasonable but rather poorly designed. Poor design does not insure eroneous results but does add to the likelyhood. I think they could have designed a simpler test that would have been both better and more comfortable for the applicant.



"I also don't buy any argument that the testing of such tweaks automatically requires substantially different protocols than 'supernatural' claims."


I have made no such argument. So ther is nothing for you not to buy there.


" Take dowsing for an example. Find the water enough times in a blind test to statistically prove that you can douse. This is no different than blindly picking a 'treated' disc or interconnect."


i don't know what dowsing is but I do know that as audible differences aproach the threshold of human hearing the results get worse. If the effect is real but barely audible any deviation form the accepted 95% probability is a joke. It could amount to the rejection of a positive result. If the JREF people are willing to stack the odds in their favor in this way for the sake of preventing an extreme fluke they aren't playing fair. Plain and simple.



" Check out their other protocols for claims such as dousing, they don't treat tweaks and treatments any differently."

Why? was there something at of the ordinary with the protocols with the challenge we have been discussing?




"Look, I'm not going to keep going back and forth with you, point by point."


You are losing your patience already?


" The bottom line is that you say they will not allow anyone to win and that their goal is not the truth, it's about making it impossible to beat them."

Yes that is what I believe as long as they have a million dollars at stake. And I think they will continue to look very reasonable so long as they are taking on claims of the paranormal. I also think that if Rani continues to make challenges against things that may be possible based on dubious opinions of his pals at JREF the more the likelyhood that they will have to drop the facade of resaonability and resort to playng dirty. This is my opinion and much of it is total speculation. I don't represent it as anything else. But this I will say about the folk at the JREF and their fans. IMO many of them are not interested in investigating claims as much as they are interested in attacking people who make such claims. IMO many of these peopela re not merely skeptics but have a vested emotional inteest in tearing down other peopels beliefs to feel superior. as long as the desire is to make people wrong rather than to investigate out of a pure desire to know better, I will remain skeptical of a certain breed of skeptics.



"Neither I or anybody else can prove a negative, so it is impossible to prove that you are wrong."

It's not about proving a negative. By the way many negatives are quite provable. I make no misrepresentations about the nature of my opinions of these challenges. At this point it is impossible to prove my opinion either way. It is very speculative in nature.

"You, however, can very easily prove that you are right."


No I can't for reasons that I have already explained.


" Choose a qualifying tweak that you can hear, when applied, and submit yourself as an applicant for the challenge."


If you want to send me some Shakti stones to audition we can go from there. But I may not hear a difference with them. Then what?


"Show us how they are unfair."


Well if they are changing the rules of statistical analysis as I thnk you have claimed to avoid extreme flukes then you have already done that for me.


" If you can't hear any of the qualifying tweaks, find a friend that does!"


You seem to have me confused with someone with nothing better to do than to deal with a million dollar offer that I suspect to be...shall we say, well protected against being surrendered. No thanks. I have productive things to do in my life.