If you did nothing and none raised their hands you would have some validation. - Norm

Follow Ups Full Thread

Topic - A/B testing methodology question - PhilNYC 07:46:50 08/20/06 (76)

Re: A/B testing methodology question - tianguis 09:57:54 08/26/06 (0)

Perfectly valid test to me. - hukkfinn 08:13:36 08/24/06 (0)

The trick is: - Presto 13:48:24 08/21/06 (0)

Do you have a polarity switch on your preamp? - Dave Pogue 10:01:12 08/21/06 (4)

Shhhh... you might attract the attention of that polarity psycho! :) [nt] - Steve Cortez 16:41:12 08/21/06 (2)

Excuse me! That's Polarity Psychoâ„¢,ok? We MUST stay abreast of such things...N/T - musetap 18:02:43 08/21/06 (0)

Lord, I hope not. (nt) - Dave Pogue 16:51:21 08/21/06 (0)

Re: Do you have a polarity switch on your preamp? - PhilNYC 10:15:25 08/21/06 (0)

A slight change in volume (SPL) will be audible ... but so what? - Richard BassNut Greene 08:58:32 08/21/06 (5)

Re: A slight change in volume (SPL) will be audible ... but so what? - PhilNYC 09:28:41 08/21/06 (4)

You didn't mention if the tweak caused a small change in volume (which could be audible) nt - Richard BassNut Greene 09:30:35 08/22/06 (3)

Re: You didn't mention if the tweak caused a small change in volume (which could be audible) nt - PhilNYC 12:01:08 08/22/06 (2)

A small change in level will often be mistaken for a change in quality - Analog Scott 12:05:11 08/22/06 (1)

Re: A small change in level will often be mistaken for a change in quality - PhilNYC 13:39:25 08/22/06 (0)

Thanks... - PhilNYC 08:43:42 08/21/06 (0)

You will NEVER EVER find a test that will satisfy your antagonists.... - clarkjohnsen 08:31:28 08/21/06 (9)

Re: You will NEVER EVER find a test that will satisfy your antagonists.... - Inmate51 09:11:02 08/21/06 (6)

"Clark is clearly being sarcastic." No, the word is "realistic". Those people will never be happy... - clarkjohnsen 09:13:23 08/21/06 (5)

I don't think you are being realistic at all. - Analog Scott 15:48:47 08/21/06 (4)

Re: I don't think you are being realistic at all. - PhilNYC 09:06:22 08/22/06 (1)

Re: I don't think you are being realistic at all. - Analog Scott 10:13:45 08/22/06 (0)

"His protocols were painfully poor for a meaningful blind test." See what I mean? - clarkjohnsen 08:43:30 08/22/06 (1)

Re: "His protocols were painfully poor for a meaningful blind test." See what I mean? - Analog Scott 10:21:53 08/22/06 (0)

I have no antagonists... ;-) - PhilNYC 08:57:34 08/21/06 (1)

Aw, c'mon... nt - clarkjohnsen 09:16:58 08/21/06 (0)

it's a pretty good test but it has a big problem - tunenut 18:51:53 08/20/06 (2)

Re: it's a pretty good test but it has a big problem - PhilNYC 19:51:34 08/20/06 (1)

Re: it's a pretty good test but it has a big problem - tunenut 21:02:50 08/20/06 (0)

Sounds like a fun session :) I'm awfully curious what the tweak was, I've tried something similar... - Steve Cortez 17:53:22 08/20/06 (0)

Re: A/B testing methodology question - Inmate51 12:58:21 08/20/06 (0)

Make lots of tests - Caymus 11:22:00 08/20/06 (0)

It was a great test, Phil; way to go. Obviously the fans of DBT don't accept it... - jeffreybehr 10:05:18 08/20/06 (3)

Re: It was a great test, Phil; way to go. Obviously the fans of DBT don't accept it... - Bob Wortman 18:15:03 08/20/06 (2)

I believe most of us would label the former as being OPEN-minded, not close-minded. - jeffreybehr 19:51:27 08/20/06 (1)

Arrogance is Them, alright. nt - clarkjohnsen 08:24:17 08/21/06 (0)

probably not because.. - Tom Tripp 09:22:12 08/20/06 (2)

Re: probably not because.. - PhilNYC 09:44:57 08/20/06 (1)

Re: probably not because.. - Tom Tripp 09:53:40 08/20/06 (0)

a better way, i think - pingong 09:14:20 08/20/06 (0)

Re: A/B testing methodology question - Pat D 09:09:16 08/20/06 (8)

Re: A/B testing methodology question - PhilNYC 09:33:03 08/20/06 (7)

Re: A/B testing methodology question - Analog Scott 11:42:53 08/20/06 (6)

Uhh...Scott...read it all. NO ONE knew WHAT he was going to do... - jeffreybehr 12:17:06 08/20/06 (5)

"Everybody was certainly aware that I was going to be implementing a tweak" - Analog Scott 12:40:46 08/20/06 (4)

Not a thing. I'm responding to your statement... - jeffreybehr 14:37:12 08/20/06 (3)

They did know what he was going to do. - Analog Scott 15:02:18 08/20/06 (2)

Re: They did know what he was going to do. - PhilNYC 15:44:53 08/20/06 (1)

That might have added some uncertainty. - Analog Scott 16:07:51 08/20/06 (0)

Re: A/B testing methodology question - Norm 08:43:09 08/20/06 (2)

Re: A/B testing methodology question - PhilNYC 08:55:12 08/20/06 (1)

Be happy! nt - Norm 08:56:16 08/20/06 (0)

Not valid - Analog Scott 08:29:05 08/20/06 (6)

Itâ€™s still better than sighted listening tests - Caymus 11:46:08 08/20/06 (1)

I disagree. One could get unfounded confidence - Analog Scott 13:23:54 08/20/06 (0)

Great response, Scott...NOT. Do you think those who raised their hands DID NOT hear a change? - jeffreybehr 09:53:51 08/20/06 (3)

Re: no isolation from social pressure - mls-stl 12:39:10 08/20/06 (0)

Who knows? - Slider 11:25:31 08/20/06 (0)

i think there is no way to know, because you have too many variables. - Analog Scott 11:13:12 08/20/06 (0)

You would get the same results if you did nothing. (nt) - Garth 08:24:44 08/20/06 (15)

What Garth is saying is: - Frihed89 10:29:08 08/20/06 (0)

Re: You would get the same results if you did nothing. (nt) - PhilNYC 08:51:27 08/20/06 (13)

Re: You would get the same results if you did nothing. (nt) - Analog Scott 11:44:31 08/20/06 (4)

I thought... - PhilNYC 12:00:23 08/20/06 (3)

Re: I thought... - Analog Scott 12:54:03 08/20/06 (0)

Phil, try to pay no attention to Scott. Your testing methodology is perfectly valid... - jeffreybehr 12:23:27 08/20/06 (1)

What a bunch of balony. - Analog Scott 12:57:18 08/20/06 (0)

If you did nothing and none raised their hands you would have some validation. - Norm 08/20/0608:55:14 08/20/06 (7)

Re: If you did nothing and none raised their hands you would have some validation. - PhilNYC 08:57:39 08/20/06 (6)

Re: Test administrator needs to be blind as well... - mls-stl 09:31:49 08/20/06 (5)

Re: Test administrator needs to be blind as well... - PhilNYC 09:49:36 08/20/06 (3)

Actually few experiments are done double blind - Norm 10:40:17 08/20/06 (2)

Re: same thing stated another way - mls-stl 11:29:55 08/20/06 (1)

I was merely pointing out that double blind testing is relatively uncommon - Norm 11:45:58 08/21/06 (0)

Good point about not generalizing to what others might hear. nt - Norm 09:40:25 08/20/06 (0)

Re: A/B testing methodology question - Todd Krieger 08:21:21 08/20/06 (1)

Re: A/B testing methodology question - PhilNYC 08:50:24 08/20/06 (0)

Follow Ups

If you did nothing and none raised their hands you would have some validation. - Norm 08/20/0608:55:14 08/20/06 (7)

Re: If you did nothing and none raised their hands you would have some validation. - PhilNYC 08:57:39 08/20/06 (6)

Re: Test administrator needs to be blind as well... - mls-stl 09:31:49 08/20/06 (5)

Re: Test administrator needs to be blind as well... - PhilNYC 09:49:36 08/20/06 (3)

Actually few experiments are done double blind - Norm 10:40:17 08/20/06 (2)

Re: same thing stated another way - mls-stl 11:29:55 08/20/06 (1)

I was merely pointing out that double blind testing is relatively uncommon - Norm 11:45:58 08/21/06 (0)

Good point about not generalizing to what others might hear. nt - Norm 09:40:25 08/20/06 (0)