In Reply to: A/B testing methodology question posted by PhilNYC on August 20, 2006 at 07:46:50:
It's not scientific and it is hard to say for sure what happened. Maybe they did hear a difference, maybe they didn't. For example:Could the listener's hear any switching?
Did the tweak make a level difference?
Did your tone of voice or body language indicate something?But the results are intriguing enough to investigate further. You could do a level matched SBT for 12 out of 16 trials.
____________________________________________________________
"Nature loves to hide."
---Heraclitus of Ephesus (trans. Wheelwright)
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- Re: A/B testing methodology question - Pat D 08/20/0609:09:16 08/20/06 (8)
- Re: A/B testing methodology question - PhilNYC 09:33:03 08/20/06 (7)
- Re: A/B testing methodology question - Analog Scott 11:42:53 08/20/06 (6)
- Uhh...Scott...read it all. NO ONE knew WHAT he was going to do... - jeffreybehr 12:17:06 08/20/06 (5)
- "Everybody was certainly aware that I was going to be implementing a tweak" - Analog Scott 12:40:46 08/20/06 (4)
- Not a thing. I'm responding to your statement... - jeffreybehr 14:37:12 08/20/06 (3)
- They did know what he was going to do. - Analog Scott 15:02:18 08/20/06 (2)
- Re: They did know what he was going to do. - PhilNYC 15:44:53 08/20/06 (1)
- That might have added some uncertainty. - Analog Scott 16:07:51 08/20/06 (0)