Home General Asylum

General audio topics that don't fit into specific categories.

[ Follow Ups ] Thread: [ Display  All  Email ] [ General Asylum ]
[ Alert Moderator ]

A/B testing methodology question

A question (with honest interest...not trying to open any arguments here) regarding an A/B testing process I tried this weekend...would this be considered a "valid" test?

I had a dozen audiophiles listen to my system this weekend. It is a highly resolving system in a well-treated listening room. I played the first 45 seconds a track of music to this group of people a couple of times so they could become familiar with the track. I then had everyone close their eyes, and asked them to raise their hands if they heard something change in the system. I did not tell anyone what tweak I was going to employ, nor did I mention anything about what kinds of changes (if any) they would be hearing. The tweak could be employed "on the fly" (no need to stop the music or reconnect anything).

At the moment I employed the tweak (about 20 seconds into the track), 6 people immediately raised their hands...and 3-4 additional people slowly raised their hands in the 4-5 seconds following the implementation of the tweak. This test was repeated 3 times (at different times into the track), and the results were pretty similar each time.

Is this a testing methodology that would satisfy those who believe double-blind testing is a necessity for drawing legitimate conclusions?

This post is made possible by the generous support of people like you and our sponsors:
  Atma-Sphere Music Systems, Inc.  

Topic - A/B testing methodology question - PhilNYC 07:46:50 08/20/06 (76)
You can not post to an archived thread.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2025, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]