In Reply to: Types of validity posted by pbarach on April 2, 2022 at 09:17:20:
I think the issue is as you put it ecological validity. It's not that the HK tests are worthless but their tests don't really relate to what normal people do when they listen to music. The main problems in the short form are:
1) people buy two stereo loudspeakers and listen in stereo (or multichannel if H/T
HK's test is in mono - one loudspeaker in the center of the room (stereo speakers are set up to the sides of listeners no directly in front of them
2) HK has a conflict of interest - they are in control of the test and control the listener training telling people what to listen to and for. People selling you the speakers and conducting tests to ensure their product wins said test is problematic.
3) with the above two problems the reliability is largely worthless because it is apples from a sour tree.
You can test 10 sports cars on rollers to show how great they are in terms of speed and acceleration and get statistically reliable results that they all beat up on a Toyota 4Runner but if the real-world driving is driving up a pothole-filled mountain - then the reliable results are kind of worthless - Ie; the results have to mirror what it is you are testing.
So it is perfectly fine for HK to test say 10 speakers in their room and claim that X percent of people chose speaker A but it's not ok to extrapolate that result to suggest that those results will extend to stereo listening in a home. Or that it would extend beyond those other 9 "loser" speakers - see point 2.
I would far prefer Psychologists to run these tests if it were feasible. Professors often have assignments for students to conduct such tests with people who feel they have some ESP ability. We need an audiophile Psych prof to assign an audio DBT as I suggested somewhere here where it is more as you say "ecologically" valid. Seems like not a terrible idea for someone to have a Master's Thesis on it.
This post is made possible by the generous support of people like you and our sponsors:
Follow Ups
- RE: Types of validity - RGA 04/4/2205:32:12 04/4/22 (5)
- conflict of interest - Analog Scott 08:01:19 04/7/22 (1)
- RE: conflict of interest - geoffkait 13:48:26 04/8/22 (0)
- one more thing - pbarach 07:59:54 04/4/22 (0)
- RE: Types of validity - pbarach 07:48:18 04/4/22 (0)
- Conflict of interest - geoffkait 06:52:32 04/4/22 (0)