Flawed Study Gets Congressional Hearing

Tomorrow, the House Small Business Committee will convene a hearing based on a study that is so flawed it could be used to teach students how not to do survey research.

Last month, we wrote about this âsurvey,â conducted by the US Chamber of Commerce, purporting to show that compliance with Sarbanes-Oxley rules would be enormously burdensome to small business. It is difficult to believe anyone who reads the actual study would reach the same conclusion. The Chamber tried to identify small businesses that might be impacted by the law and asked almost 5,000 to complete a simple on-line survey asking questions that encouraged the answers the Chamber wanted. Only 177 (3.6%) of the businesses surveyed bothered to respond.

That's not all - many of the small businesses who actually answered the survey aren't so small - 20% (of the 177 who responded) are reported to be valued at more than $200 million, and almost half at more than $75 million. But even those figures are likely to be gross underestimates of the actual worth of a business, because the Chamber used a measure ("public float") of worth that reports the portion of a firm's outstanding shares that is in the hands of public investors, as opposed to firm officers, directors, or controlling-interest investors. So the number of actual small business respondents to this survey is mighty low.

My reading is that this study says little if anything about what small business owners think. The crack survey researchers at the Chamber of Commerce don't agree, of course. Here's what they conclude:

Based on the response rate and varied demographics of participants, the Research and Analysis Center at the U.S. Chamber of Commerce is confident that the research results presented within this report can be considered to be generally representative of thoughts, opinions and perceptions of the small business universe.

Given this dismal response rate, it would be more accurate to conclude that 96% of the businesses pre-selected to answer the survey couldnât be bothered â evidence that perhaps Sarbanes-Oxley isnât the problem the Chamber is claiming.

I thought the era of Congress wasting time on phony studies was over. Members of the Small Business Committee need to reject this faux-research and recognize that it should not be the basis for policy-making.