How to read a research report you did not commission

Sooner or later someone hands you a research deck and asks you to make a decision on it. Here is what to check before you do, in the order that eliminates the most reports fastest.

1. Ask for the questionnaire

Not the findings, the instrument. A leading question produces a confident, useless answer, and the phrasing never appears in the summary. If the questionnaire is not available, you are being asked to trust a conclusion whose method you are not allowed to see, which is not research, it is an assertion with charts.

2. Look at where the sample was taken, not just how big it is

Sample size gets quoted because it is impressive. Sample composition decides whether the number means anything. A study of two thousand people that was fielded where the client is already strong will tell you the client is strong. Iraq is not one market, and a sample that treats it as one is measuring an average nobody lives in.

3. Find out who did the fieldwork and how they were supervised

Fieldwork is where the quality is won or lost. Ask who the interviewers were, how they were trained, what proportion of interviews were back-checked, and what happened to the ones that failed. A house that cannot answer this quickly has not been doing it.

4. Check whether the report contains anything the client did not want to hear

This is the fastest test of all. A study that confirms everything the commissioning team already believed is either very lucky or was not really an inquiry. Real fieldwork produces at least one uncomfortable finding, and a report worth having puts it in the summary rather than on slide forty-one.

5. See whether it ends with a recommendation or a summary

A summary tells you what was found. A recommendation says what the researcher would do in your position, and puts their judgment on the record where it can be wrong. The second is much rarer and far more useful, and it is the reason to prefer a research team that also has to live with the campaign that follows.

What to do when it fails these

Not much, honestly. A weak study is not improved by re-analysis, and the temptation to squeeze a decision out of it because it has already been paid for is how a bad number ends up in a board pack for three years. It is cheaper to field a small, well-built study than to act on a large, badly-built one.