How Difficult Is It to Do a Good Survey?

Creating a survey may seem like a simple task. Many believe that it's just a matter of asking questions, but the reality is much more complex. Formulating questions is an art that requires deep knowledge about what you want to measure and how the answers should be interpreted. And this is where a lot of them fail.3,4

The problem isn't just that a poorly formulated question can confuse the respondent. Worse yet, a bad question can generate useless data, leading to wrong decisions based on faulty information. At best, data will be irrelevant, but at worst, it can lead to erroneous decisions that seriously affect a business. Research indicates that, in many cases, errors in the formulation of surveys are due to the lack of a clear theoretical basis to guide the process.⦁

What it really takes to create a good survey†

Creating an effective survey isn't just about throwing questions into the air. Here are some of the essential steps you need to take to develop a reliable measuring instrumentation.

Generating theory-based questions†

The first step in developing a reliable survey is to have a solid theoretical foundation. This means that before writing any question, you must deeply understand the phenomenon you are measuring. Without this foundation, you'll be creating questions that probably won't capture the construct you really want to analyze.

Content validation†

Once the questions are generated, it's crucial to ensure that they actually measure what you intend to measure. This is achieved through content validation, a process that helps eliminate questions that don't correctly represent the phenomenon. If you don't go through this stage, you run the risk of introducing ambiguity or biases in your answers, compromising the validity of your results.

Reduction of items with factor analysis†

Then comes the challenge of reducing the number of questions without sacrificing data quality. For this reason, factor analysis is an indispensable tool. This process allows us to identify which questions really provide value and which don't. This way, you can ensure that every question in your questionnaire is there for a clear and justified reason.

Internal Consistency and Reliability†

A measuring tool must not only be valid, but also reliable. This refers to the internal consistency of the survey: the questions must be correlated with each other so that they all point to the same phenomenon. If your questions don't have adequate internal consistency, the results will be inconsistent and unreliable.

The risk of asking unasked questions†

One of the biggest dangers in creating surveys is falling into the trap of asking questions simply to fill in space. Poorly posed questions or questions without an adequate structure not only generate confusing answers, but they can produce dangerously incorrect data. Not only do these errors waste time and resources, but they can jeopardize decision-making based on the information that is collected. That's why it's essential to remember that it's not just about having a lot of questions, but about having the right questions.

Is there another way to measure without so much risk?

Instead of relying solely on traditional questions, a more effective alternative is to use visual approaches, such as what we offer at woku. Instead of asking directly, people can share their opinion about specific moments represented in photos or videos. Not only does this eliminate the risk of misframing questions, but it also provides a more natural experience for the respondent. In this way, we capture the essence of what people are feeling without the interference that written questions often cause.

At woku, we don't just collect opinions, but we do so in a way that minimizes biases and errors associated with traditional questionnaires. In this way, we help companies obtain clearer and more actionable data, without the danger of misformulating questions.

References 3,4

• Hinkin, T.R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1 (1), 104-121.