What to Look For When Choosing an Online Survey System

Every organisation that wants to measure customer satisfaction starts with the same question: "which system should we buy?" What actually decides the outcome is not the system but when and through which channel you collect the feedback. The right question asked at the wrong moment is just as useless. This article lists the headings to examine when evaluating a survey system.

1. Through which channel will it run?

The same survey can run through different channels, and the choice directly affects the quality of the feedback:

• Kiosk and tablet. The survey is completed directly on a kiosk or tablet placed at the service point. It can be used in the waiting area or at the exit.
• Sharing by link. The survey link is shared and completed on a mobile-friendly screen; no paper form or manual data entry.
• Measurement through integration. With smart integration options, the survey becomes part of your existing processes.

The question to ask: does the system support all three channels, or does it lock you into a single format?

Channel choice also determines who responds. A kiosk reaches everyone who was served; a link-based survey amplifies the voices of those most inclined to answer, usually the very satisfied or the very dissatisfied. Because the two channels reach different audiences, keep the channel fixed when comparing results.

2. Question design and the admin panel

Who will prepare the survey is the most overlooked cost. A panel that requires technical knowledge means asking someone for help every time a survey changes.

What you want: creating questions and answer options with drag-and-drop, adapting colours and logo to your organisation, and publishing. In other words, marketing or operations staff being able to build the survey without the software team.

Question types are part of the same ease of use: single choice, multiple choice, rating (stars), numeric score, open text. If the team needs development work to add a new question type, survey evaluation will stall.

3. How should wording and scales be built?

Even the best system cannot rescue a badly written question. Common mistakes and what to do instead:

• Leading questions. "Did you enjoy our fast service?" makes a dissatisfied answer harder. Keep the wording neutral: "How would you rate the speed of service?"
• Double-barrelled questions. "Was the staff friendly and fast?" carries two separate qualities; if one is good and the other is not, the answer becomes meaningless. Each question should carry one subject.
• One-sided scales. A scale that only runs "poor–fair–good" narrows the negative end. Both ends of the scale should be balanced and labelled.
• The midpoint decision. An odd-numbered scale (1–5) includes a "neither" option; an even-numbered scale (1–4) forces the respondent to pick a side. Use odd if you want to separate out neutral answers, even if you want to force a direction.
• Option order. In ordered lists, the first and last options get chosen more often. If you run the same survey repeatedly, reversing the option order and comparing the result is a useful check.

Kiosk surveys carry one more field detail: choose a position where staff cannot see the screen. If customers know the attendant can see the score they give, they score higher and the measurement no longer reflects the real experience.

4. Which metric are you measuring: NPS, CSAT or CES

Different metrics answer different questions and are not interchangeable: For how these metrics apply in queue management and how they relate to abandonment, see our article on measuring waiting time.

• CSAT (satisfaction). Measures satisfaction with a specific transaction. It is the proportion of positive answers to "were you satisfied with this service?". It is the most direct indicator for transaction-level improvement.
• NPS (recommendation likelihood). Based on "how likely are you to recommend this organisation to someone you know, from 0 to 10?". Scores of 9–10 count as promoters, 7–8 as passives and 0–6 as detractors; the score is the promoter share minus the detractor share. It is a relationship-level trend indicator.
• CES (effort). Measures how much effort the customer had to spend to get their business done. High effort signals churn risk independently of satisfaction.

Rather than cramming all three into one survey, choose the metric your decisions depend on and hold it fixed. If the definition or the question wording changes, comparison with the previous period becomes invalid.

5. The scale-average trap

Satisfaction scales are ordinal data; a score of "4" is not twice the satisfaction of a "2". Even so, many reports present averages and treat a 0.2 difference between two periods as a strategic finding. A more robust reading:

• Show the distribution. How many people gave the lowest score and how many the highest? A rise at the negative end shows up before the average moves.
• Use proportions. "Share who answered satisfied or very satisfied" is easier to interpret than a mean score, and it compares more honestly.
• Freeze the wording and the scale. If the question wording or the scale changes, the trend line breaks. If you must change it, keep the old data and show the break in the chart.

6. Response rate: timing is everything

Most survey systems are set up correctly but applied at the wrong moment. Two rules help:

• Ask immediately after the transaction. The experience is fresh as the customer leaves; a link sent the next day is far less likely to be answered.
• Ask before the customer leaves. A kiosk at the exit works while the customer is still in the waiting area or the moment the transaction ends. Here a five-question survey is worth more than a twenty-question one.

7. Sampling: how many responses are enough?

In a small sample, a single response visibly shifts the result. At a site collecting twelve responses a month, one person giving the lowest score drags the average down noticeably and pushes a manager towards the wrong urgency. The same measurement logic applies to parent satisfaction in settings such as schools; you can move on to digitising the student pickup process from here.

There are three ways to manage this:

• Set a minimum response threshold. Below a certain count, treat the result as indicative rather than as a basis for decisions.
• Compare branches on equal terms. Placing two branches with very different response counts side by side misleads; compare similar volumes over the same period.
• Normalise against volume. A busy unit naturally collects more responses; that does not make it better or worse.

8. The burden of open-text answers

Free-text fields produce the most valuable feedback, but they also create an obligation: without a process for reading them, the box fills up and nobody looks. Before going live, settle who reads the text, how often, how recurring themes get tagged, and which process triggers when a critical statement (relating to safety or health) arrives.

A free-text field is also a place where customers may voluntarily type personal data. The survey form should carry a "do not enter personal information" notice, and incoming text must fall within your data retention policy.

9. Reporting and automation

Who sees the results, and how often? Live tracking on a dashboard is a good start, but as an organisation scales the real need is automation: the analysis report landing automatically in the inbox of nominated managers when the survey period closes or the target response count is reached.

Being able to export the report as PDF or Excel makes it easy to move the result into a management presentation. Data you cannot export is data you cannot use.

10. Comparison, not just reporting

A single satisfaction score says little on its own; comparison says a lot. The capabilities to look for:

• Branch and unit comparison. Running the same survey at different branches or units and seeing the results side by side.
• Tracking over time. Repeating the same measurement at regular intervals to see the trend.
• Service-specific feedback. Being able to ask targeted questions about a particular service or process.

Without these three, what you have is not measurement but a list of impressions.

11. Closing the loop

When a customer gives feedback, an expectation is created. If that expectation is not met, the survey starts to look like "a form filled in for nothing" and participation drops. Three steps to close the loop:

• Tie collected feedback to a visible improvement. If the same complaint arrives three periods in a row, the problem is not in the measurement but in the action.
• Report back to the customer. A short notice at the service point or an announcement screen can carry the message that feedback made a difference.
• Cap the survey frequency. Presenting a survey after every single transaction reduces participation; define the measurement interval.

12. Data protection and retention

Survey data often contains personal data. What to ask your vendor:

• What technology is the infrastructure built on, and how are security updates tracked?
• Where is the data stored, and for how long is it retained?
• Who is able to see the survey results?
• How are export and deletion performed?

These headings affect your organisation's obligations as data controller; the final legal assessment rests with your organisation.

There is also a balance between anonymity and candour. An impression that answers can be traced back to individuals reduces negative feedback. On the other hand, fully anonymous surveys make it hard to tell repeated responses apart. Decide which questions will be linked to an identity before the survey goes live.

On the technical side, the concrete indicator to check is performance: processing thousands of responses without interruption during a peak period. On systems built on a .NET foundation, that means responses being processed within milliseconds.

13. Three things to test in the field

If the survey will run on a kiosk, three behaviours should be tested before going live:

• Network loss. If the connection drops mid-answer, is the response lost, or held on the device and sent later?
• Session timeout. If a customer pauses while answering, does the form reset or resume where they left off?
• Duplicate submission. Whether many responses can be sent from the same device, and how that shows up in the report.

These three scenarios surface in the field rather than the lab, and deserve deliberate testing during a pilot.

Three common mistakes

• Writing a long survey. Repeating a five-question survey quarterly produces more data than one twenty-question survey.
• Nobody reading the result. If reports are produced but never turned into action, the cost of measuring is wasted.
• Reading only negative feedback. Knowing what satisfied customers liked is worth as much as the complaints.

Frequently Asked Questions

How many questions should a survey have?

The strongest single driver of completion rate is length. Repeating a short five-question survey at regular intervals produces more usable data than a one-off twenty-question survey. If a comprehensive questionnaire is genuinely needed, plan it as a separate and less frequent measurement.

Should I use NPS or CSAT?

If the aim is to improve a specific transaction, CSAT is the more direct indicator. NPS fits trend tracking at the relationship level, while CES measures how much effort the customer had to spend. Rather than cramming all three into one survey, choose the metric your decisions depend on and fix it together with the question wording.

Can I compare branches with few responses?

Placing branches with very different response counts side by side misleads. At a site collecting few responses, a single low score visibly shifts the average and creates unnecessary urgency. Defining a minimum response threshold, or comparing similar volumes over the same period, is sounder.

Should the survey be anonymous?

Anonymity improves the candour of answers, but it also makes repeated responses harder to tell apart. Kiosk surveys carry one extra rule: choose a position where staff cannot see the screen. Otherwise customers know their score is visible and tend to answer inaccurately.

Conclusion

Choosing a survey system starts with "which questions will we ask" and ends with "who turns the answers into action, and when". If channel flexibility, easy question design, a high completion rate and comparable reporting are all in place, the tool is the right one.

Three responsibilities come on top: hold your chosen metric fixed, avoid drawing strategic conclusions from a small sample, and close the feedback loop. Without those three, even the best survey system produces reports for the shelf.

For the platform's full feature set see PulseFlow Survey System; to evaluate against your own purpose see our solutions.