Creating a Questionnaire

Creating Questionnaires

A Quick Guide To Creating QuestionnairesFrom The LIPA LRC

Designing a questionnaire requires planning and thought. This presentation will give you some ideas on where to start.

Is a questionnaire the best option? Before designing a questionnaire you should think carefully about what you need to find out. Only when you've figured this out can you decide if a questionnaire is an appropriate method of collecting data. Is it more suitable than an interview, focus group, or observation? Or perhaps you will decide to combine methods? If you're sure a questionnaire is what you need, read on!

Designing a questionnaire requires planning and thought. This presentation will give you some ideas on where to start.

Questionnaire / Survey, what's the correct word?! Questionnaire refers to the set of questions you are asking. This presentation will use the word questionnaire throughout. Survey refers to the wider process of gathering and analysing information from lots of people.

This quick presentation will look at:

Your Sample

Writing the Qs

Question Types

Getting Started

Checklist

Ethics

Distributing

Piloting

Your preliminary reading and research will have helped you to identify areas for further investigation. Think again about your hypothesis or essay question. Consider which questions you need to ask in order to complete the task. Try to phrase these questions as clearly and simply as possible. Avoid jargon and any terminology your respondents may not understand. Think about the types of questions you need to ask, and how you intend to analyse the responses. Consider practical points such as timescales. When is this assignment due? Are you confident you have sufficient time to design, distribute, and analyse responses? What are your options if things don't go to plan?

Getting started

How will you administer the questionnaire? Will you distribute paper copies or will it be online? Or a combination of the two? Choose the method that will best suit your respondents. There are lots of online tools available, such as Google Forms, MS Forms, or pehaps the best known, Survey Monkey. The Survey Monkey free plan allows you to ask 10 questions, and should be suitable for most undergraduate projects. Survey Monkey provides templates and their help centre is full of useful information on the design, implementation and analysis of questionnaires.

Getting started

Go To Survey Monkey

Getting started

Thinking about questions...

Questions can be open or closed Open questions allow your respondent to reply in their own words. They are a good way of obtaining qualitative information and a deeper insight into the opinions of your respondent. However, the answers can take time to analyse. Closed questions provide your respondent with options to select from. This makes the results straightforward to analyse. You can quickly count the responses and present findings in graphs and charts.

Let's look at some different question types.

Question Types

These are types of CLOSED questions

ListThe respondent is given a list of options. They can usually choose more than one.

Category The respondent is given a set of categories and chooses one.

Yes/NoA straightforward yes or no option is presented. This isn't going to offer much nuance but can be useful for indentifying broad groupings among your respondents: Do you hold a driving licence: Yes/No

Open The respondent is given the opportunity to answer in their own words.

Question Types

These are types of CLOSED questions

Likert scale A Likert scale provides the respondent with a statement and scaled levels of agreement. It provides more options for respondents who are unsure of their opinions. Strongly agree | Agree | Unsure | Disagree | Strongly Disagree The National Student Survey uses scale questions. Take a look at the 2023 NSS questions here:

ScaleMeasurement scale questions are used when you want to measure something that is not directly observable. For example, when you want to measure an attitude.

• Agree
• Undecided
• Disagree

Think about...

Question Types

Ooops! Poorly structured questions might look like... How old are you? 0 -25 25-30 30-35 35-40 Do you know who the current Foreign Secretary is? Do you believe in capital punishement? Yes / No Do you like going into the city centre and watching live music?

Example of well-structured questions Which company logo do you prefer? Please tick? Sample logo 1 Sample logo 2 No preference Are you a member of the Musicians' Union? Yes / No How often do you eat in the LIPA Canteen? Choose which is closest to your experience. Daily | Weekly | Monthly | Less than once a month

What's the problem? Click here to find out.

What's the problem? If you're 25 which box should you tick? The categories overlap so it's unclear. Also, what happens if you're over 40?

What's the problem? Click here to find out.

What's the problem? Will this question help you to obtain accurate information? If they answer "Yes" how do you know they're correct? A better question would be: Who is the current Foreign Secretary? What's the problem? This doesn't allow the respondent any room for nuance. Instead a range of options could be provided and the respondent asked to select the one closest to their own opinion. What's the problem? This is two questions. What if the respondent likes live music, but doesn't enjoy going into town? Break the question down and make it less confusing.

Question Types

Open/ClosedYou may wish to follow a closed question with an open question. This gives your respondent the opportunity to further express their views. This will hopefully encourage them to answer all your questions, rather than leaving some out, annotating their reponses or referring back to previous questions in later sections of the questionnaire. All things which add to the difficulty of eventually analysing your questionnaire results.Questionnaires typically use more closed questions than open ones. The main advantage of closed questions are that they are easier and quicker to analyse. The disadvantage is that they probably don't allow for every possibility, and they limit the freedom of your respondents. A mixture of closed and open questions is a great way of providing an effective mixture of qualitative and quantitative data on your topic from a relatively large sample of people. However, if you feel you need to ask lots of open questions, a questionnaire may not be right for you. More open and detailed interviews with fewer respondents might be a better option.

Writing the questions

Focus

Question order

Introductions

Order your questions in a logical way. Make it easy for your respondents to follow. Choose a sensible sequence for your questions. Maybe you can group them into themes? If you're asking about something that has changed over time, you could begin by establishing a respondent's opinion of something in the past, how they find it in the present, what they think of future options.

Introduce yourself and explain the purpose of your questionnaire. Ensure respondents are aware of what your research is for and how their answers will be used. Will their answers be anonymous? Tell your respondents about the ethical implications of your research.

Everything you ask in your questionnaire should be focussed on helping you to answer your overall research question. Keep your questionnaire on track by only asking relevant questions. This way, all your questions and responses will be useful.

Writing the questions

Freedom

Practical

Clarity

Ensure your questions are clearly written. Avoid jargon and technical terms, unless you are sure your respondents will understand. Is it clear to respondents what they have to do? Does the questionnaire look daunting or off-putting? Simple adjustments, such as making all your tick boxes line up nicely, can make a big difference.

Give your respondents the opportunity to answer honestly and authentically. Do not ask leading questions. Do not assume your respondent shares your opinions or feels a certain way.

Is analysing the responses going to be manageable? Will they help you to obtain the information you need? Will you be able to collate and analyse the responses in a straightforward way while meeting your deadlines. Don't bite off more than you can chew!

Sampling = the people you ask to complete your questionnaire. When designing a questionnaire think about how many responses you will need to make your research valid. Are you looking intensely at a narrow subject area? Perhaps you are only interested in a small sample of experts or people who have a particular experience. If you choose to focus on a small group of people, you need to be sure that each person responding is as appropriate and relevant as possible.

Quantitative research often uses larger sample sizes. Generally speaking, the larger the sample size, the more reliable the results. However the number of people you can question will obviously be limited by factors such as how much time you have and any costs that may be incurred. Also try to factor in non-responses. You may hope to receive 100 reponses to your questionnaire, but what happens if you only receive 80? It's better to send out more questionnaires than you need.

Piloting = trying out your questionnaire. Always test out your questionnaire. Ask friends, colleagues or family to fill it out. They can help you to refine technical aspects of the questionnaire, and spot errors. Maybe you have forgotten to include something, or the question numbering is wrong.

Importantly, they can tell you how long the questionnaire took to fill in. Too long and it may need a redesign. They may also be able to give you some more detailed feedback. Were they able to understand all the questions? Did it give them the opportunity to express an honest opinion?

How will you distribute the questionnaire? Will you distribute paper copies or will it be online? Or a combination of the two? Choose the method that will best suit your respondents. Do you need to obtain permission from your tutor before going ahead with your questionnaire? Are there any ethical concerns you need to consider? Is the purpose of your questionnaire and wider study clear to your respondents? Is it clear how you will treat their responses, will they be anonymous?

Distributing your questionnaire

How does the respondent return their completed questionnaire? Do you need to include a stamped addressed envelope? Are you writing a cover letter or providing some introductory information? You are likely to get a better response if you are able to establish some sort of personal contact. Is there a closing date? Have you communicated this clearly?It's probably best not to allow too long. Not specifying a cut off date, or setting it a long time in the future, may encourage responders to put-off filling it in.

Distributing your questionnaire

Keep a record of how many, and when, questionnaires were distributed. Do you intend to chase up non-respondents? If the responses are anonymous it will be harder to do this. Expect to receive a good response at first, which then tails off. There will inevitably be people who do not respond. Bear in mind the type of people who respond to your questionnaire may differ from those who don't. They may have a vested interest in your research area, or strong opinions they actively want to share. Is this something you need to account for in your analysis?

Distributing your questionnaire

Thinking ethically

Discuss your ideas with your tutor before you even begin to construct a questionnaire. Read LIPA's Code of Practice for Ethical Standards in Research Involving Human Participants. Does your sample require any 'extra' ethical thinking? Are they children, or a group that could be considered vulnerable? Remove questions that may cause offence. Think carefully about questions covering sensitive topics. You may wish to leave these until later in your questionnaire. If the respondent abandons the questionnaire at this point you may still have useable responses to earlier questions. Is it clear to your respondents how their information will be used? If you are offering anonymity how will you ensure this? Can they contact you if they need to? Can they withdraw a response if necessary? Do not submit your questionnaire results or analysis to an AI Tool without participant approval. For example, to ask the AI tool to help you to intepret the results. Once your questionnaire responses have been uploaded they become training material for the AI Tool and you no longer control their use.

Checklist time! Consider the list and figure out if your questionnaire is going to be effective and manageable.

Further Reading

The LRC has a great collection of books on study skills. Find out more about questionnaire and research methodologies at shelf number 371.3. Click the covers below to find the books in the LRC.

For further help & advice contact the LIPA LRCLRC@lipa.ac.uk

Be clear. If you are asking your respondents to rate something (for example 1-5) which is positive and which is negative? Be consistent. If 1 = negative and 5 = positive, make sure you stick with this scale throughout your questionnaire. Don't swap it around. Apply the same consistency whatever the scale. Don't swap from: Agree | Undecided | Disagree to Disagree | Undecided | Agree It will confuse your respondents. If they have to puzzle over your questions they may not complete the questionnaire. Or you may get answers that don't acurately reflect a respondent's true opinions.