Questionnaire

Last updated
A basic questionnaire in Thai Questionaire in Thai.png
A basic questionnaire in Thai

A questionnaire is a research instrument that consists of a set of questions (or other types of prompts) for the purpose of gathering information from respondents through survey or statistical study. A research questionnaire is typically a mix of close-ended questions and open-ended questions. Open-ended, long-term questions offer the respondent the ability to elaborate on their thoughts. The Research questionnaire was developed by the Statistical Society of London in 1838. [1] [2]

Contents

Although questionnaires are often designed for statistical analysis of the responses, this is not always the case.

Questionnaires have advantages over some other types of surveys in that they are cheap, do not require as much effort from the questioner as verbal or telephone surveys, and often have standardized answers that make it simple to compile data. [3] However, such standardized answers may frustrate users as the possible answers may not accurately represent their desired responses. [4] Questionnaires are also sharply limited by the fact that respondents must be able to read the questions and respond to them. Thus, for some demographic groups conducting a survey by questionnaire may not be concretely feasible.

History

One of the earliest questionnaires was Dean Milles' Questionnaire of 1753. [5]

Types

A distinction can be made between questionnaires with questions that measure separate variables, and questionnaires with questions that are aggregated into either a scale or index. Questionnaires with questions that measure separate variables, could, for instance, include questions on:

Questionnaires with questions that are aggregated into either a scale or index include for instance questions that measure:

Examples

Questionnaire construction

Question type

Usually, a questionnaire consists of a number of questions that the respondent has to answer in a set format. A distinction is made between open-ended and closed-ended questions. An open-ended question asks the respondent to formulate his own answer, whereas a closed-ended question asks the respondent to pick an answer from a given number of options. The response options for a closed-ended question should be exhaustive and mutually exclusive. Four types of response scales for closed-ended questions are distinguished:

A respondent's answer to an open-ended question is coded into a response scale afterward. An example of an open-ended question is a question where the testee has to complete a sentence (sentence completion item). [8]

Question sequence

In general, questions should flow logically from one to the next. To achieve the best response rates, questions should flow from the least sensitive to the most sensitive, from the factual and behavioural to the attitudinal, and from the more general to the more specific.[ citation needed ]

There typically is a flow that should be followed when constructing a questionnaire in regards to the order that the questions are asked. The order is as follows:

  1. Screens
  2. Warm-ups
  3. Transitions
  4. Skips
  5. Difficult
  6. Classification

Screens are used as a screening method to find out early whether or not someone should complete the questionnaire. Warm-ups are simple to answer, help capture interest in the survey, and may not even pertain to research objectives. Transition questions are used to make different areas flow well together. Skips include questions similar to "If yes, then answer question 3. If no, then continue to question 5." Difficult questions are towards the end because the respondent is in "response mode." Also, when completing an online questionnaire, the progress bars lets the respondent know that they are almost done so they are more willing to answer more difficult questions. Classification, or demographic question should be at the end because typically they can feel like personal questions which will make respondents uncomfortable and not willing to finish survey. [9]

Basic rules for questionnaire item construction

Multi-item scales

Labelled example of a multi-item psychometric scale as used in questionnaires Multi-item psychometric scale.jpg
Labelled example of a multi-item psychometric scale as used in questionnaires

Within social science research and practice, questionnaires are most frequently used to collect quantitative data using multi-item scales with the following characteristics: [11]

Questionnaire administration modes

Main modes of questionnaire administration include: [8]

Concerns with questionnaires

While questionnaires are inexpensive, quick, and easy to analyze, often the questionnaire can have more problems than benefits. For example, unlike interviews, the people conducting the research may never know if the respondent understood the question that was being asked. Also, because the questions are so specific to what the researchers are asking, the information gained can be minimal. [12] Often, questionnaires such as the Myers-Briggs Type Indicator, give too few options to answer; respondents can answer either option but must choose only one response. Questionnaires also produce very low return rates, whether they are mail or online questionnaires. The other problem associated with return rates is that often the people who do return the questionnaire are those who have a very positive or a very negative viewpoint and want their opinion heard. The people who are most likely unbiased either way typically do not respond because it is not worth their time.

One key concern with questionnaires is that they may contain quite large measurement errors. [13] These errors can be random or systematic. Random errors are caused by unintended mistakes by respondents, interviewers, and/or coders. Systematic error can occur if there is a systematic reaction of the respondents to the scale used to formulate the survey question. Thus, the exact formulation of a survey question and its scale is crucial, since they affect the level of measurement error. [14]

Further, if the questionnaires are not collected using sound sampling techniques, often the results can be non-representative of the population—as such a good sample is critical to getting representative results based on questionnaires. [15]

See also

Further reading

Questionnaire are of different types as per Paul: 1)Structured Questionnaire. 2)Unstructured Questionnaire. 3)Open ended Questionnaire. 4)Close ended Questionnaire. 5)Mixed Questionnaire. 6)Pictorial Questionnaire.

Related Research Articles

Psychological testing Administration of psychological tests

Psychological testing is the administration of psychological tests. Psychological tests are administered by trained evaluators. A person's responses are evaluated according to carefully prescribed guidelines. Scores are thought to reflect individual or group differences in the construct the test purports to measure. The science behind psychological testing is psychometrics.

Validity is the main extent to which a concept, conclusion or measurement is well-founded and likely corresponds accurately to the real world. The word "valid" is derived from the Latin validus, meaning strong. The validity of a measurement tool is the degree to which the tool measures what it claims to measure. Validity is based on the strength of a collection of different types of evidence described in greater detail below.

In the social sciences, scaling is the process of measuring or ordering entities with respect to quantitative attributes or traits. For example, a scaling technique might involve estimating individuals' levels of extraversion, or the perceived quality of products. Certain methods of scaling permit estimation of magnitudes on a continuum, while other methods provide only for relative ordering of the entities.

Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subject.

Survey methodology is "the study of survey methods". As a field of applied statistics concentrating on human-research surveys, survey methodology studies the sampling of individual units from a population and associated techniques of survey data collection, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.

Quantitative marketing research is the application of quantitative research techniques to the field of marketing research. It has roots in both the positivist view of the world, and the modern marketing viewpoint that marketing is an interactive process in which both the buyer and seller reach a satisfying agreement on the "four Ps" of marketing: Product, Price, Place (location) and Promotion.

Likert scale Psychometric measurement scale

A Likert scale is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, although there are other types of rating scales.

Personality test Method of assessing human personality constructs

A personality test is a method of assessing human personality constructs. Most personality assessment instruments are in fact introspective self-report questionnaire measures or reports from life records (L-data) such as rating scales. Attempts to construct actual performance tests of personality have been very limited even though Raymond Cattell with his colleague Frank Warburton compiled a list of over 2000 separate objective tests that could be used in constructing objective personality tests. One exception however, was the Objective-Analytic Test Battery, a performance test designed to quantitatively measure 10 factor-analytically discerned personality trait dimensions. A major problem with both L-data and Q-data methods is that because of item transparency, rating scales and self-report questionnaires are highly susceptible to motivational and response distortion ranging all the way from lack of adequate self-insight to downright dissimulation depending on the reason/motivation for the assessment being undertaken.

Response bias

Response bias is a general term for a wide range of tendencies for participants to respond inaccurately or falsely to questions. These biases are prevalent in research involving participant self-report, such as structured interviews or surveys. Response biases can have a large impact on the validity of questionnaires or surveys.

Computer-assisted personal interviewing (CAPI) is an interviewing technique in which the respondent or interviewer uses an electronic device to answer the questions. It is similar to computer-assisted telephone interviewing, except that the interview takes place in person instead of over the telephone. This method is usually preferred over a telephone interview when the questionnaire is long and complex. It has been classified as a personal interviewing technique because an interviewer is usually present to serve as a host and to guide the respondent. If no interviewer is present, the term Computer-Assisted Self Interviewing (CASI) may be used. An example of a situation in which CAPI is used as the method of data collection is the British Crime Survey.

Personality Assessment Inventory (PAI), developed by Leslie Morey, is a self-report 344-item personality test that assesses a respondent's personality and psychopathology. Each item is a statement about the respondent that the respondent rates with a 4-point scale. It is used in various contexts, including psychotherapy, crisis/evaluation, forensic, personnel selection, pain/medical, and child custody assessment. The test construction strategy for the PAI was primarily deductive and rational. It shows good convergent validity with other personality tests, such as the Minnesota Multiphasic Personality Inventory and the Revised NEO Personality Inventory.

In social science research, social-desirability bias is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting "good behavior" or under-reporting "bad", or undesirable behavior. The tendency poses a serious problem with conducting research with self-reports. This bias interferes with the interpretation of average tendencies as well as individual differences.

A self-report study is a type of survey, questionnaire, or poll in which respondents read the question and select a response by themselves without any outside interference. A self-report is any method which involves asking a participant about their feelings, attitudes, beliefs and so on. Examples of self-reports are questionnaires and interviews; self-reports are often used as a way of gaining participants' responses in observational studies and experiments.

Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people.

Acquiescence bias, also known as agreement bias, is a category of response bias common to survey research in which respondents have a tendency to select a positive response option or indicate a positive connotation disproportionately more frequently. Respondents do so without considering the content of the question or their 'true' preference. Acquiescence is sometimes referred to as "yea-saying" and is the tendency of a respondent to agree with a statement when in doubt. Questions affected by acquiescence bias take the following format: a stimulus in the form of a statement is presented, followed by 'agree/disagree,' 'yes/no' or 'true/false' response options. For example, a respondent might be presented with the statement "gardening makes me feel happy," and would then be expected to select either 'agree' or 'disagree.' Such question formats are favoured by both survey designers and respondents because they are straightforward to produce and respond to. The bias is particularly prevalent in the case of surveys or questionnaires that employ truisms as the stimuli, such as: "It is better to give than to receive" or "Never a lender nor a borrower be". Acquiescence bias can introduce systematic errors that affect the validity of research by confounding attitudes and behaviours with the general tendency to agree, which can result in misguided inference. Research suggests that the proportion of respondents who carry out this behaviour is between 10% and 20%.

Computer-assisted web interviewing (CAWI) is an Internet surveying technique in which the interviewee follows a script provided in a website. The questionnaires are made in a program for creating web interviews. The program allows for the questionnaire to contain pictures, audio and video clips, links to different web pages, etc. The website is able to customize the flow of the questionnaire based on the answers provided, as well as information already known about the participant. It is considered to be a cheaper way of surveying since one doesn't need to use people to hold surveys unlike computer-assisted telephone interviewing. With the increasing use of the Internet, online questionnaires have become a popular way of collecting information. The design of an online questionnaire has a dramatic effect on the quality of data gathered. There are many factors in designing an online questionnaire; guidelines, available question formats, administration, quality and ethic issues should be reviewed. Online questionnaires should be seen as a sub-set of a wider-range of online research methods.

A methodological advisor or statistical consultant provides methodological and statistical advice and guidance to clients interested in making decisions regarding the design of studies, the collection and analysis of data, and the presentation and dissemination of research findings. Trained in both methods and statistics, and communication skills, advisors may work in academia, industry, or the public sector.

With the application of probability sampling in the 1930s, surveys became a standard tool for empirical research in social sciences, marketing, and official statistics. The methods involved in survey data collection are any of a number of ways in which data can be collected for a statistical survey. These are methods that are used to collect information from a sample of individuals in a systematic way. First there was the change from traditional paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Now, face-to-face surveys (CAPI), telephone surveys (CATI), and mail surveys are increasingly replaced by web surveys.

Computer-assisted survey information collection

Computer-assisted survey information collection (CASIC) refers to a variety of survey modes that were enabled by the introduction of computer technology. The first CASIC modes were interviewer-administered, while later on computerized self-administered questionnaires (CSAQ) appeared. It was coined in 1990 as a catch-all term for survey technologies that have expanded over time.

The Mokken scale is a psychometric method of data reduction. A Mokken scale is a unidimensional scale that consists of hierarchically-ordered items that measure the same underlying, latent concept. This method is named after the political scientist Rob Mokken who suggested it in 1971.

References

  1. Gault, RH (1907). "A history of the questionnaire method of research in psychology". Research in Psychology. 14 (3): 366–383. doi:10.1080/08919402.1907.10532551.
  2. A copy of the instrument was published in the Journal of the Statistical Society, Volume 1, Issue 1, 1838, pages 5–13. "Fourth Annual Report of the Council of the Statistical Society of London". JSTOR   i315562.{{cite journal}}: Cite journal requires |journal= (help)
  3. "The Roma have a much younger population". OECD Economic Surveys: Slovak Republic. 2019-02-05. doi:10.1787/d8c7c39a-en. ISBN   9789264311350. ISSN   1999-0588. S2CID   242796295.
  4. "questions-answers-the-international-criminal-court-may-2010". doi:10.1163/2210-7975_hrd-0162-0046.{{cite journal}}: Cite journal requires |journal= (help)
  5. Fox, Adam, Parochial Queries: Printed Questionnaires and the Pursuit of Natural:Knowledge in the British Isles, 1650–1800, Edinburgh University
  6. Smedts HP, de Vries JH, Rakhshandehroo M, et al. (February 2009). "High maternal vitamin E intake by diet or supplements is associated with congenital heart defects in the offspring". BJOG. 116 (3): 416–23. doi: 10.1111/j.1471-0528.2008.01957.x . PMID   19187374. S2CID   22276050.
  7. Hogervorst, J. G.; Schouten, L. J.; Konings, E. J.; Goldbohm, R. A.; Van Den Brandt, P. A. (2007). "A Prospective Study of Dietary Acrylamide Intake and the Risk of Endometrial, Ovarian, and Breast Cancer". Cancer Epidemiology, Biomarkers & Prevention. 16 (11): 2304–2313. doi: 10.1158/1055-9965.EPI-07-0581 . PMID   18006919 . Retrieved 2013-02-18.
  8. 1 2 Mellenbergh, G.J. (2008). Chapter 10: Tests and Questionnaires: Construction and administration. In H.J. Adèr & G.J. Mellenbergh (Eds.) (with contributions by D.J. Hand), Advising on Research Methods: A consultant's companion (pp. 211–236). Huizen, The Netherlands: Johannes van Kessel Publishing.
  9. Burns, A. C., & Bush, R. F. (2010). Marketing Research. Upper Saddle River, NJ: Pearson Education.
  10. "How to Make Questionnaire". SurveyKing. Retrieved 9 April 2022.
  11. 1 2 Robinson, M. A. (2018). Using multi-item psychometric scales for research and practice in human resource management. Human Resource Management, 57(3), 739–750. https://dx.doi.org/10.1002/hrm.21852 (open-access)
  12. Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological testing: Principles, applications, and issues. Belmont, CA: Wadsworth
  13. Alwin, D. F. (2007). Margins of error: A study of reliability in survey measurement. Hoboken, Wiley
  14. Saris, W. E. and Gallhofer, I. N. (2014). Design, evaluation, and analysis of questionnaires for survey research. Second Edition. Hoboken, Wiley.
  15. Moser, Claus Adolf, and Graham Kalton. "Survey methods in social investigation." Survey methods in social investigation. 2nd Edition (1971).