Paradata

Last updated

The paradata of a data set or survey are data about the process by which the data were collected. [1] [2] Paradata of a survey are usually "administrative data about the survey." [3]

Example paradata topics about a survey include the times of day interviews were conducted, how long the interviews took, how many times there were contacts with each interviewee or attempts to contact the interviewee, the reluctance of the interviewee, and the mode of communication (such as phone, Web, email, or in person). [4] Thus there are paradata about each observation in the survey. These attributes affect the costs and management of a survey, the findings of a survey, evaluations of interviewers, and inferences one might make about non-respondents.

Paradata information can be used to help achieve the goals of a survey. For example, early responses may be mainly from one type of respondent, and the collectors knowing this can focus on reaching the other types so the survey has good coverage of the intended population. Thus survey efforts can be dynamically responsive to the paradata. [5]

In principle a survey's metadata includes its paradata.

The term is attributed to Couper (1998). [6] [7] [8]

Related Research Articles

Questionnaire construction refers to the design of a questionnaire to gather statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subject.

Survey methodology is "the study of survey methods". As a field of applied statistics concentrating on human-research surveys, survey methodology studies the sampling of individual units from a population and associated techniques of survey data collection, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.

<span class="mw-page-title-main">Likert scale</span> Psychometric measurement scale

A Likert scale is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, although there are other types of rating scales.

<span class="mw-page-title-main">Questionnaire</span> Series of questions for gathering information

A questionnaire is a research instrument that consists of a set of questions for the purpose of gathering information from respondents through survey or statistical study. A research questionnaire is typically a mix of close-ended questions and open-ended questions. Open-ended, long-term questions offer the respondent the ability to elaborate on their thoughts. The Research questionnaire was developed by the Statistical Society of London in 1838.

<span class="mw-page-title-main">C. R. Rao</span> Indian-American mathematician (born 1920)

Calyampudi Radhakrishna Rao,, commonly known as C. R. Rao, is an Indian-American mathematician and statistician. He is currently professor emeritus at Pennsylvania State University and Research Professor at the University at Buffalo. Rao has been honoured by numerous colloquia, honorary degrees, and festschrifts and was awarded the US National Medal of Science in 2002. The American Statistical Association has described him as "a living legend whose work has influenced not just statistics, but has had far reaching implications for fields as varied as economics, genetics, anthropology, geology, national planning, demography, biometry, and medicine." The Times of India listed Rao as one of the top 10 Indian scientists of all time. In 2023, Rao was awarded the International Prize in Statistics, an award often touted as the "statistics’ equivalent of the Nobel Prize". Rao is also a Senior Policy and Statistics advisor for the Indian Heart Association non-profit focused on raising South Asian cardiovascular disease awareness.

Computer-assisted personal interviewing (CAPI) is an interviewing technique in which the respondent or interviewer uses an electronic device to answer the questions. It is similar to computer-assisted telephone interviewing, except that the interview takes place in person instead of over the telephone. This method is usually preferred over a telephone interview when the questionnaire is long and complex. It has been classified as a personal interviewing technique because an interviewer is usually present to serve as a host and to guide the respondent. If no interviewer is present, the term Computer-Assisted Self Interviewing (CASI) may be used. An example of a situation in which CAPI is used as the method of data collection is the British Crime Survey.

SERVQUAL is a multi-dimensional research instrument designed to capture consumer expectations and perceptions of a service along five dimensions that are believed to represent service quality. SERVQUAL is built on the expectancy-disconfirmation paradigm, which, in simple terms, means that service quality is understood as the extent to which consumers' pre-consumption expectations of quality are confirmed or disconfirmed by their actual perceptions of the service experience. When the SERVQUAL questionnaire was first published in 1985 by a team of academic researchers, A. Parasuraman, Valarie Zeithaml and Leonard L. Berry to measure quality in the service sector, it represented a breakthrough in the measurement methods used for service quality research. The diagnostic value of the instrument is supported by the model of service quality which forms the conceptual framework for the development of the scale. The instrument has been widely applied in a variety of contexts and cultural settings and found to be relatively robust. It has become the dominant measurement scale in the area of service quality. In spite of the long-standing interest in SERVQUAL and its myriad of context-specific applications, it has attracted some criticism from researchers.

<span class="mw-page-title-main">Leslie Kish</span>

Leslie Kish was a Hungarian-American statistician and survey methodologist.

Cognitive pretesting, or cognitive interviewing, is a field research method where data is collected on how the subject answers interview questions. It is the evaluation of a test or questionnaire before it's administered. It allows survey researchers to collect feedback regarding survey responses and is used in evaluating whether the question is measuring the construct the researcher intends. The data collected is then used to adjust problematic questions in the questionnaire before fielding the survey to the full sample of people.

Computer-assisted web interviewing (CAWI) is an Internet surveying technique in which the interviewee follows a script provided in a website. The questionnaires are made in a program for creating web interviews. The program allows for the questionnaire to contain pictures, audio and video clips, links to different web pages, etc. The website is able to customize the flow of the questionnaire based on the answers provided, as well as information already known about the participant. It is considered to be a cheaper way of surveying since one doesn't need to use people to hold surveys unlike computer-assisted telephone interviewing. With the increasing use of the Internet, online questionnaires have become a popular way of collecting information. The design of an online questionnaire has a dramatic effect on the quality of data gathered. There are many factors in designing an online questionnaire; guidelines, available question formats, administration, quality and ethic issues should be reviewed. Online questionnaires should be seen as a sub-set of a wider-range of online research methods.

<span class="mw-page-title-main">Unstructured interview</span> Interview in which questions are not prearranged.

An unstructured interview or non-directive interview is an interview in which questions are not prearranged. These non-directive interviews are considered to be the opposite of a structured interview which offers a set amount of standardized questions. The form of the unstructured interview varies widely, with some questions being prepared in advance in relation to a topic that the researcher or interviewer wishes to cover. They tend to be more informal and free flowing than a structured interview, much like an everyday conversation. Probing is seen to be the part of the research process that differentiates the in-depth, unstructured interview from an everyday conversation. This nature of conversation allows for spontaneity and for questions to develop during the course of the interview, which are based on the interviewees' responses. The chief feature of the unstructured interview is the idea of probe questions that are designed to be as open as possible. It is a qualitative research method and accordingly prioritizes validity and the depth of the interviewees' answers. One of the potential drawbacks is the loss of reliability, thereby making it more difficult to draw patterns among interviewees' responses in comparison to structured interviews. Unstructured interviews are used in a variety of fields and circumstances, ranging from research in social sciences, such as sociology, to college and job interviews. Fontana and Frey have identified three types of in depth, ethnographic, unstructured interviews - oral history, creative interviews, and post-modern interviews.

<span class="mw-page-title-main">Multitrait-multimethod matrix</span> Statistical technique used to examine construct validity

The multitrait-multimethod (MTMM) matrix is an approach to examining construct validity developed by Campbell and Fiske (1959). It organizes convergent and discriminant validity evidence for comparison of how a measure relates to other measures. The conceptual approach has influenced experimental design and measurement theory in psychology, including applications in structural equation models.

<span class="mw-page-title-main">Official statistics</span> Statistics published by government agencies

Official statistics are statistics published by government agencies or other public bodies such as international organizations as a public good. They provide quantitative or qualitative information on all major areas of citizens' lives, such as economic and social development, living conditions, health, education, and the environment.

In qualitative research, a member check, also known as informant feedback or respondent validation, is a technique used by researchers to help improve the accuracy, credibility, validity, and transferability of a study. There are many subcategories of members checks, including; narrative accuracy checks, interpretive validity, descriptive validity, theoretical validity, and evaluative validity. In many member checks, the interpretation and report is given to members of the sample (informants) in order to check the authenticity of the work. Their comments serve as a check on the viability of the interpretation.

<span class="mw-page-title-main">C. F. Jeff Wu</span>

Chien-Fu Jeff Wu is the Coca-Cola Chair in Engineering Statistics and Professor in the H. Milton Stewart School of Industrial and Systems Engineering at the Georgia Institute of Technology. He is known for his work on the convergence of the EM algorithm, resampling methods such as the bootstrap and jackknife, and industrial statistics, including design of experiments, and robust parameter design.

With the application of probability sampling in the 1930s, surveys became a standard tool for empirical research in social sciences, marketing, and official statistics. The methods involved in survey data collection are any of a number of ways in which data can be collected for a statistical survey. These are methods that are used to collect information from a sample of individuals in a systematic way. First there was the change from traditional paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). Now, face-to-face surveys (CAPI), telephone surveys (CATI), and mail surveys are increasingly replaced by web surveys.

<span class="mw-page-title-main">Interview (research)</span> Research technique

An interview in qualitative research is a conversation where questions are asked to elicit information. The interviewer is usually a professional or paid researcher, sometimes trained, who poses questions to the interviewee, in an alternating series of usually brief questions and answers. They can be contrasted with focus groups in which an interviewer questions a group of people and observes the resulting conversation between interviewees, or surveys which are more anonymous and limit respondents to a range of predetermined answer choices. In addition, there are special considerations when interviewing children. In phenomenological or ethnographic research, interviews are used to uncover the meanings of central themes in the life world of the subjects from their own point of view.

The Joint Program in Survey Methodology was established at the University of Maryland, College Park in 1993, a collaboration between that University, the University of Michigan, and Westat. Today JPSM offers coursework on-site and online, offering MS and Ph.D. degrees, certificate programs, and short courses on the subjects of Survey Methodology, Survey Statistics, and Data Science.

<span class="mw-page-title-main">Frauke Kreuter</span> German sociologist and statistician

Frauke Kreuter is a German sociologist and statistician. She is a professor of the Joint Program in Survey Methodology (JPSM) of the University of Maryland, College Park and a professor in statistics and data science at the Ludwig Maximilian University of Munich, Germany. Her research in survey methodology includes work on sampling error and observational error.

Jill A. Dever is an American statistician specializing in survey methodology who works as a senior researcher and senior director in the division for statistical & data sciences at RTI International.

References

  1. O'Reilly, Jim. Paradata and Blaise: A Review of Recent Applications and Research. http://ibuc2009.blaiseusers.org/papers/7d.pdf
  2. Groves, R.M.; Heeringa, S.G. (2006). "Responsive design for household surveys: tools for actively controlling survey errors and costs" (PDF). Journal of the Royal Statistical Society, Series A. 169 (3): 439–457. doi:10.1111/j.1467-985X.2006.00423.x. hdl: 2027.42/71787 .
  3. Safir, Black, and Steinbach. 2001. Using Paradata to Examine the Effects of Interviewer Characteristics on Survey Response and Data Quality, Proceedings of the Annual Meeting of the American Statistical Association, http://www.amstat.org/sections/SRMS/Proceedings/y2001/Proceed/00620.pdf
  4. Taylor, Beth L. 2008. The 2006 National Health Interview Survey (NHIS) Paradata File: Overview and Applications. Section of Survey Research Methods--JSM 2008. http://www.amstat.org/sections/SRMS/proceedings/y2008/Files/301266.pdf
  5. Frauke Kreuter (ed.). 2013. Improving Surveys with Paradata: Analytic Uses of Process Information. ISBN   978-0-470-90541-8
  6. Frauke Kreuter; Mick Couper; Lars Lyberg. The use of paradata to monitor and manage survey data collection. Section on Survey Research Methods – JSM 2010. (online at amstat.org), citing Couper, M. 1998. Measuring survey quality in a CASIC environment. In Proceedings of the Section on Survey Research Methods of the American Statistical Association.
  7. Fritz Scheuren. 2000. Macro and Micro Paradata for Survey Assessment. 1999 NSAF Collection of Papers. (online at urban.org), citing Couper, M. 1998. Measuring survey quality in a CASIC environment. In Proceedings of the Section on Survey Research Methods of the American Statistical Association.
  8. Mick P. Couper. 1998. Measuring survey quality in a CASIC environment. In Proceedings of the Section on Survey Research Methods of the American Statistical Association. http://www.asasrms.org/Proceedings/papers/1998_006.pdf