Event sampling methodology

Last updated

Event sampling methodology (ESM) refers to a diary study. ESM is also known as ecological momentary assessment (EMA) or experience sampling methodology . ESM includes sampling methods that allow researchers to study ongoing experiences and events by taking assessments one or more times per day per participant (n=1) in the naturally occurring social environment. [1] [2] ESM enables researchers to study the prevalence of behaviors, promote theory development, and to serve an exploratory role. The frequent sampling of events inherent in ESM enables researchers to measure the typology of activity and detect the temporal and dynamic fluctuations of experiences (e.g., at work, or in a relationship). The popularity of ESM as a new form of research design increased over the recent years, because it addresses the shortcomings of cross-sectional research which cannot detect intra-individual variances and processes across time and cause-effect relationships. [3] In ESM, participants are asked to record their experiences and perceptions in a paper or electronic diary. Diary studies allow for the studying of events that occur naturally but are difficult to examine in the lab. For conducting event sampling, SurveySignal and Expimetrics. [4] are becoming popular platforms for social science researchers.

Contents

Some authors also use the term experience sampling to encompass passive data derived from sources such as smartphones, wearable sensors, the Internet of Things, email and social media that do not require explicit input from participants. [5] These methods can be advantageous as they impose less demand on participants improving compliance and allowing data to be collected for much longer periods, are less likely to change the behaviour being studied and allow data to be sampled at much high rates and with greater precision. Many research questions can benefit from both active and passive forms of experience sampling. [6]

Types

There are three types of ESM procedures:

  1. Signal contingent – some form of notification, such as SMS text messaging, [7] or beeping from a pager or wristwatch, notifies participants to record data at either fixed or random intervals. [2] An advantage of this type of ESM is minimization of recall bias. An additional advantage when used in conjunction with online data collection is improved response rates and timeliness by sending a reminder signal if the participant does not respond within a specified time. [7]
  2. Event contingent – participant records any occurrence of certain pre-determined events. This form is most useful when the event is fairly infrequent. It also allows flexibility for tracking events that typically fluctuate on their own. Clear definition(s) for the events of interest are important for not distorting the frequency and allowing the participant to make an easy distinction as to whether or not the event has occurred. [2]
  3. Interval contingent – records data according to the passing of a certain period of time. Typically, participants are asked to self-report on the behavior of interest at pre-determined intervals [8] which are determined on the basis of either theoretical or logical units of time. [3] The selection of the interval itself is crucial for not leading to skewed perception of the behavior, but it also is important that it is not taxing on the participants. [9] A day is the most commonly used sampling unit. [2]

Strengths

There are many strengths to using ESM. The first is that it highlights the possible situations and roles that behavior may be contingent upon. That is, it serves as a demonstration of the interaction between the person and the context and provides insight to the contingencies of behavior. [8] A second strength of ESM is that it provides ecological validity because the data is collected in the participant's natural environment, and this allows greater generalizability of the resulting data. Another strength is that it provides an excellent tool to measure individual differences and the differences the emerge or change over time. The fact that ESM avoid some of the typical problems of ESM is a fourth strength. With typical self-report measures, there can be concerns with memory or recall bias where participants may have difficulty recalling the past. A final strength of ESM is that is serves as a demonstration of the usefulness of multi methods assessment. The results and application of ESM are most useful when they are used in combination with global reports, for instance. [8]

Limitations

ESM has several disadvantages. One of the disadvantages of ESM is it can sometimes be perceived as invasive and intrusive by participants. ESM also leads to possible self-selection bias. It may be that only certain types of individuals are willing to participate in this type of study creating an unrepresentative sample. Another concern is related to participant cooperation. Participants may not be actually fill out their diaries at the specified times and concern has been raised difference in diary format may be influential in compliance. [10] However recent research [10] found that research that focuses on mean levels, between-person differences, and correlations among variables are less likely to be impacted by the difference between electronic and paper diary methods. Research reported that some participants experience the repeated assessments as taxing. [3] Further concerns are related to the fact that ESM may substantively change the phenomenon being studied. Reactivity or priming effects may occur, such that repeated measurement may cause changes in the participants' experiences. This method of sampling data is also highly vulnerable to common method variance.

Further, it is important to think about whether or not an appropriate dependent variable is being used in an ESM design. For example, it might be logical to use ESM in order to answer research questions which involve dependent variables with a great deal of variation throughout the day. Thus, variables such as change in mood, change in stress level, or the immediate impact of particular events may be best studied using ESM methodology. However, it is not likely that utilizing ESM will yield meaningful predictions when measuring someone performing a repetitive task throughout the day, when outcomes are long-term in nature (e.g., coronary heart problems), or inherently stable variables.

Statistical techniques

The possible techniques for analyzing data from ESM are important to consider. ESM produces a large amount of data which is one issue, but there are also considerations to factor into the analysis decision such as nesting, serial dependence, and an imbalance in the number and variance of data points. [2] Options range from calculating composites, using regression analyses, or modeling through multilevel or hierarchically nested models. [2] The most frequently used modeling options for diary data include [9] Hierarchical Linear Modeling (HLM) software, [11] MLWin, [12] and vector autoregression. [13]

See also

Related Research Articles

Statistical bias, in the mathematical field of statistics, is a systematic tendency in which the methods used to gather data and generate statistics present an inaccurate, skewed or biased depiction of reality. Statistical bias exists in numerous stages of the data collection and analysis process, including: the source of the data, the methods used to collect the data, the estimator chosen, and the methods used to analyze the data. Data analysts can take various measures at each stage of the process to reduce the impact of statistical bias in their work. Understanding the source of statistical bias can help to assess whether the observed results are close to actuality. Issues of statistical bias has been argued to be closely linked to issues of statistical validity.

<span class="mw-page-title-main">Sensor</span> Converter that measures a physical quantity and converts it into a signal

A sensor is a device that produces an output signal for the purpose of sensing a physical phenomenon.

Survey methodology is "the study of survey methods". As a field of applied statistics concentrating on human-research surveys, survey methodology studies the sampling of individual units from a population and associated techniques of survey data collection, such as questionnaire construction and methods for improving the number and accuracy of responses to surveys. Survey methodology targets instruments or procedures that ask one or more questions that may or may not be answered.

Experimental psychology refers to work done by those who apply experimental methods to psychological study and the underlying processes. Experimental psychologists employ human participants and animal subjects to study a great many topics, including sensation, perception, memory, cognition, learning, motivation, emotion; developmental processes, social psychology, and the neural substrates of all of these.

A cohort study is a particular form of longitudinal study that samples a cohort, performing a cross-section at intervals through time. It is a type of panel study where the individuals in the panel share a common characteristic.

<span class="mw-page-title-main">UK Biobank</span> Long-term biobank study of 500,000 people

UK Biobank is a large long-term biobank study in the United Kingdom (UK) which is investigating the respective contributions of genetic predisposition and environmental exposure to the development of disease. It began in 2006.

A longitudinal study is a research design that involves repeated observations of the same variables over long periods of time. It is often a type of observational study, although it can also be structured as longitudinal randomized experiment.

In medical research, social science, and biology, a cross-sectional study is a type of observational study that analyzes data from a population, or a representative subset, at a specific point in time—that is, cross-sectional data.

<span class="mw-page-title-main">Confounding</span> Variable or factor in causal inference

In causal inference, a confounder is a variable that influences both the dependent variable and independent variable, causing a spurious association. Confounding is a causal concept, and as such, cannot be described in terms of correlations or associations. The existence of confounders is an important quantitative explanation why correlation does not imply causation. Some notations are explicitly designed to identify the existence, possible existence, or non-existence of confounders in causal relationships between elements of a system.

<span class="mw-page-title-main">Research design</span> Overall strategy utilized to carry out research

Research design refers to the overall strategy utilized to carry out research that defines a succinct and logical plan to tackle established research question(s) through the collection, interpretation, analysis, and discussion of data.

In statistics, missing data, or missing values, occur when no data value is stored for the variable in an observation. Missing data are a common occurrence and can have a significant effect on the conclusions that can be drawn from the data.

<span class="mw-page-title-main">Quasi-experiment</span> Empirical interventional study

A quasi-experiment is an empirical interventional study used to estimate the causal impact of an intervention on target population without random assignment. Quasi-experimental research shares similarities with the traditional experimental design or randomized controlled trial, but it specifically lacks the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment.

The experience sampling method (ESM), also referred to as a daily diary method, or ecological momentary assessment (EMA), is an intensive longitudinal research methodology that involves asking participants to report on their thoughts, feelings, behaviors, and/or environment on multiple occasions over time. Participants report on their thoughts, feelings, behaviors, and/or environment in the moment or shortly thereafter. Participants can be given a journal with many identical pages. Each page can have a psychometric scale, open-ended questions, or anything else used to assess their condition in that place and time. ESM studies can also operate fully automatized on portable electronic devices or via the internet. The experience sampling method was developed by Suzanne Prescott during doctoral work at University of Chicago's Committee on Human Development with assistance from her dissertation advisor Mihaly Csikszentmihalyi. Early studies that used ESM were coauthored by fellow students Reed W. Larson and Ronald Graef, whose dissertations both used the method.

In medicine and psychology, clinical significance is the practical importance of a treatment effect—whether it has a real genuine, palpable, noticeable effect on daily life.

User experience evaluation (UXE) or user experience assessment (UXA) refers to a collection of methods, skills and tools utilized to uncover how a person perceives a system before, during and after interacting with it. It is non-trivial to assess user experience since user experience is subjective, context-dependent and dynamic over time. For a UXA study to be successful, the researcher has to select the right dimensions, constructs, and methods and target the research for the specific area of interest such as game, transportation, mobile, etc.

Subjective well-being (SWB) is a self-reported measure of well-being, typically obtained by questionnaire.

Diary studies is a research method that collects qualitative information by having participants record entries about their everyday lives in a log, diary or journal about the activity or experience being studied. This collection of data uses a longitudinal technique, meaning participants are studied over a period of time. This research tool, although not being able to provide results as detailed as a true field study, can still offer a vast amount of contextual information without the costs of a true field study. Diary studies are also known as experience sampling or ecological momentary assessment (EMA) methodology.

"How Nuts Are The Dutch" is an online research platform that became established on 13 December 2013 in the Netherlands. HowNutsAreTheDutch has been designed by researchers from the University of Groningen to support self-measurement of mental health for the entire population of The Netherlands. The project shows how crowdsourcing can be used for studying mental health in the general population. HowNutsAreTheDutch provides automatic personalized feedback on filled-out questionnaires to provide participants with more basic insight in their mental health, including a comparison with the scores of other participants. HowNutsAreTheDutch is meant to reduce mental health stigma and to promote a discrete categorization of mental health, by showing that all people have both personal strengths and weaknesses, and that most psychological characteristics are distributed continuously in the population. A background goal is to develop and evaluate personalized interventions to improve mood-related problems and social-emotional functioning. In the first year 13000 inhabitants of the Netherlands and Belgium participated.

Susana Mourato is a professor of environmental economics at the London School of Economics and Political Science. She holds a leader position at the Grantham Research Institute on Climate Change and the Environment.

References

  1. ↑ Csikszentmihalyi, M. (July 2014). Validity and Reliability of the Experience-Sampling Method. Vol. 175. New York: Springer. pp. 526–36. doi:10.1097/00005053-198709000-00004. ISBN   978-94-017-9087-1. PMID   3655778.{{cite book}}: |journal= ignored (help)
  2. 1 2 3 4 5 6 Reis, H. T., & Gable, S. L. (2000). Event-sampling and other methods for studying everyday experience. In H. T. Reis, C. M. Judd, H. T. Reis, C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 190–222). New York, NY US: Cambridge University Press.
  3. 1 2 3 van der Krieke, L; Blaauw, FJ; Emerencia, AC; Schenk, HM; Slaets, JP; Bos, EH; de Jonge, P; Jeronimus, BF (2016). "Temporal Dynamics of Health and Well-Being: A Crowdsourcing Approach to Momentary Assessments and Automated Generation of Personalized Feedback (2016)" (PDF). Psychosomatic Medicine. 79 (2): 213–223. doi:10.1097/PSY.0000000000000378. PMID   27551988. S2CID   10955232.
  4. ↑ "Expimetrics".
  5. ↑ Nielson, D. M.; Smith, T. A.; Sreekumar, V.; Dennis, S.; Sederberg, P. B. (2015). "Human hippocampus represents space and time during retrieval of real-world memories". Proceedings of the National Academy of Sciences. 112 (35): 11078–11083. Bibcode:2015PNAS..11211078N. doi: 10.1073/pnas.1507104112 . PMC   4568259 . PMID   26283350.
  6. ↑ Blaauw, FJ; Schenk, HM; Jeronimus, BF; van der Krieke, L; de Jonge, P; Aiello, M; Emerencia, AC (2016). "Let's get Physiqual - an intuitive and generic method to combine sensor technology with ecological momentary assessments (2016)" (PDF). Journal of Biomedical Informatics. 63: 141–149. doi: 10.1016/j.jbi.2016.08.001 . PMID   27498066.
  7. 1 2 Hofmann, W., & Patel, P. V. (2015). SurveySignal: A convenient solution for experience sampling research using participants’ own smartphones. Social Science Computer Review, 33, 235-253. http://journals.sagepub.com/doi/pdf/10.1177/0894439314525117
  8. 1 2 3 Scollon, C.; Kim-Prieto, C.; Diener, E. (2003). "Experience sampling: Promises and pitfalls, strengths and weaknesses". Journal of Happiness Studies. 4 (1): 5–34. doi:10.1023/a:1023605205115. S2CID   145310487.
  9. 1 2 Bolger, N.; Davis, A.; Rafaeli, E. (2003). "Diary methods: Capturing life as it is lived". Annual Review of Psychology. 54: 579–616. doi:10.1146/annurev.psych.54.101601.145030. PMID   12499517. S2CID   29202034.
  10. 1 2 Green, A. S.; Rafaeli, E.; Bolger, N.; Shrout, P. E.; Reis, H. T. (2006). "Paper or plastic? Data equivalence in paper and electronic diaries". Psychological Methods. 11 (1): 87–105. doi:10.1037/1082-989x.11.1.87. PMID   16594769. S2CID   7126703.
  11. ↑ Bryk A.S., & Raudenbush S.W., Congdon RT. 1996. HLM: Hierarchical Linear and Nonlinear Modeling with the HLM/2L and HLM/3L Programs. Chicago: Scientific Software Int.
  12. ↑ Prosser R, Rasbash J, Goldstein H. 1996. MLn User’s Guide. London: Inst. Educ.
  13. ↑ Krieke, LV; Jeronimus, BF; Blaauw, FJ; Wanders, RB; Emerencia, AC; Schenk, HM; Vos, SD; Snippe, E; Wichers, M; Wigman, JT; Bos, EH; Wardenaar, KJ; Jonge, PD (2016). "HowNutsAreTheDutch (HoeGekIsNL): A crowdsourcing study of mental symptoms and strengths (2015)" (PDF). International Journal of Methods in Psychiatric Research. 25 (2): 123–144. doi:10.1002/mpr.1495. PMC   6877205 . PMID   26395198. Archived from the original (PDF) on 2019-08-02. Retrieved 2019-01-13.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.