Satisficing

Last updated

Satisficing is a decision-making strategy or cognitive heuristic that entails searching through the available alternatives until an acceptability threshold is met, without necessarily maximizing any specific objective. [1] The term satisficing, a portmanteau of satisfy and suffice, [2] was introduced by Herbert A. Simon in 1956, [3] [4] although the concept was first posited in his 1947 book Administrative Behavior . [5] [6] Simon used satisficing to explain the behavior of decision makers under circumstances in which an optimal solution cannot be determined. He maintained that many natural problems are characterized by computational intractability or a lack of information, both of which preclude the use of mathematical optimization procedures. He observed in his Nobel Prize in Economics speech that "decision makers can satisfice either by finding optimum solutions for a simplified world, or by finding satisfactory solutions for a more realistic world. Neither approach, in general, dominates the other, and both have continued to co-exist in the world of management science". [7]

Contents

Simon formulated the concept within a novel approach to rationality, which posits that rational choice theory is an unrealistic description of human decision processes and calls for psychological realism. He referred to this approach as bounded rationality. Moral satisficing is a branch of bounded rationality that views moral behavior as based on pragmatic social heuristics rather than on moral rules or optimization principles. These heuristics are neither good nor bad per se, but only in relation to the environments in which they are used. [8] Some consequentialist theories in moral philosophy use the concept of satisficing in a similar sense, though most call for optimization instead.

In decision-making research

Two traditions of satisficing exist in decision-making research: Simon’s program of studying how individuals or institutions rely on heuristic solutions in the real world, and the program of finding optimal solutions to problems simplified by convenient mathematical assumptions (so that optimization is possible). [9]

Heuristic Satisficing

Heuristic satisficing refers to the use of aspiration levels when choosing from different paths of action. By this account, decision-makers select the first option that meets a given need or select the option that seems to address most needs rather than the "optimal" solution. The basic model of aspiration-level adaptation is as follows: [10]

Step 1: Set an aspiration level α.

Step 2: Choose the first option that meets or exceeds α.

Step 3: If no option has satisfied a after time β, then change α by an amount γ and continue until a satisfying option is found.

Example: Consider pricing commodities. An analysis of 628 used car dealers showed that 97% relied on a form of satisficing. [11] Most set the initial price α in the middle of the prize range of comparable cars and lowered the price if the car was not sold after 24 days (β) by about 3% (γ). A minority (19%), mostly smaller dealerships, set a low intitial price and kept it unchanged (no Step 3). The car dealers adapted the parameters to their business environment. For instance, they decreased the waiting time b by about 3% for each additional competitor in the area.

Note that aspiration-level adaptation is a process model of actual behavior rather than an as-if optimization model, and accordingly requires an analysis of how people actually make decisions. It allows for predicting surprising effects such as the “cheap twin paradox,” where two similar cars have substantially different price tags in the same dealership.[4] The reason is that one car entered the dealership earlier and had at least one change in price at the time the second car arrived.

Example: A task is to sew a patch onto a pair of blue pants. The best needle to do the threading is a 4-cm-long needle with a 3-millimeter eye. This needle is hidden in a haystack along with 1,000 other needles varying in size from 1 cm to 6 cm. Satisficing claims that the first needle that can sew on the patch is the one that should be used. Spending time searching for that one specific needle in the haystack is a waste of energy and resources.

A crucial determinant of a satisficing decision strategy concerns the construction of the aspiration level. In many circumstances, the individual may be uncertain about the aspiration level.

Example: An individual who only seeks a satisfactory retirement income may not know what level of wealth is required—given uncertainty about future prices—to ensure a satisfactory income. In this case, the individual can only evaluate outcomes on the basis of their probability of being satisfactory. If the individual chooses that outcome which has the maximum chance of being satisfactory, then this individual's behavior is theoretically indistinguishable from that of an optimizing individual under certain conditions. [12] [13] [14]

Another key issue concerns an evaluation of satisficing strategies. Although often regarded as an inferior decision strategy, specific satisficing strategies for inference have been shown to be ecologically rational, that is in particular decision environments, they can outperform alternative decision strategies. [15]

Satisficing also occurs in consensus building when the group looks towards a solution everyone can agree on even if it may not be the best.

Example: A group spends hours projecting the next fiscal year's budget. After hours of debating they eventually reach a consensus, only to have one person speak up and ask if the projections are correct. When the group becomes upset at the question, it is not because this person is wrong to ask, but rather because the group has already come up with a solution that works. The projection may not be what will actually come, but the majority agrees on one number and thus the projection is good enough to close the book on the budget.

Optimization

One popular method for rationalizing satisficing is optimization when all costs, including the cost of the optimization calculations themselves and the cost of getting information for use in those calculations, are considered. As a result, the eventual choice is usually sub-optimal in regard to the main goal of the optimization, i.e., different from the optimum in the case that the costs of choosing are not taken into account.

As a form of optimization

Alternatively, satisficing can be considered to be just constraint satisfaction, the process of finding a solution satisfying a set of constraints, without concern for finding an optimum. Any such satisficing problem can be formulated as an (equivalent) optimization problem using the indicator function of the satisficing requirements as an objective function. More formally, if X denotes the set of all options and SX denotes the set of "satisficing" options, then selecting a satisficing solution (an element of S) is equivalent to the following optimization problem

where Is denotes the Indicator function of S, that is

A solution sX to this optimization problem is optimal if, and only if, it is a satisficing option (an element of S). Thus, from a decision theory point of view, the distinction between "optimizing" and "satisficing" is essentially a stylistic issue (that can nevertheless be very important in certain applications) rather than a substantive issue. What is important to determine is what should be optimized and what should be satisficed. The following quote from Jan Odhnoff's 1965 paper is appropriate: [16]

In my opinion there is room for both 'optimizing' and 'satisficing' models in business economics. Unfortunately, the difference between 'optimizing' and 'satisficing' is often referred to as a difference in the quality of a certain choice. It is a triviality that an optimal result in an optimization can be an unsatisfactory result in a satisficing model. The best things would therefore be to avoid a general use of these two words.

Applied to the utility framework

In economics, satisficing is a behavior which attempts to achieve at least some minimum level of a particular variable, but which does not necessarily maximize its value. [17] The most common application of the concept in economics is in the behavioral theory of the firm, which, unlike traditional accounts, postulates that producers treat profit not as a goal to be maximized, but as a constraint. Under these theories, a critical level of profit must be achieved by firms; thereafter, priority is attached to the attainment of other goals.

More formally, as before if X denotes the set of all options s, and we have the payoff function U(s) which gives the payoff enjoyed by the agent for each option. Suppose we define the optimum payoff U* the solution to

with the optimum actions being the set O of options such that U(s*) = U* (i.e. it is the set of all options that yield the maximum payoff). Assume that the set O has at least one element.

The idea of the aspiration level was introduced by Herbert A. Simon and developed in economics by Richard Cyert and James March in their 1963 book A Behavioral Theory of the Firm . [18] The aspiration level is the payoff that the agent aspires to: if the agent achieves at least this level it is satisfied, and if it does not achieve it, the agent is not satisfied. Let us define the aspiration level A and assume that AU*. Clearly, whilst it is possible that someone can aspire to something that is better than the optimum, it is in a sense irrational to do so. So, we require the aspiration level to be at or below the optimum payoff.

We can then define the set of satisficing options S as all those options that yield at least A: sSif and only ifAU(s). Clearly since AU*, it follows that O S. That is, the set of optimum actions is a subset of the set of satisficing options. So, when an agent satisfices, then she will choose from a larger set of actions than the agent who optimizes. One way of looking at this is that the satisficing agent is not putting in the effort to get to the precise optimum or is unable to exclude actions that are below the optimum but still above aspiration.

An equivalent way of looking at satisficing is epsilon-optimization (that means you choose your actions so that the payoff is within epsilon of the optimum). If we define the "gap" between the optimum and the aspiration as ε where ε = U*A. Then the set of satisficing options S(ε) can be defined as all those options s such that U(s) ≥ U*ε.

Other applications in economics

Apart from the behavioral theory of the firm, applications of the idea of satisficing behavior in economics include the Akerlof and Yellen model of menu cost, popular in New Keynesian macroeconomics. [19] [20] Also, in economics and game theory there is the notion of an Epsilon-equilibrium, which is a generalization of the standard Nash equilibrium in which each player is within ε of his or her optimal payoff (the standard Nash-equilibrium being the special case where ε = 0). [21]

Endogenous aspiration levels

What determines the aspiration level may be derived from past experience (some function of an agent's or firm's previous payoffs), or some organizational or market institutions. For example, if we think of managerial firms, the managers will be expected to earn normal profits by their shareholders. Other institutions may have specific targets imposed externally (for example state-funded universities in the UK have targets for student recruitment).

An economic example is the Dixon model of an economy consisting of many firms operating in different industries, where each industry is a duopoly. [22] The endogenous aspiration level is the average profit in the economy. This represents the power of the financial markets: in the long-run firms need to earn normal profits or they die (as Armen Alchian once said, "This is the criterion by which the economic system selects survivors: those who realize positive profits are the survivors; those who suffer losses disappear" [23] ). We can then think what happens over time. If firms are earning profits at or above their aspiration level, then they just stay doing what they are doing (unlike the optimizing firm which would always strive to earn the highest profits possible). However, if the firms are earning below aspiration, then they try something else, until they get into a situation where they attain their aspiration level. It can be shown that in this economy, satisficing leads to collusion amongst firms: competition between firms leads to lower profits for one or both of the firms in a duopoly. This means that competition is unstable: one or both of the firms will fail to achieve their aspirations and hence try something else. The only situation which is stable is one where all firms achieve their aspirations, which can only happen when all firms earn average profits. In general, this will only happen if all firms earn the joint-profit maximizing or collusive profit. [24]

In personality and happiness research

Some research has suggested that satisficing/maximizing and other decision-making strategies, like personality traits, have a strong genetic component and endure over time. This genetic influence on decision-making behaviors has been found through classical twin studies, in which decision-making tendencies are self-reported by each member of a twinned pair and then compared between monozygotic and dizygotic twins. [25] This implies that people can be categorized into "maximizers" and "satisficers", with some people landing in between.

The distinction between satisficing and maximizing not only differs in the decision-making process, but also in the post-decision evaluation. Maximizers tend to use a more exhaustive approach to their decision-making process: they seek and evaluate more options than satisficers do to achieve greater satisfaction. However, whereas satisficers tend to be relatively pleased with their decisions, maximizers tend to be less happy with their decision outcomes. This is thought to be due to limited cognitive resources people have when their options are vast, forcing maximizers to not make an optimal choice. Because maximization is unrealistic and usually impossible in everyday life, maximizers often feel regretful in their post-choice evaluation. [26]

In survey methodology

As an example of satisficing, in the field of social cognition, Jon Krosnick proposed a theory of statistical survey satisficing which says that optimal question answering by a survey respondent involves a great deal of cognitive work and that some people would use satisficing to reduce that burden. [27] [28] Some people may shortcut their cognitive processes in two ways:

Likelihood to satisfice is linked to respondent ability, respondent motivation and task difficulty.

Regarding survey answers, satisficing manifests in:

See also


Related Research Articles

<span class="mw-page-title-main">Herbert A. Simon</span> American political scientist (1916–2001)

Herbert Alexander Simon was an American scholar whose work influenced the fields of computer science, economics, and cognitive psychology. His primary research interest was decision-making within organizations and he is best known for the theories of "bounded rationality" and "satisficing". He received the Turing Award in 1975 and the Nobel Memorial Prize in Economic Sciences in 1978. His research was noted for its interdisciplinary nature, spanning the fields of cognitive science, computer science, public administration, management, and political science. He was at Carnegie Mellon University for most of his career, from 1949 to 2001, where he helped found the Carnegie Mellon School of Computer Science, one of the first such departments in the world.

<span class="mw-page-title-main">Rational choice model</span> Sociological theory

The rational choice model, also called rational choice theory refers to a set of guidelines that help understand economic and social behaviour. The theory originated in the eighteenth century and can be traced back to the political economist and philosopher Adam Smith. The theory postulates that an individual will perform a cost–benefit analysis to determine whether an option is right for them. Rational choice theory looks at three concepts: rational actors, self interest and the invisible hand.

Zero-sum game is a mathematical representation in game theory and economic theory of a situation that involves two competing entities, where the result is an advantage for one side and an equivalent loss for the other. In other words, player one's gain is equivalent to player two's loss, with the result that the net improvement in benefit of the game is zero.

<span class="mw-page-title-main">Cognitive bias</span> Systematic pattern of deviation from norm or rationality in judgment

A cognitive bias is a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own "subjective reality" from their perception of the input. An individual's construction of reality, not the objective input, may dictate their behavior in the world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, and irrationality.

A heuristic or heuristic technique is any approach to problem solving that employs a pragmatic method that is not fully optimized, perfected, or rationalized, but is nevertheless "good enough" as an approximation or attribute substitution. Where finding an optimal solution is impossible or impractical, heuristic methods can be used to speed up the process of finding a satisfactory solution. Heuristics can be mental shortcuts that ease the cognitive load of making a decision.

Heuristic reasoning is often based on induction, or on analogy ... Induction is the process of discovering general laws  ... Induction tries to find regularity and coherence ... Its most conspicuous instruments are generalization, specialization, analogy. [...] Heuristic discusses human behavior in the face of problems [... that have been] preserved in the wisdom of proverbs.

Bounded rationality is the idea that rationality is limited when individuals make decisions, and under these limitations, rational individuals will select a decision that is satisfactory rather than optimal.

<span class="mw-page-title-main">Behavioral economics</span> Academic discipline

Behavioral economics is the study of the psychological factors involved in the decisions of individuals or institutions, and how these decisions deviate from those implied by traditional economic theory.

Managerial economics is a branch of economics involving the application of economic methods in the organizational decision-making process. Economics is the study of the production, distribution, and consumption of goods and services. Managerial economics involves the use of economic theories and principles to make decisions regarding the allocation of scarce resources. It guides managers in making decisions relating to the company's customers, competitors, suppliers, and internal operations.

<span class="mw-page-title-main">Gerd Gigerenzer</span> German cognitive psychologist

Gerd Gigerenzer is a German psychologist who has studied the use of bounded rationality and heuristics in decision making. Gigerenzer is director emeritus of the Center for Adaptive Behavior and Cognition (ABC) at the Max Planck Institute for Human Development, Berlin, director of the Harding Center for Risk Literacy, University of Potsdam, and vice president of the European Research Council (ERC).

Utility maximization was first developed by utilitarian philosophers Jeremy Bentham and John Stuart Mill. In microeconomics, the utility maximization problem is the problem consumers face: "How should I spend my money in order to maximize my utility?" It is a type of optimal decision problem. It consists of choosing how much of each available good or service to consume, taking into account a constraint on total spending (income), the prices of the goods and their preferences.

<span class="mw-page-title-main">Bellman equation</span> Necessary condition for optimality associated with dynamic programming

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes. The equation applies to algebraic structures with a total ordering; for algebraic structures with a partial ordering, the generic Bellman's equation can be used.

Backward induction is the process of determining a sequence of optimal choices by reasoning from the endpoint of a problem or situation back to its beginning using individual events or actions. Backward induction involves examining the final point in a series of decisions and identifying the optimal process or action required to arrive at that point. This process continues backward until the best action for every possible point along the sequence is determined. Backward induction was first utilized in 1875 by Arthur Cayley, who discovered the method while attempting to solve the secretary problem.

Info-gap decision theory seeks to optimize robustness to failure under severe uncertainty, in particular applying sensitivity analysis of the stability radius type to perturbations in the value of a given estimate of the parameter of interest. It has some connections with Wald's maximin model; some authors distinguish them, others consider them instances of the same principle.

In game theory, the traveler's dilemma is a non-zero-sum game in which each player proposes a payoff. The lower of the two proposals wins; the lowball player receives the lowball payoff plus a small bonus, and the highball player receives the same lowball payoff, minus a small penalty. Surprisingly, the Nash equilibrium is for both players to aggressively lowball. The traveler's dilemma is notable in that naive play appears to outperform the Nash equilibrium; this apparent paradox also appears in the centipede game and the finitely-iterated prisoner's dilemma.

Heuristics is the process by which humans use mental shortcuts to arrive at decisions. Heuristics are simple strategies that humans, animals, organizations, and even machines use to quickly form judgments, make decisions, and find solutions to complex problems. Often this involves focusing on the most relevant aspects of a problem or situation to formulate a solution. While heuristic processes are used to find the answers and solutions that are most likely to work or be correct, they are not always right or the most accurate. Judgments and decisions based on heuristics are simply good enough to satisfy a pressing need in situations of uncertainty, where information is incomplete. In that sense they can differ from answers given by logic and probability.

<span class="mw-page-title-main">A Behavioral Theory of the Firm</span> Book by Richard Cyert

The behavioral theory of the firm first appeared in the 1963 book A Behavioral Theory of the Firm by Richard M. Cyert and James G. March. The work on the behavioral theory started in 1952 when March, a political scientist, joined Carnegie Mellon University, where Cyert was an economist.

Cognitive bias mitigation is the prevention and reduction of the negative effects of cognitive biases – unconscious, automatic influences on human judgment and decision making that reliably produce reasoning errors.

Ecological rationality is a particular account of practical rationality, which in turn specifies the norms of rational action – what one ought to do in order to act rationally. The presently dominant account of practical rationality in the social and behavioral sciences such as economics and psychology, rational choice theory, maintains that practical rationality consists in making decisions in accordance with some fixed rules, irrespective of context. Ecological rationality, in contrast, claims that the rationality of a decision depends on the circumstances in which it takes place, so as to achieve one's goals in this particular context. What is considered rational under the rational choice account thus might not always be considered rational under the ecological rationality account. Overall, rational choice theory puts a premium on internal logical consistency whereas ecological rationality targets external performance in the world. The term ecologically rational is only etymologically similar to the biological science of ecology.

In behavioural sciences, social rationality is a type of decision strategy used in social contexts, in which a set of simple rules is applied in complex and uncertain situations.

Maximization is a style of decision-making characterized by seeking the best option through an exhaustive search through alternatives. It is contrasted with satisficing, in which individuals evaluate options until they find one that is "good enough".

References

  1. Colman, Andrew (2006). A Dictionary of Psychology . New York: Oxford University Press. p.  670. ISBN   978-0-19-861035-9.
  2. Manktelow, Ken (2000). Reasoning and Thinking. Hove: Psychology Press. p. 221. ISBN   978-0863777080.
  3. Simon, Herbert A. (1956). "Rational Choice and the Structure of the Environment" (PDF). Psychological Review . 63 (2): 129–138. CiteSeerX   10.1.1.545.5116 . doi:10.1037/h0042769. PMID   13310708. S2CID   8503301. (page 129: "Evidently, organisms adapt well enough to 'satisfice'; they do not, in general, 'optimize'."; page 136: "A 'satisficing' path, a path that will permit satisfaction at some specified level of all its needs.")
  4. Artinger, Florian M.; Gigerenzer, Gerd; Jacobs, Perke (2022). "Satisficing: Integrating Two Traditions". Journal of Economic Literature. 60 (2): 598–635. doi:10.1257/jel.20201396. hdl: 21.11116/0000-0007-5C2A-4 . ISSN   0022-0515. S2CID   249320959.
  5. Brown, Reva (2004). "Consideration of the Origin of Herbert Simon's Theory of 'Satisficing' (1933-1947)". Management Decision. 42 (10): 1240–1256. doi:10.1108/00251740410568944.
  6. Simon, Herbert A. (1947). Administrative Behavior: a Study of Decision-Making Processes in Administrative Organization (1st ed.). New York: Macmillan. OCLC   356505.
  7. Simon, Herbert A. (1979). "Rational decision making in business organizations". The American Economic Review. 69 (4): 493–513. JSTOR   1808698.
  8. Gigerenzer, Gerd (2011-04-15), "Moral Satisficing: Rethinking Moral Behavior as Bounded Rationality", Heuristics, Oxford University Press, pp. 203–221, retrieved 2024-09-13
  9. Artinger, Florian M.; Gigerenzer, Gerd; Jacobs, Perke (2022-06-01). "Satisficing: Integrating Two Traditions". Journal of Economic Literature. 60 (2): 598–635. doi:10.1257/jel.20201396. hdl: 21.11116/0000-0007-5C2A-4 . ISSN   0022-0515.
  10. Reb, Jochen; Luan, Shenghua; Gigerenzer, Gerd (2024-05-14). Smart Management. The MIT Press. ISBN   978-0-262-37857-4.
  11. Artinger, Florian M; Gigerenzer, Gerd (2024-07-23). "How heuristic pricing shapes the aggregate market: the "Cheap Twin Paradox"". Industrial and Corporate Change. doi: 10.1093/icc/dtae025 . ISSN   0960-6491.
  12. Castagnoli, E.; LiCalzi, M. (1996). "Expected Utility without Utility" (PDF). Theory and Decision. 41 (3): 281–301. doi:10.1007/BF00136129. hdl: 10278/4143 . S2CID   154464803.
  13. Bordley, R.; LiCalzi, M. (2000). "Decision Analysis Using Targets Instead of Utility Functions". Decisions in Economics & Finance. 23 (1): 53–74. doi:10.1007/s102030050005. hdl: 10278/3610 . S2CID   11162758.
  14. Bordley, R.; Kirkwood, C. (2004). "Preference Analysis with Multiattribute Performance Targets". Operations Research. 52 (6): 823–835. doi:10.1287/opre.1030.0093.
  15. Gigerenzer, Gerd; Goldstein, Daniel G. (October 1996). "Reasoning the fast and frugal way: Models of bounded rationality". Psychological Review. 103 (4): 650–669. CiteSeerX   10.1.1.174.4404 . doi:10.1037/0033-295x.103.4.650. PMID   8888650.
  16. Odhnoff, Jan (1965). "On the Techniques of Optimizing and Satisficing". The Swedish Journal of Economics. 67 (1): 24–39. doi:10.2307/3439096. JSTOR   3439096.
  17. Dixon, Huw (2001). "Artificial Intelligence and Economic Theory" (PDF). Surfing Economics: Essays for the Inquiring Economist. New York: Palgrave. ISBN   978-0-333-76061-1.
  18. Cyert, Richard; March, James G. (1992). A Behavioral Theory of the Firm (2nd ed.). Wiley-Blackwell. ISBN   978-0-631-17451-6.
  19. Akerlof, George A.; Yellen, Janet L. (1985). "Can Small Deviations from Rationality Make Significant Differences to Economic Equilibria?". American Economic Review . 75 (4): 708–720. JSTOR   1821349.
  20. Akerlof, George A.; Yellen, Janet L. (1985). "A Near-rational Model of the Business Cycle, with Wage and Price Inertia". The Quarterly Journal of Economics . 100 (5): 823–838. doi:10.1093/qje/100.Supplement.823.
  21. Dixon, H. (1987). "Approximate Bertrand Equilibria in a replicated Industry". Review of Economic Studies . 54 (1): 47–62. doi:10.2307/2297445. JSTOR   2297445.
  22. Dixon, H. (2000). "Keeping Up with the Joneses: Competition and the Evolution of Collusion". Journal of Economic Behavior and Organization . 43 (2): 223–238. doi:10.1016/S0167-2681(00)00117-7.
  23. Alchian, A. (1950). "Uncertainty, Evolution and Economic Theory". Journal of Political Economy . 58 (3): 211–222. doi:10.1086/256940. JSTOR   1827159. S2CID   36045710.
  24. Dixon (2000), Theorem 1 page 228. for a non-technical explanation see Chapter 8, Surfing Economics by Dixon H
  25. Simonson, I.; Sela, A. (2011). "On the heritability of consumer decision making: An exploratory approach for studying genetic effects on judgment and choice". Journal of Consumer Research. 37 (6): 951–966. doi:10.1086/657022.
  26. Schwartz, B.; Ward, A.; Monterosso, J.; Lyubomirsky, S.; White, K.; Lehman, D. R. (2002). "Maximizing versus satisficing: Happiness is a matter of choice" (PDF). Journal of Personality and Social Psychology. 83 (5): 1178–1197. doi:10.1037/0022-3514.83.5.1178. PMID   12416921.
  27. Krosnick, Jon A. (1991). "Response strategies for coping with the cognitive demands of attitude measures in surveys". Applied Cognitive Psychology. 5 (3): 213–236. doi:10.1002/acp.2350050305. ISSN   0888-4080.
  28. Krosnick, Jon A. (1999). "Survey research". Annual Review of Psychology. 50 (1): 537–567. doi:10.1146/annurev.psych.50.1.537. ISSN   0066-4308. PMID   15012463.

Further reading