Robust decision-making

Robust decision-making (RDM) is an iterative decision analytics framework that aims to help identify potential robust strategies, characterize the vulnerabilities of such strategies, and evaluate the tradeoffs among them. ^{[1] [2] [3]} RDM focuses on informing decisions under conditions of what is called "deep uncertainty", that is, conditions where the parties to a decision do not know or do not agree on the system models relating actions to consequences or the prior probability distributions for the key input parameters to those models. ^{[2] : 1011}

History

A wide variety of concepts, methods, and tools have been developed to address decision challenges that confront a large degree of uncertainty. One source of the name "robust decision" was the field of robust design popularized primarily by Genichi Taguchi in the 1980s and early 1990s. ^{[4] [5]} Jonathan Rosenhead and colleagues were among the first to lay out a systematic decision framework for robust decisions, in their 1989 book Rational Analysis for a Problematic World. ^[6] Similar themes have emerged from the literatures on scenario planning, robust control, imprecise probability, and info-gap decision theory and methods. An early review of many of these approaches is contained in the Third Assessment Report of the Intergovernmental Panel on Climate Change, published in 2001.

Application

Robust decision-making (RDM) is a particular set of methods and tools developed over the last decade, primarily by researchers associated with the RAND Corporation, designed to support decision-making and policy analysis under conditions of deep uncertainty.

While often used by researchers to evaluate alternative options, RDM is designed and is often employed as a method for decision support, with a particular focus on helping decision-makers identify and design new decision options that may be more robust than those they had originally considered. Often, these more robust options represent adaptive decision strategies designed to evolve over time in response to new information. In addition, RDM can be used to facilitate group decision-making in contentious situations where parties to the decision have strong disagreements about assumptions and values. ^[7]

RDM approaches have been applied to a wide range of different types of decision challenges. A study in 1996 addressed adaptive strategies for reducing greenhouse gas emissions. ^[8] More recent studies include a variety of applications to water management issues, ^{[9] [10] [11]} evaluation of the impacts of proposed U.S. renewable energy requirements,^{[ citation needed ]} a comparison of long-term energy strategies for the government of Israel,^{[ citation needed ]} an assessment of science and technology policies the government of South Korea might pursue in response to increasing economic competition from China,^{[ citation needed ]} and an analysis of Congress' options for reauthorizing the Terrorism Risk Insurance Act (TRIA).^{[ citation needed ]}

Differences between RDM and traditional expected-utility analysis

RDM rests on three key concepts that differentiate it from the traditional subjective expected utility decision framework: multiple views of the future, a robustness criterion, and reversing the order of traditional decision analysis by conducting an iterative process based on a vulnerability-and-response-option rather than a predict-then-act decision framework.^{[ citation needed ]}

First, RDM characterizes uncertainty with multiple views of the future. In some cases, these multiple views will be represented by multiple future states of the world. RDM can also incorporate probabilistic information, but rejects the view that a single joint probability distribution represents the best description of a deeply uncertain future. Rather RDM uses ranges or, more formally, sets of plausible probability distributions to describe deep uncertainty.

Second, RDM uses robustness rather than optimality as a criterion to assess alternative policies. The traditional subjective utility framework ranks alternative decision options contingent on best estimated probability distributions. In general, there is a best (i.e., highest ranked) option. RDM analyses have employed several different definitions of robustness. These include: trading a small amount of optimum performance for less sensitivity to broken assumptions, good performance compared to the alternatives over a wide range of plausible scenarios, and keeping options open. ^[2] All incorporate some type of satisficing criteria and, in contrast to expected utility approaches, all generally describe tradeoffs rather than provide a strict ranking of alternative options.

Third, RDM employs a vulnerability-and-response-option analysis framework to characterize uncertainty and to help identify and evaluate robust strategies. This structuring of the decision problem is a key feature of RDM. The traditional decision analytic approach follows what has been called a predict-then-act approach ^[12] that first characterizes uncertainty about the future, and then uses this characterization to rank the desirability of alternative decision options. Importantly, this approach characterizes uncertainty without reference to the alternative options. In contrast, RDM characterizes uncertainty in the context of a particular decision. That is, the method identifies those combinations of uncertainties most important to the choice among alternative options and describes the set of beliefs about the uncertain state of the world that are consistent with choosing one option over another. This ordering provides cognitive benefits in decision support applications, allowing stakeholders to understand the key assumptions underlying alternative options before committing themselves to believing those assumptions. ^[13]

Conditions for robust decision-making

Robust decision methods seem most appropriate under three conditions: when the uncertainty is deep as opposed to well characterized, when there is a rich set of decision options, and the decision challenge is sufficiently complex that decision-makers need simulation models to trace the potential consequences of their actions over many plausible scenarios.

When the uncertainty is well characterized, then traditional expected utility (predict-then-act) analyses are often most appropriate. In addition, if decision-makers lack a rich set of decision options they may have little opportunity to develop a robust strategy and can do no better than a predict-then-act analysis. ^[2]

If the uncertainty is deep and a rich set of options is available, traditional qualitative scenario methods may prove most effective if the system is sufficiently simple or well understood that decision-makers can accurately connect potential actions to their consequences without the aid of simulation models.

Analytic tools for robust decision-making

RDM is not a recipe of analytic steps, but rather a set of methods that can be combined in varying ways for specific decisions to implement the concept. Two key items in this toolkit are described below: exploratory modeling and scenario discovery.

Exploratory modeling

Many RDM analyses use an exploratory modeling approach, ^[14] with computer simulations used not as a device for prediction, but rather as a means for relating a set of assumptions to their implied consequences. The analyst draws useful information from such simulations by running them many times using an appropriate experimental design over the uncertain input parameters to the model(s), collecting the runs in a large database of cases, and analyzing this database to determine what policy-relevant statements can be supported. RDM represents a particular implementation of this concept. An RDM analysis typically creates a large database of simulation model results, and then uses this database to identify vulnerabilities of proposed strategies and the tradeoffs among potential responses. This analytic process provides several practical advantages:

• The database of cases provides a concrete representation of the concept of a multiplicity of plausible futures.
• Running a simulation multiple times in the forward direction can simplify the analytic challenge of representing adaptive strategies in many practical applications because it separates the running of the simulation from the analysis needed to evaluate alternative decision options using the simulation. In contrast, some optimization methods make it difficult to include many types of feedbacks in a simulation.
• The exploratory modeling concept makes it possible to use a wide variety of decision approaches using diverse types simulation models within a common analytic framework (depending on what seems most appropriate for a particular decision application). Within this common framework RDM analyses have used traditional sequential decision approaches, rule-based descriptions of adaptive strategies, real options representations, complicated optimal economic growth models, spreadsheet models, agent-based models, and organization's existing suites of simulation models such as one used by the U.S. government to forecast the future state of the social security trust fund.
• The database of cases simplifies the comparison of alternative decision frameworks because one can apply these frameworks to an identical set of model results. For instance, one can place a joint probability distribution across the cases in a database, conduct an expected utility analysis, and compare the results to an RDM analysis using the same database.

Scenario discovery

RDM analyses often employ a process called scenario discovery to facilitate the identification of vulnerabilities of proposed strategies. ^{[13] [15]} The process begins by specifying some performance metric, such as the total cost of a policy or its deviation from optimality (regret), which can be used to distinguish those cases in the results database where the strategy is judged successful from those where it is judged unsuccessful. Statistical or data-mining algorithms are applied to the database to generate simple descriptions of regions in the space of uncertain input parameters to the model that best describe the cases where the strategy is unsuccessful. That is, the algorithm for describing these cases is tuned to optimize both the predictability and interpretability by decision-makers. The resulting clusters have many characteristics of scenarios and can be used to help decision-makers understand the vulnerabilities of the proposed policies and potential response options. A review conducted by the European Environment Agency of the rather sparse literature evaluating how scenarios actually perform in practice when used by organizations to inform decisions identified several key weaknesses of traditional scenario approaches.^{[ citation needed ]} Scenario-discovery methods are designed to address these weaknesses. ^[13] In addition, scenario discovery supports analysis for multiple stressors because it characterizes vulnerabilities as combinations of very different types of uncertain parameters (e.g. climate, economic, organizational capabilities, etc.).

Software support

There is several software available to perform RDM analysis. RAND Corporation has developed CARS for exploratory modeling and the sdtoolkit R package for scenario discovery. The EMA Workbench, developed at Delft University of Technology, provides extensive exploratory modeling and scenario discovery capabilities in Python. ^[16] OpenMORDM is an open source R package for RDM that includes support for defining more than one performance objective. ^[17] OpenMORDM facilitates exploring the impact of different robustness criteria, including both regret-based (e.g., minimizing deviation in performance) and satisficing-based (e.g., satisfying performance constraints) criteria. Rhodium is an open source Python package that supports similar functionality to the EMA Workbench and to OpenMORDM, but also allows its application on models written in C, C++, Fortran, R and Excel, as well as the use of several multi-objective evolutionary algorithms. ^[18]

See also

• Design rationale
• Problem structuring methods
• System dynamics

References

1. Mahmoudi, Amin; Abbasi, Mehdi; Deng, Xiaopeng (2022). "A novel project portfolio selection framework towards organizational resilience: Robust Ordinal Priority Approach". Expert Systems with Applications. 188: 116067. doi:10.1016/j.eswa.2021.116067. ISSN   0957-4174. PMC   9928571 . PMID   36818824.
2. Lempert, Robert J.; Collins, Myles T. (August 2007). "Managing the risk of uncertain threshold responses: comparison of robust, optimum, and precautionary approaches". Risk Analysis . 27 (4): 1009–1026. doi:10.1111/j.1539-6924.2007.00940.x. PMID   17958508. S2CID   1722147. Robust decision making describes a variety of approaches that differ from traditional optimum expected utility analysis in that they characterize uncertainty with multiple representations of the future rather than a single set of probability distributions and use robustness, rather than optimality, as a decision criterion. (1011-1012)
3. Croskerry, Pat (August 2009). "A universal model of diagnostic reasoning". Academic Medicine . 84 (8): 1022–1028. doi: 10.1097/ACM.0b013e3181ace703 . PMID   19638766. Robust decision making is more analytical than intuitive. It adopts a systematic approach to remove uncertainty within the resources available to make safe and effective decisions. (1023)
4. Phadke, Madhav Shridhar (1989). Quality engineering using robust design. Englewood Cliffs, NJ: Prentice Hall. ISBN   978-0137451678. OCLC   19455232.
5. Ullman, David G. (2006). Making robust decisions: decision management for technical, business, and service teams. Victoria, BC: Trafford Publishing. p. 35. ISBN   9781425109561. OCLC   81600845.
6. Mingers, John; Rosenhead, Jonathan, eds. (2001) [1989]. Rational analysis for a problematic world revisited: problem structuring methods for complexity, uncertainty and conflict (2nd ed.). Chichester, UK; New York: John Wiley & Sons. ISBN   978-0471495239. OCLC   46601256.
7. Lempert, Robert J.; Popper, Steven W. (2005). "High-performance government in an uncertain world". In Klitgaard, Robert E.; Light, Paul C. (eds.). High-performance government: structure, leadership, incentives. Santa Monica, CA: RAND Corporation. pp.  113–138. ISBN   978-0833037404. OCLC   57344300.
8. Lempert, Robert J.; Schlesinger, Michael E.; Bankes, Steve C. (June 1996). "When we don't know the costs or the benefits: adaptive strategies for abating climate change". Climatic Change . 33 (2): 235–274. Bibcode:1996ClCh...33..235L. CiteSeerX   10.1.1.20.9055 . doi:10.1007/BF00140248. S2CID   7945822.
9. Groves, David G.; Davis, Martha; Wilkinson, Robert; Lempert, Robert J. (2008). "Planning for climate change in the Inland Empire, Southern California" (PDF). Water Resources IMPACT. 10 (4): 14–17.
10. Dessai, Suraje; Hulme, Mike (February 2007). "Assessing the robustness of adaptation decisions to climate change uncertainties: A case study on water resources management in the East of England". Global Environmental Change . 17 (1): 59–72. doi:10.1016/j.gloenvcha.2006.11.005.
11. Weaver, Christopher P.; Lempert, Robert J.; Brown, Casey; Hall, John A.; Revell, David; Sarewitz, Daniel (January 2013). "Improving the contribution of climate model information to decision making: the value and demands of robust decision frameworks" (PDF). Wiley Interdisciplinary Reviews: Climate Change. 4 (1): 39–60. doi:10.1002/wcc.202. S2CID   53679056.
12. Lempert, Robert J.; Nakicenovic, Nebojsa; Sarewitz, Daniel; Schlesinger, Michael (July 2004). "Characterizing climate-change uncertainties for decision-makers: an editorial essay". Climatic Change . 65 (1–2): 1–9. doi:10.1023/B:CLIM.0000037561.75281.b3. S2CID   153473117.
13. Bryant, Benjamin P.; Lempert, Robert J. (January 2010). "Thinking inside the box: a participatory, computer-assisted approach to scenario discovery". Technological Forecasting and Social Change . 77 (1): 34–49. doi:10.1016/j.techfore.2009.08.002.
14. Bankes, Steve (June 1993). "Exploratory modeling for policy analysis". Operations Research . 41 (3): 435–449. doi: 10.1287/opre.41.3.435 . JSTOR   171847.
15. Groves, David G.; Lempert, Robert J. (February 2007). "A new analytic method for finding policy-relevant scenarios". Global Environmental Change . 17 (1): 73–85. doi:10.1016/j.gloenvcha.2006.11.006. S2CID   510560.
16. Kwakkel, Jan H.; Pruyt, Erik (March 2013). "Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty". Technological Forecasting and Social Change . 80 (3): 419–431. doi:10.1016/j.techfore.2012.10.005.
17. Hadka, David; Herman, Jonathan; Reed, Patrick; Keller, Klaus (December 2015). "An open source framework for many-objective robust decision making". Environmental Modelling & Software. 74: 114–129. doi: 10.1016/j.envsoft.2015.07.014 .
18. Hadjimichael, Antonia; Gold, David; Hadka, David; Reed, Patrick (9 June 2020). "Rhodium: Python Library for Many-Objective Robust Decision Making and Exploratory Modeling". Journal of Open Research Software. 8: 12. doi: 10.5334/jors.293 .

External links

• EMA workbench, an "exploratory modeling workbench" by Delft University of Technology
• OpenMORDM, an R package for multiobjective robust decision making
• sdtoolkit, scenario discovery toolkit for robust decision-making