Probability box

For other uses of "p-box", see P-box.

A p-box (probability box).

A probability box (or p-box) is a characterization of uncertain numbers consisting of both aleatoric and epistemic uncertainties that is often used in risk analysis or quantitative uncertainty modeling where numerical calculations must be performed. Probability bounds analysis is used to make arithmetic and logical calculations with p-boxes.

An example p-box is shown in the figure at right for an uncertain number x consisting of a left (upper) bound and a right (lower) bound on the probability distribution for x. The bounds are coincident for values of x below 0 and above 24. The bounds may have almost any shape, including step functions, so long as they are monotonically increasing and do not cross each other. A p-box is used to express simultaneously incertitude (epistemic uncertainty), which is represented by the breadth between the left and right edges of the p-box, and variability (aleatory uncertainty), which is represented by the overall slant of the p-box.

Interpretation

Probability that x is 2.5 or less is between 4% and 36%

95th percentile is between 9 and 16

Dual interpretation of p-boxes

There are dual interpretations of a p-box. It can be understood as bounds on the cumulative probability associated with any x-value. For instance, in the p-box depicted at right, the probability that the value will be 2.5 or less is between 4% and 36%. A p-box can also be understood as bounds on the x-value at any particular probability level. In the example, the 95th percentile is sure to be between 9 and 16.

If the left and right bounds of a p-box are sure to enclose the unknown distribution, the bounds are said to be rigorous, or absolute. The bounds may also be the tightest possible such bounds on the distribution function given the available information about it, in which case the bounds are therefore said to be best-possible. It may commonly be the case, however, that not every distribution that lies within these bounds is a possible distribution for the uncertain number, even when the bounds are rigorous and best-possible.

Mathematical definition

P-boxes are specified by left and right bounds on the distribution function (or, equivalently, the survival function) of a quantity and, optionally, additional information constraining the quantity's mean and variance to specified intervals, and specified constraints on its distributional shape (family, unimodality, symmetry, etc.). A p-box represents a class of probability distributions consistent with these constraints.

A distribution function on the real numbers ${\displaystyle \mathbb {R} }$, is a function ${\displaystyle D:\mathbb {R} \rightarrow [0,1],}$ for which D(x) ≤ D(y) whenever x < y, and the limit of D at +∞ is 1 and the limit at −∞ is 0. A p-box is a set of distributions functions F satisfying the following constraints, for specified distribution functions FF, and specified bounds m₁ ≤ m₂ on the expected value of the distribution and specified bounds v₁ ≤ v₂ on the variance of the distribution.

${\displaystyle {\begin{aligned}&{\underline {F}}(x)\leq F(x)\leq {\overline {F}}(x),\\[4pt]&m_{1}\leq \int _{-\infty }^{\infty }x\,\mathrm {d} F(x)\leq m_{2}\\[4pt]&v_{1}\leq \int _{-\infty }^{\infty }x^{2}\,\mathrm {d} F(x)-\left(\int _{-\infty }^{\infty }x\,\mathrm {d} F(x)\right)^{2}\leq v_{2}\\[4pt]&F\in \mathbf {F} \end{aligned}}}$

where integrals of the form ${\textstyle \int _{-\infty }^{\infty }\cdots \,\mathrm {d} F(x)}$ are Riemann–Stieltjes integrals.

Thus, the constraints are that the distribution function F falls within prescribed bounds, the mean of the distribution is in the interval m, the variance of the distribution is in the interval v, and the distribution is within some admissible class of distributions F. The Riemann–Stieltjes integrals do not depend on the differentiability of F.

P-boxes serve the same role for random variables that upper and lower probabilities serve for events. In robust Bayes analysis ^[1] a p-box is also known as a distribution band. ^{[2] [3]} A p-box can be constructed as a closed neighborhood of a distribution ${\displaystyle F\in \mathbb {D} }$ under the Kolmogorov, Lévy or Wasserstein metric. A p-box is a crude but computationally convenient kind of credal set. Whereas a credal set is defined solely in terms of the constraint F as a convex set of distributions (which automatically determine F, F, m, and v, but are often very difficult to compute with), a p-box usually has a loosely constraining specification of F, or even no constraint so that F = ${\displaystyle \mathbb {D} }$. Calculations with p-boxes, unlike credal sets, are often quite efficient, and algorithms for all standard mathematical functions are known.

A p-box is minimally specified by its left and right bounds, in which case the other constraints are understood to be vacuous as ${\textstyle \left\{{\overline {F}},{\underline {F}},[-\infty ,+\infty ],[0,+\infty ],\mathbb {D} \right\}.}$ Even when these ancillary constraints are vacuous, there may still be nontrivial bounds on the mean and variance that can be inferred from the left and right edges of the p-box.

Where p-boxes come from

P-boxes may arise from a variety of kinds of incomplete information about a quantity, and there are several ways to obtain p-boxes from data and analytical judgment.

Distributional p-boxes

When a probability distribution is known to have a particular shape (e.g., normal, uniform, beta, Weibull, etc.) but its parameters can only be specified imprecisely as intervals, the result is called a distributional p-box, or sometimes a parametric p-box. Such a p-box is usually easy to obtain by enveloping extreme distributions given the possible parameters. For instance, if a quantity is known to be normal with mean somewhere in the interval [7,8] and standard deviation within the interval [1,2], the left and right edges of the p-box can be found by enveloping the distribution functions of four probability distributions, namely, normal(7,1), normal(8,1), normal(7,2), and normal(8,2), where normal(μ,σ) represents a normal distribution with mean μ and standard deviation σ. All probability distributions that are normal and have means and standard deviations inside these respective intervals will have distribution functions that fall entirely within this p-box. The left and right bounds enclose many non-normal distributions, but these would be excluded from the p-box by specifying normality as the distribution family.

Distribution-free p-boxes

Even if the parameters such as mean and variance of a distribution are known precisely, the distribution cannot be specified precisely if the distribution family is unknown. In such situations, envelopes of all distributions matching given moments can be constructed from inequalities such as those due to Markov, Chebyshev, Cantelli, or Rowe ^{[4] [5]} that enclose all distribution functions having specified parameters. These define distribution-free p-boxes because they make no assumption whatever about the family or shape of the uncertain distribution. When qualitative information is available, such as that the distribution is unimodal, the p-boxes can often be tightened substantially. ^[6]

P-boxes from imprecise measurements

When all members of a population can be measured, or when random sample data are abundant, analysts often use an empirical distribution to summarize the values. When those data have non-negligible measurement uncertainty represented by interval ranges about each sample value, an empirical distribution may be generalized to a p-box. ^[7] Such a p-box can be specified by cumulating the lower endpoints of all the interval measurements into a cumulative distribution forming the left edge of the p-box, and cumulating the upper endpoints to form the right edge. The broader the measurement uncertainty, the wider the resulting p-box.

Interval measurements can also be used to generalize distributional estimates based on the method of matching moments or maximum likelihood, that make shape assumptions such as normality or lognormality, etc. ^{[7] [8]} Although the measurement uncertainty can be treated rigorously, the resulting distributional p-box generally will not be rigorous when it is a sample estimate based on only a subsample of the possible values. But, because these calculations take account of the dependence between the parameters of the distribution, they will often yield tighter p-boxes than could be obtained by treating the interval estimates of the parameters as unrelated as is done for distributional p-boxes.

Confidence bands

There may be uncertainty about the shape of a probability distribution because the sample size of the empirical data characterizing it is small. Several methods in traditional statistics have been proposed to account for this sampling uncertainty about the distribution shape, including Kolmogorov–Smirnov ^[9] and similar ^[10] confidence bands, which are distribution-free in the sense that they make no assumption about the shape of the underlying distribution. There are related confidence-band methods that do make assumptions about the shape or family of the underlying distribution, which can often result in tighter confidence bands. ^{[11] [12] [13]} Constructing confidence bands requires one to select the probability defining the confidence level, which usually must be less than 100% for the result to be non-vacuous. Confidence bands at the (1 − α)% confidence level are defined such that, (1 − α)% of the time they are constructed, they will completely enclose the distribution from which the data were randomly sampled. A confidence band about a distribution function is sometimes used as a p-box even though it represents statistical rather than rigorous or sure bounds. This use implicitly assumes that the true distribution, whatever it is, is inside the p-box.

An analogous Bayesian structure is called a Bayesian p-box, ^{[14] [15]} which encloses all distributions having parameters within a subset of parameter space corresponding to some specified probability level from a Bayesian analysis of the data. This subset is the credible region for the parameters given the data, which could be defined as the highest posterior probability density region, or the lowest posterior loss region, or in some other suitable way. To construct a Bayesian p-box one must select a prior distribution, in addition to specifying the credibility level (analogous to a confidence level).

C-boxes

C-boxes (or confidence structures ^[16] ) are estimators of fixed, real-valued quantities that depend on random sample data and encode Neyman ^[17] confidence intervals at every level of confidence. ^{[18] [19] [16]} They characterize the inferential uncertainty about the estimate in the form of a collection of focal intervals (or sets), each with associated confidence (probability) mass. This collection can be depicted as a p-box and can project the confidence interpretation through probability bounds analysis.

Unlike traditional confidence intervals which cannot usually be propagated through mathematical calculations, c-boxes can be used in calculations in ways that preserve the ability to obtain arbitrary confidence intervals for the results. ^{[20] [19]} For instance, they can be used to compute probability boxes for both prediction and tolerance distributions.

C-boxes can be computed in a variety of ways directly from random sample data. There are confidence boxes for both parametric problems where the family of the underlying distribution from which the data were randomly generated is known (including normal, lognormal, exponential, Bernoulli, binomial, Poisson), and nonparametric problems in which the shape of the underlying distribution is unknown. ^[20] Confidence boxes account for the uncertainty about a parameter that comes from the inference from observations, including the effect of small sample size, but also potentially the effects of imprecision in the data and demographic uncertainty which arises from trying to characterize a continuous parameter from discrete data observations.

C-boxes are closely related to several other concepts. They are comparable to bootstrap distributions, ^[21] and are imprecise generalizations of traditional confidence distributions such as Student's t-distribution. Like it, c-boxes encode frequentist confidence intervals for parameters of interest at every confidence level. They are analogous to Bayesian posterior distributions in that they characterize the inferential uncertainty about statistical parameters estimated from sparse or imprecise sample data, but they can have a purely frequentist interpretation that makes them useful in engineering because they offer a guarantee of statistical performance through repeated use. In the case of the Bernoulli or binomial rate parameter, the c-box is mathematically equivalent to Walley's imprecise beta model ^{[22] [23]} with the parameter s=1, which is a special case of the imprecise Dirichlet process, a central idea in robust Bayes analysis.

Unlike confidence bands which are confidence limits about an entire distribution function at some particular confidence level, c-boxes encode confidence intervals about a fixed quantity at all possible confidence levels at the same time.

Envelopes of possible distributions

When there are multiple possible probability distributions that might describe a variable, and an analyst cannot discount any of them based on available information, a p-box can be constructed as the envelope of the various cumulative distributions. ^{[24] [25]} It is also possible to account for the uncertainty about which distribution is the correct one with a sensitivity study, but such studies become more complex as the number of possible distributions grows, and combinatorially more complex as the number of variables about which there could be multiple distributions increases. An enveloping approach is more conservative about this uncertainty than various alternative approaches to handle the uncertainty which average together distributions in stochastic mixture models or Bayesian model averages. The unknown true distribution is likely to be within the class of distributions encompassed by the p-box. In contrast, assuming the true distribution is one of the distributions being averaged, the average distribution is sure to be unlike the unknown true distribution.

P-boxes from calculation results

P-boxes can arise from computations involving probability distributions, or involving both a probability distribution and an interval, or involving other p-boxes. For example, the sum of a quantity represented by a probability distribution and a quantity represented by an interval will generally be characterized by a p-box. ^[26] The sum of two random variables characterized by well-specified probability distributions is another precise probability distribution typically only when the copula (dependence function) between the two summands is completely specified. When their dependence is unknown or only partially specified, the sum will be more appropriately represented by a p-box because different dependence relations lead to many different distributions for the sum. Kolmogorov originally asked what bounds could be placed about the distribution of a sum when nothing is known about the dependence between the distributions of the addends. ^[27] The question was only answered in the early 1980s. Since that time, formulas and algorithms for sums have been generalized and extended to differences, products, quotients and other binary and unary functions under various dependence assumptions. ^{[27] [28] [29] [30] [31] [32] [33]}

These methods, collectively called probability bounds analysis, provide algorithms to evaluate mathematical expressions when there is uncertainty about the input values, their dependencies, or even the form of mathematical expression itself. The calculations yield results that are guaranteed to enclose all possible distributions of the output variable if the input p-boxes were also sure to enclose their respective distributions. In some cases, a calculated p-box will also be best-possible in the sense that only possible distributions are within the p-box, but this is not always guaranteed. For instance, the set of probability distributions that could result from adding random values without the independence assumption from two (precise) distributions is generally a proper subset of all the distributions admitted by the computed p-box. That is, there are distributions within the output p-box that could not arise under any dependence between the two input distributions. The output p-box will, however, always contain all distributions that are possible, so long as the input p-boxes were sure to enclose their respective underlying distributions. This property often suffices for use in risk analysis.

Special cases

Precise probability distributions and intervals are special cases of p-boxes, as are real values and integers. Because a probability distribution expresses variability and lacks incertitude, the left and right bounds of its p-box are coincident for all x-values at the value of the cumulative distribution function (which is a non-decreasing function from zero to one). Mathematically, a probability distribution F is the degenerate p-box {F, F, E(F), V(F), F}, where E and V denote the expectation and variance operators. An interval expresses only incertitude. Its p-box looks like a rectangular box whose upper and lower bounds jump from zero to one at the endpoints of the interval. Mathematically, an interval [a, b] corresponds to the degenerate p-box {H(a), H(b), [a, b], [0, (b–a)²/4], ${\displaystyle \mathbb {D} }$}, where H denotes the Heaviside step function. A precise scalar number c lacks both kinds of uncertainty. Its p-box is just a step function from 0 to 1 at the value c; mathematically this is {H(c), H(c), c, 0, H(c)}.

Applications

P-boxes and probability bounds analysis have been used in many applications spanning many disciplines in engineering and environmental science, including:

• Engineering design ^[34]
• Expert elicitation ^[35]
• Analysis of species sensitivity distributions ^[36]
• Sensitivity analysis in aerospace engineering of the buckling load of the frontskirt of the Ariane 5 launcher ^[37]
• ODE models of chemical reactor dynamics ^{[38] [39]}
• Pharmacokinetic variability of inhaled VOCs ^[40]
• Groundwater modeling ^[41]
• Bounding failure probability for series systems ^[42]
• Heavy metal contamination in soil at an ironworks brownfield ^{[43] [44]}
• Uncertainty propagation for salinity risk models ^[45]
• Power supply system safety assessment ^[46]
• Contaminated land risk assessment ^[47]
• Engineered systems for drinking water treatment ^[48]
• Computing soil screening levels ^[49]
• Human health and ecological risk analysis by the U.S. EPA of PCB contamination at the Housatonic River Superfund site ^{[50] [51]}
• Environmental assessment for the Calcasieu Estuary Superfund site ^[52]
• Aerospace engineering for supersonic nozzle thrust ^[53]
• Verification and validation in scientific computation for engineering problems ^[54]
• Toxicity to small mammals of environmental mercury contamination ^[55]
• Modeling travel time of pollution in groundwater ^[56]
• Reliability analysis ^[57]
• Endangered species assessment for reintroduction of Leadbeater's possum ^[58]
• Exposure of insectivorous birds to an agricultural pesticide ^[59]
• Climate change projections ^{[43] [60] [61]}
• Waiting time in queuing systems ^[62]
• Extinction risk analysis for spotted owl on the Olympic Peninsula ^[63]
• Biosecurity against introduction of invasive species or agricultural pests ^[64]
• Finite-element structural analysis ^{[65] [66] [67]}
• Cost estimates ^[68]
• Nuclear stockpile certification ^[69]
• Fracking risks to water pollution ^[70]

Criticisms

No internal structure. Because a p-box retains little information about any internal structure within the bounds, it does not elucidate which distributions within the p-box are most likely, nor whether the edges represent very unlikely or distinctly likely scenarios. This could complicate decisions in some cases if an edge of a p-box encloses a decision threshold.

Loses information. To achieve computational efficiency, p-boxes lose information compared to more complex Dempster–Shafer structures or credal sets. ^[24] In particular, p-boxes lose information about the mode (most probable value) of a quantity. This information could be useful to keep, especially in situations where the quantity is an unknown but fixed value.

Traditional probability sufficient. Some critics of p-boxes argue that precisely specified probability distributions are sufficient to characterize uncertainty of all kinds. For instance, Lindley has asserted, "Whatever way uncertainty is approached, probability is the only sound way to think about it." ^{[71] [72]} These critics argue that it is meaningless to talk about 'uncertainty about probability' and that traditional probability is a complete theory that is sufficient to characterize all forms of uncertainty. Under this criticism, users of p-boxes have simply not made the requisite effort to identify the appropriate precisely specified distribution functions.

Possibility theory can do better. Some critics contend that it makes sense in some cases to work with a possibility distribution rather than working separately with the left and right edges of p-boxes. They argue that the set of probability distributions induced by a possibility distribution is a subset of those enclosed by an analogous p-box's edges. ^{[73] [74]} Others make a counterargument that one cannot do better with a possibility distribution than with a p-box. ^[75]

See also

• uncertain number
• interval
• cumulative probability distribution
• upper and lower probabilities
• credal set
• risk analysis
• uncertainty propagation
• probability bounds analysis
• Dempster–Shafer theory and the section on Dempster–Shafer structure
• imprecise probability
• simultaneous confidence bands on distribution and survival functions using likelihood ratios ^[13]
• pointwise binomial confidence intervals for F(X) for a given X ^[76]

References

1. Berger, J. O. (1984). "The robust Bayesian viewpoint." Pages 63–144 in Robustness of Bayesian analyses, edited by J.B. Kadane, Elsevier Science.
2. Basu, S. (1994). "Variations of posterior expectations for symmetric unimodal priors in a distribution band." Sankhyā: The Indian Journal of Statistics, Series A56: 320–334.
3. Basu, S., and A. DasGupta (1995). "Robust Bayesian analysis with distribution bands". Statistics and Decisions13: 333–349.
4. Rowe, N.C. (1988). Absolute bounds on the mean and standard deviation of transformed data for constant-sign-derivative transformations. SIAM Journal of Scientific and Statistical Computing9: 1098–1113.
5. Smith, J.E. (1995). Generalized Chebychev inequalities: theory and applications in decision analysis. Operations Research43: 807–825.
6. Zhang, J. and D. Berleant (2005). Arithmetic on random variables: squeezing the envelopes with new joint distribution constraints. Pages 416–422 in Proceedings of the Fourth International Symposium On Imprecise Probabilities and Their Applications (ISIPTA '05), Carnegie Mellon University, Pittsburgh, July 20–23, 2005.
7. Ferson, S., V. Kreinovich, J. Hajagos, W. Oberkampf, and L. Ginzburg (2007). Experimental Uncertainty Estimation and Statistics for Data Having Interval Uncertainty. Sandia National Laboratories, SAND 2007-0939, Albuquerque, NM.
8. Xiang, G., V. Kreinovich and S. Ferson, (2007). Fitting a normal distribution to interval and fuzzy data. Pages 560–565 in Proceedings of the 26th International Conference of the North American Fuzzy Information Processing Society NAFIPS'2007, M. Reformat and M.R. Berthold (eds.).
9. Kolmogorov, A. (1941). Confidence Limits for an Unknown Distribution Function. Annals of Mathematical Statistics12: 461–463.
10. Owen, A.B. (1995). Nonparametric likelihood confidence bands for a distribution function. Journal of the American Statistical Association90: 516–521.
11. Cheng, R.C.H., and T.C. Iles (1983). Confidence bands for cumulative distribution functions of continuous random variables. Technometrics25: 77–86.
12. Cheng, R.C.H., B.E. Evans and J.E. Williams (1988). Confidence band estimations for distributions used in probabilistic design. International Journal of Mechanical Sciences30: 835–845.
13. Murphy, S.A. (1995). Likelihood ratio-based confidence intervals in survival analysis. Journal of the American Statistical Association90: 1399–1405.
14. Montgomery, V. (2009). New Statistical Methods in Risk Assessment by Probability Bounds. Ph.D. dissertation, Durham University, UK.
15. Montgomery, V.J., F.P.A. Coolen, and A.D.M. Hart (2009) Bayesian probability boxes in risk assessment. Journal of Statistical Theory and Practice 3: 69–83.
16. M.S. Balch (2012). Mathematical foundations for a theory of confidence structures. International Journal of Approximate Reasoning 53: 1003–1019.
17. J. Neyman (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society A237: 333–380.
18. Confidence boxes website.
19. Ferson, S., M. Balch, K. Sentz, and J. Siegrist. 2013. Computing with confidence. Proceedings of the 8th International Symposium on Imprecise Probability: Theories and Applications, edited by F. Cozman, T. Denoeux, S. Destercke and T. Seidenfeld. SIPTA, Compiègne, France.
20. Ferson, S., J O'Rawe and M. Balch. 2014. Computing with confidence: imprecise posteriors and predictive distributions. Proceedings of the International Conference on Vulnerability and Risk Analysis and Management and International Symposium on Uncertainty Modeling and Analysis.
21. B. Efron (1998). R.A. Fisher in the 21st century. Statistical Science 13: 95–122.
22. P. Walley (1996). Inferences from multinomial data: learning about a bag of marbles. Journal of the Royal Statistical Society, Series B 58: 3–57.
23. J.-M. Bernard (2005). An introduction to the imprecise Dirichlet model for multinomial data. International Journal of Approximate Reasoning 39: 123–150.
24. Ferson, S., V. Kreinovich, L. Ginzburg, D.S. Myers, and K. Sentz (2003). Constructing Probability Boxes and Dempster–Shafer Structures Archived 2011-07-22 at the Wayback Machine . Sandia National Laboratories, SAND2002-4015, Albuquerque, NM.
25. Fu, G., D. Butler, S.-T. Khu, and S. Sun (2011). Imprecise probabilistic evaluation of sewer flooding in urban drainage systems using random set theory. Water Resources Research47: W02534.
26. Ferson, S., and L.R. Ginzburg (1996). Different methods are needed to propagate ignorance and variability. Reliability Engineering and Systems Safety54: 133–144.
27. Frank, M.J., R.B. Nelsen and B. Schweizer (1987). Best-possible bounds for the distribution of a sum—a problem of Kolmogorov. Probability Theory and Related Fields74: 199–211.
28. Yager, R.R. (1986). Arithmetic and other operations on Dempster–Shafer structures. International Journal of Man-machine Studies25: 357–366.
29. Williamson, R.C., and T. Downs (1990). Probabilistic arithmetic I: Numerical methods for calculating convolutions and dependency bounds. International Journal of Approximate Reasoning4: 89–158.
30. Berleant, D. (1993). Automatically verified reasoning with both intervals and probability density functions. Interval Computations1993 (2) : 48–70.
31. Berleant, D., G. Anderson, and C. Goodman-Strauss (2008). Arithmetic on bounded families of distributions: a DEnv algorithm tutorial. Pages 183–210 in Knowledge Processing with Interval and Soft Computing, edited by C. Hu, R.B. Kearfott, A. de Korvin and V. Kreinovich, Springer ( ISBN   978-1-84800-325-5).
32. Berleant, D., and C. Goodman-Strauss (1998). Bounding the results of arithmetic operations on random variables of unknown dependency using intervals. Reliable Computing4: 147–165.
33. Ferson, S., R. Nelsen, J. Hajagos, D. Berleant, J. Zhang, W.T. Tucker, L. Ginzburg and W.L. Oberkampf (2004). Dependence in Probabilistic Modeling, Dempster–Shafer Theory, and Probability Bounds Analysis. Sandia National Laboratories, SAND2004-3072, Albuquerque, NM.
34. Aughenbaugh, J. M., and C.J.J. Paredis (2007). Probability bounds analysis as a general approach to sensitivity analysis in decision making under uncertainty Archived 2012-03-21 at the Wayback Machine . SAE 2007 Transactions Journal of Passenger Cars: Mechanical Systems, (Section 6)116: 1325–1339, SAE International, Warrendale, Pennsylvania.
35. Flander, L., W. Dixon, M. McBride, and M. Burgman. (2012). Facilitated expert judgment of environmental risks: acquiring and analysing imprecise data. International Journal of Risk Assessment and Management16: 199–212.
36. Dixon, W.J. (2007). The use of Probability Bounds Analysis for Characterising and Propagating Uncertainty in Species Sensitivity Distributions. Technical Report Series No. 163, Arthur Rylah Institute for Environmental Research, Department of Sustainability and Environment. Heidelberg, Victoria, Australia.
37. Oberguggenberger, M., J. King and B. Schmelzer (2007). Imprecise probability methods for sensitivity analysis in engineering. Proceedings of the 5th International Symposium on Imprecise Probability: Theories and Applications, Prague, Czech Republic.
38. Enszer, J.A., Y. Lin, S. Ferson, G.F. Corliss and M.A. Stadtherr (2011). Probability bounds analysis for nonlinear dynamic process models. AIChE Journal57: 404–422.
39. Enszer, Joshua Alan, (2010). Verified Probability Bound Analysis for Dynamic Nonlinear Systems. Dissertation, University of Notre Dame.
40. Nong, A., and K. Krishnan (2007). Estimation of interindividual pharmacokinetic variability factor for inhaled volatile organic chemicals using a probability-bounds approach. Regulatory Toxicology and Pharmacology 48: 93–101.
41. Guyonnet, D., F. Blanchard, C. Harpet, Y. Ménard, B. Côme and C. Baudrit (2005). Projet IREA—Traitement des incertitudes en évaluation des risques d'exposition, Annexe B, Cas «Eaux souterraines». Rapport BRGM/RP-54099-FR, Bureau de Recherches Géologiques et Minières, France. Archived 2012-03-11 at the Wayback Machine
42. Fetz, Thomas; Tonon, Fulvio (2008). "Probability bounds for series systems with variables constrained by sets of probability measures". International Journal of Reliability and Safety. 2 (4): 309. doi:10.1504/IJRS.2008.022079.
43. Augustsson, A., M. Filipsson, T. Öberg, B. Bergbäck (2011). Climate change—an uncertainty factor in risk analysis of contaminated land. Science of the Total Environment409: 4693–4700.
44. Baudrit, C., D. Guyonnet, H. Baroudi, S. Denys and P. Begassat (2005). Assessment of child exposure to lead on an ironworks brownfield: uncertainty analysis. 9th International FZK/TNO Conference on Contaminated Soil – ConSoil2005, Bordeaux, France, pages 1071–1080.
45. Dixon, W.J. (2007). Uncertainty Propagation in Population Level Salinity Risk Models. Technical Report Technical Report Series No. 164, Arthur Rylah Institute for Environmental Research. Heidelberg, Victoria, Australia
46. Karanki, D.R., H.S. Kushwaha, A.K. Verma, and S. Ajit. (2009). Uncertainty analysis based on probability bounds (p-box) approach in probabilistic safety assessment. Risk Analysis29: 662–75.
47. Sander, P., B. Bergbäck and T. Öberg (2006). Uncertain numbers and uncertainty in the selection of input distributions—Consequences for a probabilistic risk assessment of contaminated land. Risk Analysis26: 1363–1375.
48. Minnery, J.G., J.G. Jacangelo, L.I. Boden, D.J. Vorhees and W. Heiger-Bernays (2009). Sensitivity analysis of the pressure-based direct integrity test for membranes used in drinking water treatment. Environmental Science and Technology43(24): 9419–9424.
49. Regan, H.M., B.E. Sample and S. Ferson (2002). Comparison of deterministic and probabilistic calculation of ecological soil screening levels. Environmental Toxicology and Chemistry 21: 882–890.
50. U.S. Environmental Protection Agency (Region I), GE/Housatonic River Site in New England
51. Moore, Dwayne R.J.; Breton, Roger L.; Delong, Tod R.; Ferson, Scott; Lortie, John P.; MacDonald, Drew B.; McGrath, Richard; Pawlisz, Andrzej; Svirsky, Susan C.; Teed, R. Scott; Thompson, Ryan P.; Whitfield Aslund, Melissa (2016). "Ecological risk assessment for mink and short-tailed shrew exposed to PCBS, dioxins, and furans in the Housatonic River area". Integrated Environmental Assessment and Management. 12 (1): 174–184. doi:10.1002/ieam.1661. PMID   25976918.
52. U.S. Environmental Protection Agency (Region 6 Superfund Program), Calcasieu Estuary Remedial Investigation Archived January 20, 2011, at the Wayback Machine
53. Roy, C.J., and M.S. Balch (2012). A holistic approach to uncertainty quantification with application to supersonic nozzle thrust. International Journal for Uncertainty Quantification2: 363-381. doi:10.1615/Int.J.UncertaintyQuantification.2012003562.
54. Oberkampf, W.L., and C. J. Roy. (2010). Verification and Validation in Scientific Computing. Cambridge University Press.
55. Regan, H.M., B.K. Hope, and S. Ferson (2002). Analysis and portrayal of uncertainty in a food web exposure model. Human and Ecological Risk Assessment8: 1757–1777.
56. Ferson, S., and W.T. Tucker (2004). Reliability of risk analyses for contaminated groundwater. Groundwater Quality Modeling and Management under Uncertainty, edited by S. Mishra, American Society of Civil Engineers Reston, VA.
57. Crespo, Luis G.; Kenny, Sean P.; Giesy, Daniel P. (2013). "Reliability analysis of polynomial systems subject to p-box uncertainties". Mechanical Systems and Signal Processing. 37 (1–2): 121–136. Bibcode:2013MSSP...37..121C. doi:10.1016/j.ymssp.2012.08.012.
58. Ferson, S., and M. Burgman (1995). Correlations, dependency bounds and extinction risks. Biological Conservation73: 101–105.
59. Ferson, S., D.R.J. Moore, P.J. Van den Brink, T.L. Estes, K. Gallagher, R. O'Connor and F. Verdonck. (2010). Bounding uncertainty analyses. Pages 89–122 in Application of Uncertainty Analysis to Ecological Risks of Pesticides, edited by W. J. Warren-Hicks and A. Hart. CRC Press, Boca Raton, Florida.
60. Kriegler, E., and H. Held (2005). Utilizing belief functions for the estimation of future climate change. International Journal of Approximate Reasoning39: 185–209.
61. Kriegler, E. (2005). Imprecise probability analysis for integrated assessment of climate change, Ph.D. dissertation, Universität Potsdam, Germany.
62. Batarseh, O.G.Y., (2010). An Interval Based Approach to Model Input Uncertainty in Discrete-event Simulation. Ph.D. dissertation, University of Central Florida.
63. Goldwasser, L., L. Ginzburg and S. Ferson (2000). Variability and measurement error in extinction risk analysis: the northern spotted owl on the Olympic Peninsula. Pages 169–187 in Quantitative Methods for Conservation Biology, edited by S. Ferson and M. Burgman, Springer-Verlag, New York.
64. Hayes, K.R. (2011).Uncertainty and uncertainty analysis methods: Issues in quantitative and qualitative risk modeling with application to import risk assessment ACERA project (0705). Report Number: EP102467, CSIRO, Hobart, Australia.
65. Zhang, H., R.L. Mullen, and R.L. Muhanna (2010). Finite element structural analysis using imprecise probabilities based on p-box representation. Proceedings of the 4th International Workshop on Reliable Engineering Computing (REC 2010).
66. Zhang, H., R. Mullen, R. Muhanna (2012). Safety Structural Analysis with Probability-Boxes. International Journal of Reliability and Safety6: 110–129.
67. Patelli, E; de Angelis, M (2015). "Line sampling approach for extreme case analysis in presence of aleatory and epistemic uncertainties". Safety and Reliability of Complex Engineered Systems. pp. 2585–2593. doi:10.1201/b19094-339. ISBN   978-1-138-02879-1.
68. Mehl, Christopher H. (2013). "P-boxes for cost uncertainty analysis". Mechanical Systems and Signal Processing. 37 (1–2): 253–263. Bibcode:2013MSSP...37..253M. doi:10.1016/j.ymssp.2012.03.014.
69. Sentz, K., and S. Ferson (2011). Probabilistic bounding analysis in the quantification of margins and uncertainties. Reliability Engineering and System Safety96: 1126–1136.
70. Rozell, Daniel J., and Sheldon J. Reaven (2012). Water pollution risk associated with natural gas extraction from the Marcellus Shale. Risk Analysis32: 1382–1393.
71. Lindley, D. V. (2006). Understanding Uncertainty . Hoboken, New Jersey: John Wiley & Sons. p.  71. ISBN   978-0-470-04383-7.
72. "Dennis Lindley - Wikiquote".
73. Baudrit, C., D. Dubois, H. Fargier (2003). Représentation de la connaissance probabiliste incomplète. Pages 65–72 in Actes Rencontres Francophones sur la Logique Floue et ses Applications (LFA'03), Tours, France. Cépaduès-Éditions.
74. Baudrit, C. (2005.) Représentation et propagation de connaissances imprécises et incertaines : Application à l'évaluation des risques liés aux sites et aux sols pollués. Ph.D. dissertation, Université Paul Sabatier, Toulouse III.
75. Troffaes, M.C.M., and S. Destercke (2011). Probability boxes on totally preordered spaces for multivariate modelling. International Journal of Approximate Reasoning (in press).
76. Meeker, W.Q., and L.A. Escobar (1998). Statistical Methods for Reliability Data, John Wiley and Sons, New York.

Additional references

• Baudrit, C., and D. Dubois (2006). Practical representations of incomplete probabilistic knowledge. Computational Statistics & Data Analysis51: 86–108.
• Baudrit, C., D. Dubois, D. Guyonnet (2006). Joint propagation and exploitation of probabilistic and possibilistic information in risk assessment. IEEE Transactions on Fuzzy Systems14: 593–608.
• Bernardini, A., and F. Tonon (2009). Extreme probability distributions of random/fuzzy sets and p-boxes. International Journal of Reliability and Safety3: 57–78. (alternative link)
• Destercke, S., D. Dubois and E. Chojnacki (2008). Unifying practical uncertainty representations – I: Generalized p-boxes. International Journal of Approximate Reasoning49: 649–663 .
• Dubois, D. (2010). (Commentary) Representation, propagation, and decision issues in risk analysis under incomplete probabilistic information. Risk Analysis30: 361–368. doi : 10.1111/j.1539-6924.2010.01359.x.
• Dubois, D., and D. Guyonnet (2011). Risk-informed decision-making in the presence of epistemic uncertainty. International Journal of General Systems40: 145–167.
• Guyonnet, D., F. Blanchard, C. Harpet, Y. Ménard, B. Côme and C. Baudrit (2005). Projet IREA—Traitement des incertitudes en évaluation des risques d'exposition. Rapport BRGM/RP-54099-FR, Bureau de Recherches Géologiques et Minières, France.