Method of moments for estimating uncertainty distributions

Uncertainty theory is a branch of mathematics for modeling human uncertainty, and uncertain statistics is a methodology for collecting and interpreting expert’s experimental data by uncertainty theory. In order to estimate uncertainty distributions via experts’ experimental data, this paper will present a method of moments and design a numerical method to find moment estimate of unknown parameters.

Probability theory is a branch of mathematics concerned with analysis of random phenomena. The probability distribution describes the range of possible values and the probability of a random variable. In order to determine the probability distribution, random statistics (i.e., the classical mathematical statistics) was proposed as a methodology for collecting and interpreting the test data with correlative information in a system by probability theory. On the other hand, the fuzzy set theory via membership function initiated by Zadeh [1] is a mathematical model of vague qualitative or qualitative data, frequently generated by means of natural language. In order to determine the membership function, fuzzy statistics was presented including fuzzy point estimation by a fuzzy decision-making approach and so on.

However, a lot of surveys showed that some information and knowledge represented usually by human language like ‘about 1,000 km,’ ‘roughly 60 kg,’ ‘high speed,’ and ‘small size’ are neither randomness nor fuzziness. When the sample size is too small (even no-sample) to estimate a probability distribution, we have to invite some domain experts to evaluate their belief degree that each event will occur. Since human beings usually overweigh unlikely events, the belief degree may have much larger variance than the real frequency. Perhaps some people think that the belief is a subjective probability or fuzzy membership degree. However, the examples [2] tell us that it is inappropriate because probability theory and fuzzy set theory may lead to counterintuitive results in this case.

In order to model this type of imprecise quantities, uncertainty theory was founded by Liu [3] in 2007 and refined by Liu [4] in 2010 and became a branch of mathematics based on the normality, duality, subadditivity, and product axioms. Based on Liu’s uncertainty theory, some basic and important theoretical work of uncertainty theory such as uncertain process [5], uncertain calculus [6], uncertain differential equation [6], uncertain logic [7], uncertain inference [8], and uncertain risk analysis [9] have been established. Meanwhile, as an application of uncertainty theory, Liu [10] proposed a spectrum of uncertain programming and applied uncertain programming to system reliability design, facility location problem, vehicle routing problem, project scheduling problem, and so on. Other references related to uncertainty theory are Gao [11], Gao et al. [12], Peng and Iwamura [13], and Liu and Ha [14]. Nowadays, uncertainty theory was well developed on both theory section and practice section. For exploring the recent developments of uncertainty theory, the readers may consult the book [15].

One important issue in uncertainty theory is how to determine uncertainty distribution of an uncertain variable. In order to answer this question, uncertain statistics was firstly presented by Liu [4] in 2010, and is a methodology for collecting and interpreting expert’s experimental data by uncertainty theory. In uncertain statistics, Liu [4] first suggested an empirical uncertainty distribution and proposed a principle of least squares as the method for estimating the unknown parameters based on the expert’s experimental data. After that, Chen and Ralescu [16] employed uncertain statistics to estimate the travel distance between Beijing and Tianjin. In 2012, Wang et al. recast the Delphi method as a process to determine the uncertainty distributions [17] and presented a method of uncertain hypothesis testing for determining if the views of two domain experts are identical (i.e., wether or not they have the same uncertainty distribution) [18].

Based on experts’ experimental data and the empirical uncertainty distribution, the k th sample moment is defined in this paper. Then, a method of moments for estimating the unknown parameters of uncertainty distribution is proposed. The remainder of this paper is organized as follows. The next section is intended to introduce some concepts in uncertainty theory as they are needed. Some basic concepts of uncertain statistics are introduced in Section ‘Beginning of uncertain statistics.’ The uncertain moment method for estimating parameters is proposed in Section ‘Method of moments.’ A numerical method to find the moment estimate of unknown parameters is designed in Section ‘Numerical method.’ Finally, a conclusion is drawn in Section ‘Conclusions.’

In this section, we will introduce some useful definitions about uncertain measure, uncertain variable, uncertain moment, and so on.

Let Γ be a nonempty set, and L be a σ-algebra over Γ. Each element Λ∈L is called an event. A number M(Λ) indicates the level that Λ will occur. Uncertain measure M was introduced as a set function satisfying the following three axioms (Liu [3]):

Axiom 1. M{Γ}=1.

Axiom 2. M{Λ}+M{Λ^c}=1 for any event Λ.

Axiom 3. For every countable sequence of events {Λ _i }, we have

Although probability measure satisfies the above three axioms, probability theory is not a special case of uncertainty theory because product probability measure does not satisfy the following product axiom (Liu [3]):

Axiom 4. Let (Γ _k ,L _k ,M _k ) be uncertainty spaces for k=1,2,…. The product uncertain measure M is an uncertain measure satisfying

where Λ _k are arbitrarily chosen events from L _k for k=1,2,…, respectively.

The concept of uncertain variable was introduced by Liu [3] as a measurable function from an uncertainty space (Γ,L,M) to the set of real numbers. The expected value operator of an uncertain variable was defined by Liu [3] as

provided that at least one of the two integrals is finite. The k th moment of an uncertain variable ξ is defined by E[ξ^k], where k is a positive integer.

For any x∈ℜ, the function Φ(x)=M{ξ≤x} is called the uncertainty distribution of uncertain variable ξ, Peng and Iwamura [13] presented a sufficient and necessary condition of uncertainty distribution that a function Φ:ℜ→[ 0,1] is an uncertainty distribution if and only if it is an increasing function except Φ(x)≡0 and Φ(x)≡1. Moreover, if the inverse function Φ⁻¹(α) exists and is unique for each α∈(0,1), then Φ(x) is called regular, and the inverse function Φ⁻¹(α) is called the inverse uncertainty distribution of ξ. If an uncertain variable ξ has regular distribution Φ(x) and its expected value exists, then the expected value may be presented by the following formula:

In addition, Liu [4] proved a series of operational laws of independent uncertain variables for calculating the uncertainty distribution of monotone function of uncertain variables. By these operational laws, the uncertainty distributions of sum, difference, product, quotient, maximum, and minimum for n uncertain variables are easy obtained. The detail contents are shown in [4] or [15].

Uncertain statistics is based on the expert’s experimental data rather than historical data. One question is how to obtain expert’s experimental data. Liu [4] designed a questionnaire survey for collecting expert’s experimental data, that is, we invite one or more domain experts who are asked to complete a questionnaire about the meaning of an uncertain variable ξ like ‘about 1,000 km’ individually.

We first ask the domain expert to choose a possible value x that the uncertain variable ξ may take. Then, we quiz him ‘how likely is ξ less than x?’ and denote his belief degree by α. Thus, we obtain an expert’s experimental data (x, α) from the domain expert. Repeating the above process, we obtain the expert’s experimental data.

Let (x₁, α₁),(x₂, α₂),…,(x _n , α _n ) be the expert’s experimental data that meet the following condition:

Based on the above data, Liu [4] presented the following empirical uncertainty distribution:

Assume that an uncertainty distribution to be determined has a known functional form with one or more unknown parameters like Φ(x;θ₁, θ₂,⋯, θ _p ) where θ₁, θ₂,⋯, θ _p are unknown parameters. How do we estimate those unknown parameters? Liu [4] presented the principle of least squares, which says that the unknown parameters θ _i , i=1,2,…, p are the solution of the minimization problem,

Let ξ be an uncertain variable with uncertainty distribution Φ(x;θ₁, θ₂,…, θ _p ) where θ₁, θ₂,…, θ _p are unknown parameters. If the uncertainty distribution Φ(x _i ;θ₁, θ₂,…, θ _p ) is regular, then Φ⁻¹ is the inverse uncertainty distribution of ξ. Assume that we have some expert’s experimental data (x₁, α₁),(x₂, α₂),…,(x _n , α _n ) satisfying the condition (1). Then, for any α _i , we have M{ξ≤Φ⁻¹(α _i )}=α _i , i=1,2,…, n. It means that the principle of least squares can be changed into the following form:

where the unknown parameters θ _i , i=1,2,…, p are the solution of the minimization problem.

If the inverse uncertainty distribution of uncertain variable ξ is simple and easy to calculate, we should solve the optimization problem (4) rather than the optimization problem (3) to estimate the unknown parameters. For example, let (x₁, α₁),(x₂, α₂),⋯,(x _n , α _n ) be the expert’s experimental data, and the uncertain variable ξ to be determined be a normal uncertain variable, which has the following uncertainty distribution:

where e and σ are two unknown parameters. The inverse uncertainty distribution of normal uncertain variable ξ is

By solving the optimization problem (4), it is easy to obtain the following estimate values:

and

where \bar{x}=(x_1+x_2+\cdots +x_n)/n.

In this section, a method of moments based on expert’s experimental data is proposed to estimate the unknown parameters. Firstly, we present the k th moment of the empirical uncertainty distribution (2), which is the uncertainty distribution of some uncertain variable ξ.

Theorem 1. Let (x _i , α _i ), i=1,2,…, n be the expert’s experimental data that meet the following condition:

Then for any positive integer k, the uncertain variable ξ with the empirical uncertainty distribution (2) has the k th moment

Proof. Since 0≤x₁<x₂<⋯<x _n , by (2), we have

The theorem is proved.

Definition 1. Let ξ be an uncertain variable. For any positive integer k , if the k th moment of ξ exists, then E[ξ^k] is called the k th theoretical moment of ξ.

Definition 2. Let (x₁, α₁),(x₂, α₂),⋯,(x _n , α _n ) be the expert’s experimental data that meet the condition (5). For any positive integer k,

is called the k th sample moment.

Secondly, we introduce a method of moments for estimating the unknown parameter, the description of the method is presented as follows.

Method of moments Let ξ be an uncertain variable with uncertainty distribution Φ(x;θ₁, θ₂,…, θ _p ) where θ₁, θ₂,…, θ _p are unknown parameters. Let (x₁, α₁), (x₂, α₂),…,(x _n , α _n ) be the expert’s experiment data satisfying the condition (5). Let E[ξ^k] and \overline{{\xi }^k},k=1,2,\dots ,p be the k th theoretical and k th sample moments, respectively. As a general procedure for estimating the unknown parameters θ₁, θ₂,…, θ _p , a system of equations is presented:

Since E\left[{\xi }^k\right]=k{\int }_0^{+\infty }{x}^{k-1}(1-\Phi (x;{\theta }_1,{\theta }_2,\dots ,{\theta }_p\left)\right)\mathit{\text{dx}}, the above system of equations is equivalent to the following form:

A solution of the equations, θ₁, θ₂,…, θ _p , is called a moment estimate. We denote the estimate values by \widehat{{\theta }_1},\widehat{{\theta }_2},\dots ,\widehat{{\theta }_p}, respectively.

Example 1.Assume (x _i , α _i ), i=1,2,…, n are the expert’s experimental data satisfying the condition (5) and the uncertainty distribution of uncertain variable ξ has a functional form with one unknown parameter a as follows:

By Equation 8, the estimate value of the unknown parameter a is the solution of the equation

where \bar{\xi } is defined by Equation 7. Since E\left[\xi \right]=\frac{1}{3a^2}, we have

Hence, we obtain the estimate value of the unknown parameter a,

Assume that the expert’s experiment data are obtained as follows:

By Equation 12, we have the moment estimate value

and the moment estimate distribution is

Example 2.Let ξ be an uncertain variable with uncertainty distribution

where a, b are two unknown parameters. Assume (x _i , α _i ), i=1,2,…, n are the expert’s experimental data satisfying the condition (5). By Equation 8, we will solve the following system of equations:

where \bar{\xi } and \overline{{\xi }^2} are defined by Equation 7. In addition, the inverse uncertainty distribution is {\Phi }^{-1}\left(\alpha \right)=\frac{\alpha -b}{a}. We have

and

Thus, we have the following system of equations

Then the unique solution of the above equations

is the moment estimate value of unknown parameters a and b, and the moment estimate distribution is

Assume that we have the following expert’s experimental data:

From Equations 7 and 14, we have moment estimate values â=0.2442, \widehat{b}=-0.0519 and the moment estimate distribution is Φ(x)=0.2442x−0.0519.

In this section, we introduce Newton’s method to estimate the moment estimate value. A general algorithm is proposed to solve the parameter estimate problem as follows.

Assume (x _i , α _i ), i=1,2,…, n are the expert’s experimental data satisfying the condition (5) and Φ(x;θ₁, θ₂,…, θ _p ) is the uncertainty distribution of uncertain variable ξ with unknown parameters θ₁, θ₂,…, θ _p . The numerical method is to solve the Equations 8,

where \bar{\xi },\overline{{\xi }^2},\dots ,\overline{{\xi }^p} are the sample moments, which are constants calculated by Equation 7.

Denote a function F:{ℝ}^p\to {ℝ}^p,

where

The Equation 8 is equivalent to F(θ₁, θ₂,…, θ _p )=0.

If the function F is differentiable and the Jacobian matrix of F is easily computed, the procedure can be summarized as follows:

Step 1. Estimate the k th sample moments by Equation (7), k=1,2,…, p.

Step 2. Give an initial value of parameters {\theta }^0=\left({\theta }_1^0,{\theta }_2^0,\dots ,{\theta }_p^0\right).

Step 3. Compute the converse matrix J _F (θ^k)⁻¹ of the Jacobian matrix J _F (θ^k), k=0,1,2,…, where

The integrals in the equation can be computed by some numerical methods.

Step 4. Calculate θ^k+1 by the following equation:

Step 5. Repeat the third to fourth steps when |θ^k+1−θ^k|>α and k<M, where α and M are given positive numbers.

Step 6. Report the last θⁿ as the solution, if |θⁿ−θⁿ⁻¹|≤α.

Remark 1.If the distribution Φ(x;θ₁, θ₂,⋯, θ _p ) is continuous differentiable of the parameters, we have

Remark 2.If the function F is not differentiable or the Jacobian matrix is hard to compute, we can compute

instead of

where i, j=1,2,…, p, and h₁, h₂,…, h _p are given small positive numbers.

Example 3.Assume that an uncertainty distribution has a lognormal form with two unknown parameters e and σ, that is,

and the expert’s experimental data are (0.6, 0.1), (1.0, 0.3), (1.5, 0.4), (2.0, 0.6), (2.8, 0.8), (3.6, 0.9). We have

and

where f\left(x\right|e,\sigma )=exp\left(\frac{\pi (e-lnx)}{\sqrt{3}\sigma }\right). All of these integrals are computed by Simpson formula, and the initial value is (0.6,0.5).

A run of the algorithm shows that the approximate solution of the equation F(e, σ)=0 is ê=0.50 and \widehat{\sigma }=0.47, which leads to the moment estimate distribution

Uncertain statistics is a methodology for collecting and interpreting expert’s experimental data by uncertainty theory. The method of moments in this paper is a new way to estimate the unknown parameters in uncertainty distribution. This method depends on the expert’s experimental data and is easy to be realistic in the real experiment.

This work was supported by National Natural Science Foundation of China (No. 61073121) and Hebei Natural Science Foundation (No. G2013402063 and No. F2012402037).

Affiliations

1. School of Science, Hebei University of Engineering, Handan, 056038, China

   • Xiaosheng Wang

2. School of Applied Mathematics, Central University of Finance and Economics, Beijing, 100081, China

   • Zixiong Peng

Corresponding author

Correspondence to Xiaosheng Wang.

Reprints and Permissions