On the Inverted Gamma Distribution
Salah H. Abid^{*}, Saja A. Al-Hassany
Mathematics Department, Education College, Al-Mustansiriya University, Baghdad, Iraq
Email address:
To cite this article:
Salah H. Abid, Saja A. Al-Hassany. On the Inverted Gamma Distribution. International Journal of Systems Science and Applied Mathematics. Vol. 1, No. 3, 2016, pp. 16-22. doi: 10.11648/j.ijssam.20160103.11
Received: August 29, 2016; Accepted: September 14, 2016; Published: September 28, 2016
Abstract: If a random variable follows a particular distribution then the distribution of the reciprocal of that random variable is called inverted distribution. In this paper we studied some issues related with inverted gamma distribution which is the reciprocal of the gamma distribution.We provide forms for the characteristic function, rth raw moment, skewness, kurtosis, Shannon entropy, relative entropy and Rényi entropy function. This paper deals also with the determination of R = P[Y < X] when X and Y are two independent inverted gamma distributions (IGD) with different scale parameters and different shape parameters. Different methods to estimate inverted gamma distribution parameters are studied, Maximum Likelihood estimator, Moments estimator, Percentile estimator, least square estimator and weighted least square estimator. An empirical study is conducted to compare among these methods.
Keywords: Inverted Gamma Distribution, Characteristic Function, Stress-Strength, Shannon Entropy, Relative Entropy, Rényi Entropy, MLE, Percentile Estimator
1. Introduction
The inverted gamma distribution is a two-parameter family of continuous probability distributions on the positive real line, which is the distribution of the reciprocal of a variable distributed according to the gamma distribution. Perhaps the chief use of the inverted gamma distribution is in Bayesian statistics, where the distribution arises as the marginal posterior distribution for the unknown variance of a normal distribution if an uninformative prior is used; and as an analytically tractable conjugate prior if an informative prior is required.
However, it is common among Bayesians to consider an alternative parameterization of the normal distribution in terms of the precision, defined as the reciprocal of the variance, which allows the gamma distribution to be used directly as a conjugate prior. Other Bayesians prefer to parameterize the inverted gamma distribution differently, as a scaled inverse chi-squared distribution.
Giron and Castillo [4] in 2001 defined the generalized Behrens-Fisher distribution is as the convolution of two Student distributions and is related to the inverted-gamma distribution by means of a representation theorem as a scale mixture of normals where the mixing distribution is a convolution of two inverted-gamma distributions. One important result in this paper establishes that for odd degrees of freedom the Behrens-Fisher distribution is distributed as a finite mixture of Student distributions. This result follows from the main theorem concerning the form of the convolution of inverted-gamma distributions with demi-integer shape parameter.
Witkovsky in 2001 [9] presented a formula for evaluation of the distribution of independent inverted Gamma random variables by one dimensional numerical integration. This method is applied to computation of the generalized p-values used for exact significance testing and interval estimation of the parameter of interest in the Behrens-Fisher problem and for variance components in balanced mixed linear model.
Li et al. in 2008 [6] studied the geometric structure of the inverse Gamma manifold from the viewpoint of information geometry and give the Kullback divergence, the J-divergence and the geodesic equations. Also, some applications of the inverse Gamma distribution are provided.
Ali et al. in 2008 [3] defined skew-symmetric distributions based on the double inverted.
Gamma, double inverted Weibull and double inverted compound gamma distributions, all of which have symmetric density about zero. Expressions are derived for the probability density function (pdf), cumulative distribution function (cdf) and the moments of these distributions. They referred that some of these quantities could not be evaluated in closed forms and they used special functions to express them.
Woo in 2012 [10] derived distributions of ratio for two independent gamma variables and two independent inverted gamma variables and then we observe the skewness of two ratio densities. We then consider inference on reliability in two independent gamma random variables and two independent inverted gamma random variables each having known shape parameters.
Abdulah and Elsalloukh in 2012 [1] introduced a new class of asymmetric probability densities, the Epsilon Skew Inverted Gamma (ESIG) distribution. They applied it to analyze skewed and bimodality data. In 2014 [2] the same authors presented basic properties of this distribution, such as the pdf, cdf, and moments are presented. addition, computational forms of parameters estimation of MLE and MME are used. Finally, they illustrated the theory of ESIG distribution by modeling some real data.
Llera and Beckmann in 2016 [7] introduced five different algorithms based on method of moments, maximum likelihood and full Bayesian estimation for learning the parameters of the Inverse Gamma distribution. They also provided an expression for the KL divergence for Inverse Gamma distributions which allows us to quantify the estimation accuracy of each of the algorithms. All the presented algorithms in this paper are novel. The most relevant novelties include the first conjugate prior for the Inverse Gamma shape parameter which allows analytical Bayesian inference, and two very fast algorithms, a maximum likelihood and a Bayesian one, both based on likelihood approximation. In order to compute expectations under the proposed distributions, they used the Laplace approximation. The introduction of these novel Bayesian estimators opens the possibility of including Inverse Gamma distributions into more complex Bayesian structures, e.g. variational Bayesian mixture models. The algorithms introduced in this paper are computationally compared using synthetic data and interesting relationships between the maximum likelihood and the Bayesian approaches are derived.
The probability density function (PDF) and the cumulative distribution function (CDF) for the inverted gamma random variable are respectively,
(1)
(2)
Where >0 is the scale parameter and >0 is the shape parameter. , is the upper incomplete gamma function and is complete gamma function.
In this paper we will refer to Inverted Gamma distribution by , which is mean that the random variable follow Inverted Gamma distribution with parameters and .
The reliability function and hazard rate function of are respectively,
=, then
(3)
(4)
where is the lower incomplete gamma function
The rth raw moment of can be obtained as,
E() ==== = (5)
Then, the mean and variance of random variable are respectively,
(6)
(7)
The mode of is obtained as follows,
(8)
The skewness and The excess kurtosis are respectively
(9)
(10)
The characteristic function of is,
(11)
2. Shannon and Relative Entropies
An entropy of a random variable is a measure of variation of the uncertainty. The Shannon entropy of random variable can be found as follows,
=
=, then
=
=, then,
,
H=
=
=
And =(α), then
(12)
Rѐnyi entropy for a random variable can be derived as,
=
Now, since =, then,
=
=
=
=
(13)
The relative entropy (or the Kullback–Leibler divergence) is a measure of the difference between two probability distributions and . It is not symmetric in and . In applications, typically represents the "true" distribution of data, observations, or a precisely calculated theoretical distribution, while typically represents a theory, model, description, or approximation of . Specifically, the Kullback–Leibler divergence of from , denoted D_{KL}(‖), is a measure of the information gained when one revises ones beliefs from the prior probability distribution to the posterior probability distribution. More exactly, it is the amount of information that is lost when is used to approximate ,defined operationally as the expected extra number of bits required to code samples from using a code optimized for rather than the code optimized for .
So, relative entropy for a random variable can be found as follows,
=
Since, = and =, then,
==, then,
=
=
=
(14)
3. Stress-Strength Reliability
Inferences about R = P[Y < X], where X and Y are two independent random variables, is very common in the reliability literature. For example, if X is the strength of a component which is subject to a stress Y, then R is a measure of system performance and arises in the context of mechanical reliability of a system. The system fails if and only if at any time the applied stress is greater than its strength.
Let Y and X be the stress and the strength random variables, independent of each other, follow respectively and then,
=
=
, and
R=
= then,
R=
=
=
=
=, then,
R=
(15)
, R can be found as follows as a second way,
=
=
Since, =
=
=
=
(16)
4. Parameters Estimation of Inverse Gamma Distribution
The main aim of this section is to study different estimators of the unknown parameters of IG distribution.
4.1. The Maximum Likelihood Estimator (MLE)
If is a random sample from , then the likelihood and log likelihood functions are respectively,
=
=
(17)
(18)
(19)
=
=
=
(20)
once we get α ̂_MLE numerically from (20) we substitute it' s value in (19) to get β ̂_(MLE )
4.2. The Exact Method of Moments Estimator (EMME)
Here we provide the method of moments estimators of the parameters of a (IG) distribution when both are unknown.
Since the mean and variance of X which is follow IG (, are defined in (6) and (7) respectively, then the coefficient of variation is,
=
The CV is independent of scale parameter β,
By equating the sample , we obtain:
=
(21)
=
by substituting (21) in (6) we get the EMME of as follows,
(22)
4.3. The Approximate Method of Moments Estimators (AMME)
Since the mean and mode of X which is follow IG (, are defined in (6) and (8) respectively, then
(23)
= = , then,
(24)
is independent of the scale parameter , then, after calculating the sample mode and the sample mean and substituting their values in (24), One can get the AMME of , say
(25)
by substituting (25) in (6) we get the AMME of as follows:
(26)
4.4. Estimators Based on Percentiles (PE)
Kao in (1959) [5] originally explored this method by using the graphical approximation to the best linear unbiased estimators. The estimators can be obtained by fitting a straight line to the theoretical points obtained from the distribution function and the sample percentile points. In the case of a IG distribution, it is possible to use the same concept to obtain the estimators of α and β based on percentiles because of the structure of its distribution function.
Firstly, we find numerically the value of x=F^ (-1) (pi, α, β), where F is defined in (2) and pi is the estimate of , then can be obtained by minimizing
(27)
Equation (d.1) is a nonlinear function of and .It is possible to use some nonlinear regression techniques to estimate and simultaneously, where = is the most used estimator of
4.5. Least Squares Estimator (LSE)
This method was originally suggested by Swain, Venkatraman and Wilson (1988) [8] to estimate the parameteres of beta distribution. Therefore in the case of IG distribution, the least squares estimators of , Say respectively, can be obtained by minimizing,
(28)
With respect to
4.6. Weighted Least Squares Estimators (WLSE)
The weighted least squares estimators of say can be obtained
(29)
5. The Empirical Study and Discussions
We conduct extensive simulations to compare the performances of the different methods, stated in section 4, for estimating unknown parameters of Inverted Gamma distribution, mainly with respect to their mean square errors (MSE) for different sample sizes and for different parameters values.
The experiments are conducted according to run size . We reported the results for (small sample), (moderate sample) and (large sample) and for the following different values of and ,
| 0.6 | 1 | 0.9 | 1.2 | 0.3 |
| 1 | 0.6 | 0.9 | 0.3 | 1.2 |
The results are reported in table (1). From the table, we observe that,
1) The MSE's decrease as sample size increases in all methods of estimation. It verifies the asymptotic unbiasedness and consistency of all the estimators.
2) It can be said that the estimation of scale parameters are more accurate for the smaller values of those parameters whereas the estimation of shape parameters are more accurate for the larger values of those parameters. in other words, MSE's increase as scale parameter increases whereas MSE's increase as shape parameter decreases.
3) The performances of LSE, EMME and AMME are according to their order.
4) The performances of EMME's and AMME's are close to each other.
5) For small (n=10) sample size and moderate (n=20) sample size, it is observed that PE works the best for both of the two parameters whereas the second best method is MLE.
6) For large (n=50, 100) sample size, it is observed that MLE works the best from all other methods to estimate the shape parameter whereas the second best method is PE. PE works the best from all other methods to estimate the scale parameter whereas the second best method is MLE.
Table 1. Empirical MSE to estimate the IG distribution parameters and .
6. Summary and Conclusions
In view of the great importance of Gamma distributions in statistical analysis, the inverted gamma distribution (IGD) is considered here. For IGD we derived exact formulas of hazard function, characteristic function, rth raw moment, skewness, kurtosis, Shannon entropy function, relative entropy, quantile function and stress-strength reliability. Different methods to estimate inverted gamma distribution parameters are studied, Maximum Likelihood estimator, Moments estimator, Percentile estimator, least square estimator and weighted least square estimator. An empirical study was conducted to compare among these methods. It seemed to us that the Percentile estimator is the best one for small and moderate samples and it is also the best to estimate the scale parameter for large samples, whereas the maximum likelihood estimator is the best to estimate the shape parameter for large samples.
References