ABSTRACT

This book deals with the development of methodology for the analysis of truncated and censored sample data. It is primarily intended as a handbook for practitioners who need simple and efficient methods for the analysis of incomplete sample data.

chapter 1|1 pages

INTRODUCTION Preliminary Considerations

1.1 PRELIMINARY CONSIDERATIONS of the sample space are, depending on the nature of the restriction, It is perhaps more accurate

chapter 1|2 pages

2A HISTORICAL ACCOUNT

of modem of American trotting horses. Sample data were extracted from Wallace's

chapter |2 pages

of truncation

of truncation. of n observations, x T, where T is of truncation. of a total of N observations of which n are fully measured while c < T, whereas for each of Tis a fixed (known) of censoring. In Type II samples, T that is, the (c+ l)st of size N. of a total of N ob- > of I

chapter 1|1 pages

5 LIKELIHOOD FUNCTIONS

of an unrestricted (i.e., complete) distribution with parameters

chapter 2|1 pages

Singly Truncated and Singly Censored Samples from the Normal Distribution

2.1 PRELIMINARY REMARKS of estimators for

chapter |1 pages

21x-JL ] exp - oo<x<oo, (2.2.1)

(-cr-) f,.,

chapter |1 pages

) and ) are defined by (2.2.3), and of course = F(T).

of the standard normal distribution. 2.3 MOMENT ESTIMATORS FOR SINGLY TRUNCATED SAMPLES of the truncated normal population.

chapter x|5 pages

= T

= cr(Q -

chapter |9 pages

=.X-

2.4.2 An Illustrative Example Example 2.4.1. A complete sample of 40 observations was selected from of random observations from a normal population

chapter 2|1 pages

6 SAMPLING ERRORS OF ESTIMATES

of expected values of the second-order partial derivatives of the

chapter |4 pages

= whereas for Type I right censored

chapter |1 pages

of miles of service we have

of 12.00 of 50 units was selected from the screened pro- of Figure 2.1). fl and &are calculated from (2.3.9) and (2.3.12) as = + 0.00916(9.35- = 1.1907, rl = 9.35 - = 9.37. It follows that = 1.091 = (12.00 -= 2.41.

chapter 3|13 pages

Multirestricted Samples from the Normal Distribution

3.1 INTRODUCTION of the normal distribution are derived for of the complete distribution, the truncation points are = Tz -

chapter 3|2 pages

2.3 Illustrative Examples

= -2.50 = of curves of the two

chapter 3|4 pages

3 DOUBL V CENSORED SAMPLES

of (3.3.1), differentiate with respect to these parameters, and = 0.

chapter |2 pages

52.97671-l--0.000 026

-52.9767 1791.2545 3.4 PROGRESSIVELY CENSORED SAMPLES 2a i=I + 2: +

chapter |5 pages

of iterations required for a specified degree of accuracy will to

of which are considered later +A.) -1.0 -0.083 -0.205 -0.5

chapter |5 pages

yl), j = 1,

chapter |2 pages

of this statement is left as

chapter 4|1 pages

Linear Estimators

of error. In order to eliminate bias in small samples from dis-

chapter |2 pages

of significant proportions, and the elimination of bias more than compensates

of products of the complete (uncensored) ob- of right censored observations. It follows that N + n,

chapter |10 pages

of 9* is = = (v;). (4.3.11)

(W'V- It then follows that E'fX (lnV- -i and < J). = c= c, and 1V-E = E'V- It follows that

chapter |5 pages

= ( + ( + (

of the standard errors thus are = + + + + + =

chapter 5|2 pages

Truncated and Censored Samples from the Weibull Distribution

of prominence in the field of reliability and life testing where samples

chapter |1 pages

of sample specimens have failed, the

chapter |2 pages

+ cTIn T

chapter |1 pages

> I , the maximum likelihood estimating

8 are --"'- = 0,

chapter |5 pages

of a progressively censored sample is

+ (3 - + const. < and where

chapter |4 pages

It is, of course, necessary that we solve the three equations of (6.4.4) si-jl, d-). A straightforward trial-and-error iterative

of the first two of these equations are identical to the first two of (6.4.4). The third of the preceding equations, which in this case of (6.4.4), can be written in an expanded form as + + aE(Z

chapter 6|2 pages

6 ERRORS OF ESTIMATES

of an estimate of the threshold parameter of total size N should approximately equal the of a corresponding estimate from a complete sample. This result was

chapter 7|3 pages

Truncated and Censored Samples from the Inverse Gaussian and the Gamma Distributions

7.1 THE INVERSE GAUSSIAN DISTRIBUTION

chapter |1 pages

(x-"{) (x-

-exp- + const. (7.l.II) -,-,

chapter |2 pages

+ 3) -

-2 L

chapter 7|3 pages

.1.3 Maximum Likelihood Estimators for Truncated Samples

---'--'-- ----=0

chapter 7|2 pages

2.2 Maximum Likelihood Estimators for Censored Samples of a progressively censored sample from a gamma

2: ln(x; - > I, maximum likelihood estimating equations may be obtained by 1 ~ ~ c· aF

chapter |1 pages

= = = = +

= I, but it may sometimes be of these estimating equations > are

chapter |2 pages

a--- nl + ln(x;- y)- = 0,

jl(p)- aF(D

chapter |2 pages

of a

nplnl3 ~In[ I -

chapter |6 pages

= i,

Of course, Tis a lower bound on all and we

chapter 8|3 pages

Truncated and Censored Samples from the Exponential and the Extreme Value Distributions

8.1 THE EXPONENTIAL DISTRIBUTION 8.1.1 = 0, and in these cases the

chapter |1 pages

--.- ----

chapter |1 pages

= 0 for all j, = n and ST

[F(nJ '. of size n from a truncated dis- t 7) = Thus [expk

chapter |2 pages

of (8 .1.18) become

of and in small samples, the of the preceding equations is identical with the second equation

chapter |1 pages

of survivors immediately of a corresponding sample item. Estimates of the hazard or

= 100 ( , of the W eibull distribution is of both sides of the first equation of (8.1.31), we obtain of ln (x -of + = H- Ink ·

chapter 8|1 pages

2 THE EXTREME VALUE DISTRIBUTION

of rainfall, flood flow, earthquake, and other of material, cor- of extreme value is of limiting distributions, which approximate = exp [ - = exp (>0), and (>0) are parameters. of "extreme value" distributions. Many authors consider it to be "the"

chapter |3 pages

(1943). It was Gumbel who pioneered application

of the two-parameter We ibull distribution is o < < > o. > o. (8.2.4) of the Type I distribution of greatest extreme values is (8.2.5)

chapter |1 pages

> Let c (N -

of a sample as thus described from a distribution that is of least extreme values is * = +

chapter |2 pages

of these equations. With & determined from equation (8.2.18), we

(5.3.6) for calculating the Weibull estimate 8. of censoring cj items are removed (censored) from further -nina+ -'-a-

chapter 9|2 pages

Truncated and Censored Samples from the Rayleigh Distribution

9.1 INTRODUCTION of acoustical of which is normally distributed (0, u of the Rayleigh distribution (i.e., the pdf of X) follows as

chapter |4 pages

f(x; 1, 0') =

of variation, a of this = 0.7978850', = 0.3633800'

chapter |3 pages

= f -

chapter 0|5 pages

72 (z) and J 3 (z) 0.6

r-...

chapter 9|1 pages

6 SOME CONCLUDING REMARKS

of the of the estimators presented in Chapter 5 for Weibull of Weibull parameters might of Rayleigh

chapter 10|2 pages

Truncated and Censored Samples from the Pareto Distribution

10.1 of economics who formulated it ( 1897)

chapter |4 pages

of the Pareto distribution as ~ is a degenerate form of the two-parameter exponential distribution (8.1.1) in which

of the pdf and the cdf with a of this of a are included in Table 10.1 for selected values of this argument. More complete

chapter |1 pages

of the Pareto distribution, sample sizes

I. 96 , I + 1.96

chapter |1 pages

Maximum Likelihood Estimates of (10.3.6), we have 1010, and from

143.2632- of as It follows that = 20.66, and the approximate 95% CI is It is noted that differences between the MLE and the MMLE calculated from of its smaller bias. However, readers are again reminded that the only

chapter |1 pages

HETEROTYPIC

chapter 11|3 pages

5 DETERMINING THE DISTRIBUTION TYPE of distribution can be established from the original Pearson criteria, or from

= 0; of the same sign, D f.l;, D = and = a of Figure of Craig (1936).

chapter |2 pages

of H* suggests

of (11.4.1) plus (11.4.2) to obtain h* = 38.600670, of the sample data based on these Shook's Graduation For Right Singly For Complete Sample Truncated Sample 159.95 79.9 0 0.2 89.9 12.8

chapter |2 pages

of the quadratic form in the exponent is the of the variance-covariance matrix llaijll and has the positive determinant

of accepted specimens, and c = N - of rejected Nand care unknown. Only n, the number of acceptances, is known. In selected samples, full measurement of the screening ---------===----

chapter |2 pages

aL 1

chapter |2 pages

of the multivariate

of the associated variates. Accordingly, of Chapters 2 and 3 are applicable here just as they of size N. of maximum likelihood estimators for parameters of the multivariate normal

chapter |3 pages

Complete Truncated Censored Parameters Sample Estimates Estimates Estimates -1.379 -1.342 138.2353 138.4883 138.2376

Sample Sample 67.6664 67.7033 67.6794 1.7857 1.6927 1. 5235 1. 5172 1. 5318 0.5239 0.5265 0.5318 0.5446 0. 4872 0.4924 0.7339 0.7053 0.7037 N • 119 n "' 108 Asymptotic Variances* 3.377 1.836 0.695 1. 959 1.041

chapter 13|1 pages

Truncated and Censored Samples from Discrete Distributions

13.1 of zero are not observed. As an example, consider the distribution of the of children per family in developing nations, where records are maintained

chapter 13|2 pages

2.2 Singly Left Truncated Samples

l, 2, = -nA + = = 0, - = A [ + f(a 1)].

chapter |1 pages

dIn L = [[(a -1)_-

2 26 + (l = 0. (13.2.29)

chapter x|1 pages

+ S (

xis the mean of the n uncensored observations. 13.2.6 Doubly Censored Samples-Total Number of Censored Observations Known, But Not the Number in Each Tail Separately + IW. + P(d + + (1 f(a 1))].

chapter |3 pages

ni _ n [f(a - + f(d)]

= _ ni _ n [f(a - + [f(a + n [ !_(d) ] + I) 1 -+ (! ))<flnL ni

chapter |2 pages

+ k.

of all two-parameter /(0) It follows that = = =

chapter 0|1 pages

----

+ ni ---=

chapter 13|8 pages

3.2 An Illustrative Example

= 0.2113 and = = = 13.4 THE BINOMIAL DISTRIBUTION = 0, 1,2, = I - + n(n -

chapter |2 pages

of defectives found and the number of items inspected be recorded of the paired values (zy

of defectives found in the ith accepted lot (z; of defectives found in each rejected lot. This sample could be described of the paired values (z;, y;), i 1, 2, . . . , m

chapter 10|1 pages

, k 4, = 20, 5, 16

0.0000 7 0.0000 0.10 0.0128 7.74 0.10 0.0432 17.57 0.15 0.0450 8.09 0.15 0.1702 17.86 0.20 0.1209 8.35 0.20 0.3704 17.49 0.0000 0.01 0.0466 78.99 0.02 0.2156 74.70 0.0000 60 0.03 0.4319 68.02 0.01 0.0224 59.65 0.04 0.6252 60.54

chapter |1 pages

= = 2, K= =

100, k 120, 0.0000 0 0.0000 0.01 0.0794 97.76 0.01 0.0330 119.1 0.02 0.3233 89.37 0.02 0.2200 112.9 0.03 0.5802 77.94 0.03 0.4867 101.4

chapter 14|2 pages

2.1 Misclassification in the Poisson Distribution

of x + 1 were reported as x = k with probability of defects per item, becomes + 2) + + 1)], x + 1)!, x k + 1, = 1, 2,

chapter |1 pages

of (14.2.3) gives

chapter |2 pages

0.189 + (2.092)(0.393) = o.o

chapter |2 pages

+ 1 ] 2 " : nN

chapter |1 pages

PiJ.e--

14.2.6 An Illustrative Example-Misclassified Binomial Data 14.3 An example generated by Cohen ( 1960a) consisted of N of defectives in samples of n 40 from a = of + = = 0.00000075, V(e) = 0.0017, Cov({J, e) = 0.0000025, and = 0.07.

chapter 14|1 pages

3 INFLATED ZERO DISTRIBUTIONS

chapter |1 pages

= ln[I-w(l-

+ nlnw = 2: x;ln,

chapter |3 pages

of L, differentiate with respect to the parameters, and

---+-=0 e e ·

chapter 5|3 pages

+ Total

of(l4.3.15), we calculate = 23/56 = 0.4107, and from the = = = = of Neyman's contageous distributions and they calculated expected fre-

chapter |30 pages

Appendix: Tables

Distribution Functions

chapter |3 pages

:'\a3

1.5 1.6 3.4 .991473 .990795

chapter |1 pages

Glossary

chapter |1 pages

THE GREEK ALPHABET

ll 3 H eta K kappa A lambda N v nu

chapter |1 pages

functions-

chapter |1 pages

of a

of censored observations in of observations censored at of variation = of of the W eibull shape pa- of the gamma function.

chapter |2 pages

of censored observations

chapter 286|2 pages

Glossary

chapter |1 pages

Bibliography

of Poisson, binomial and negative

chapter |1 pages

of contageous distributions when ap-

of fitting the truncated negative binomial of the parameters of the distribution-a reconsideration. Austral. J. Statist., 3, 185-190.

chapter C|2 pages

, and Whitten, B. (1983) The standardized inverse

of Michigan, Ann Arbor. of truncated

chapter L|4 pages

and Moore, A. H. (1967) Asymptotic variances and covariances

7. Kotz, S., Johnson, N. L., and Read, C. B., eds. Wiley, New York, of estimating the mean and standard

chapter |2 pages

This page intentionally left blank