序贯概率比检验Sequential Probability Ratio Test

序贯概率比检验(Sequential probability ratio test，SPRT)

什么是序贯概率比检验

数理统计学的一个分支，其名称源出于亚伯拉罕·瓦尔德在1947年发表的一本同名著作，它研究的对象是所谓“序贯抽样方案”，及如何用这种抽样方案得到的样本去作统计推断。序贯抽样方案是指在抽样时，不事先规定总的抽样个数(观测或实验次数),而是先抽少量样本,根据其结果，再决定停止抽样或继续抽样、抽多少，这样下去，直至决定停止抽样为止。反之，事先确定抽样个数的那种抽样方案，称为固定抽样方案。

例如，一个产品抽样检验方案规定按批抽样品20件，若其中不合格品件数不超过 3，则接收该批，否则拒收。在此，抽样个数20是预定的，是固定抽样。若方案规定为：第一批抽出3个，若全为不合格品，拒收该批,若其中不合格品件数为x1<3，则第二批再抽3-x1个,若全为不合格品，则拒收该批，若其中不合格品数为 x2<3-x1,则第三批再抽3-x1-x2个,这样下去,直到抽满20件或抽得 3个不合格品为止。这是一个序贯抽样方案，其效果与前述固定抽样方案相同，但抽样个数平均讲要节省些。此例中，抽样个数是随机的，但有一个不能超过的上限20。有的序贯抽样方案，其可能抽样个数无上限，例如，序贯概率比检验的抽样个数就没有上限。

 

Sequential Probability Ratio Test for Reliability Demonstration

 

The Sequential Probability Ratio Test (SPRT) was developed by Abraham Wald more than a half century ago [​​1​​]. It is widely used in quality control in manufacturing and detection of anomalies in medical trials. In this article, we will explain the theory behind this method and illustrate its use in reliability engineering, especially in reliability demonstration test design. An example using the SPRT report template in ​​Weibull++​​ is provided.

SPRT Theory

SPRT was originally developed as an inspection tool to determine whether a given lot meets the production requirements. Basically, a sequential test is a method by which items are tested in sequence (one after another). The test results are reviewed after each test. Two tests of significance are applied to the data accumulated up to that time.

Concept of SPRT

Let's first use a simple example to explain the principal behind SPRT. Two vendors provide the same component to a company. Although the components from the two companies look exactly the same, their lifetime distributions are different. Components from vendor A have a mean life of μ₁ = 15, and components made by vendor B have a mean life of μ₂ = 20. An unlabeled box of components was received by the company. We want to determine if the components are from vendor A or from vendor B by conducting a test. The test should meet the following requirements:

• If the component is indeed from vendor A, the chance of making a wrong claim that it is from vendor B should be less than α₁ = 0.01.
• If the component is indeed from vendor B, the chance of making a wrong claim that it is from vendor A should be less than α₂ = 0.05.

Therefore, we need to conduct two statistical hypothesis tests. Since we know μ₂ > μ₁, the two tests are one-sided tests. The first test is for vendor A:

The second one is for vendor B:

These two separate hypothesis tests are shown graphically below:

The top plot is for the first hypothesis test (vendor A). C₁ is the critical value at a significance level of α₁. If we take some samples and the sample mean is less than C₁, then we accept , which is that the components are from vendor A. Otherwise, we accept , that the components are not from vendor A.

The bottom plot is for the second test (vendor B). C₂ is the critical value at a significance level of α₂. If a sample mean is greater than C₂, then we accept  that the component is from vendor B; otherwise, we accept , that the component is not from vendor B.

When we take samples for the life test, the resulting sample mean has one of the following values:

• Assume a sample with mean  was drawn. For the test for vendor A, since it is less than C₁, we accept that μ = μ₁. For the test for vendor B, since it is less than C₂, we accept that μ < μ₂. The test is ended and we conclude that the component is from vendor A.
• Assume a sample with mean  was drawn. For the test for vendor A, since it is greater than C₁, we reject that μ = μ₁. For the test for vendor B, since it is greater than C₂, we accept that μ = μ₂. The test is ended and we conclude that the component is from vendor B.
• Assume a sample with mean  was drawn. For the test for vendor A, since it is less than C₁, we accept that μ = μ₁. For the test for vendor B, since it is greater than C₂, we accept that μ = μ₂. We conclude the component is from both vendor A and vendor B, which is impossible. Therefore, the test is not ended and more samples are needed.

With more and more samples, the sample mean will be closer to the true population mean. The test will end with a conclusion either from vendor A or from vendor B. This is the principal behind a sequential test. A sequential probability ratio test is based on this idea.

Calculation of SPRT

Now assume the lifetime t of the component follows an exponential distribution. Let θ_A = μ_A for vendor A and θ_B = μ_B for vendor B. The probability density function (pdf) of the exponential distribution is:

(1)

For an observed failure time t, if it is from vendor A, then the “probability” of observing it is:

(2)

where Δt is a very small time duration around t.

If the observation is from vendor B, then the “probability” of observing it is:

(3)

If the component is from vendor A, then Eqn. (2) will likely have a larger value than the one given in Eqn. (3), and vice versa.

The logarithm of the ratio of the above two probabilities is given by:

(4)

When there are more samples, the log-likelihood ratio becomes:

(5)

If the ratio is greater than a critical value U, then the chance that the samples are from vendor B is much larger than the chance that the samples are from vendor A. We can conclude that the samples are from vendor B.

If the ratio is less than a critical value L, then the chance that the samples are from vendor A is much larger than the chance that the samples are from vendor B. We can conclude that the samples are from vendor A.

If the ratio is between L and U, then no conclusion can be made. More samples are needed. The decision is made based on the following formula:

(6)

But what are the values for U and L? U and L are determined based on the two significance levels α₁ and α₂. The significance level is also called a Type I error. For details on Type I and Type II errors, please refer to ​​When the ratio is less than L, weaccept vendor A:

(7)

When we accept vendor A, the probability of making the right decision (the component is from vendor A) should be greater than 1-α₁, as required by the hypothesis test. The probability of making the wrong decision (the component is actually from vendor B) should be less than α₂. Here α₁ is the Type I error α and α₂ is the Type II error β for the hypothesis test for vendor A.

Please note that Type I and Type II errors are related to a given statistical hypothesis test. Since SPRT combines two hypothesis tests together, it is very important to determine which one is the Type I error and which one is the Type II error.

When vendor A is accepted, based on the requirement for the Type I and Type II errors, we have:

(8)

From Eqns. (7) and (8), we set:

(9)

Similarly, when the ratio is larger than U, we accept vendor B:

(10)

When we accept vendor B, the probability of making the right decision (the component is indeed from vendor B) should be greater than 1-α₂. The probability of making the wrong decision (the component is actually from vendor A) should be less than α₁. Here α₂ is the Type I error and α₁ is the Type II error for the hypothesis test for vendor B. Therefore, we have:

(11)

From Eqns. (7) and (8), we can set:

(12)

Combining all the above equations, we get the decision formula for SPRT as the follows:

(13)

Which is:

(14)

SPRT for Weibull Distribution

SPRT can be used for any distribution. The likelihood ratio can be calculated based on the assumed distribution. In this section, we will use the Weibull distribution to illustrate how it is used in a reliability requirement test. The probability density function for a Weibull distribution is given by:

(15)

where:

• η is the scale parameter.
• b is the shape parameter. Note that here we do not use the traditional notation beta for the shape parameter because beta is used for the Type II error in this article.

Assume we want to test if a component is from a Weibull population with parameters of b₁ and η₁, or from a population with parameters of b₂ and η₂(η_{2  >}  η₁). The two likelihood functions are:

(16)

and

(17)

The likelihood ratio is:

(18)

When b₁ = b₂, the above equation becomes:

(19)

The decision equation for the log-likelihood ratio R is:

(20)

L and U are given in Eqns. (9) and (12). Therefore, Eqn. (20) becomes:

(21)

Example

Let's assume the lifetime of a component is described by a Weibull distribution with the shape parameter b = 1.5. We will use SPRT to determine if the component meets the following reliability requirements:

• A target reliability of 92% at 200 hours. If the component meets or exceeds the target reliability, the chance of rejecting it (i.e., Type I error or α error) should be less than 0.05. This is comparable to α₂ in the previous section.
• A minimum reliability of 82% at 200 hours. If the component’s reliability is 82% or less, the probability of accepting it (i.e., Type II error or β error) should be less than 0.1. This is comparable to α₁ in the previous section.

Our objectives are to:

• Calculate the acceptance and rejection line for the SPRT test.
• Determine whether to accept or reject the component based on a series of observed failure times.

Solution Using Manual Calculations

2. Calculate η₁ and η₂ based on the reliability requirements. The reliability function for a Weibull distribution is given by:
   Therefore, η₂ equals:
    
   and η₁ equals:
   
3. Enter the calculated values for η₁ and η₂ into the decision equations from Eqn. (21).
   The equation becomes:
   
4. Calculate  for each observed failure time. The observed time from each sequential test and the calculated results are shown next. (Note that in the table, negative rejection values were adjusted to 0.)

ID	Ti	T	Acceptance Value	Rejection Value	Decision
1	629	15,775.24	76,651.04125	0	Continue
2	369	22,863.5	97,964.88014	0	Continue
3	685	40,791.66	119,278.719	0	Continue
4	270	45,228.22	140,592.5579	14,209.44008	Continue
5	682	63,038.74	161,906.3968	35,523.27897	Continue
6	194	65,740.84	183,220.2357	56,837.11786	Continue
7	113	66,942.05	204,534.0746	78,150.95675	Reject

The plot of the data is shown next. The component is rejected at a failure time of 113 hours.

 

Solution Using the Weibull SPRT Template in Weibull++

The Synthesis version of Weibull++ includes a report template for calculating the SPRT results using a Weibull distribution and generating a plot of the results. To use the template:

2. Add a new analysis workbook in an existing project, by choosing Insert > Reports and Plots > Analysis Workbook.
3. Select the Based on Existing Template check box and then choose Weibull SPRT Template on the Standard tab.

2. Click OK, then click Yes to create the workbook.
3. Enter the reliability and risk requirement values, and the observed failure times in the white cells. The results and plot are shown next.

The resulting report provides all of the information that you obtained by doing the calculations manually and it automatically creates a plot.

Conclusion

Many published materials on SPRT only provide the simplified final formulas, such as Eqn. (21), for specific distributions for ease of use. In this article, we reviewed the basic theory of SPRT and illustrated its use in reliability engineering. It can be seen that it is a general tool that can be used for any distribution. Once the theory is understood, it is an easy task to develop your own SPRT for your applications.

Reference

[1] A. Wald, Sequential Analysis, John Wiley & Sons, Inc, New York, 1947.