PSA Screening and Early Detection. Part 1 - Guides
PSA Screening and Early Detection. Part 2 - Key Points on PSA
PSA Screening and Early Detection. Part 3. Current Environment [previous]
PSA Screening and Early Detection - Part 4. Diagnostic Testing Concepts [current]
PSA Screening and Early Detection - Part 5. More Diagnostic Testing Concepts [next]
In this part we discuss diagnostic testing terms prevalence, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) and discuss the accuracy of PSA testing in those terms. The links just given are to definitions that the reader may wish to review first.
Accuracy of TSA Test. The bars at the bottom of the accompanying figure (click it to see it in full resolution) show the distribution of PSA values found in the Prostate Cancer Prevention Trial (PCPT). This trial was a randomized, double-blind, placebo-controlled study of the efficacy of finasteride in preventing prostate cancer in 18,882 men aged 55 years or older. PCPT is discussed in [PMID: 16622122] [Full Text]. The data itself is derived from Table 3 of [PMID: 15998892] [Full Text] (or Table 1 here). The percentage of healthy men in each PSA range are shown as blue bars and the percentage of diseased men in each PSA range are shown as red bars. The blue bars and the red bars each sum to 100%. Below each bar the range of PSA values for that bar is indicated such that an open parenthesis below the bar means that that end of the range is not included in the range and a closed bracket means that that end of the range is included. Thus looking at the blue (3.0,4.0] bar we see that if we had 100 healthy men that 11.7 of them would have a PSA greater than 3 and less than or equal to 4. Looking at the red (3.0,4.0] bar we see that if we had a 100 diseased men that 7.1 of them would have a PSA greater than 3 and less than or equal to 4.
Pick a cutoff point of 4.0, say, so that test results above it are postive, i.e. indicative of prostate cancer, and test results below 4.0 are negative, i.e. indicative of no prostate cancer.
The sensitivity is the fraction of correct PSA tests among those with prostate cancer. Thus it equals the sum of the red bars to the right of the cutoff point. For example, if the cutoff point is 4.0 then the sensitivity equals the sum of the red bars (4,6], (6,8] and (8,10] and >10 which is 15.9% + 2.9% + 0.8% + 0.9% = 20.5% . Above the red bars we have shown the sensitivity as a red line graph. Note that with a cutoff of 4.0 only 20.5% of all cancers will be found but if we lower the cutoff to 2.5 we find 40.5% of the cancers (twice as many) which is why cutoff levels are being revised downwards to 2.5 by many organizations. Although a single PSA test will miss many cancers some of the missed ones will be caught via DRE or on subsequent checkups so the overall situation is not as unfavorable as it seems here.
The specificity is the fraction of correct PSA tests among those who are healthy. It equals the sum of the blue bars to the left of the cutoff point which in the case of a cutoff of 4.0 is 38.9% + 19.8% + 13.8% + 8.6% + 5.6% + 7.1% = 93.8% . It is shown as the blue line. If we reduce the cutoff to 2.5 then the specificity drops to 81.1%.
Thus the sum of the red bars to the right of the cutoff point is the sensitivity and represents the percentage of correct tests among the group of subjects who have cancer while the sum of the blue bars to the left of the cutoff point is the specificity and represents the percentage of correct tests among the group of subjects who are healthy. Sensitivity and specificity are obtained from the frequencies by summation while the frequencies can be recovered from the sensitivity and specificity by taking differences of successive sensitivity or specificity values. If we move the cutoff point to the left then the sensitivity increases as additional red bars get included in the sum while the specificity declines as fewer blue bars are included in the sum. Thus there is a tradeoff between sensitivity and specificity with both critically depending on the cutoff point.
One important point is that we only need to look at the diseased population to calculate the sensitivity and we only need to look at the healthy population to calculate the specificity. The one aspect that is not addressed by looking at each such population separately is that the diseased population is much smaller than the healthy population. The prevalence is the fraction of the population under test having prostate cancer. The sensitivity and the specificity are independent of prevalence. If two different countries have different prevalence rates of prostate cancer they will still have the same sensitivity and specificity (if all other factors are the same -- although usually all other factors will not be the same). Vollmer indicates that the prevalence of prostate cancer is about 1%, 5% and 12% for men aged 55, 65 and 75 and we will use the 5% figure as our overall prevalence among men of prostate cancer testing age. If the prevalence were 5% then among 100,000 men 5,000 would have cancer while 95,500 would not.
Using a cutoff of 2.5 we can use the above data to form a 2x2 table which categorizes 100 men as Cancer vs. Healthy and positive test for cancer vs. negative test.
Prevalence = 5.0%
Sensitivity = 40.5%
Specificity = 81.1%
-----Group----
Test Cancer Healthy Total
Positive 2 18 20
Negative 3 77 80
Total 5 95 100
The entries can be calculated as follows:
- the grand total of the table is set to 100%
- the first column total is grand total x prevalence = 100 x 5% = 5%
- the second column total is grand total - first column total = 100% - 5% = 95%
- the diagonal elements are the sensitivity (0.405) and specificity (0.811) times the corresponding column totals, i.e. 0.405 x 5 = 2.025 and 0.811 x 95 = 77.045 respectively
- the off diagonal elements are filled in to make the columns sum
- the rows are summed to form the row totals
The negative predictive value (NPV) is the chance of not having cancer among those who test negative for it. It answers the question: if I receive a negative PSA test what is the chance I am free of cancer? It is the lower right element of the table (not counting the total row and column) expressed as a fraction of its row sum. 77/80 = 96%
Comparing sensitivity/specificity with PPV/NPV.
The sensitivity and specificity are the diagonal elements of the count matrix as a fraction of their column sums while the PPV and NPV are the diagonal elements of the count matrix as a fraction of their row sums.
The sensitivity and specificity do not depend on the prevalence but the PPV and NPV do depend on the prevalence. Thus sensitivity and specificity are pure measures of the test itself while PPV and NPV partly measure the test but are also influenced by the makeup of the population being tested.
Also noteworthy is how low the positive predictive value (PPV) is. That is there is only a 10% of chance of cancer if you have a positive test. This implies that 90% of the biopsies performed will be needless.
The sensitivity of 40.5% implies that nearly 60% of those with cancer will be missed by the PSA test.
The specificity of 81.1% implies that nearly 20% healthy men will be tested positive and require further testing to undergo needless biopsy.
Changing the Cutoff. In both the case of sensitivity/specificity and PPV/NPV the cutoff value plays a crucial role. Changing the cutoff value would change all four quantities in the 2x2 count table (however, it would not change the column totals of the count table). Another thing we could do would be to fix the specificity and then calculate the implied cutoff and sensitivity. For example if we fixed the specificity at 80.1% then the blue line in the chart above shows that the cutoff would need to be set to 2.5 and the red line shows that with a cutoff of 2.5 the implied sensitivity is 40.5%.
As a second example, suppose we wish to raise the sensitivity of the test from 40.5% to at least 80%. Then from the red line on the graph above we see that reducing the cutoff point to 1 would do that but from the blue line we see that that would also decrease the specificity to 38.9% -- i.e. the cost of this is that over 60% of healthy men would undergo biopsies, all needlessly.
Although in the above we have focused on the PSA Test in isolation, in reality
- if the PSA Test were negative and the digital rectal exam (DRE) were positive a biopsy would still be given so the combined sensitivity of PSA Test plus DRE would be greater than the sensitivity of the PSA Test alone.
- if the PSA Test were positive but the DRE were negative one could have a Free PSA test done (provided the PSA Test gives a score of between 4 and 10). Free PSA is a component of PSA which is minimally affected by cancer so its proportion of total PSA helps gauge whether the high PSA was due to cancer or some other cause, such as inflammation or other prostatic disease. This article by McDermed discusses the sensitivity and specificity of the Free PSA Test.
- Different data sets yield somewhat different results. For example, the following modification of the previous table
-----Group----
Test Cancer Healthy Total
Positive 3 5 8
Negative 2 90 92
Total 5 95 100
is based on
which is found in the Ontario Government information [here].
With this data the sensitivity is 3/5 = 60% and the specificity is 90/95 = 95%. The prevalence is 5/100 = 5% which is the same as our assumed prevalence previously.
Another dataset reported in this publication from the American Association of Family Physicians [link] is slightly different again:
-----Group----
Test Cancer Healthy Total
Positive 3 7 10
Negative 1 89 90
Total 4 96 100
In this case the sensitivity is substantially higher at 3/4 = 75%, the specificity = 89/96 = 93% and the prevalence is 4/100 = 4%.
Code
The R code to produce the chart and various calculations in this post is provided below for those interested. Just copy and paste it into a running R session or it can be pasted into an online R session at Rweb .
Lines <- "PSA Sensitivity Specificity
1.1 83.4 38.9
1.6 67 58.7
2.1 52.6 72.5
2.6 40.5 81.1
3.1 32.2 86.7
4.1 20.5 93.8
6.1 4.6 98.5
8.1 1.7 99.4
10.1 0.9 99.7
"
DF <- read.table(textConnection(Lines), header = TRUE)
nam <- c("[0,1]", "(1,1.5]", "(1.5,2]", "(2,2.5]", "(2.5,3]", "(3,4]",
"(4,6]", "(6,8]", "(8,10]", ">10")
left <- c(0, DF$PSA - 0.1)
right <- c(DF$PSA - 0.1, Inf)
Nam <- paste("(", left, ",", right, "]", sep = "")
dd <- abs(apply(rbind(c(100, 0), DF[-1], c(0, 100)), 2, diff))
sens <- 100 - cumsum(dd[,1])
spec <- cumsum(dd[,2])
rownames(dd) <- nam
barplot(t(dd), beside = TRUE)
bp <- barplot(t(dd), beside = TRUE, cex.names = 0.7,
ylim = c(0, 120), legend = c("Diseased", "Healthy"), density = 20,
angle = c(45, -45), col = c(2, 4))
legend("right", legend = c("Sensitivity", "Specificity"), lty = 1, pch = 20, col = c(2,4))
# sensitivity and specification lines
lines(bp[1,], sens, col=2, type="o", pch=20)
lines(bp[2,], spec, col=4, type="o", pch=20)
text(bp[1,1:6], sens[1:6], pos=4, sprintf("Pr[PSA\76%.1f] = %.1f%%",right[1:6],sens[1:6]), cex=.7)
text(bp[2,1:6], spec[1:6], pos=4, sprintf("Pr[PSA\74%.1f] = %.1f%%",right[1:6],spec[1:6]), cex=.7)
title(main = "PSA Distribution - PCPT Trial", xlab = PSA ~ (ng / ml),
ylab = "Percent")
# place text above bars
dx <- .25
dy <- .2
txt <- round(dd[,1], 1)
txt <- sprintf("%.1f", dd[,1])
# text(bp[1,], dd[,1], round(100*dd[,1]), cex = .7)
text(bp[1,] + dx, dd[,1] + dy, txt, cex = .6, pos = 3, srt = 90, adj = c(2, 0))
txt <- round(dd[,2], 1)
txt <- sprintf("%.1f", dd[,2])
text(bp[2,] + dx, dd[,2] + dy, txt, cex = .6, pos = 3, srt = 90)
# Table showing frequency of diseased and healthy
# and sensitivity and specificity
cbind(PCa = dd[,1], Healthy = dd[,2], Sens = sens, Spec = spec)
tab <- function(sens, spec, N = 100, prev = 0.05) {
mm <- N*matrix(c(prev*sens, prev*(1-sens), (1-prev)*(1-spec),
(1-prev)*spec), 2)
mm2 <- round(cbind(rbind(mm, colSums(mm)), c(rowSums(mm), sum(mm))))
cat("Prevalence:", prev, "\n")
cat("Sensitivity:", sens, "\n")
cat("Specificity:", spec, "\n")
cat("PPV:", mm2[1,1]/mm2[1,3], "\n")
cat("NPV:", mm2[2,2]/mm2[2,3], "\n")
cat("Detection Rate:", mm2[1,1]/N, "\n")
cat("Detection Prevalence:", mm2[1,3]/N, "\n")
dimnames(mm2) <- list(Test = c("Positive", "Negative", "Total"),
Status = c("Cancer", "Healthy", "Total"))
print(mm2)
print(prop.table(mm, 1))
print(prop.table(mm, 2))
}
tab(sens[4]/100, spec[4]/100)
tab(0.60, 0.95)
tab(0.75, 0.93, prev = 0.04)
PSA Screening and Early Detection. Part 1 - Guides
PSA Screening and Early Detection. Part 2 - Key Points on PSA
PSA Screening and Early Detection. Part 3. Current Environment [previous]
PSA Screening and Early Detection - Part 4. Diagnostic Testing Concepts [current]
PSA Screening and Early Detection - Part 5. More Diagnostic Testing Concepts [next]
No comments:
Post a Comment