JBJS welcomes reader comments on published articles. Letters to the Editor are reviewed by JBJS editors but are not peer-reviewed. To submit your letter, please follow the "submit a response" link that appears in the content box at the upper right of the full text of the article.

Letters to the Editor to:

Scientific Articles:
Nelson V. Greidanus, Bassam A. Masri, Donald S. Garbuz, S. Darrin Wilson, M. Gavan McAlinden, Min Xu, and Clive P. Duncan
Use of Erythrocyte Sedimentation Rate and C-Reactive Protein Level to Diagnose Infection Before Revision Total Knee Arthroplasty. A Prospective Evaluation
J Bone Joint Surg Am 2007; 89: 1409-1416 [Abstract] [Full text] [PDF]
*Letters to the Editor: Submit a response to this article

Electronic letters published:

[Read Letter to the Editor] Optimal Operating Points Determination for ESR and CRP
Nelson V. Greidanus, Bassam A. Masri, Donald S. Garbuz, S. Darrin Wilson, M. Gavan McAlinden, Min Xu, and Clive P. Duncan   (22 October 2007)
[Read Letter to the Editor] Optimal Operating Points Determination
Zong-I Lin, M.D., Yu-Min Lin, M.D., Taichung Veterans General Hospital, TAIWAN   (17 October 2007)

Optimal Operating Points Determination for ESR and CRP 22 October 2007
Previous Letter to the Editor Top
Nelson V. Greidanus,
Orthopaedic Surgeon
University of British Columbia,
Bassam A. Masri, Donald S. Garbuz, S. Darrin Wilson, M. Gavan McAlinden, Min Xu, and Clive P. Duncan

Send letter to journal:
Re: Optimal Operating Points Determination for ESR and CRP

nelson.greidanus{at}vch.ca Nelson V. Greidanus, et al.

The stated purpose of our published manuscript was “to prospectively evaluate the diagnostic test characteristics of the erythrocyte sedimentation rate(ESR) and C-reactive protein(CRP) level for the diagnosis of infection prior to revision total knee arthroplasty” (1). We collected data prospectively and evaluated the diagnostic performance of the ESR and CRP against the so-called ‘gold standard’ diagnosis of infection---positive growth of bacteria on culture of intra-articular fluid or tissue specimen. Microsoft Excel(Redmond, Washington) and SPSS(version 10.0; SPSS) software were employed to tabulate, graphically illustrate, and analyze the data including the development of receiver-operator-characteristic (ROC) curves for the respective ESR and CRP tests. Our goals were to plot the ROC curves, identify optimal cutpoints for the ESR and CRP, and to objectively illustrate the performance of these tests using standardized nomenclature with 95% confidence intervals. We selected to illustrate the ‘ideal cutpoints’ alongside the so-called ‘traditional cutpoints’ for ESR and CRP so that the readership may understand the differences in test performance should they choose to use cutpoints from the historical literature or those recommended from the hip arthroplasty literature(2).

Dr. Lin et al. identify a number of methods for ROC curve evaluation and ‘cut-point’ determination. While numerous methods are available for determining the optimal cut-point there is no evidence that one method is superior or is the so-called ‘gold standard’(3). One method that is commonly utilized is to use the apex of the ROC curve (upper left corner of the curve) as the optimal cut-point as this represents the point at which sensitivity and specificity are optimized. This is the method that we used, which corresponds with the Youden index(4). While we recommend the selection of cutpoints of 22.5 mm/hr for ESR and 13.5mg/L for CRP on the basis of this method we do not claim statistical superiority (p<.05) of this cutpoint over the traditional cutpoints of of 30mm/hr and 10mg/L for ESR and CRP respectively. This is evident from the overlapping 95% confidence intervals in Table 3. The variance seen in the ESR and CRP point estimates and our modest sample size do not permit for us to demonstrate statistical superiority of our proposed cutpoints over the traditional cutpoints reported for ESR and CRP. However, to our knowledge, our report is the first to demonstrate an ‘evidence-based’ determination of optimal cutpoint for the ESR and CRP to diagnose infection prior to revision knee arthroplasty. Our data and analytic methods suggest that these cutpoints are 22.5mm/hr for ESR and 13.5mg/L for CRP.

When performing ‘area under curve’(AUC) analyses we calculated the AUC (with 95% confidence intervals) for ESR ROC as 0.925 (0.884-0.967) and for CRP ROC as 0.908 (0.857-0.960). This AUC data does not prove statistical superiority of one test over the other but rather demonstrates that both the ESR and CRP are ‘highly accurate’ diagnostic tests as defined by criteria of(5). Our calculations and subsequent recommendations for using ESR and CRP together in ‘combination testing’ (for improved sensitivity and specificity) are based on Bayesion methods as recommended by Black et al.(6). We did not develop ROC curves and AUC analyses for the 'combination testing' of ESR and CRP and therefore are unable to comment on AUC differences which might occur in such scenarios.

In summary, we have demonstrated objective ‘evidence-based’ cutpoints for ESR and CRP in the population of patients assessed for revision total knee arthroplasty. Furthermore, we have developed and demonstrated ROC curves which prove that the ESR and CRP tests are highly accurate when evaluating for possible infection. Finally, we have suggested ‘combination testing’ of the ESR and CRP as a method to increase the diagnostic information from these tests whether screening patients for the possibility of infection or seeking confirmation of a presumptive diagnosis of infection.

References:

1. Greidanus NV, Masri BA, Garbuz DS, Wilson SD, McAlinden MG., Xu M, Duncan CP. Use of erythrocyte sedimentation rate and C-reactive protein level to diagnose infection before revision total knee arthroplasty. A prospective evaluation. J Bone Joint Surg Am 2007;89:1409-1416.

2. Spangehl MJU, Masri BA, O'Connell JX, Duncan CP. Prospective analysis of preoperative and intraoperative investigations for the diagnosis of injection at the sites of two hundred and two evision total hip arthroplasties. J bone Joint Surg Am. 1999;81:672-83.

3. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000;45:23-41.

4. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32-35.

5. Swetts JA. Measuring the accuracy of diagnostic systems. Science 1988; 240: 1285-1293.

6. Black ER, Panzer RJ, Mayewski RJ, Griner PF. Characteristics of diagnostic tests and principles for their use in quantitative decision making. In: Black ER, Bordley DR, Tape TG, Panzer RJ. Diagnostic strategies of common medical problems. Philadelphia: American College of Physicians; 1999. p 8-10
Optimal Operating Points Determination 17 October 2007
Next Letter to the Editor Top
Zong-I Lin, M.D.,
Attending Physician
Chung Shan Medical University Hospital, TAIWAN,
Yu-Min Lin, M.D., Taichung Veterans General Hospital, TAIWAN

Send letter to journal:
Re: Optimal Operating Points Determination

l4321{at}ms2.hinet.net Zong-I Lin, M.D., et al.

To The Editor:

In their recent article, "Use of Erythrocyte Sedimentation Rate and C-Reactive Protein Level to Diagnose Infection Before Revision Total Knee Arthroplasty. A Prospective Evaluation" Greidanus et al.(1) did not publish the data of the area under the curve (AUC) of the respective receiver-operating characteristic (ROC) curve of erythrocyte sedimentation rate (ESR), C-reactive protein (CRP) level, ESR or CRP positive, and ESR and CRP level positive. They also did not compare the AUC between ESR or CRP positive and ESR and CRP level positive by statistical tests.

We found, with or without receiving antibiotics, there were no statistically significant difference between the overall accuracy of 22.5 mm/hr for ESR and of 30 mm/hr for ESR, of 13.5 mg/L for CRP and of 10 mg/L for CRP, of 22.5 mm/hr for ESR or 13.5 mg/L for CRP and of 30 mm/hr for ESR or 10 mg/L for CRP, and of 22.5 mm/hr for ESR and 13.5 mg/L for CRP and of 30 mm/hr for ESR and 10 mg/L for CRP. The combination of ESR and CRP tests provides better diagnostic information statistically when selecting 22.5 mm/hr for ESR and 13.5 mg/L for CRP, not 30 mm/hr for ESR and 10 mg/L for CRP.

In addition, Greidanus et al.(1) also did not mention clearly which method they used to determine the optimal operating point of ESR and CRP, probably by choosing cut-points at the apex of the ROC. We rechecked the cut-off points of ESR and CRP by the Youden index (J), maximum {Sensitivity + Specificity -1}(2), and we reached the same conclusion. We could not, however, perform the t test to compare the difference because of lacking of the complete data set, and we wonder about the statistical significance. The criteria for the optimal cut-off determination include a plot of sensitivity and specificity as a function of the cut-off value, Youden index, odds ratio, likelihood ratio, efficiency, kappa, misclassification-cost term, optimized likelihood ratio for a given prevalence, slope approach, and many more(3). We wish the authors will provide details about their method for determining the optimal cut-off points and the statistical significance.

The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

References:

1. Greidanus NV, Masri BA, Garbuz DS, Wilson SD, McAlinden MG., Xu M, Duncan CP. Use of erythrocyte sedimentation rate and C-reactive protein level to diagnose infection before revision total knee arthroplasty. A prospective evaluation. J Bone Joint Surg Am 2007;89:1409-1416.

2. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32-35.

3. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000;45:23-41.