Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter January 20, 2021

Error simulation modeling to assess the effects of bias and precision on bilirubin measurements used to screen for neonatal hyperbilirubinemia

  • Jose C. Jara Aguirre , Andrew P. Norgan , Walter J. Cook and Brad S. Karon EMAIL logo

Abstract

Objectives

Error simulation models have been used to understand the relationship between analytical performance and clinical outcomes. We developed an error simulation model to understand the effects of method bias and precision on misclassification rate for neonatal hyperbilirubinemia using an age-adjusted risk assessment tool.

Methods

For each of 176 measured total bilirubin (TSBM) values, 10,000 simulated total bilirubin (TBS) values were generated at each combination of bias and precision conditions for coefficient of variation (CV) between 1 and 15%, and for biases between −51.3 μmol/L and 51.3 μmol/L (−3 and 3 mg/dL) fixed bias. TBS values were analyzed to determine if they were in the same risk zone as the TSBM value. We then calculated sensitivity and specificity for prediction of ≥75th percentile for postnatal age values as a function of assay bias and precision, and determined the rate of critical errors (≥95th percentile for age TSBM with <75th percentile TBS).

Results

A sensitivity >95% for predicting ≥75th percentile bilirubin values was observed when there is a positive fixed bias of greater than 17.1 μmol/L (1.0 mg/dL) and CV is maintained ≤10%. A specificity >70% for predicting <75th percentile bilirubin values was observed when positive systematic bias was 17.1 μmol/L (1 mg/dL) or less at CV ≤ 10%. Critical errors did not occur with a frequency >0.2% until negative bias was −17.1 μmol/L (−1 mg/dL) or lower.

Conclusions

A positive systematic bias of 17.1 μmol/L (1 mg/dL) may be optimal for balancing sensitivity and specificity for predicting ≥75th percentile TSB values. Negative systematic bias should be avoided to allow detection of high risk infants and avoid critical classification errors.


Corresponding author: Brad S. Karon, MD, PhD, Department of Laboratory Medicine and Pathology, Mayo Clinic, 200 First Street SW., Rochester, MN 55905, USA, Phone: +1507- 538- 4921, Fax: +1507- 538- 7060, E-mail:

  1. Research funding: None declared.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Ethical approval: The samples were collected as part of a separate study previously approved by the Mayo Clinic Institutional Review Board.

References

1. Maisels, MJ, Bhutani, VK, Bogen, D, Newman, TB, Stark, AR, Watchko, JF. Hyperbilirubinemia in the newborn infant > or =35 weeks’ gestation: an update with clarifications. Pediatrics 2009;124:1193–8. https://doi.org/10.1542/peds.2009-0329.10.1542/peds.2009-0329Search in Google Scholar PubMed

2. Guidelines for detection, management and prevention of hyperbilirubinemia in term and late preterm newborn infants (35 or more weeks’ gestation) - Summary. Paediatr Child Health 2007;12:401–18.10.1093/pch/12.5.401Search in Google Scholar PubMed PubMed Central

3. American Academy of Pediatrics Subcommittee on Hyberbilirubinemia. Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics 2004;114:297–316.10.1542/peds.114.1.297Search in Google Scholar PubMed

4. Rennie, J, Burman-Roy, S, Murphy, MS, Guideline Development, G. Neonatal jaundice: summary of NICE guidance. BMJ 2010;340:c2409. https://doi.org/10.1136/bmj.c2409.Search in Google Scholar PubMed

5. Bhutani, VK, Johnson, L, Sivieri, EM. Predictive ability of a predischarge hour-specific serum bilirubin for subsequent significant hyperbilirubinemia in healthy term and near-term newborns. Pediatrics 1999;103:6–14. https://doi.org/10.1542/peds.103.1.6.Search in Google Scholar PubMed

6. Karon, BS, Teske, A, Santrach, PJ, Cook, WJ. Evaluation of the BiliChek noninvasive bilirubin analyzer for prediction of serum bilirubin and risk of hyperbilirubinemia. Am J Clin Pathol 2008;130:976–82. https://doi.org/10.1309/ajcprx1e3nwcxhmz.Search in Google Scholar PubMed

7. Kolman, KB, Mathieson, KM, Frias, C. A comparison of transcutaneous and total serum bilirubin in newborn Hispanic infants at 35 or more weeks of gestation. J Am Board Fam Med 2007;20:266–71. https://doi.org/10.3122/jabfm.2007.03.060172.Search in Google Scholar PubMed

8. Taylor, JA, Burgos, AE, Flaherman, V, Chung, EK, Simpson, EA, Goyal, NK, et al.. Utility of decision rules for transcutaneous bilirubin measurements. Pediatrics 2016;137:e20153032. https://doi.org/10.1542/peds.2015-3032.Search in Google Scholar PubMed PubMed Central

9. Boyd, JC, Bruns, DE. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem 2001;47:209–14. https://doi.org/10.1093/clinchem/47.2.209.Search in Google Scholar

10. Karon, BS, Boyd, JC, Klee, GG. Glucose meter performance criteria for tight glycemic control estimated by simulation modeling. Clin Chem 2010;56:1091–7. https://doi.org/10.1373/clinchem.2010.145367.Search in Google Scholar PubMed

11. Karon, BS, Boyd, JC, Klee, GG. Empiric validation of simulation models for estimating glucose meter performance criteria for moderate levels of glycemic control. Diabetes Technol Therapeut 2013;15:996–1003. https://doi.org/10.1089/dia.2013.0086.Search in Google Scholar

12. Lyon, AW, Kavsak, PA, Lyon, OA, Worster, A, Lyon, ME. Simulation models of misclassification error for single thresholds of high-sensitivity cardiac troponin I due to assay bias and imprecision. Clin Chem 2017;63:585–92. https://doi.org/10.1373/clinchem.2016.265058.Search in Google Scholar

13. EBMcalc: An educational medical reference. Neonatal hyperbilirubinemia assessment. https://ebmcalccom/NewbornHyperbilihtm [Accessed 24 Dec 20].Search in Google Scholar

14. Bromiker, R, Bin-Nun, A, Schimmel, MS, Hammerman, C, Kaplan, M. Neonatal hyperbilirubinemia in the low-intermediate-risk category on the bilirubin nomogram. Pediatrics 2012;130:e470–5. https://doi.org/10.1542/peds.2012-0005.Search in Google Scholar

15. Dai, J, Parry, DM, Krahn, J. Transcutaneous bilirubinometry: its role in the assessment of neonatal jaundice. Clin Biochem 1997;30:1–9. https://doi.org/10.1016/s0009-9120(96)00131-2.Search in Google Scholar

16. Sheridan-Pereira, M, Gorman, W. Transcutaneous bilirubinometry: an evaluation. Arch Dis Child 1982;57:708–10. https://doi.org/10.1136/adc.57.9.708.Search in Google Scholar PubMed PubMed Central

17. Rubaltelli, FF, Gourley, GR, Loskamp, N, Modi, N, Roth-Kleiner, M, Sender, A, et al.. Transcutaneous bilirubin measurement: a multicenter evaluation of a new device. Pediatrics 2001;107:1264–71. https://doi.org/10.1542/peds.107.6.1264.Search in Google Scholar PubMed

18. Taylor, JA, Burgos, AE, Flaherman, V, Chung, EK, Simpson, EA, Goyal, NK, et al.. Discrepancies between transcutaneous and serum bilirubin measurements. Pediatrics 2015;135:224–31. https://doi.org/10.1542/peds.2014-1919.Search in Google Scholar PubMed PubMed Central

19. Jara Aguirre, JC, Wockenfus, AM, Krekelberg, BJ, Fine, KL, Cook, WJ, Karon, BS. Implementation of a reformulated Roche(R) bilirubin Gen.3 reagent did not affect the relationship between BiliChek transcutaneous and Roche total serum bilirubin. Clin Biochem 2019;63:148–51. https://doi.org/10.1016/j.clinbiochem.2018.10.001.Search in Google Scholar PubMed

20. Lo, SF, Doumas, BT, Ashwood, ER, College of American Pathologists. Bilirubin proficiency testing using specimens containing unconjugated bilirubin and human serum: results of a College of American Pathologists study. Arch Pathol Lab Med 2004;128:1219–23.10.5858/2004-128-1219-BPTUSCSearch in Google Scholar PubMed

21. Lo, SF, Doumas, BT. The status of bilirubin measurements in U.S. laboratories: why is accuracy elusive? Semin Perinatol 2011;35:141–7. https://doi.org/10.1053/j.semperi.2011.02.008.Search in Google Scholar PubMed

22. Gu, D, Wang, Y, Ren, B, Wang, L, Zhang, K, Yuan, Y. Comparison of three routine methods for the measurement of serum bilirubin in a China laboratory. Clin Lab 2018;64:1485–90. https://doi.org/10.7754/clin.lab.2018.180333.Search in Google Scholar PubMed

23. Lano, IM, Lyon, AW, Wang, L, Ruskin, R, Lyon, ME. Comparative evaluation of neonatal bilirubin using Radiometer whole blood co-oximetry and plasma bilirubin methods from Roche Diagnostics and Ortho Clinical Diagnostics. Clin Biochem 2018;53:88–92. https://doi.org/10.1016/j.clinbiochem.2017.12.009.Search in Google Scholar PubMed

24. Kuzniewicz, MW, Greene, DN, Walsh, EM, McCulloch, CE, Newman, TB. Association between laboratory calibration of a serum bilirubin assay, neonatal bilirubin levels, and phototherapy use. JAMA Pediatr 2016;170:557–61. https://doi.org/10.1001/jamapediatrics.2015.4944.Search in Google Scholar PubMed

25. Medicare. Medicaid and CLIA programs; regulations implementing the clinical laboratory improvement Amendments of 1988 (CLIA). Fed Regist 1992;28:7002–186.Search in Google Scholar

26. Karon, BS, Wockenfus, AM, Hartung, KJ, Scott, RJ, Carter, SD, Jaffe, AS. Comparing analytical outliers and the percent of emergency department patients with results above the 99th percentile upper reference limit for 2 conventional and one high sensitivity troponin assay. Clin Biochem 2018;53:104–9. https://doi.org/10.1016/j.clinbiochem.2018.01.001.Search in Google Scholar PubMed


Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2020-1640).


Received: 2020-11-02
Accepted: 2020-12-28
Published Online: 2021-01-20
Published in Print: 2021-05-26

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 29.3.2024 from https://www.degruyter.com/document/doi/10.1515/cclm-2020-1640/html
Scroll to top button