References

vsp

Вопросы современной педиатрии

Current Pediatrics

1682-55271682-5535

Издательство «ПедиатрЪ»

10.15690/vsp.v22i2.2557

vsp-3184

Research Article

РЕДАКЦИОННАЯ СТАТЬЯ

EDITORIAL

Прозрачная отчетность о многофакторной предсказательной модели для индивидуального прогнозирования или диагностики (TRIPOD): разъяснения и уточнения

Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and Elaboration. Translation into Russian

https://orcid.org/0000-0003-2118-004X

Moons

K. G.M.

Moons

Karel G.M.

Julius Center for Health Sciences and Primary Care

PhD, Julius Centre for Health Sciences and Primary Care, UMC Utrecht

PO Box 85500, 3508 GA Utrecht, Утрехт

Julius Center for Health Sciences and Primary Care

PO Box 85500, 3508 GA Utrecht

K.G.M.Moons@umcutrecht.nl

https://orcid.org/0000-0002-7183-4083

Altman

D. G.

Altman

Douglas G.

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre

Oxford OX3 7LD, Оксфорд

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre

Oxford OX3 7LD, Оксфорд

https://orcid.org/0000-0003-4026-4345

Reitsma

J. B.

Reitsma

Johannes B.

Julius Center for Health Sciences and Primary Care

PO Box 85500, 3508 GA Utrecht, Утрехт

Julius Center for Health Sciences and Primary Care

PO Box 85500, 3508 GA Utrecht

https://orcid.org/0000-0003-3118-6859

Loannidis

J. P.A.

Loannidis

John P.A.

Stanford Prevention Research Center, School of Medicine

291 Campus Drive, Room LK3C02, Li Ka Shing Building, 3rd Floor, Stanford, CA 943055101, Стэнфорд

291 Campus Drive, Room LK3C02, Li Ka Shing Building, 3rd Floor, Stanford, CA 943055101

https://orcid.org/0000-0001-5879-6193

Macaskill

Petra

Screening & Test Evaluation Program (STEP), School of Public Health, Edward Ford Building (A27), Sydney Medical School

Sydney, NSW 2006, Сидней

Screening & Test Evaluation Program (STEP), School of Public Health, Edward Ford Building (A27), Sydney Medical School

Sydney, NSW 2006

https://orcid.org/0000-0002-7787-0122

Steyerberg

Е. W.

Steyerberg

Ewout W.

Department of Public Health

PO Box 2040, 3000 CA, Rotterdam, Роттердам

Department of Public Health

PO Box 2040, 3000 CA, Rotterdam

https://orcid.org/0000-0003-1525-6503

Vickers

А. J.

Vickers

Andrew J.

Department of Epidemiology and Biostatistics

307 East 63rd Street, 2nd Floor, Box 44, New York, NY 10065, Нью-Йорк

Department of Epidemiology and Biostatistics

307 East 63rd Street, 2nd Floor, Box 44, New York, NY 10065

https://orcid.org/0000-0002-2200-039X

Ransohoff

D. F.

Ransohoff

David F.

Departments of Medicine and Epidemiology

4103 Bioinformatics, CB 7080, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7080, Чапел-Хилл

Departments of Medicine and Epidemiology

4103 Bioinformatics, CB 7080, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7080

https://orcid.org/0000-0002-2772-2316

Collins

G. S.

Collins

Gary S.

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre

Oxford OX3 7LD, Оксфорд

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre

Oxford OX3 7LD

University Medical Center UtrechtНидерландыUniversity Medical Center UtrechtNetherlands

University of OxfordВеликобританияUniversity of OxfordUnited Kingdom

Stanford UniversityСоединённые Штаты АмерикиStanford Prevention Research Center, School of Medicine, Stanford UniversityUnited States

University of SydneyАвстралияUniversity of SydneyAustralia

Erasmus MC-University Medical Center RotterdamНидерландыErasmus MC-University Medical Center RotterdamNetherlands

Memorial Sloan Kettering Cancer CenterСоединённые Штаты АмерикиMemorial Sloan Kettering Cancer CenterUnited States

University of North Carolina at Chapel HillСоединённые Штаты АмерикиUniversity of North Carolina at Chapel HillUnited States

2023

07052023

222109187

2023

Moons K.G., Altman D.G., Reitsma J.B., Loannidis J.P., Macaskill P., Steyerberg Е.W., Vickers А.J., Ransohoff D.F., Collins G.S.

Moons K.G., Altman D.G., Reitsma J.B., Loannidis J.P., Macaskill P., Steyerberg E.W., Vickers A.J., Ransohoff D.F., Collins G.S.

Данная работа распространяется под лицензией Creative Commons Attribution 4.0.

This work is licensed under a Creative Commons Attribution 4.0 License.

https://vsp.spr-journal.ru/jour/article/view/3184

.Руководство TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) содержит контрольный перечень из 22 пунктов рекомендаций, предложенных для повышения качества отчетности по исследованиям, в которых разрабатывали, проверяли или обновляли предсказательные модели для диагностики или прогнозирования. Руководство TRIPOD направлено на повышение прозрачности отчета об исследовании предсказательной модели независимо от использованных методов. Этот документ с пояснениями и уточнениями включает обоснование, разъяснения значений каждого пункта рекомендаций, обсуждение важности прозрачной отчетности для оценки риска систематических ошибок и клинической полезности предсказательной модели. Каждая рекомендация руководства TRIPOD подробно объясняется, приводятся опубликованные примеры правильного представления результатов. Документ также содержит ценную справочную информацию, которую следует учитывать при разработке, проведении и анализе исследований предсказательных моделей. Рекомендуем авторам включать в свои работы все пункты контрольного перечня, что облегчит оценку исследования редакторами, рецензентами, читателями и исследователями, проводящими систематическое обобщение результатов таких исследований. Контрольный перечень TRIPOD также доступен по адресу: www.tripod-statement.org.

Данная статья является переводом на русский язык. Оригинальная статья опубликована в Annals of Internal Medicine. 2015;162(1):W1–W73. doi: https://doi.org/10.7326/M14-0698. Перевод и повторная публикация осуществлены с разрешения правообладателя. Перевод и научное редактирование выполнены д.м.н. Р.Т. Сайгито вым (ORCID: https://orcid.org/0000-0002-8915-6153). Перевод впервые опубликован в Digital Diagnostics. doi: https://doi.org/10.17816/DD110794. Публикуется с незначительными изменениями, связанными с литературным редактированием текста перевода.

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org. This article is the translation in to Russian by Dr. Ruslan Saygitov (ORCID: https://orcid.org/0000-0002-8915-6153) from the original published in [Ann Intern Med. 2015;162:W1-W73. doi: https://doi.org/10.7326/M14-0698].

Разработка контрольного перечня и рекомендаций выполнена без прямой спонсорской поддержки. Заседание в июне 2011 г. проведено при частичной поддержке National Institute for Health Research Senior Investigator Award во главе с доктором Altman, а также грантов со стороны Cancer Research UK (C5529) и Netherlands Organization for Scientific Research (ZONMW 918.10.615 и 91208004). Доктора Collins и Altman получили частичную финансовую поддержку со стороны Medical Research Council (G1100513). Доктор Altman является членом Medical Research Council Prognosis Research Strategy (PROGRESS) Partnership (G0902393/99558)

There was no explicit funding for the development of this checklist and guidance document. The consensus meeting in June 2011 was partially funded by a National Institute for Health Research Senior Investigator Award held by Dr. Altman, Cancer Research UK (grant C5529), and the Netherlands Organization for Scientific Research (ZONMW 918.10.615 and 91208004). Drs. Collins and Altman are funded in part by the Medical Research Council (grant G1100513). Dr. Altman is a member of the Medical Research Council Prognosis Research Strategy PROGRESS) Partnership (G0902393/99558)

References1

Moons KG, Royston P, Vergouwe Y, et al. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338:b375. doi: https://doi.org/10.1136/bmj.b375

Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.

Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. Applications and methodological standards. N Engl J Med. 1985;313(13):793–799. doi: https://doi.org/10.1056/NEJM198509263131306

Dorresteijn JA, Visseren FL, Ridker PM, et al. Estimating treatment effects for individual patients based on the results of randomised clinical trials. BMJ. 2011;343:d5888. doi: https://doi.org/10.1136/bmj.d5888

Hayward RA, Kent DM, Vijan S, Hofer TP. Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol. 2006;6:18. doi: https://doi.org/10.1186/1471-2288-6-18

Kattan MW, Vickers AJ. Incorporating predictions of individual patient risk in clinical trials. Urol Oncol. 2004;22(4):348–352. doi: https://doi.org/10.1016/j.urolonc.2004.04.012

Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007;298(10):1209–1212. doi: https://doi.org/10.1001/jama.298.10.1209

Riley RD, Hayden JA, Steyerberg EW, et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2):e1001380. doi: https://doi.org/10.1371/journal.pmed.1001380

Steyerberg EW, Moons KG, van der Windt DA, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. doi: https://doi.org/10.1371/journal.pmed.1001381

Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ. 2009; 338:b604. doi: https://doi.org/10.1136/bmj.b604

Collins GS, Altman DG. Identifying patients with undetected renal tract cancer in primary care: an independent and external validation of QCancer® (Renal) prediction model. Cancer Epidemiol. 2013;37(2):115–120. doi: https://doi.org/10.1016/j.canep.2012.11.005

Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15(4):361–387. doi: https://doi.org/10.1002/(SICI)1097-0258(19960229)15:43.0.CO;2-4

Canet J, Gallart L, Gomar C, et al. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology. 2010;113(6):1338–1350. doi: https://doi.org/10.1097/ALN.0b013e3181fc6e0a

Nashef SA, Roques F, Sharples LD, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41:734–744; discussion 744–745. doi: https://doi.org/10.1093/ejcts/ezs043

Schulze MB, Hoffmann K, Boeing H, et al. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care. 2007;30(3): 510–515. doi: https://doi.org/10.2337/dc06-2089

Hippisley-Cox J, Coupland C, Robson J, et al. Predicting risk of type 2 diabetes in England and Wales: prospective derivation and validation of QDScore. BMJ. 2009;338:b880. doi: https://doi.org/10.1136/bmj.b880

D’Agostino RB, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117(6):743–753. doi: https://doi.org/10.1161/CIRCULATIONAHA.107.699579

North RA, McCowan LM, Dekker GA, et al. Clinical risk prediction for pre-eclampsia in nulliparous women: development of model in international prospective cohort. BMJ. 2011;342:d1875. doi: https://doi.org/10.1136/bmj.d1875

Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009; 338:b605. doi: https://doi.org/10.1136/bmj.b605

Moons KG, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012;98(9):691–698. doi: https://doi.org/10.1136/heartjnl-2011-301247

Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11): 1085–1094. doi: https://doi.org/10.1016/j.jclinepi.2008.04.008

Steyerberg EW, Pencina MJ, Lingsma HF, et al. Assessing the incremental value of diagnostic and prognostic markers: a review and illustration. Eur J Clin Invest. 2012;42(2):216–228. doi: https://doi.org/10.1111/j.1365-2362.2011.02562.x

Steyerberg EW, Bleeker SE, Moll HA, et al. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003;56(5):441–447. doi: https://doi.org/10.1016/s0895-4356(03)00047-7

Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med. 2000;19(8):1059–1079. doi: https://doi.org/10.1002/(sici)1097-0258(20000430)19:83.0.co;2-0

Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JD. Prognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Med Decis Making. 2001;21(1):45–56. doi: https://doi.org/10.1177/0272989X0102100106

Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453–473. doi: https://doi.org/10.1002/(sici)1097-0258(20000229)19:43.0.co;2-5

Ioannidis JPA, Khoury MJ. Improving validation practices in “omics” research. Science. 2011;334(6060):1230–1232. doi: https://doi.org/10.1126/science.1211811

Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6):515–524. doi: https://doi.org/10.7326/0003-4819-130-6-199903160-00016

McGinn TG, Guyatt GH, Wyer PC, et al. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA. 2000;284: 79–84. doi: https://doi.org/10.1001/jama.284.1.79

Taylor JM, Ankesrt DP, Andridge RR. Validation of biomarkerbased risk prediction models. Clin Cancer Res. 2008;14(19): 5977–5983. doi: https://doi.org/10.1158/1078-0432.CCR-07-4534

Janssen KJ, Moons KG, Kalkman CJ, et al. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61(1):76–86. doi: https://doi.org/10.1016/j.jclinepi.2007.04.018

Steyerberg EW, Harrell FE, Borsboom GJ, et al. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–871. doi: https://doi.org/10.1016/s0895-4356(01)00341-9

Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–209. doi: https://doi.org/10.7326/0003-4819-144-3-200602070-00009

Bouwmeester W, Zuithoff NP, Mallett S, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9(5):1–12. doi: https://doi.org/10.1371/journal.pmed.1001221

Rabar S, Lau R, O’Flynn N, et al. Risk assessment of fragility fractures: summary of NICE guidance. BMJ. 2012;345:e3698. doi: https://doi.org/10.1136/bmj.e3698

National Institute for Health and Care Excellence. Lipid modification: cardiovascular risk assessment and the modification of blood lipids for the primary and secondary prevention of cardiovascular disease. Clinical guideline CG67. London: National Institute for Health and Care Excellence; 2008. Available online: http://guidance.nice.org.uk/CG67. Accessed on October 30, 2011.

National Osteoporosis Foundation. Clinician’s guide to prevention and treatment of osteoporosis. Washington DC: National Osteoporsis Foundation; 2010. Available online: http://nof.org/hcp/clinicians-guide. Accessed on January 17, 2013.

National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Third report of the National Cholesterol Education Program (NCEP) Expert Panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report. Circulation. 2002;106:3143–3421.

Goldstein LB, Adams R, Alberts MJ, et al. Primary prevention of ischemic stroke: a guideline from the American Heart Association/ American Stroke Association Stroke Council: cosponsored by the Atherosclerotic Peripheral Vascular Disease Interdisciplinary Working Group; Cardiovascular Nursing Council; Clinical Cardiology Council; Nutrition, Physical Activity, and Metabolism Council; and the Quality of Care and Outcomes Research Interdisciplinary Working Group. Circulation. 2006;113:e873–e923. doi: https://doi.org/10.1161/01.STR.0000223048.70103.F1

Lackland DT, Elkind MS, D’Agostino R, et al. Inclusion of stroke in cardiovascular risk prediction instruments: a statement for healthcare professionals from the American Heart Association/ American Stroke Association. Stroke. 2012;43(7):1998–2027. doi: https://doi.org/10.1161/STR.0b013e31825bcdac

Perel P, Edwards P, Wentz R, Roberts I. Systematic review of prognostic models in traumatic brain injury. BMC Med Inform Decis Mak. 2006;6:38. doi: https://doi.org/10.1186/1472-6947-6-38

Shariat SF, Karakiewicz PI, Margulis V, Kattan MW. Inventory of prostate cancer predictive tools. Curr Opin Urol. 2008;18(3): 279–296. doi: https://doi.org/10.1097/MOU.0b013e3282f9b3e5

Altman DG. Prognostic models: a methodological framework and review of models for breast cancer. Cancer Invest. 2009;27(3): 235–243. doi: https://doi.org/10.1080/07357900802572110

van Dieren S, Beulens JW, Kengne AP, et al. Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review. Heart. 2012;98(5):360–369. doi: https://doi.org/10.1136/heartjnl-2011-300734

Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103. doi: https://doi.org/10.1186/1741-7015-9-103

Ettema RG, Peelen LM, Schuurmans MJ, et al. Prediction mode ls for prolonged intensive care unit stay after cardiac surgery: systematic review and validation study. Circulation. 2010;122:682–689. doi: https://doi.org/10.1161/CIRCULATIONAHA.109.926808

Collins GS, Moons KG. Comparing risk prediction models. BMJ. 2012;344:e3186.

Siontis GC, Tzoulaki I, Siontis KC, Ioannidis JP. Comparisons of established risk prediction models for cardiovascular disease: systematic review. BMJ. 2012;344:e3318. doi: https://doi.org/10.1136/bmj.e3186

Seel RT, Steyerberg EW, Malec JF, et al. Developing and evaluating prediction models in rehabilitation populations. Arch Phys Med Rehabil. 2012;93 8 Suppl S138–S153. doi: https://doi.org/10.1016/j.apmr.2012.04.021

Green SM, Schriger DL, Yealy DM. Methodologic standards for interpreting clinical decision rules in emergency medicine: 2014 update. Ann Emerg Med. 2014;64:286–291. doi: https://doi.org/10.1016/j.annemergmed.2014.01.016

Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Intern Med. 2007;146:450–453. doi: https://doi.org/10.7326/0003-4819-146-6-200703200-00154

Groves T, Godlee F. Open science and reproducible research. BMJ. 2012;344:e4383. doi: https://doi.org/10.1136/bmj.e4383

Collins GS, Omar O, Shanyinde M, Yu LM. A systematic review finds prediction models for chronic kidney were poorly reported and often developed using inappropriate methods. J Clin Epidemiol. 2013;66:268–277. doi: https://doi.org/10.1016/j.jclinepi.2012.06.020

Mallett S, Royston P, Dutton S, et al. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010;8:20. doi: https://doi.org/10.1186/1741-7015-8-20

Mallett S, Royston P, Waters R, et al. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21. doi: https://doi.org/10.1186/1741-7015-8-21

Burton A, Altman DG. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. Br J Cancer. 2004;91(1):4–8. doi: https://doi.org/10.1038/sj.bjc.6601907

Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models. Ann Intern Med. 1993;118(3):201–210. doi: https://doi.org/10.7326/0003-4819-118-3-199302010-00009

Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277(6):488–494.

Steurer J, Haller C, Häuselmann H, et al. Clinical value of prognostic instruments to identify patients with an increased risk for osteoporotic fractures: systematic review. PLoS One. 2011;6(5):e19994. doi: https://doi.org/10.1371/journal.pone.0019994

van Dijk WD, Bemt L, Haak-Rongen S, et al. Multidimensional prognostic indices for use in COPD patient care. A systematic review. Respir Res. 2011;12(1):151. doi: https://doi.org/10.1186/1465-9921-12-151

Hayden JA, Côté P, Bombardier C. Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med. 2006;144(6):427–437. doi: https://doi.org/10.7326/0003-4819-144-6-200603210-00010

Meads C, Ahmed I, Riley RD. A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance. Breast Cancer Res Treat. 2012;132(2):365–377. doi: https://doi.org/10.1007/s10549-011-1818-2

Mushkudiani NA, Hukkelhoven CW, Hernández AV, et al. A sys tematic review finds methodological improvements neces sary for prognostic models in determining traumatic brain injury outcomes. J Clin Epidemiol. 2008;61(4):331–343. doi: https://doi.org/10.1016/j.jclinepi.2007.06.011

Rehn M, Perel P, Blackhall K, Lossius HM. Prognostic models for the early care of trauma patients: a systematic review. Scand J Trauma Resusc Emerg Med. 2011;19:17. doi: https://doi.org/10.1186/1757-7241-19-17

Siontis GC, Tzoulaki I, Ioannidis JP. Predicting death: an empirical evaluation of predictive tools for mortality. Arch Intern Med. 2011;171(19):1721–1726. doi: https://doi.org/10.1001/archinternmed.2011.334

Medlock S, Ravelli ACJ, Tamminga P, et al. Prediction of mortality in very premature infants: a systematic review of prediction models. PLoS One. 2011;6(9):e23441. doi: https://doi.org/10.1371/journal.pone.0023441

Maguire JL, Kulik DM, Laupacis A, et al. Clinical prediction rules for children: a systematic review. Pediatrics. 2011;128(3): e666–e677. doi: https://doi.org/10.1542/peds.2011-0043

Kulik DM, Uleryk EM, Maguire JL. Does this child have appendicitis? A systematic review of clinical prediction rules for children with acute abdominal pain. J Clin Epidemiol. 2013;66(1): 95–104. doi: https://doi.org/10.1016/j.jclinepi.2012.09.004

Kulik DM, Uleryk EM, Maguire JL. Does this child have bacterial meningitis? A systematic review of clinical prediction rules for children with suspected bacterial meningitis. J Emerg Med. 2013;45: 508–519. doi: https://doi.org/10.1016/j.jemermed.2013.03.042

Jacob M, Lewsey JD, Sharpin C, et al. Systematic review and validation of prognostic models in liver transplantation. Liver Transpl. 2005;11(7):814–825. doi: https://doi.org/10.1002/lt.20456

Hussain A, Choukairi F, Dunn K. Predicting survival in thermal injury: a systematic review of methodology of composite prediction models. Burns. 2013;39(5):835–850. doi: https://doi.org/10.1016/j.burns.2012.12.010

Haskins R, Rivett DA, Osmotherly PG. Clinical prediction rules in the physiotherapy management of low back pain: a systematic review. Man Ther. 2012;17(1):9–21. doi: https://doi.org/10.1016/j.math.2011.05.001

Echouffo-Tcheugui JB, Kengne AP. Risk models to predict chronic kidney disease and its progression: a systematic review. PLoS Med. 2012;9(11):e1001344. doi: https://doi.org/10.1371/journal.pmed.1001344

Echouffo-Tcheugui JB, Batty GD, Kivimäki M, Kengne AP. Risk models to predict hypertension: a systematic review. PLoS One. 2013; 8(7):e67370. doi: https://doi.org/10.1371/journal.pone.0067370

Anothaisintawee T, Teerawattananon Y, Wiratkapun C, et al. Risk prediction models of breast cancer: a systematic review of model performances. Breast Cancer Res Treat. 2012;133(1):1–10. doi: https://doi.org/10.1007/s10549-011-1853-z

van Oort L, van den Berg T, Koes BW, et al. Preliminary state of development of prediction models for primary care physical therapy: a systematic review. J Clin Epidemiol. 2012;65(12):1257–1266. doi: https://doi.org/10.1016/j.jclinepi.2012.05.007

Tangri N, Kitsios GD, Inker LA, et al. Risk prediction models for patients with chronic kidney disease: a systematic review. Ann Intern Med. 2013;158(8):596–603. doi: https://doi.org/10.7326/0003-4819-158-8-201304160-00004

van Hanegem N, Breijer MC, Opmeer BC, et al. Prediction models in women with postmenopausal bleeding: a systematic review. Womens Health (Lond Engl). 2012;8(3):251–262. doi: https://doi.org/10.2217/whe.12.10

Minne L, Ludikhuize J, de Jonge E, et al. Prognostic models for predicting mortality in elderly ICU patients: a systematic review. Intensive Care Med. 2011;37(8):1258–1268. doi: https://doi.org/10.1007/s00134-011-2265-6

Leushuis E, van der Steeg JW, Steures P, et al. Prediction models in reproductive medicine: a critical appraisal. Hum Reprod Update. 2009;15(5):537–552. doi: https://doi.org/10.1093/humupd/dmp013

Jaja BN, Cusimano MD, Etminan N, et al. Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review. Neurocrit Care. 2013;18(1):143–153. doi: https://doi.org/10.1007/s12028-012-9792-z

Wlodzimirow KA, Eslami S, Chamuleau RA, et al. Prediction of poor outcome in patients with acute liver failure-systematic review of prediction models. PLoS One. 2012;7(12):e50952. doi: https://doi.org/10.1371/journal.pone.0050952

Phillips B, Wade R, Stewart LA, Sutton AJ. Systematic review and meta-analysis of the discriminatory performance of risk prediction rules in febrile neutropaenic episodes in children and young people. Eur J Cancer. 2010;46(16):2950–2964. doi: https://doi.org/10.1016/j.ejca.2010.05.024

Rubin KH, Friis-Holmberg T, Hermann AP, et al. Risk assessment tools to identify women with increased risk of osteoporotic fracture: complexity or simplicity? A systematic review. J Bone Miner Res. 2013;28(8):1701–1717. doi: https://doi.org/10.1002/jbmr.1956

Abbasi A, Peelen LM, Corpeleijn E, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900. doi: https://doi.org/10.1136/bmj.e5900

Braband M, Folkestad L, Clausen NG, et al. Risk scoring systems for adults admitted to the emergency department: a systematic review. Scand J Trauma Resusc Emerg Med. 2010;18:8. doi: https://doi.org/10.1186/1757-7241-18-8

Maguire JL, Boutis K, Uleryk EM, et al. Should a head-injured child receive a head CT scan? A systematic review of clinical prediction rules. Pediatrics. 2009;124(1):e145–e154. doi: https://doi.org/10.1542/peds.2009-0075

Vuong K, McGeechan K, Armstrong BK, Cust AE. Risk prediction models for incident primary cutaneous melanoma: a systematic review. JAMA Dermatol. 2014;150(4):434–444. doi: https://doi.org/10.1001/jamadermatol.2013.8890

Ahmed I, Debray TP, Moons KG, Riley RD. Developing and validating risk prediction models in an individual participant data meta-analysis. BMC Med Res Methodol. 2014;14:3. doi: https://doi.org/10.1186/1471-2288-14-3

Huen SC, Parikh CR. Predicting acute kidney injury after cardiac surgery: a systematic review. Ann Thorac Surg. 2012;93(1): 337–341. doi: https://doi.org/10.1016/j.athoracsur.2011.09.010

Calle P, Cerro L, Valencia J, Jaimes F. Usefulness of severity scores in patients with suspected infection in the emergency department: a systematic review. J Emerg Med. 2012;42(4): 379–391. doi: https://doi.org/10.1016/j.jemermed.2011.03.033

Usher-Smith JA, Emery J, Kassianos AP, Walter FM. Risk prediction models for melanoma: a systematic review. Cancer Epidemiol Biomarkers Prev. 2014;23(8):1450–1463. doi: https://doi.org/10.1158/1055-9965.EPI-14-0295

Warnell I, Chincholkar M, Eccles M. Predicting perioperative mortality after oesophagectomy: a systematic review of performance and methods of multivariate models. Br J Anaesth. 2015;114(1): 32–43. doi: https://doi.org/10.1093/bja/aeu294

Silverberg N, Gardner AJ, Brubacher J, et al. Systematic review of multivariable prognostic models for mild traumatic brain injury. J Neurotrauma. 2015;32(8):517–526. doi: https://doi.org/10.1089/neu.2014.3600

Delebarre M, Macher E, Mazingue F, et al. Which decision rules meet methodological standards in children with febrile neutropenia? Results of a systematic review and analysis. Pediatr Blood Cancer. 2014; 61(10):1786–1791. doi: https://doi.org/10.1002/pbc.25106

Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: https://doi.org/10.1136/bmj.c332

von Elm E, Altman DG, Egger M, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624): 806–808. doi: https://doi.org/10.1136/bmj.39335.541782.AD

McShane LM, Altman DG, Sauerbrei W, et al. Reporting recom mendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst. 2005;97(16):1180–1184. doi: https://doi.org/10.1093/jnci/dji237

Gallo V, Egger M, McCormack V, et al. STrengthening the Reporting of OBservational studies in Epidemiology — Molecular Epidemiology (STROBE-ME): an extension of the STROBE statement. Eur J Clin Invest. 2012;42(1):1–16. doi: https://doi.org/10.1111/j.1365-2362.2011.02561.x

100

Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD Initiative. Radiology. 2003;226:575–580. doi: https://doi.org/10.1016/S0009-9260(03)00258-7

101

Janssens AC, Ioannidis JP, vanDuijn CM, et al. Strengthening the reporting of genetic risk prediction studies: the GRIPS statement. Eur J Clin Invest. 2011;41(9):1004–1009. doi: https://doi.org/10.1111/j.1365-2362.2011.02494.x

102

Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606. doi: https://doi.org/10.1136/bmj.b606

103

Moons KG, Kengne AP, Woodward M, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio) marker. Heart. 2012;98(9): 683–690. doi: https://doi.org/10.1136/heartjnl-2011-301246

104

Labarère J, Bertrand R, Fine MJ. How to derive and validate clinical prediction models for use in intensive care medicine. Intensive Care Med. 2014;40(4):513–527. doi: https://doi.org/10.1007/s00134-014-3227-6

105

Tzoulaki I, Liberopoulos G, Ioannidis JP. Use of reclassification for assessment of improved prediction: an empirical evaluation. Int J Epidemiol. 2011;40(4):1094–1105. doi: https://doi.org/10.1093/ije/dyr013

106

Peters SA, Bakker M, den Ruijter HM, Bots ML. Added value of CAC in risk stratification for cardiovascular events: a systematic review. Eur J Clin Invest. 2012;42(1):110–116. doi: https://doi.org/10.1111/j.1365-2362.2011.02555.x

107

Wallace E, Smith SM, Perera-Salazar R, et al. Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med Inform Decis Mak. 2011;11:62. doi: https://doi.org/10.1186/1472-6947-11-62

108

Altman DG, McShane LM, Sauerbrei W, Taube SE. Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration. BMC Med. 2012;10:51. doi: https://doi.org/10.1186/1741-7015-10-51

109

Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328(7441): 702–708. doi: https://doi.org/10.1136/bmj.328.7441.702

110

Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet. 1974;2(7872):81–84. doi: https://doi.org/10.1016/s0140-6736(74)91639-0

111

Farrell B, Godwin J, Richards S, Warlow C. The United Kingdom transient ischaemic attack (UK-TIA) aspirin trial: final results. J Neurol Neurosurg Psychiatry. 1991;54(12):1044–1054. doi: https://doi.org/10.1136/jnnp.54.12.1044

112

Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression and Survival Analysis. New York: Springer; 2001.

113

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2): e1000217. doi: https://doi.org/10.1371/journal.pmed.1000217

114

Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis: the TRIPOD statement. Ann Intern Med. 2015;162(1):55–63. doi: https://doi.org/10.7326/M14-0697

115

Morise AP, Haddad WJ, Beckner D. Development and validation of a clinical score to estimate the probability of coronary artery disease in men and women presenting with suspected coronary disease. Am J Med. 1997;102(4):350–356. doi: https://doi.org/10.1016/s0002-9343(97)00086-7

116

Dehing-Oberije C, Yu S, DeRuysscher D, et al. Development and external validation of prognostic model for 2-year survival of non-small-cell lung cancer patients treated with chemoradiotherapy. Int J Radiat Oncol Biol Phys. 2009;74(2):355–362. doi: https://doi.org/10.1016/j.ijrobp.2008.08.052

117

Collins GS, Altman DG. Predicting the 10 year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2. BMJ. 2012;344:e4181. doi: https://doi.org/10.1136/bmj.e4181

118

Michikawa T, Inoue M, Sawada N, et al. Development of a prediction model for 10-year risk of hepatocellular carcinoma in middle-aged Japanese: the Japan Public Health Center-based Prospective Study Cohort II. Prev Med. 2012;55(2):137–143. doi: https://doi.org/10.1016/j.ypmed.2012.05.017

119

Morise AP, Detrano R, Bobbio M, Diamond GA. Development and validation of a logistic regression-derived algorithm for estimating the incremental probability of coronary artery disease before and after exercise testing. J Am Coll Cardiol. 1992;20(5):1187–1196. doi: https://doi.org/10.1016/0735-1097(92)90377-y

120

D’Agostino RB, Grundy S, Sullivan LM, Wilson P. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187. doi: https://doi.org/10.1001/jama.286.2.180

121

Beck DH, Smith GB, Pappachan JV, Millar B. External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive Care Med. 2003;29(2): 249–526. doi: https://doi.org/10.1007/s00134-002-1607-9

122

Collins GS, de Groot JA, Dutton S, et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol. 2014;14:40. doi: https://doi.org/10.1186/1471-2288-14-40

123

Perel P, Prieto-Merino D, Shakur H, et al. Predicting early death in patients with traumatic bleeding: development and validation of prognostic model. BMJ. 2012;345:e5166. doi: https://doi.org/10.1136/bmj.e5166

124

Stiell IG, Greenberg GH, McKnight RD, et al. Decision rules for the use of radiography in acute ankle injuries. Refinement and prospective validation. JAMA. 1993;269(9):1127–1132. doi: https://doi.org/10.1001/jama.269.9.1127

125

Holland JL, Wilczynski NL, Haynes RB. Optimal search strategies for identifying sound clinical prediction studies in EMBASE. BMC Med Inform Decis Mak. 2005;5:11. doi: https://doi.org/10.1186/1472-6947-5-11

126

Ingui BJ, Rogers MA. Searching for clinical prediction rules in MEDLINE. J Am Med Inform Assoc. 2001;8(4):391–397. doi: https://doi.org/10.1136/jamia.2001.0080391

127

Wong SS, Wilczynski NL, Haynes RB, Ramkissoonsingh R. Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE. AMIA Annu Symp Proc. 2003; 2003:728–732.

128

Geersing GJ, Bouwmeester W, Zuithoff P, et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS One. 2012;7(2):e32844. doi: https://doi.org/10.1371/journal.pone.0032844

129

Keogh C, Wallace E, O’Brien KK, et al. Optimized retrieval of primary care clinical prediction rules from to establish a Web-based register. J Clin Epidemiol. 2011;64(8):848–860. doi: https://doi.org/10.1016/j.jclinepi.2010.11.011

130

Rietveld RP, terRiet G, Bindels PJ, et al. Predicting bacterial cause in infectious conjunctivitis: cohort study on informativeness of combinations of signs and symptoms. BMJ. 2004;329(7459): 206–210. doi: https://doi.org/10.1136/bmj.38128.631319.AE

131

Poorten VV, Hart A, Vauterin T, et al. Prognostic index for patients with parotid carcinoma: international external validation in a Belgian-German database. Cancer. 2009;115(3):540–550. doi: https://doi.org/10.1002/cncr.24015

132

Moynihan R, Glassock R, Doust J. Chronic kidney disease controversy: how expanding definitions are unnecessarily labelling many people as diseased. BMJ. 2013;347:f4298. doi: https://doi.org/10.1136/bmj.f4298

133

Moynihan R, Henry D, Moons KG. Using evidence to combat overdiagnosis and overtreatment: evaluating treatments, tests, and disease definitions in the time of too much. PLoS Med. 2014;11(7):e1001655. doi: https://doi.org/10.1371/journal.pmed.1001655

134

Dowling S, Spooner CH, Liang Y, et al. Accuracy of Ottawa Ankle Rules to exclude fractures of the ankle and midfoot in children: a meta-analysis. Acad Emerg Med. 2009;16(4):277–287. doi: https://doi.org/10.1111/j.1553-2712.2008.00333.x.

135

Bachmann LM, Kolb E, Koller MT, et al. Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ. 2003;326(7386):417. doi: https://doi.org/10.1136/bmj.326.7386.417.

136

Büller HR, Ten Cate-Hoek AJ, Hoes AW, et al. Safely ruling out deep venous thrombosis in primary care. Ann Intern Med. 2009;150(4):229–235.

137

Sparks AB, Struble CA, Wang ET, et al. Noninvasive prenatal detection and selective analysis of cell-free DNA obtained from maternal blood: evaluation for trisomy 21 and trisomy 18. Am J Obstet Gynecol. 2012;206(4):319.e1–9. doi: https://doi.org/10.1016/j.ajog.2012.01.030

138

Ankerst DP, Boeck A, Freedland SJ, et al. Evaluating the PCPT risk calculator in ten international biopsy cohorts: results from the Prostate Biopsy Collaborative Group. World J Urol. 2012;30(2): 181–187. doi: https://doi.org/10.1007/s00345-011-0818-5

139

Hippisley-Cox J, Coupland C, Vinogradova Y, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008;336(7659):1475–1482. doi: https://doi.org/10.1136/bmj.39609.449676.25

140

Conroy RM, Pyörälä K, Fitzgerald AP, et al. Estimation of tenyear risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987–1003. doi: https://doi.org/10.1016/s0195-668x(03)00114-3

141

Califf RM, Woodlief LH, Harrell FE, et al. Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. Am Heart J. 1997;133(6):630–639. doi: https://doi.org/10.1016/s0002-8703(97)70164-9

142

McCowan C, Donnan PT, Dewar J, et al. Identifying suspected breast cancer: development and validation of a clinical prediction rule. Br J Gen Pract. 2011;61:e205–e214. doi: https://doi.org/10.3399/bjgp11X572391

143

Campbell HE, Gray AM, Harris AL, et al. Estimation and external validation of a new prognostic model for predicting recurrence-free survival for early breast cancer patients in the UK. Br J Cancer. 2010;103(6):776–786. doi: https://doi.org/10.1038/sj.bjc.6605863

144

Wilson PW, D’Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factor categories. Circulation. 1998; 97(18):1837–1847. doi: https://doi.org/10.1161/01.cir.97.18.1837

145

Kengne AP, Patel A, Marre M, et al. Contemporary model for cardiovascular risk prediction in people with type 2 diabetes. Eur J Cardiovasc Prev Rehabil. 2011;18(3):393–398. doi: https://doi.org/10.1177/1741826710394270

146

Appelboam A, Reuben AD, Benger JR, et al. Elbow extension test to rule out elbow fracture: multicentre, prospective validation and observational study of diagnostic accuracy in adults and children. BMJ. 2008;337:a2428. doi: https://doi.org/10.1136/bmj.a2428

147

Puhan MA, Hansel NN, Sobradillo P, et al; International COPD Cohorts Collaboration Working Group. Largescale international validation of the ADO index in subjects with COPD: an individual subject data analysis of 10 cohorts. BMJ Open. 2012;2(6):e002152. doi: https://doi.org/10.1136/bmjopen-2012-002152

148

Knottnerus JA. The Evidence Base of Clinical Diagnosis. London: BMJ Books; 2002.

149

Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol. 2003; 56(11):1118–1128. doi: https://doi.org/10.1016/s0895-4356(03)00206-3

150

Grobbee DE, Hoes AW. Clinical Epidemiology: Principles, Methods, and Applications for Clinical Research. London: Jones and Bartlett Publishers; 2009.

151

Sackett DL, Tugwell P, Guyatt GH. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2d ed. Boston: Little, Brown; 1991.

152

Biesheuvel CJ, Vergouwe Y, Oudega R, et al. Advantages of the nested case-control design in diagnostic research. BMC Med Res Methodol. 2008;8:48. doi: https://doi.org/10.1186/1471-2288-8-48

153

Knottnerus JA, Dinant GJ. Medicine based evidence, a prerequisite for evidence based medicine. BMJ. 1997;315(7116): 1109–1110. doi: https://doi.org/10.1136/bmj.315.7116.1109

154

Knottnerus JA, vanWeel C, Muris JW. Evaluation of diagnostic procedures. BMJ. 2002;324(7335):477–480. doi: https://doi.org/10.1136/bmj.324.7335.477

155

Rutjes AW, Reitsma JB, Vandenbroucke JP, et al. Casecontrol and two-gate designs in diagnostic accuracy studies. Clin Chem. 2005;51(8):1335–1341. doi: https://doi.org/10.1373/clinchem.2005.048595

156

Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282(11):1061–1066. doi: https://doi.org/10.1001/jama.282.11.1061

157

van Zaane B, Vergouwe Y, Donders AR, Moons KG. Comparison of approaches to estimate confidence intervals of post-test probabilities of diagnostic test results in a nested case-control study. BMC Med Res Methodol. 2012;12:166. doi: https://doi.org/10.1186/1471-2288-12-166

158

Lumbreras B, Parker LA, Porta M, et al. Overinterpretation of clinical applicability in molecular diagnostic research. Clin Chem. 2009;55(4):786–794. doi: https://doi.org/10.1373/clinchem.2008.121517

159

Tzoulaki I, Siontis KC, Ioannidis JP. Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ. 2011;343:d6829. doi: https://doi.org/10.1136/bmj.d6829

160

Greving JP, Wermer MJ, Brown RD, et al. Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol. 2014;13(1):59–66. doi: https://doi.org/10.1016/S1474-4422(13)70263-1

161

Collins GS, Altman DG. Predicting the adverse risk of statin treatment: an independent and external validation of Qstatin risk scores in the UK. Heart. 2012;98(14):1091–1097. doi: https://doi.org/10.1136/heartjnl-2012-302014

162

Glickman SW, Shofer FS, Wu MC, et al. Development and validation of a prioritization rule for obtaining an immediate 12-lead electrocardiogram in the emergency department to identify ST-elevation myocardial infarction. Am Heart J. 2012;163(3): 372–382. doi: https://doi.org/10.1016/j.ahj.2011.10.021

163

Debray TP, Koffijberg H, Lu D, et al. Incorporating published univariable associations in diagnostic and prognostic modeling. BMC Med Res Methodol. 2012;12:121. doi: https://doi.org/10.1186/1471-2288-12-121

164

Debray TP, Koffijberg H, Vergouwe Y, et al. Aggregating published prediction models with individual participant data: a comparison of different approaches. Stat Med. 2012;31(23):2697-712. doi: https://doi.org/10.1002/sim.5412

165

Debray TP, Moons KG, Abo-Zaid GM, et al. Individual participant data meta-analysis for a binary outcome: onestage or two-stage? PLoS One. 2013;8(4):e60650. doi: https://doi.org/10.1371/journal.pone.0060650

166

Debray TP, Moons KG, Ahmed I, et al. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat Med. 2013;32(18):3158–3180. doi: https://doi.org/10.1002/sim.5732

167

Bouwmeester W, Twisk JW, Kappen TH, et al. Prediction models for clustered data: comparison of a random intercept and standard regression model. BMC Med Res Methodol. 2013;13:19. doi: https://doi.org/10.1186/1471-2288-13-19

168

Bouwmeester W, Moons KG, Happen TH, et al. Internal validation of risk models in clustered data: a comparison of bootstrap schemes. Am J Epidemiol. 2013;177(11):1209–1217. doi: https://doi.org/10.1093/aje/kws396

169

Rosner B, Qiu W, Lee ML. Assessing discrimination of risk prediction rules in a clustered data setting. Lifetime Data Anal. 2013; 19(2):242–256. doi: https://doi.org/10.1007/s10985-012-9240-6

170

van Klaveren D, Steyerberg EW, Perel P, Vergouwe Y. Assessing discriminative ability of risk models in clustered data. BMC Med Res Methodol. 2014;14:5. doi: https://doi.org/10.1186/1471-2288-14-5

171

van Klaveren D, Steyerberg EW, Vergouwe Y. Interpretation of concordance measures for clustered data. Stat Med. 2014;33(4):714–716. doi: https://doi.org/10.1002/sim.5928

172

Sanderson J, Thompson SG, White IR, et al. Derivation and assessment of risk prediction models using case-cohort data. BMC Med Res Methodol. 2013;13:113. doi: https://doi.org/10.1186/1471-2288-13-113

173

Ganna A, Reilly M, de Faire U, et al. Risk prediction measures for case-cohort and nested case-control designs: an application to cardiovascular disease. Am J Epidemiol. 2012;175(7):715–724. doi: https://doi.org/10.1093/aje/kwr374

174

Kulathinal S, Karvanen J, Saarela O, Kuulasmaa K. Casecohort design in practice — experiences from the MORGAM Project. Epidemiol Perspect Innov. 2007;4:15. doi: https://doi.org/10.1186/1742-5573-4-15

175

Kengne AP, Beulens JW, Peelen LM, et al. Non-invasive risk scores for prediction of type 2 diabetes (EPIC-InterAct): a validation of existing models. Lancet Diabetes Endocrinol. 2014;2(1):19–29. doi: https://doi.org/10.1016/S2213-8587(13)70103-7

176

Alba AC, Agoritsas T, Jankowski M, et al. Risk prediction models for mortality in ambulatory heart failure patients: a systematic review. Circ Heart Fail. 2013;6(5):881–889. doi: https://doi.org/10.1161/CIRCHEARTFAILURE.112.000043

177

Arkenau HT, Barriuso J, Olmos D, et al. Prospective validation of a prognostic score to improve patient selection for oncology phase I trials. J Clin Oncol. 2009;27(16):2692–2696. doi: https://doi.org/10.1200/JCO.2008.19.5081

178

Ronga A, Vaucher P, Haasenritter J, et al. Development and validation of a clinical prediction rule for chest wall syndrome in primary care. BMC Fam Pract. 2012;13:74. doi: https://doi.org/10.1186/1471-2296-13-74

179

Martinez JA, Belastegui A, Basabe I, et al. Derivation and validation of a clinical prediction rule for delirium in patients admitted to a medical ward: an observational study. BMJ Open. 2012;2(5):e001599. doi: https://doi.org/10.1136/bmjopen-2012-001599

180

Rahimi K, Bennett D, Conrad N, et al. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail. 2014;2(5):440–446. doi: https://doi.org/10.1016/j.jchf.2014.04.008

181

Ebell MH, Afonson AM, Gonzales R, et al. Development and validation of a clinical decision rule for the diagnosis of influenza. J Am Board Fam Med. 2012;25(1):55–62. doi: https://doi.org/10.3122/jabfm.2012.01.110161

182

Counsell C, Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis. 2001;12(3): 159–170. doi: https://doi.org/10.1159/000047699

183

Knottnerus JA. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. J Clin Epidemiol. 2002;55(12):1201–1206. doi: https://doi.org/10.1016/s0895-4356(02)00528-0

184

Moreno R, Apolone G. Impact of different customization strategies in the performance of a general severity score. Crit Care Med. 1997;25(12):2001–2008. doi: https://doi.org/10.1097/00003246-199712000-00017

185

Tu JV, Austin PC, Walld R, et al. Development and validation of the Ontario acute myocardial infarction mortality prediction rules. J Am Coll Cardiol. 2001;37(4):992–997. doi: https://doi.org/10.1016/s0735-1097(01)01109-3

186

Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol. 2010;172(8):971–980. doi: https://doi.org/10.1093/aje/kwq223

187

Kappen TH, Vergouwe Y, van Klei WA, et al. Adaptation of clinical prediction models for application in local settings. Med Decis Making. 2012;32(3):E1–E10. doi: https://doi.org/10.1177/0272989X12439755

188

Oudega R, Hoes AW, Moons KG. The Wells rule does not adequately rule out deep venous thrombosis in primary care patients. Ann Intern Med. 2005;143(2):100–107. doi: https://doi.org/10.7326/0003-4819-143-2-200507190-00008

189

Knottnerus JA, Leffers P. The influence of referral patterns on the characteristics of diagnostic tests. J Clin Epidemiol. 1992;45(10):1143–1154. doi: https://doi.org/10.1016/0895-4356(92)90155-g

190

Knottnerus JA. The effects of disease verification and referral on the relationship between symptoms and diseases. Med Decis Making. 1987;7(3):139–148. doi: https://doi.org/10.1177/0272989X8700700304

191

Eberhart LH, Morin AM, Guber D, et al. Applicability of risk scores for postoperative nausea and vomiting in adults to paediatric patients. Br J Anaesth. 2004;93(3):386–392. doi: https://doi.org/10.1093/bja/aeh221

192

Debray TP, Vergouwe Y, Koffijberg H, et al. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol. 2015;68(3):279–289. doi: https://doi.org/10.1016/j.jclinepi.2014.06.018

193

Klemke CD, Mansmann U, Poenitz N, et al. Prognostic factors and prediction of prognosis by the CTCL Severity Index in mycosis fungoides and Sézary syndrome. Br J Dermatol. 2005;153(1): 118–124. doi: https://doi.org/10.1111/j.1365-2133.2005.06676.x

194

Tay SY, Thoo FL, Sitoh YY, et al. The Ottawa Ankle Rules in Asia: validating a clinical decision rule for requesting X-rays in twisting ankle and foot injuries. J Emerg Med. 1999;17(6):945–947. doi: https://doi.org/10.1016/s0736-4679(99)00120-1

195

Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59(10):1087–1091. doi: https://doi.org/10.1016/j.jclinepi.2006.01.014

196

Groenwold RH, White IR, Donders AR, et al. Missing covariate data in clinical research: when and when not to use the missingindicator method for analysis. CMAJ. 2012;184(11):1265–1269. doi: https://doi.org/10.1503/cmaj.110977

197

Janssen KJ, Donders AR, Harrell FE, et al. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol. 2010;63(7):721–727. doi: https://doi.org/10.1016/j.jclinepi.2009.12.008

198

Janssen KJ, Vergouwe Y, Donders AR, et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem. 2009;55(5):994–1001. doi: https://doi.org/10.1373/clinchem.2008.115345

199

Moons KG, Donders RA, Stijnen T, Harrell FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59(10):1092–1101. doi: https://doi.org/10.1016/j.jclinepi.2006.01.009

200

Sterne JA, White IR, Carlin JB, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. doi: https://doi.org/10.1136/bmj.b2393

201

Vergouwe Y, Royston P, Moons KG, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol. 2010;63(2):205–214. doi: https://doi.org/10.1016/j.jclinepi.2009.03.017

202

Hemingway H, Croft P, Perel P, et al. Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:35595. doi: https://doi.org/10.1136/bmj.e5595

203

Liew SM, Doust J, Glasziou P. Cardiovascular risk scores do not account for the effect of treatment: a review. Heart. 2011;97(9): 689–697. doi: https://doi.org/10.1136/hrt.2010.220442

204

Simon R, Altman DG. Statistical aspects of prognostic factor studies in oncology. Br J Cancer. 1994;69(6):979–985. doi: https://doi.org/10.1038/bjc.1994.192

205

Landefeld CS, Goldman L. Major bleeding in outpatients treated with warfarin: incidence and prediction by factors known at the start of outpatient therapy. Am J Med. 1989;87(2):144–152. doi: https://doi.org/10.1016/s0002-9343(89)80689-8

206

Schuit E, Groenwold RH, Harrell FE, et al. Unexpected predictoroutcome associations in clinical prediction research: causes and solutions. CMAJ. 2013;185(10):E499–E505. doi: https://doi.org/10.1503/cmaj.120812

207

Wong J, Taljaard M, Forster AJ, et al. Addition of timedependent covariates to a survival model significantly improved predictions for daily risk of hospital death. J Eval Clin Pract. 2013;19(2):351–357. doi: https://doi.org/10.1111/j.1365-2753.2012.01832.x

208

Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297(6):611–619. doi: https://doi.org/10.1001/jama.297.6.611

209

Reitsma JB, Rutjes AW, Khan KS, et al. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol. 2009;62(8):797–806. doi: https://doi.org/10.1016/j.jclinepi.2009.02.005

210

Massing MW, Simpson RJ, Rautaharju PM, et al. Usefulness of ventricular premature complexes to predict coronary heart disease events and mortality (from the Atherosclerosis Risk In Communities cohort). Am J Cardiol. 2006;98(12):1609–1612. doi: https://doi.org/10.1016/j.amjcard.2006.06.061

211

Craig JC, Williams GJ, Jones M, et al. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: prospective cohort study of 15 781 febrile illnesses. BMJ. 2010;340:c1594. doi: https://doi.org/10.1136/bmj.c1594

212

Todenhofer T, Renninger M, Schwentner C, et al. A new prognostic model for cancer-specific survival after radical cystectomy including pretreatment thrombocytosis and standard pathological risk factors. BJU Int. 2012;110(11 Pt B):E533–E540. doi: https://doi.org/10.1111/j.1464-410X.2012.11231.x

213

Boggs DA, Rosenberg L, Pencina MJ, et al. Validation of a breast cancer risk prediction model developed for Black women. J Natl Cancer Inst. 2013;105(5):361–367. doi: https://doi.org/10.1093/jnci/djt008

214

Knottnerus JA, Buntinx F. The Evidence Base of Clinical Diagnosis: Theory and Methods of Diagnostic Research. Hoboken, NJ: Wiley-Blackwell; 2009.

215

Naaktgeboren CA, de Groot JA, van Smeden M, et al. Evaluating diagnostic accuracy in the face of multiple reference standards. Ann Intern Med. 2013;159(3):195–202. doi: https://doi.org/10.7326/0003-4819-159-3-201308060-00009

216

Bertens LC, Broekhuizen BD, Naaktgeboren CA, et al. Use of expert panels to define the reference standard in diagnostic research: a systematic review of published methods and reporting. PLoS Med. 2013;10(10):e1001531. doi: https://doi.org/10.1371/journal.pmed.1001531

217

Naaktgeboren CA, Bertens LC, van Smeden M, et al. Value of composite reference standards in diagnostic research. BMJ. 2013; 347:f5605. doi: https://doi.org/10.1136/bmj.f5605

218

de Groot JA, Bossuyt PM, Reitsma JB, et al. Verification problems in diagnostic accuracy studies: consequences and solutions. BMJ. 2011;343:d4770. doi: https://doi.org/10.1136/bmj.d4770

219

de Groot JA, Dendukuri N, Janssen KJ, et al. Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies. Am J Epidemiol. 2012;175(8):847–853. doi: https://doi.org/10.1093/aje/kwr383

220

Rutjes AW, Reitsma JB, DiNisio M, et al. Evidence of bias and variation in diagnostic accuracy studies. CMAJ. 2006;174(4): 469–476. doi: https://doi.org/10.1503/cmaj.050090

221

Rouzier R, Pusztai L, Delaloge S, et al. Nomograms to predict pathologic complete response and metastasis-free survival after preoperative chemotherapy for breast cancer. J Clin Oncol. 2005;23(33):8331–8339. doi: https://doi.org/10.1200/JCO.2005.01.2898

222

Elliott J, Beringer T, Kee F, et al. Predicting survival after treatment for fracture of the proximal femur and the effect of delays to surgery. J Clin Epidemiol. 2003;56(8):788–795. doi: https://doi.org/10.1016/s08954356(03)00129-x

223

Adams LA, Bulsara M, Rossi E, et al. Hepascore: an accurate validated predictor of liver fibrosis in chronic hepatitis C infection. Clin Chem. 2005;51(10):1867–1873. doi: https://doi.org/10.1373/clinchem.2005.048389

224

Hess EP, Brison RJ, Perry JJ, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med. 2012;59(2):115–125. doi: https://doi.org/10.1016/j.annemergmed.2011.07.026

225

Moons KG, Grobbee DE. When should we remain blind and when should our eyes remain open in diagnostic studies? J Clin Epidemiol. 2002;55(7):633–636. doi: https://doi.org/10.1016/s0895-4356(02)00408-0

226

Rutjes AW, Reitsma JB, Coomarasamy A, et al. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health Technol Assess. 2007;11(50):iii, ix–51. doi: https://doi.org/10.3310/hta11500

227

Kaijser J, Sayasneh A, Van Hoorde K, et al. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis. Hum Reprod Update. 2014;20(3):449–452. doi: https://doi.org/10.1093/humupd/dmt059

228

Kaul V, Friedenberg FK, Braitman LE, et al. Development and validation of a model to diagnose cirrhosis in patients with hepatitis C. Am J Gastroenterol. 2002;97(10):2623–2628. doi: https://doi.org/10.1111/j.1572-0241.2002.06040.x

229

Halbesma N, Jansen DF, Heymans MW, et al. Development and validation of a general population renal risk score. Clin J Am Soc Nephrol. 2011;6(7):1731–1738. doi: https://doi.org/10.2215/CJN.08590910

230

Beyersmann J, Wolkewitz M, Schumacher M. The impact of time-dependent bias in proportional hazards modelling. Stat Med. 2008;27(30):6439–6454. doi: https://doi.org/10.1002/sim.3437

231

van Walraven C, Davis D, Forster AJ, Wells GA. Time-dependent bias was common in survival analyses published in leading clinical journals. J Clin Epidemiol. 2004;57(7):672–682. doi: https://doi.org/10.1016/j.jclinepi.2003.12.008

232

Rochon J. Issues in adjusting for covariates arising postrandomization in clinical trials. Drug Inf J. 1999;33:1219–1228.

233

D’Agostino RB. Beyond baseline data: the use of time-varying covariates. J Hypertens. 2008;26(4):639–640. doi: https://doi.org/10.1097/HJH.0b013e3282fcbc22

234

Scheike TH. Time-varying effects in survival analysis. In: Advances in Survival Analysis. Rao CR, ed. Amsterdam: Elsevier; 2004. pp 61–68.

235

Sun GW, Shook TL, Kay GL. Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. J Clin Epidemiol. 1996;49(8):907–916. doi: https://doi.org/10.1016/0895-4356(96)00025-x

236

Rutten FH, Vonken EJ, Cramer MJ, et al. Cardiovascular magnetic resonance imaging to identify left-sided chronic heart failure in stable patients with chronic obstructive pulmonary disease. Am Heart J. 2008;156(3):506–512. doi: https://doi.org/10.1016/j.ahj.2008.04.021

237

Hess EP, Perry JJ, Calder LA, et al. Prospective validation of a modified thrombolysis in myocardial infarction risk score in emergency department patients with chest pain and possible acute coronary syndrome. Acad Emerg Med. 2010;17(4):368–375. doi: https://doi.org/10.1111/j.1553-2712.2010.00696.x

238

Begg CB. Biases in the assessment of diagnostic tests. Stat Med. 1987;6(4):411–423. doi: https://doi.org/10.1002/sim.4780060402

239

Elmore JG, Wells CK, Howard DH, Feinstein AR. The impact of clinical history on mammographic interpretations. JAMA. 1997; 277(1):49–52.

240

Loy CT, Irwig L. Accuracy of diagnostic tests read with and without clinical information: a systematic review. JAMA. 2004;292(13):1602–1609. doi: https://doi.org/10.1001/jama.292.13.1602

241

Loewen P, Dahir K. Risk of bleeding with oral anticoagulants: an updated systematic review and performance analysis of clinical prediction rules. Ann Hematol. 2011;90(10):1191–1200. doi: https://doi.org/10.1007/s00277-011-1267-3

242

Sheth T, Butler C, Chow B, et al. The coronary CT angiography vision protocol: a prospective observational imaging cohort study in patients undergoing non-cardiac surgery. BMJ Open. 2012;2(4):e001474. doi: https://doi.org/10.1136/bmjopen-2012-001474

243

Hippisley-Cox J, Coupland C. Identifying patients with suspected pancreatic cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract. 2012;62(594):e38–e45. doi: https://doi.org/10.3399/bjgp12X616355

244

Holmes JF, Mao A, Awasthi S, et al. Validation of a prediction rule for the identification of children with intra-abdominal injuries after blunt torso trauma. Ann Emerg Med. 2009;54(4):528–533. doi: https://doi.org/10.1016/j.annemergmed.2009.01.019

245

Peduzzi P, Concato J, Feinstein AR, Holford TR. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol. 1995;48(12):1503–1512. doi: https://doi.org/10.1016/0895-4356(95)00048-8

246

Peduzzi P, Concato J, Kemper E, et al. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–1379. doi: https://doi.org/10.1016/s0895-4356(96)00236-3

247

Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165(6):710–718. doi: https://doi.org/10.1093/aje/kwk052

248

Feinstein AR. Multivariable Analysis. New Haven, CT: Yale University Press; 1996.

249

Schumacher M, Holländer N, Schwarzer G, et al. Prognostic factor studies. In: Handbook of Statistics in Clinical Oncology. Crowley J, Hoering A, eds. 3rd ed. London: Chapman and Hall/CRC; 2012. pp. 415–470.

250

Courvoisier DS, Combescure C, Agoritsas T, et al. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64(9): 993–1000. doi: https://doi.org/10.1016/j.jclinepi.2010.11.012

251

Jinks RC. Sample size for multivariable prognostic models. PhD thesis. University College London; 2012.

252

Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–138. doi: https://doi.org/10.1097/EDE.0b013e3181c30fb2

253

Steyerberg EW, Calster BV, Pencina MJ. Performance measures for prediction models and markers: evaluation of predictions and classifications. Rev Esp Cardiol (Engl Ed). 2011;64(9):788–794. doi: https://doi.org/10.1016/j.recesp.2011.04.017

254

Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58(5):475–483. doi: https://doi.org/10.1016/j.jclinepi.2004.06.017

255

Audigé L, Bhandari M, Kellam J. How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand. 2004;75(2):184–194. doi: https://doi.org/10.1080/00016470412331294445

256

Genders TS, Steyerberg EW, Hunink MG, et al. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ. 2012;344:e3485. doi: https://doi.org/10.1136/bmj.e3485

257

Thompson DO, Hurtado TR, Liao MM, et al. Validation of the Simplified Motor Score in the out-ofhospital setting for the prediction of outcomes after traumatic brain injury. Ann Emerg Med. 2011;58(5):417–425. doi: https://doi.org/10.1016/j.annemergmed.2011.05.033

258

Ambler G, Omar RZ, Royston P, et al. Generic, simple risk stratification model for heart valve surgery. Circulation. 2005;112(2):224–231. doi: https://doi.org/10.1161/CIRCULATIONAHA.104.515049

259

Mackinnon A. The use and reporting of multiple imputation in medical research — a review. J Intern Med. 2010;268(6):586–593. doi: https://doi.org/10.1111/j.1365-2796.2010.02274.x

260

Hussain A, Dunn KW. Predicting length of stay in thermal burns: a systematic review of prognostic factors. Burns. 2013;39(7): 1331–1340. doi: https://doi.org/10.1016/j.burns.2013.04.026

261

Tangri N, Stevens LA, Griffith J, et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553–1559. doi: https://doi.org/10.1001/jama.2011.451

262

Steyerberg EW, Mushkudiani N, Perel P, et al. Predicting outcome after traumatic brain injury: development and internatio nal validation of prognostic scores based on admission characteristics. PLoS Med. 2008;5(8):e165. doi: https://doi.org/10.1371/journal.pmed.0050165

263

Tammemagi CM, Pinsky PF, Caporaso NE, et al. Lung cancer risk prediction: Prostate, Lung, Colorectal And Ovarian Cancer Screening Trial models and validation. J Natl Cancer Inst. 2011;103(13): 1058–1068. doi: https://doi.org/10.1093/jnci/djr173

264

Altman DG, Lausen B, Sauerbrei W, Schumacher M. Dangers of using “optimal” cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst. 1994;86(11):829–835. doi: https://doi.org/10.1093/jnci/86.11.829

265

Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25(1):127–141. doi: https://doi.org/10.1002/sim.2331

266

Royston P, Sauerbrei W. Multivariable Model-Building: A Pragmatic Approach to Regression Analysis Based on Fractional Polynomials for Modelling Continuous Variables. Chichester: John Wiley; 2008.

267

Veerbeek JM, Kwakkel G, van Wegen EE, et al. Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke. 2011;42(5):1482–1488. doi: https://doi.org/10.1161/STROKEAHA.110.604090

268

Lubetzky-Vilnai A, Ciol M, McCoy SW. Statistical analysis of clinical prediction rules for rehabilitation interventions: current state of the literature. Arch Phys Med Rehabil. 2014;95(1):188–196. doi: https://doi.org/10.1016/j.apmr.2013.08.242

269

Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–1931. doi: https://doi.org/10.1093/eurheartj/ehu207

270

Ioannidis JP. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640–648. doi: https://doi.org/10.1097/EDE.0b013e31818131e7

271

Hrynaszkiewicz I, Norton ML, Vickers AJ, Altman DG. Preparing raw clinical data for publication: guidance for journal editors, authors, and peer reviewers. Trials. 2010;11:9. doi: https://doi.org/10.1186/1745-6215-11-9

272

Hosmer DW, Lemeshow S. Applied Logistic Regression. New York: Wiley; 2000.

273

Vittinghoff E. Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models. New York: Springer; 2005.

274

Hosmer DW, Lemeshow S, May S. Applied Survival Analysis: Regression Modelling of Time-To-Event Data. Hoboken, NJ: WileyInterscience; 2008.

275

Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer; 2001.

276

Kuhn M, Johnson K. Applied Predictive Modelling. New York: Springer; 2013.

277

Andersen PK, Skovgaard LT. Regression With Linear Predictors. New York: Springer; 2010.

278

Hippisley-Cox J, Coupland C, Vinogradova Y, et al. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007;335(7611):136. doi: https://doi.org/10.1136/bmj.39261.471806.55

279

Moreno L, Krishnan JA, Duran P, Ferrero F. Development and validation of a clinical prediction rule to distinguish bacterial from viral pneumonia in children. Pediatr Pulmonol. 2006;41(4): 331–337. doi: https://doi.org/10.1002/ppul.20364

280

Anderson KM, Odell PM, Wilson PW, Kannel WB. Cardiovascular disease risk profiles. Am Heart J. 1991;121(1 Pt 2):293–298. doi: https://doi.org/10.1016/0002-8703(91)90861-b

281

Royston P, Parmar MK. Flexible parametric proportionalhazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21(15):2175–2197. doi: https://doi.org/10.1002/sim.1203

282

Hans D, Durosier C, Kanis JA, et al. Assessment of the 10-year probability of osteoporotic hip fracture combining clinical risk factors and heel bone ultrasound: the EPISEM prospective cohort of 12,958 elderly women. J Bone Miner Res. 2008;23(7):1045–1051. doi: https://doi.org/10.1359/jbmr.080229

283

Bohensky MA, Jolley D, Pilcher DV, et al. Prognostic models based on administrative data alone inadequately predict the survival outcomes for critically ill patients at 180 days posthospital discharge. J Crit Care. 2012;27(4):422.e11–e21. doi: https://doi.org/10.1016/j.jcrc.2012.03.008

284

Barrett TW, Martin AR, Storrow AB, et al. A clinical prediction model to estimate risk for 30-day adverse events in emergency department patients with symptomatic atrial fibrillation. Ann Emerg Med. 2011;57(1):1–12. doi: https://doi.org/10.1016/j.annemergmed.2010.05.031

285

Krijnen P, van Jaarsveld BC, Steyerberg EW, et al. A cli nical prediction rule for renal artery stenosis. Ann Intern Med. 1998; 129(9):705–711. doi: https://doi.org/10.7326/0003-4819-129-9-199811010-00005

286

Smits M, Dippel DW, Steyerberg EW, et al. Predicting intracranial traumatic findings on computed tomography in patients with minor head injury: the CHIP prediction rule. Ann Intern Med. 2007;146(6):397–405. doi: https://doi.org/10.7326/0003-4819-146-6-200703200-00004

287

Moons KG, Donders AR, Steyerberg EW, Harrell FE. Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example. J Clin Epidemiol. 2004;57(12):1262–1270. doi: https://doi.org/10.1016/j.jclinepi.2004.01.020

288

Mantel N. Why stepdown procedures in variable selection? Technometrics. 1970;12:621–625.

289

Bleeker SE, Moll HA, Steyerberg EW, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826–832. doi: https://doi.org/10.1016/s0895-4356(03)00207-5

290

Steyerberg EW, Borsboom GJ, van Houwelingen HC, et al. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23(16): 2567–2586. doi: https://doi.org/10.1002/sim.1844

291

van Houwelingen HC, Sauerbrei W. Cross-validation, shrinkage and variable selection in linear regression revisited. Open J Statist. 2013;3:79–102. doi: https://doi.org/10.4236/OJS.2013.32011

292

Sauerbrei W, Boulesteix AL, Binder H. Stability investigations of multivariable regression models derived from low- and highdimensional data. J Biopharm Stat. 2011;21(6):1206–1231. doi: https://doi.org/10.1080/10543406.2011.629890

293

Harrell FE, Lee KL, Califf RM, et al. Regression modelling strate gies for improved prognostic prediction. Stat Med. 1984;3(2): 143–152. doi: https://doi.org/10.1002/sim.4780030207

294

van Houwelingen JC, LeCessie S. Predictive value of statistical models. Stat Med. 1990;9(11):1303–1325. doi: https://doi.org/10.1002/sim.4780091109

295

Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: a comparison of resampling methods. Bioinformatics. 2005;21(15): 3301–3307. doi: https://doi.org/10.1093/bioinformatics/bti499

296

Chatfield C. Model uncertainty, data mining and statistical inference. J R Stat Soc A. 1995;158(3):419–466. doi: https://doi.org/10.2307/2983440

297

Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Stat Med. 2007; 26(30):5512–5528. doi: https://doi.org/10.1002/sim.3148

298

Heymans MW, van Buuren S, Knol DL, et al. Variable selection under multiple imputation using the bootstrap in a prognostic study. BMC Med Res Meth. 2007;7:33. doi: https://doi.org/10.1186/1471-2288-7-33

299

Castaldi PJ, Dahabreh IJ, Ioannidis JP. An empirical assessment of validation practices for molecular classifiers. Brief Bioinform. 2011;12(3):189–202. doi: https://doi.org/10.1093/bib/bbq073

300

Varma S, Simon R. Bias in error estimation when using crossvalidation for model selection. BMC Bioinformatics. 2006;7:91. doi: https://doi.org/10.1186/1471-2105-7-91

301

Vach K, Sauerbrei W, Schumacher M. Variable selection and shrinkage: comparison of some approaches. Stat Neerl. 2001; 55(1):53–75. doi: https://doi.org/10.4236/OJS.2013.32011

302

Lin IF, Chang WP, Liao YN. Shrinkage methods enhanced the accuracy of parameter estimation using Cox models with small number of events. J Clin Epidemiol. 2013;66(7):743–751. doi: https://doi.org/10.1016/j.jclinepi.2013.02.002

303

Ambler G, Seaman S, Omar RZ. An evaluation of penalised survival methods for developing prognostic models with rare events. Stat Med. 2012;31(11–12):1150–1161. doi: https://doi.org/10.1002/sim.4371

304

Yourman LC, Lee SJ, Schonberg MA, et al. Prognostic indices for older adults: a systematic review. JAMA. 2012;307(2):182–192. doi: https://doi.org/10.1001/jama.2011.1966

305

Spelt L, Andersson B, Nilsson J, Andersson R. Prognostic models for outcome following liver resection for colorectal cancer metastases: a systematic review. Eur J Surg Oncol. 2012;38(1): 16–24. doi: https://doi.org/10.1016/j.ejso.2011.10.013

306

Nam RK, Kattan MW, Chin JL, et al. Prospective multiinstitutional study evaluating the performance of prostate cancer risk calculators. J Clin Oncol. 2011;29(22):2959–2964. doi: https://doi.org/10.1200/JCO.2010.32.6371

307

Meffert PJ, Baumeister SE, Lerch MM, et al. Development, external validation, and comparative assessment of a new diagnostic score for hepatic steatosis. Am J Gastroenterol. 2014;109(9): 1404–1414. doi: https://doi.org/10.1038/ajg.2014.155

308

Collins GS, Altman DG. Identifying patients with undetected colorectal cancer: an independent validation of QCancer (Colorectal). Br J Cancer. 2012;107(2):260–265. doi: https://doi.org/10.1038/bjc.2012.266

309

Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. doi: https://doi.org/10.1186/1471-2288-13-33

310

Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26(8):1364–1370. doi: https://doi.org/10.1200/JCO.2007.12.9791

311

Zivanovic O, Jacks LM, Iasonos A, et al. A nomogram to predict postresection 5-year overall survival for patients with uterine leiomyosarcoma. Cancer. 2012;118(3):660–669. doi: https://doi.org/10.1002/cncr.26333

312

Kanis JA, Oden A, Johnell O, et al. The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18(8):1033–1046. doi: https://doi.org/10.1007/s00198-007-0343-y

313

Papaioannou A, Morin S, Cheung AM, et al. 2010 clinical practice guidelines for the diagnosis and management of osteoporosis in Canada: summary. CMAJ. 2010;182(17):1864–1873. doi: https://doi.org/10.1503/cmaj.100771

314

Collins GS, Michaëlsson K. Fracture risk assessment: state of the art, methodologically unsound, or poorly reported? Curr Osteoporos Rep. 2012;10(3):199–207. doi: https://doi.org/10.1007/s11914-012-0108-1

315

Collins GS, Mallett S, Altman DG. Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores. BMJ. 2011;342:d3651. doi: https://doi.org/10.1136/bmj.d3651

316

Järvinen TL, Jokihaara J, Guy P, et al. Conflicts at the heart of the FRAX tool. CMAJ. 2014;186(3):165–167. doi: https://doi.org/10.1503/cmaj.121874

317

Balmaña J, Stockwell DH, Steyerberg EW, et al. Prediction of MLH1 and MSH2 mutations in Lynch syndrome. JAMA. 2006;296(12):1469–1478. doi: https://doi.org/10.1001/jama.296.12.1469

318

Bruins Slot MH, Rutten FH, van der Heijden GJ, et al. Diagnosing acute coronary syndrome in primary care: comparison of the physicians’ risk estimation and a clinical decision rule. Fam Pract. 2011;28(3):323–328. doi: https://doi.org/10.1093/fampra/cmq116

319

Suarthana E, Vergouwe Y, Moons KG, et al. A diagnostic model for the detection of sensitization to wheat allergens was developed and validated in bakery workers. J Clin Epidemiol. 2010;63(9): 1011–1019. doi: https://doi.org/10.1016/j.jclinepi.2009.10.008

320

Uno H, Cai T, Pencina MJ, et al. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med. 2011;30(10):1105–1117. doi: https://doi.org/10.1002/sim.4154

321

Akazawa K. Measures of explained variation for a regression model used in survival analysis. J Med Syst. 1997;21(4):229–238. doi: https://doi.org/10.1023/a:1022884504683

322

Choodari-Oskooei B, Royston P, Parmar MK. A simulation study of predictive ability measures in a survival model I: explained variation measures. Stat Med. 2012;31(23):2627–2643. doi: https://doi.org/10.1002/sim.4242

323

Heller G. A measure of explained risk in the proportional ha zards model. Biostatistics. 2012;13(2):315–325. doi: https://doi.org/10.1093/biostatistics/kxr047

324

Korn EL, Simon R. Measures of explained variation for survival data. Stat Med. 1990;9(5):487–503. doi: https://doi.org/10.1002/sim.4780090503

325

Mittlböck M, Schemper M. Explained variation for logistic regression. Stat Med. 1996;15(19):1987–1997. doi: https://doi.org/10.1002/(SICI)1097-0258(19961015)15:193.0.CO;2-9

326

Royston P. Explained variation for survival models. Stata Journal. 2006;6(1):83–96. doi: https://doi.org/10.1177/1536867X0600600105

327

Schemper M. Predictive accuracy and explained variation. Stat Med. 2003;22(14):2299–2308. doi: https://doi.org/10.1002/sim.1486

328

Schemper M, Henderson R. Predictive accuracy and explained variation in Cox regression. Biometrics. 2000;56(1):249–255. doi: https://doi.org/10.1111/j.0006-341x.2000.00249.x

329

Schemper M, Stare J. Explained variation in survival analysis. Stat Med. 1996;15(19):1999–2012. doi: https://doi.org/10.1002/(SICI)1097-0258(19961015)15:193.0.CO;2-D

330

Gerds T, Schumacher M. Consistent estimation of the expected Brier score in general survival models with right-censored event times. Biom J. 2006;48(6):1029–1040. doi: https://doi.org/10.1002/bimj.200610301

331

Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010;63(8):938–939. doi: https://doi.org/10.1016/j.jclinepi.2009.11.009

332

Gerds TA, Cai T, Schumacher M. The performance of risk prediction models. Biom J. 2008;50(4):457–479. doi: https://doi.org/10.1002/bimj.200810443

333

Royston P, Sauerbrei W. A new measure of prognostic separation in survival data. Stat Med. 2004;23(5):723–748. doi: https://doi.org/10.1002/sim.1621

334

DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845.

335

Demler OV, Pencina MJ, D’Agostino RB. Misuse of DeLong test to compare AUCs for nested models. Stat Med. 2012;31(23): 2577–2587. doi: https://doi.org/10.1002/sim.5328

336

Moonesinghe SR, Mythen MG, Das P, et al. Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: qualitative systematic review. Anesthesiology. 2013;119(4):959–981. doi: https://doi.org/10.1097/ALN.0b013e3182a4e94d

337

Wallace E, Stuart E, Vaughan N, et al. Risk prediction models to predict emergency hospital admission in community-dwelling adults: a systematic review. Med Care. 2014;52(8):751–765. doi: https://doi.org/10.1097/MLR.0000000000000171

338

Widera C, Pencina MJ, Bobadilla M, et al. Incremental prognostic value of biomarkers beyond the GRACE (Global Registry of Acute Coronary Events) score and high-sensitivity cardiac troponin T in non-ST-elevation acute coronary syndrome. Clin Chem. 2013;59(10):1497–1505. doi: https://doi.org/10.1373/clinchem.2013.206185

339

Pencina MJ, D’Agostino RB, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–172. doi: https://doi.org/10.1002/sim.2929

340

Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–935. doi: https://doi.org/10.1161/CIRCULATIONAHA.106.672402

341

Hlatky MA, Greenland P, Arnett DK, et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009;119(17):2408–2416. doi: https://doi.org/10.1161/CIRCULATIONAHA.109.192278

342

Cook NR. Assessing the incremental role of novel and emerging risk factors. Curr Cardiovasc Risk Rep. 2010;4(2):112–119. doi: https://doi.org/10.1007/s12170-010-0084-x

343

Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Med Res Methodol. 2011;11:13. doi: https://doi.org/10.1186/1471-2288-11-13

344

Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795–802. doi: https://doi.org/10.7326/0003-4819-150-11-200906020-00007

345

Cook NR, Paynter NP. Performance of reclassification statistics in comparing risk prediction models. Biom J. 2011;53(2):237–258. doi: https://doi.org/10.1002/bimj.201000078

346

Cook NR. Clinically relevant measures of fit? A note of caution. Am J Epidemiol. 2012;176(6):488–491. doi: https://doi.org/10.1093/aje/kws208

347

Pencina MJ, D’Agostino RB, Pencina KM, et al. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176(6):473–481. doi: https://doi.org/10.1093/aje/kws207

348

Pencina MJ, D’Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med. 2010;48(12):1703–1711. doi: https://doi.org/10.1515/CCLM.2010.340

349

Van Calster B, Vickers AJ, Pencina MJ, et al. Evaluation of markers and risk prediction models: overview of relationships between NRI and decision-analytic measures. Med Decis Making. 2013;33(4): 490–501. doi: https://doi.org/10.1177/0272989X12470757

350

Hilden J, Gerds TA. A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index. Stat Med. 2014;33(19):3405–3414. doi: https://doi.org/10.1002/sim.5804

351

Pepe MS. Problems with risk reclassification methods for evaluating prediction models. Am J Epidemiol. 2011;173(11): 1327–1335. doi: https://doi.org/10.1093/aje/kwr013

352

Mihaescu R, van Zitteren M, van Hoek M, et al. Improvement of risk prediction by genomic profiling: reclassification measures versus the area under the receiver operating characteristic curve. Am J Epidemiol. 2010;172(3):353–361. doi: https://doi.org/10.1093/aje/kwq122

353

Mühlenbruch K, Heraclides A, Steyerberg EW, et al. Assessing improvement in disease prediction using net reclassification improvement: impact of risk cut-offs and number of risk categories. Eur J Epidemiol. 2013;28(1):25–33. doi: https://doi.org/10.1007/s10654-012-9744-0

354

Pepe M, Fang J, Feng Z, et al. The Net Reclassification Index (NRI): a Misleading Measure of Prediction Improvement with Miscalibrated or Overfit Models. UW Biostatistics Working Paper Series. Working Paper 392. Madison, WI: University of Wisconsin; 2013.

355

Vickers AJ, Pepe M. Does the net reclassification improvement help us evaluate models and markers? Ann Intern Med. 2014;160(2):136–137. doi: https://doi.org/10.7326/M13-2841

356

Hilden J. Commentary: On NRI, IDI, and “good-looking” statistics with nothing underneath. Epidemiology. 2014;25(2): 265–267. doi: https://doi.org/10.1097/EDE.0000000000000063

357

Leening MJ, Vedder MM, Witteman JCM, et al. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann Intern Med. 2014;160(2):122–131. doi: https://doi.org/10.7326/M13-1522

358

Al-Radi OO, Harrell FE, Caldarone CA, et al. Case complexity scores in congenital heart surgery: a comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg. 2007;133(4):865–875. doi: https://doi.org/10.1016/j.jtcvs.2006.05.071

359

Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012;157(4):294–295. doi: https://doi.org/10.7326/0003-4819-157-4-201208210-00014

360

Van Calster B, Vickers AJ. Calibration of risk prediction models: impact on decision-analytic performance. Med Decis Making. 2015;35(2):162–169. doi: https://doi.org/10.1177/0272989X14547233

361

Vickers AJ. Decision analysis for the evaluation of diagnostic tests, prediction models and molecular markers. Am Stat. 2008;62(4): 314–320. doi: https://doi.org/10.1198/000313008X370302

362

Vickers AJ, Cronin AM, Kattan MW, et al. Clinical benefits of a multivariate prediction model for bladder cancer: a decision analytic approach. Cancer. 2009;115(23):5460–5469. doi: https://doi.org/10.1002/cncr.24615

363

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6): 565–574. doi: https://doi.org/10.1177/0272989X06295361

364

Baker SG. Putting risk prediction in perspective: relative utility curves. J Natl Cancer Inst. 2009;101(22):1538–1542. doi: https://doi.org/10.1093/jnci/djp353

365

Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009;172(4):729–748. doi: https://doi.org/10.1111/j.1467-985X.2009.00592.x

366

Baker SG, Kramer BS. Evaluating a new marker for risk prediction: decision analysis to the rescue. Discov Med. 2012; 14(76):181–188.

367

Moons KG, de Groot JA, Linnet K, et al. Quantifying the added value of a diagnostic test or marker. Clin Chem. 2012;58(10): 1408–1417. doi: https://doi.org/10.1373/clinchem.2012.182550

368

Held U, Bové DS, Steurer J, Held L. Validating and updating a risk model for pneumonia — a case study. BMC Med Res Methodol. 2012;12:99. doi: https://doi.org/10.1186/1471-2288-12-99

369

Cindolo L, Chiodini P, Gallo C, et al. Validation by calibration of the UCLA integrated staging system prognostic model for nonmetastatic renal cell carcinoma after nephrectomy. Cancer. 2008;113(1):65–71. doi: https://doi.org/10.1002/cncr.23517

370

Baart AM, Atsma F, McSweeney EN, et al. External validation and updating of a Dutch prediction model for low hemoglobin deferral in Irish whole blood donors. Transfusion. 2014;54(3 Pt 2): 762–769. doi: https://doi.org/10.1111/trf.12211

371

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–89. doi: https://doi.org/10.1016/S0140-6736(09)60329-9

372

Janssen KJ, Vergouwe Y, Kalkman CJ, et al. A simple method to adjust clinical prediction models to local circumstances. Can J Anaesth. 2009;56(3):194–201. doi: https://doi.org/10.1007/s12630-009-9041-x

373

van Houwelingen HC. Validation. calibration, revision and combination of prognostic survival models. Stat Med. 2000;19(24): 3401–3415. doi: https://doi.org/10.1002/1097-0258(20001230)19:243.0.co;2-2

374

Manola J, Royston P, Elson P, et al. Prognostic model for survival in patients with metastatic renal cell carcinoma: results from the International Kidney Cancer Working Group. Clin Cancer Res. 2011;17(16):5443–5450. doi: https://doi.org/10.1158/1078-0432.CCR-11-0553

375

Krupp NL, Weinstein G, Chalian A, et al. Validation of a transfusion prediction model in head and neck cancer surgery. Arch Otolaryngol Head Neck Surg. 2003;129(12):1297–1302. doi: https://doi.org/10.1001/archotol.129.12.1297

376

Morra E, Cesana C, Klersy C, et al. Clinical characteristics and factors predicting evolution of asymptomatic IgM monoclonal gammopathies and IgM-related disorders. Leukemia. 2004;18(9):1512–1517. doi: https://doi.org/10.1038/sj.leu.2403442

377

Kelder JC, Cramer MJ, van Wijngaarden J, et al. The diagnostic value of physical examination and additional testing in primary care patients with suspected heart failure. Circulation. 2011;124(25):2865–2873. doi: https://doi.org/10.1161/CIRCULATIONAHA.111.019216

378

Haybittle JL, Blamey RW, Elston CW, et al. A prognostic index in primary breast cancer. Br J Cancer. 1982;45(3):361–366. doi: https://doi.org/10.1038/bjc.1982.62

379

Tang EW, Wong CK, Herbison P. Global Registry of Acute Coronary Events (GRACE) hospital discharge risk score accurately predicts long-term mortality post acute coronary syndrome. Am Heart J. 2007;153(1):29–35. doi: https://doi.org/10.1016/j.ahj.2006.10.004

380

Bang H, Edwards AM, Bomback AS, et al. Development and validation of a patient selfassessment score for diabetes risk. Ann Intern Med. 2009;151(11):775–783. doi: https://doi.org/10.7326/0003-4819-151-11-200912010-00005

381

Chen L, Magliano DJ, Balkau B, et al. AUSDRISK: an Australian Type 2 Diabetes Risk Assessment Tool based on demographic, lifestyle and simple anthropometric measures. Med J Aust. 2010;192(4): 197–202. doi: https://doi.org/10.5694/j.1326-5377.2010.tb03507.x

382

Starmans R, Muris JW, Fijten GH, et al. The diagnostic value of scoring models for organic and non-organic gastrointestinal disease, including the irritable-bowel syndrome. Med Decis Making. 1994;14(3):208–216. doi: https://doi.org/10.1177/0272989X9401400302

383

Tzoulaki I, Seretis A, Ntzani EE, Ioannidis JP. Mapping the expanded often inappropriate use of the Framingham Risk Score in the medical literature. J Clin Epidemiol. 2014;67(5):571–577. doi: https://doi.org/10.1016/j.jclinepi.2013.10.021

384

Harrison DA, Rowan KM. Outcome prediction in critical care: the ICNARC model. Curr Opin Crit Care. 2008;14(5):506–512. doi: https://doi.org/10.1097/MCC.0b013e328310165a

385

Kanaya AM, WasselFyr CL, de Rekeneire N, et al. Predicting the development of diabetes in older adults: the derivation and validation of a prediction rule. Diabetes Care. 2005;28(2):404–408. doi: https://doi.org/10.2337/diacare.28.2.404

386

Stephens JW, Ambler G, Vallance P, et al. Cardiovascular risk and diabetes. Are the methods of risk prediction satisfactory? Eur J Cardiovasc Prev Rehabil. 2004;11(6):521–528. doi: https://doi.org/10.1097/01.hjr.0000136418.47640.bc

387

Cogswell R, Kobashigawa E, McGlothlin D, et al. Validation of the Registry to Evaluate Early and Long-Term Pulmonary Arterial Hypertension Disease Management (REVEAL) pulmonary hypertension prediction model in a unique population and utility in the prediction of long-term survival. J Heart Lung Transplant. 2012;31(11):1165–1170. doi: https://doi.org/10.1016/j.healun.2012.08.009

388

Eagle KA, Lim MJ, Dabbous OH, et al. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. 2004;291(22):2727–2733. doi: https://doi.org/10.1001/jama.291.22.2727

389

Geersing GJ, Erkens PM, Lucassen WA, et al. Safe exclusion of pulmonary embolism using the Wells rule and qualitative d-dimer testing in primary care: prospective cohort study. BMJ. 2012; 345:e6564. doi: https://doi.org/10.1136/bmj.e6564

390

Collins GS, Altman DG. Identifying patients with undetected gastro-oesophageal cancer in primary care: external validation of QCancer® (Gastro-Oesophageal). Eur J Cancer. 2013;49(5): 1040–1048. doi: https://doi.org/10.1016/j.ejca.2012.10.023

391

de Vin T, Engels B, Gevaert T, et al. Stereotactic radiotherapy for oligometastatic cancer: a prognostic model for survival. Ann Oncol. 2014;25(2):467–471. doi: https://doi.org/10.1093/annonc/mdt537

392

Bernasconi P, Klersy C, Boni M, et al. World Health Organization classification in combination with cytogenetic markers improves the prognostic stratification of patients with de novo primary myelodysplastic syndromes. Br J Haematol. 2007;137(3):193–205. doi: https://doi.org/10.1111/j.1365-2141.2007.06537.x

393

Schemper M, Smith TL. A note on quantifying follow-up in studies of failure time. Control Clin Trials. 1996;17(4):343–346. doi: https://doi.org/10.1016/0197-2456(96)00075-x

394

Echouffo-Tcheugui JB, Woodward M, Kengne AP. Predicting a post-thrombolysis intracerebral hemorrhage: a systematic review. J Thromb Haemost. 2013;11(5):862–871. doi: https://doi.org/10.1111/jth.12186

395

Le Gal G, Righini M, Roy PM, et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med. 2006;144(3):165–171. doi: https://doi.org/10.7326/0003-4819-144-3-200602070-00004

396

Davis JL, Worodria W, Kisembo H, et al. Clinical and radiographic factors do not accurately diagnose smear-negative tuberculosis in HIV-infected inpatients in Uganda: a cross-sectional study. PLoS One. 2010;5(3):e9859. doi: https://doi.org/10.1371/journal.pone.0009859

397

Ji R, Shen H, Pan Y, et al. Risk score to predict gastrointestinal bleeding after acute ischemic stroke. BMC Gastroenterol. 2014; 14:130. doi: https://doi.org/10.1186/1471-230X-14-130

398

Marrugat J, Subirana I, Ramos R, et al. Derivation and validation of a set of 10-year cardiovascular risk predictive functions in Spain: the FRESCO Study. Prev Med. 2014;61:66–74. doi: https://doi.org/10.1016/j.ypmed.2013.12.031

399

Hensgens MP, Dekkers OM, Goorhuis A, et al. Predicting a complicated course of Clostridium difficile infection at the bedside. Clin Microbiol Infect. 2014;20(5):O301–O308. doi: https://doi.org/10.1111/1469-0691.12391

400

Hak E, Wei F, Nordin J, et al. Development and validation of a clinical prediction rule for hospitalization due to pneumonia or influenza or death during influenza epidemics among communitydwelling elderly persons. J Infect Dis. 2004;189(3):450–458. doi: https://doi.org/10.1086/381165

401

Vandenbroucke JP, von Elm E, Altman DG, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology. 2007;18(6):805–835. doi: https://doi.org/10.1097/EDE.0b013e3181577511

402

Schnabel RB, Sullivan LM, Levy D, et al. Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. Lancet. 2009;373(9665): 739–745. doi: https://doi.org/10.1016/S0140-6736(09)60443-8

403

Lang TA, Altman DG. Basic statistical reporting for articles published in clinical medical journals: the SAMPL guidelines. In: Science Editors’ Handbook. Smart P, Maisonneuve H, Polderman A, eds. European Association of Science Editors; 2013.

404

Binder H, Sauerbrei W, Royston P. Comparison between splines and fractional polynomials for multivariable model building with continuous covariates: a simulation study with continuous response. Stat Med. 2013;32(13):2262–2277. doi: https://doi.org/10.1002/sim.5639

405

Harrison DA, Parry GJ, Carpenter JR, et al. A new risk prediction model for critical care: the Intensive Care National Audit & Research Centre (ICNARC) model. Crit Care Med. 2007;35(4):1091–1098. doi: https://doi.org/10.1097/01.CCM.0000259468.24532.44

406

Brady AR, Harrison D, Black S, et al. Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United Kingdom. Pediatrics. 2006;117(4): e733–e742. doi: https://doi.org/10.1542/peds.2005-1853

407

Kuijpers T, van der Windt DA, van der Heijden GJ, et al. A prediction rule for shoulder pain related sick leave: a prospective cohort study. BMC Musculoskelet Disord. 2006;7:97. doi: https://doi.org/10.1186/1471-2474-7-97

408

Pocock SJ, McCormack V, Gueyffier F, et al. A score for predicting risk of death from cardiovascular disease in adults with raised blood pressure, based on individual patient data from randomised controlled trials. BMJ. 2001;323(7304):75–81. doi: https://doi.org/10.1136/bmj.323.7304.75

409

Casikar I, Lu C, Reid S, Condous G. Prediction of successful expec tant management of first trimester miscarriage: development and validation of a new mathematical model. Aust N Z J Obstet Gynaecol. 2013;53(1):58–63. doi: https://doi.org/10.1111/ajo.12053

410

Godoy G, Chong KT, Cronin A, et al. Extent of pelvic lymph node dissection and the impact of standard template dissection on nomogram prediction of lymph node involvement. Eur Urol. 2011;60(2):195–201. doi: https://doi.org/10.1016/j.eururo.2011.01.016

411

Bradburn MJ, Clark TG, Love SB, Altman DG. Survival analysis part II: multivariate data analysis — an introduction to concepts and methods. Br J Cancer. 2003;89(3):431–436. doi: https://doi.org/10.1038/sj.bjc.6601119

412

Wells P, Anderson D, Rodger M, et al. Derivation of a simple clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb Haemost. 2000;83(3):416–420.

413

Cole TJ. Scaling and rounding regression coefficients to integers. Appl Stat. 1993;42(1):261–268. doi: https://doi.org/10.2307/2347432

414

Sullivan LM, Massaro JM, D’Agostino RB. Presentation of multivariate data for clinical use: the Framingham study risk score functions. Stat Med. 2004;23(10):1631–1660. doi: https://doi.org/10.1002/sim.1742

415

Moons KG, Harrell FE, Steyerberg EW. Should scoring rules be based on odds rati os or regression coefficients? J Clin Epidemiol. 2002;55(10):1054–1055. doi: https://doi.org/10.1016/s0895-4356(02)00453-5

416

Nijman RG, Vergouwe Y, Thompson M, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:f1706. doi: https://doi.org/10.1136/bmj.f1706

417

Royston P, Altman DG. Visualizing and assessing discrimina tion in the logistic regression model. Stat Med. 2010;29(24): 2508–2520. doi: https://doi.org/10.1002/sim.3994

418

Taş U, Steyerberg EW, Bierma-Zeinstra SM, et al. Age, gender and disability predict future disability in older people: the Rotterdam Study. BMC Geriatrics. 2011;11:22. doi: https://doi.org/10.1186/1471-2318-11-22

419

Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148(3):839–843. doi: https://doi.org/10.1148/radiology.148.3.6878708

420

Pencina MJ, D’Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi: https://doi.org/10.1002/sim.4085

421

Pepe MS, Janes H. Reporting standards are needed for evaluations of risk reclassification. Int J Epidemiol. 2011;40(4): 1106–1108. doi: https://doi.org/10.1093/ije/dyr083

422

Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010;37(1):31–38. doi: https://doi.org/10.1053/j.seminoncol.2009.12.004

423

Sanders MS, de Jonge RC, Terwee CB, et al. Addition of host genetic variants in a prediction rule for post meningitis hearing loss in childhood: a model updating study. BMC Infect Dis. 2013;13:340. doi: https://doi.org/10.1186/1471-2334-13-340

424

Kramer AA, Zimmerman JE. A predictive model for the early identification of patients at risk for a prolonged intensive care unit length of stay. BMC Med Inform Decis Mak. 2010;10:27. doi: https://doi.org/10.1186/1472-6947-10-27

425

Neely D, Feinglass J, Wallace WH. Developing a predictive model to assess applicants to an internal medicine residency. J Grad Med Educ. 2010;2(1):129–132. doi: https://doi.org/10.4300/JGME-D-09-00044.1

426

Ioannidis JP. Limitations are not properly acknowledged in the scientific literature. J Clin Epidemiol. 2007;60(4):324–329. doi: https://doi.org/10.1016/j.jclinepi.2006.09.011

427

Horton R. The hidden research paper. JAMA. 2002;287(21): 2775–2778. doi: https://doi.org/10.1001/jama.287.21.2775

428

Docherty M, Smith R. The case for structuring the discussion of scientific papers. BMJ. 1999;318(7193):1224–1225. doi: https://doi.org/10.1136/bmj.318.7193.1224

429

Ioannidis JP. Research needs grants, funding and money — missing something? Eur J Clin Invest. 2012;42(4):349–351. doi: https://doi.org/10.1111/j.1365-2362.2011.02617.x

430

Janssens AC, Ioannidis JP, Bedrosian S, et al. Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration. Eur J Clin Invest. 2011;41(9):1010–1035. doi: https://doi.org/10.1111/j.1365-2362.2011.02493.x

431

Collins GS. Cardiovascular disease risk prediction in the UK. Primary Care Cardiovascular Journal. 2013;6:125–128.

432

Collins GS, Altman DG. An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. BMJ. 2009;339:b2584. doi: https://doi.org/10.1136/bmj.b2584

433

Collins GS, Altman DG. An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ. 2010;340:c2442. doi: https://doi.org/10.1136/bmj.c2442

434

Perry JJ, Sharma M, Sivilotti ML, et al. Prospective validation of the ABCD2 score for patients in the emergency department with transient ischemic attack. CMAJ. 2011;183(10):1137–1145. doi: https://doi.org/10.1503/cmaj.101668

435

Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA. 1998;280(3):280–282. doi: https://doi.org/10.1001/jama.280.3.280

436

Ioannidis JP, Polyzos NP, Trikalinos TA. Selective discussion and transparency in microarray research findings for cancer outcomes. Eur J Cancer. 2007;43(13):1999–2010. doi: https://doi.org/10.1016/j.ejca.2007.05.019

437

Van den Bosch JE, Moons KG, Bonsel GJ, Kalkman CJ. Does measurement of preoperative anxiety have added value for predicting postoperative nausea and vomiting? Anesth Analg. 2005;100(5):1525–1532. doi: https://doi.org/10.1213/01.ANE.0000149325.20542.D4

438

Kappen TH, Moons KG, van Wolfswinkel L, et al. Impact of risk assessments on prophylactic antiemetic prescription and the incidence of postoperative nausea and vomiting: a clusterrandomized trial. Anesthesiology. 2014;120(2):343–354. doi: https://doi.org/10.1097/ALN.0000000000000009

439

Poldervaart JM, Reitsma JB, Koffijberg H, et al. The impact of the HEART risk score in the early assessment of patients with acute chest pain: design of a stepped wedge, cluster randomised trial. BMC Cardiovasc Disord. 2013;13:77. doi: https://doi.org/10.1186/1471-2261-13-77

440

Hutchings HA, Evans BA, Fitzsimmons D, et al. Predictive risk stratification model: a progressive cluster-randomised trial in chronic conditions management (PRISMATIC) research protocol. Trials. 2013; 14:301. doi: https://doi.org/10.1186/1745-6215-14-301

441

Ioannidis JP. More than a billion people taking statins? Potential implications of the new cardiovascular guidelines. JAMA. 2014;311(5):463–464. doi: https://doi.org/10.1001/jama.2013.284657

442

Ioannidis JP, Tzoulaki I. What makes a good predictor? The evidence applied to coronary artery calcium score. JAMA. 2010; 303(16):1646–1647. doi: https://doi.org/10.1001/jama.2010.503

443

Mrdovic I, Savic L, Krljanac G, et al. Predicting 30-day major adverse cardiovascular events after primary percutaneous coronary intervention. The RISK-PCI score. Int J Cardiol. 2013;162(3): 220–227. doi: https://doi.org/10.1016/j.ijcard.2011.05.071

444

Ridker PM, Paynter NP, Rifai N, et al. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation. 2008;118(22):2243–2251. doi: https://doi.org/10.1161/CIRCULATIONAHA.108.814251

445

World Medical Association. Declaration of Geneva. Available online: www.wma.net/en/30publications/10policies/g1. Accessed on June 24, 2008.

446

Council for International Organizations of Medical Sciences. International ethical guidelines for biomedical research involving human subjects. Bull Med Ethics. 2002;(182):17–23.

447

Arnold DH, Gebretsadik T, Abramo TJ, et al. The Acute Asthma Severity Assessment Protocol (AASAP) study: objectives and methods of a study to develop an acute asthma clinical prediction rule. Emerg Med J. 2012;29(6):444–450. doi: https://doi.org/10.1136/emj.2010.110957

448

Azagra R, Roca G, Encabo G, et al. Prediction of absolute risk of fragility fracture at 10 years in a Spanish population: validation of the WHO FRAX tool in Spain. BMC Musculoskelet Disord. 2011;12:30. doi: https://doi.org/10.1186/1471-2474-12-30

449

Collins SP, Lindsell CJ, Jenkins CA, et al. Risk stratification in acute heart failure: rationale and design of the STRATIFY and DECIDE studies. Am Heart J. 2012;164(6):825–834. doi: https://doi.org/10.1016/j.ahj.2012.07.033

450

Hafkamp-de Groen E, Lingsma HF, Caudri D, et al. Predicting asthma in preschool children with asthma symptoms: study rationale and design. BMC Pulm Med. 2012;12:65. doi: https://doi.org/10.1186/1471-2466-12-65

451

Hess EP, Wells GA, Jaffe A, Stiell IG. A study to derive a clinical decision rule for triage of emergency department patients with chest pain: design and methodology. BMC Emerg Med. 2008;8:3. doi: https://doi.org/10.1186/1471-227X-8-3

452

Horisberger T, Harbarth S, Nadal D, et al. G-CSF and IL-8 for early diagnosis of sepsis in neonates and critically ill children — safety and cost effectiveness of a new laboratory prediction model: study protocol of a randomized controlled trial [ISRCTN91123847]. Crit Care. 2004;8:R443–R450. doi: https://doi.org/10.1186/cc2971

453

Liman TG, Zietemann V, Wiedmann S, et al. Prediction of vascular risk after stroke — protocol and pilot data of the Prospective Cohort with Incident Stroke (PROSCIS). Int J Stroke. 2013;8(6): 484–490. doi: https://doi.org/10.1186/cc2971

454

Mann DM, Kannry JL, Edonyabo D, et al. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implement Sci. 2011;6:109. doi: https://doi.org/10.1186/1748-5908-6-109

455

Meijs MF, Bots ML, Vonken EJ, et al. Rationale and design of the SMART Heart study: a prediction model for left ventricular hypertrophy in hypertension. Neth Heart J. 2007;15(9):295–298. doi: https://doi.org/10.1007/BF03086003

456

Mrdovic I, Savic L, Perunicic J, et al. Development and validation of a risk scoring model to predict net adverse cardiovascular outcomes after primary percutaneous coronary intervention in patients pretreated with 600 mg clopidogrel: rationale and design of the RISK-PCI study. J Interv Cardiol. 2009;22(4):320–328. doi: https://doi.org/10.1111/j.1540-8183.2009.00476.x

457

Nee RJ, Vicenzino B, Jull GA, et al. A novel protocol to develop a prediction model that identifies patients with nerve-related neck and arm pain who benefit from the early introduction of neural tissue management. Contemp Clin Trials. 2011;32(5):760–770. doi: https://doi.org/10.1016/j.cct.2011.05.018

458

Pita-Fernández S, Pértega-Diaz S, Valdés-Cañedo F, et al. Incidence of cardiovascular events after kidney transplantation and cardiovascular risk scores: study protocol. BMC Cardiovasc Disord. 2011;11:2. doi: https://doi.org/10.1186/1471-2261-11-2

459

Sanfelix-Genoves J, Peiro S, Sanfelix-Gimeno G, et al. Development and validation of a population-based prediction scale for osteoporotic fracture in the region of Valencia, Spain: the ESOSVAL-R study. BMC Public Health. 2010;10:153. doi: https://doi.org/10.1186/1471-2458-10-153

460

Siebeling L, terRiet G, van der Wal WM, et al. ICE COLD ERIC — International collaborative effort on chronic obstructive lung disease: exacerbation risk index cohorts — study protocol for an international COPD cohort study. BMC Pulm Med. 2009;9:15. doi: https://doi.org/10.1186/1471-2466-9-15

461

Canadian CT Head and C-Spine (CCC) Study Group. Canadian C-Spine Rule study for alert and stable trauma patients: I. Background and rationale. CJEM. 2002;4(2):84–90.

462

Canadian CT Head and C-Spine (CCC) Study Group. Canadian C-Spine Rule study for alert and stable trauma patients: II. Study objectives and methodology. CMAJ. 2002;4(3):185–193.

463

van Wonderen KE, van der Mark LB, Mohrs J, et al. Prediction and treatment of asthma in preschool children at risk: study design and baseline data of a prospective cohort study in general practice (ARCADE). BMC Pulm Med. 2009;9:13. doi: https://doi.org/10.1186/1471-2466-9-13

464

Waldron CA, Gallacher J, van der Weijden T, et al. The effect of different cardiovascular risk presentation formats on intentions, understanding and emotional affect: a randomised controlled trial using a web-based risk formatter (protocol). BMC Med Inform Decis Mak. 2010;10:41. doi: https://doi.org/10.1186/1472-6947-10-41

465

Laine C, Guallar E, Mulrow C, et al. Closing in on the truth about recombinant human bone morphogenetic protein-2: evidence synthesis, data sharing, peer review, and reproducible research. Ann Intern Med. 2013;158(12):916–918. doi: https://doi.org/10.7326/0003-4819-158-12-201306180-00012

466

Peng RD. Reproducible research and Biostatistics. Biostatistics. 2009;10(3):405–408. doi: https://doi.org/10.1093/biostatistics/kxp014

467

Keiding N. Reproducible research and the substantive context. Biostatistics. 2010;11(3):376–378. doi: https://doi.org/10.1093/biostatistics/kxq033

468

Vickers AJ. Whose data set is it anyway? Sharing raw data from randomized trials. Trials. 2006;7:15. doi: https://doi.org/10.1186/1745-6215-7-15

469

Riley RD, Abrams KR, Sutton AJ, et al. Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future. Br J Cancer. 2003;88(8): 1191–1198. doi: https://doi.org/10.1038/sj.bjc.6600886

470

Riley RD, Sauerbrei W, Altman DG. Prognostic markers in cancer: the evolution of evidence from single studies to metaanalysis, and beyond. Br J Cancer. 2009;100(8):1219–1229. doi: https://doi.org/10.1038/sj.bjc.6604999

471

Riley RD, Simmonds MC, Look MP. Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol. 2007;60(5):431–439. doi: https://doi.org/10.1016/j.jclinepi.2006.09.009

472

Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. BMJ. 2009;339:b4184. doi: https://doi.org/10.1136/bmj.b4184

473

Groves T. BMJ policy on data sharing. BMJ. 2010;340:c564. doi: https://doi.org/10.1136/bmj.c564

474

Marchionni L, Afsari B, Geman D, Leek JT. A simple and reproducible breast cancer prognostic test. BMC Genomics. 2013; 14:336. doi: https://doi.org/10.1186/1471-2164-14-336

475

Loder E, Groves T, Macauley D. Registration of observational studies. BMJ. 2010;340:c950. doi: https://doi.org/10.1136/bmj.c950

476

Chavers S, Fife D, Wacholtz M, et al. Registration of Observational Studies: perspectives from an industry-based epidemiology group. Pharmacoepidemiol Drug Saf. 2011;20(10):1009–1013. doi: https://doi.org/10.1002/pds.2221

477

Should protocols for observational studies be registered? Lancet. 2010;375(9712):348. doi: https://doi.org/10.1016/S0140-6736(10)60148-1

478

Altman DG. The time has come to register diagnostic and prognostic research. Clin Chem. 2014;60(4):580–582. doi: https://doi.org/10.1373/clinchem.2013.220335

479

The registration of observational studies — when metaphors go bad. Epidemiology. 2010;21(5):607–609. doi: https://doi.org/10.1097/EDE.0b013e3181eafbcf

480

Sørensen HT, Rothman KJ. The prognosis of research. BMJ. 2010;340:c703. doi: https://doi.org/10.1136/bmj.c703

481

Vandenbroucke JP. Registering observational research: second thoughts. Lancet. 2010;375(9719):982–983. doi: https://doi.org/10.1016/S0140-6736(10)60437-0

482

Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: Is it time? CMAJ. 2010;182(15):1638–1642. doi: https://doi.org/10.1503/cmaj.092252

483

Lenzer J. Majority of panelists on controversial new cholesterol guideline have current or recent ties to drug manufacturers. BMJ. 2013;347:f6989. doi: https://doi.org/10.1136/bmj.f6989

484

Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP. Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ. 2013;347:f5535. doi: https://doi.org/10.1136/bmj.f5535

485

Simera I. Get the content right: following reporting guidelines will make your research paper more complete, transparent and usable. J Pak Med Assoc. 2013;63(2):283–285.

486

Simera I, Kirtley S, Altman DG. Reporting clinical research: guidance to encourage accurate and transparent research reporting. Maturitas. 2012;72(1):84–87. doi: https://doi.org/10.1016/j.maturitas.2012.02.012

487

Simera I, Moher D, Hirst A, et al. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010; 8:24. doi: https://doi.org/10.1186/1741-7015-8-24

488

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–269.

489

Little J, Higgins JP, Ioannidis JP, et al. STrengthening the REporting of Genetic Association Studies (STREGA): an extension of the STROBE statement. PLoS Med. 2009;6(2):e22. doi: https://doi.org/10.1371/journal.pmed.1000022

490

Kilkenny C, Browne W, Cuthill IC, et al. Animal research: reporting in vivo experiments: the ARRIVE guidelines. J Gene Med. 2010;12(7):561–563. doi: https://doi.org/10.1002/jgm.1473

491

Gagnier JJ, Kienle G, Altman DG, et al. The CARE guidelines: consensus-based clinical case reporting guideline development. J Med Case Rep. 2013;7:223. doi: https://doi.org/10.1186/1752-1947-7-223

492

Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study. BMC Med Res Methodol. 2010;10:7. doi: https://doi.org/10.1186/1471-2288-10-7

493

Little RJ, Rubin DB. Statistical Analysis With Missing Data. Hoboken, NJ: Wiley; 2002.

494

Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: J. Wiley & Sons; 1987.

495

White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–399. doi: https://doi.org/10.1002/sim.4067

496

Harel O, Pellowski J, Kalichman S. Are we missing the importance of missing values in HIV prevention randomized clinical trials? Review and recommendations. AIDS Behav. 2012;16(6): 1382–1393. doi: https://doi.org/10.1007/s10461-011-0125-6

497

Schafer JL. Multiple imputation: a primer. Stat Methods Med Res. 1999;8(1):3–15. doi: https://doi.org/10.1177/096228029900800102

498

Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009;9:57. doi: https://doi.org/10.1186/1471-2288-9-57

499

van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999;18(6):681–694. doi: https://doi.org/10.1002/(sici)1097-0258(19990330)18:63.0.co;2-r

500

Wood AM, White IR, Royston P. How should variable selection be performed with multiply imputed data? Stat Med. 2008;27(17): 3227–3246. doi: https://doi.org/10.1002/sim.3177

501

Turner EL, Dobson JE, Pocock SJ. Categorisation of continuous risk factors in epidemiological publications: a survey of current practice. Epidemiol Perspect Innov. 2010;7:9. doi: https://doi.org/10.1186/1742-5573-7-9

502

van Walraven C, Hart RG. Leave ‘em alone — why continuous variables should be analyzed as such. Neuroepidemiology. 2008;30(3):138–139. doi: https://doi.org/10.1159/000126908

503

Vickers AJ, Lilja H. Cutpoints in clinical chemistry: time for fundamental reassessment. Clin Chem. 2009;55(1):15–17. doi: https://doi.org/10.1373/clinchem.2008.114694

504

Bennette C, Vickers A. Against quantiles: categorization of continuous variables in epidemiologic research, and its discontents. BMC Med Res Methodol. 2012;12:21. doi: https://doi.org/10.1186/1471-2288-12-21

505

Dawson NV, Weiss R. Dichotomizing continuous variables in statistical analysis: a practice to avoid. Med Decis Making. 2012; 32(2):225–226. doi: https://doi.org/10.1177/0272989X12437605

506

Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. Appl Stat. 1994;43(3):429–467. doi: https://doi.org/10.2307/2986270

507

Harrell FE, Lee KL, Pollock BG. Regression models in clinical studies: determining relationships between predictors and response. J Natl Cancer Inst. 1988;80(15):1198–1202. doi: https://doi.org/10.1093/jnci/80.15.1198

508

Schumacher M, Binder H, Gerds T. Assessment of survival prediction models based on microarray data. Bioinformatics. 2007;23(14): 1768–1774. doi: https://doi.org/10.1093/bioinformatics/btm232

509

Subramanian J, Simon R. Gene expression-based prognostic signatures in lung cancer: ready for clinical use? J Natl Cancer Inst. 2010;102(7):464–474. doi: https://doi.org/10.1093/jnci/djq025

510

Dupuy A, Simon RM. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. J Natl Cancer Inst. 2007;99(2):147–157. doi: https://doi.org/10.1093/jnci/djk018

511

Boulesteix AL. Validation in bioinformatics and molecular medicine. Brief Bioinform. 2011;12(3):187–188. doi: https://doi.org/10.1093/bib/bbr027

512

Jelizarow M, Guillemot V, Tenenhaus A, et al. Over-optimism in bioinformatics: an illustration. Bioinformatics. 2010;26(16): 1990–1998. doi: https://doi.org/10.1093/bioinformatics/btq323

513

Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298–1301. doi: https://doi.org/10.1016/j.urology.2010.06.019

514

Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33(3):517–535. doi: https://doi.org/10.1002/sim.5941

515

Crowson CS, Atkinson EJ, Therneau TM. Assessing calibration of prognostic risk scores. Stat Methods Med Res. 2016;25(4): 1692–1706. doi: https://doi.org/10.1177/0962280213497434

516

Vach W. Calibration of clinical prediction rules does not just assess bias. J Clin Epidemiol. 2013;66(11):1296–1301. doi: https://doi.org/10.1016/j.jclinepi.2013.06.003

517

Miller ME, Hui SL, Tierney WM. Validation techniques for logistic-regression models. Stat Med. 1991;10(8):1213–1226. doi: https://doi.org/10.1002/sim.4780100805

518

Cox DR. Two further applications of a model for binary regression. Biometrika. 1958;45:562–565.

519

D’Agostino RB, Nam BH. Evaluation of the performance of survival analysis models: discrimination and calibration measures. In: Handbook of Statistics, Survival Methods. Balakrishnan N, Rao CR, eds. Amsterdam: Elsevier; 2004. pp. 1–25.

520

Grønnesby JK, Borgan O. A method for checking regression models in survival analysis based on the risk score. Lifetime Data Anal. 1996;2(4):315–328. doi: https://doi.org/10.1007/BF00127305

521

Bertolini G, D’Amico R, Nardi D, et al. One model, several results: the paradox of the Hosmer-Lemeshow goodness-of-fit test for the logistic regression model. J Epidemiol Biostat. 2000;5(4):251–253.

522

Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–2056. doi: https://doi.org/10.1097/01.CCM.0000275267.64078.B0

523

Marcin JP, Romano PS. Size matters to a model’s fit. Crit Care Med. 2007;35(9):2212–2213. doi: https://doi.org/10.1097/01.CCM.0000281522.70992.EF

524

Bannister CA, Poole CD, Jenkins-Jones S, et al. External validation of the UKPDS risk engine in incident type 2 diabetes: a need for new type 2 diabetes-specific risk equations. Diab Care. 2014;37(2):537–545. doi: https://doi.org/10.2337/dc13-1159

525

Van Hoorde K, Vergouwe Y, Timmerman D, et al. Assessing calibration of multinomial risk prediction models. Stat Med. 2014; 33(15):2585–2596. doi: https://doi.org/10.1002/sim.6114

526

Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54(1):17–23. doi: https://doi.org/10.1373/clinchem.2007.096529

527

Pencina MJ, D’Agostino RB, Song L. Quantifying discrimination of Framingham risk functions with different survival C statistics. Stat Med. 2012;31(15):1543–1553. doi: https://doi.org/10.1002/sim.4508

528

Van Calster B, Van Belle V, Vergouwe Y, et al. Extending the c-statistic to nominal polytomous outcomes: the polytomous discrimination index. Stat Med. 2012;31(23):2610–2626. doi: https://doi.org/10.1002/sim.532

529

Wolbers M, Blanche P, Koller MT, et al. Concordance for prognostic models with competing risks. Biostatistics. 2014;15(3): 526–539. doi: https://doi.org/10.1093/biostatistics/kxt059

530

Pencina MJ, D’Agostino RB, Demler OV. Novel metrics for evaluating improvement in discrimination: net reclassification and integrated discrimination improvement for normal variables and nested models. Stat Med. 2012;31(2):101–113. doi: https://doi.org/10.1002/sim.4348

531

Bradburn MJ, Clark TG, Love SB, Altman DG. Survival analysis part III: multivariate data analysis — choosing a model and assessing its adequacy and fit. Br J Cancer. 2003;89(4):605–611. doi: https://doi.org/10.1038/sj.bjc.6601120

532

Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for the systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014;11(10): e1001744. doi: https://doi.org/10.1371/journal.pmed.1001744

533

Moons K.G.M., Altman D.G., Reitsma J.B. и др. Прозрачная отчетность о многофакторной предсказательной модели для индивидуального прогнозирования или диагностики (TRIPOD): разъяснения и уточнения // Digital Diagnostics. — 2022. — Т. 3. — № 3. — C. 232–322. — doi: https://doi.org/10.17816/DD110794

Moons KGM, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Translation in to Russian. Digital Diagnostics. 2022;3(3):232–322. doi: https://doi.org/10.17816/DD110794

The authors declare that there are no conflicts of interest present.