Inflammatory bowel disease (IBD) is characterized by chronic inflammation of the gastro-intestinal tract with two main subtypes, Crohn’s Disease (CD) and Ulcerative Colitis (UC), each without a cure, resulting in life-long management often associated with impaired quality of life.1,2 IBD can manifest early in life with up to 10 to 20% of cases diagnosed before 20 years of age.3,4 As the etiology of IBD remains unclear, researchers have relied on descriptive epidemiologic studies to track the development and impact of IBD on health.
While single-center studies are the most common data sources of disease registries to track chronic diseases such as IBD, they are often limited in size and scope.5 Routinely collected health data, such as health administrative data are another source of information for these epidemiologic studies, as they are increasingly available and have offered population-level insights to IBD and other chronic diseases such as diabetes and rheumatoid arthritis.6–8 However, these data are not collected originally for research purposes and present unique methodological challenges such as a lack of direct diagnosis data.9
Canada has amongst the highest incidence of IBD in the world.10 In addition, universal coverage of hospital and physician services means comprehensive, population-wide administrative data are available.11 Moreover, in British Columbia, British Columbia Children’s Hospital (BCCH) serves as the only tertiary care pediatric hospital in the province and maintains a comprehensive prospective IBD clinical registry.12 Consequently, this provided us the unique opportunity to assess the epidemiology of pediatric IBD using both clinical registry and health administrative data.
To compare the application of health administrative and clinical registry data in determining the epidemiology of IBD in BC, we conducted a population-based retrospective cohort study of all children <18 years of age who were diagnosed with IBD between 1996 and 2008. We applied a previously validated Ontario-derived administrative data algorithm13 to identify patients with childhood-onset IBD and to determine changes in incidence and prevalence. Next, we compared these rates with those derived using the BCCH clinical registry in order to determine how these databases and epidemiological assessments characterize the pediatric IBD population in BC.
Materials and Methods
Administrative Data Cohort and Case Selection
This was a population-based, retrospective cohort study of all children <18 years diagnosed with IBD in British Columbia (BC), the third most populous province in Canada.14 The administrative database contains all pediatric patients between fiscal years (FY) 1993 to 2008 (April 1, 1993 to March 31, 2008). The administrative database is a linked data set consisting of Medical Services Plan (MSP) capturing physician encounters,15 Canadian Institute of Health Information (CIHI) Discharge Abstract Database (DAD) capturing hospital encounters,16 and PharmaNet capturing prescribed drugs from the BC Ministry of Health.17 This study was approved by the BC Children’s and Women’s Research Ethics Board and all data were anonymized, maintained with confidentiality, and complied to data protection and privacy regulations.
Creating the administrative data cohort involved a two-stage process. First, the BC Ministry of Health searched full provincial MSP and DAD databases for the period from FY 1993 to 2008 for all individuals <18 years. We chose to use <18 years to reflect the age at which adolescents transfer to adult gastroenterologist care. Ministry analysts used a broad screening algorithm to select potential IBD cases: persons having at least (a) one IBD-related physician contact, or (b) endoscopy and at least one IBD-coded hospitalization within a 4-year period (ICD-9-CM diagnosis codes 555.x, 556.x, 556.9 and ICD-10-CA codes of K50.x, K51.x, K.52.3).
In the second stage, we applied the validated case definition developed by Benchimol et al13 to the Ministry-supplied data to identify cases with IBD, assign their diagnosis (UC, CD, or type unclassifiable based on available codes), and determine incidence date. Under this definition, children who had a sigmoidoscopy or colonoscopy were selected as cases if they had 4 IBD-related outpatient physician contacts or 2 IBD-coded hospitalizations within a 3-year period; children with no record of sigmoidoscopy or colonoscopy required 7 physician contacts or 3 hospitalizations in 3 years to qualify. Incidence date was defined by the first encounter in the qualifying cluster. The diagnosis (CD, UC, or IBD-U) associated with five of the last seven most recent outpatient visits was used to determine if children had CD, UC, or IBD-U.
BC Children’s Hospital Clinical Registry Cohort
Patients with incident IBD at British Columbia Children’s Hospital (BCCH) were extracted on November 13th, 2019 from a prospectively maintained, clinical pediatric database containing all children with IBD diagnosed or followed by the pediatric GI group at BCCH. For the BCCH clinical registry cohort, patients were considered to have IBD if they were diagnosed using standard clinical, laboratory, endoscopic, and radiologic criteria.18 As the only pediatric tertiary care hospital in BC with all academic pediatric gastroenterologists in attendance, we expected that children seen at BCCH represent the vast majority of the pediatric IBD cases in BC. Information recorded included age, year of diagnosis, and IBD subtype. Diagnoses per year were tabulated and separated by age group (1–5 years, 6–9 years, and 10–17 years).
Incidence and Prevalence Estimates
We separated incident and prevalent cases based on a look-back period of 3 years, also validated in the aforementioned Ontario study.13 If there were no ICD codes in those years, cases were considered incident; otherwise they were prevalent. We separated the incident cases each year during the study period into 3 groups based on age: 1 to 5 years, 6 to 9 years, and 10 to 17 years. In computing incidence per 100,000 population for each age group in each study year we used the annual age-specific population estimates provided by BC Statistics with interpolation adjustments to create values for July 1 (the mid-point) of each fiscal year.19 We standardized the incidence values for each fiscal year for each age group to the year 1996 to adjust for the changing composition of BC’s pediatric population throughout the study period. Additionally, standardized point prevalence was calculated for July 1 of each year and included all living children <18 years with IBD, censoring at death or aging out of the pediatric cohort.
We reported descriptive statistics for the identified cases in each age subgroup: mean and median age at diagnosis and percentage with UC/CD as their first diagnosis. We analyzed the incidence trends of the 3 age groups during the study period by fitting a series of Poisson regression models to the annual standardized incidence rates. We assessed the goodness of fit of each model and compared the quality of the models using the Akaike Information Criterion (AIC). We obtained the 95% confidence intervals for the annual standardized incidence values from the best model that included time. All regression models were fitted using STATA version 10. We similarly computed and plotted standardized IBD prevalence for each age group (July 1 of each year FY) taking into account years in which individuals were not living in BC, as determined from their health services utilization and Vital Statistics records. In the BCCH cohort, a residency check was unable to be accrued due to the limitations of the registry data. However, patients were excluded from the prevalence counts as they aged out into adult care. Using the DISTRATE procedure in STATA we computed 95% confidence intervals for incidence and prevalence estimates based on the gamma distribution as developed by Fay and Feuer.20 All other statistical methodology was performed using R version 3.5.3 and RStudio Version 1.1.463.
Descriptive Characteristics of BC’s Pediatric IBD Population
The BC Pediatric IBD population was defined using both health administrative data provided by the BC Ministry of Health and the BCCH clinical registry for all admitted children12 (Figure 1). Using administrative data, 1341 children satisfied the case definition requirements from fiscal years 1996 to 2008 (Table 1). Of these identified cases, 343 were prevalent (identified prior to 1996 or had IBD encounters during the three-year look-back interval) and 998 were incident cases during the study period.
Table 1 Descriptive Characteristics of BC’s Pediatric Cases Identified Using Health Administrative and BCCH Clinical Registry Data
Figure 1 Flow-chart of pediatric IBD case identification in BC from 1996 to 2008 using administrative data and the BCCH clinical registry cohorts.
The total follow-up time for all 1341 cases in the 3 age groups was 11,475 person-years. Eighty-seven percent of the cases were aged 10 to 17 with a median age of 14 years. Overall, the proportion of cases with CD was greater than UC (58% vs 29%, with 13% IBDU, p = 0.04). There were no significant differences in IBD subtype between age groups except in the youngest age group, where the proportion of children with UC was greater than those with CD (p = 0.02).
In the BCCH cohort, 49 were considered prevalent, 818 were incident cases between 1996 and 2008 (Table 1). Seventy-one percent of the cases were aged 10 to 17 and the overall median age was 12 years of age. Similar to the administrative data cohort, the proportion of CD patients was higher than UC (69% CD vs 22% UC and 9% IBDU, p = 0.03).
Incidence and Prevalence of Pediatric IBD in BC Using Administrative Data
Temporal trends in incidence from the administrative data cohort are shown in Table 2 and Figure 2A. For the whole cohort, the standardized incidence of IBD increased from 7.1 (95% CI 5.5–9.2) per 100,000 children in 1996 to 10.3 (95% CI 8.2–12.7) per 100,000 children in 2008. The greatest numeric change in rates was observed between 1998 and 2008 (5.9 to 10.3 per 100,000); however, this increase was not significant (p = 0.15). Although the temporal increase in incidence was not significant within each age group, the incidence for the oldest group was significantly higher than that of the other groups at all timepoints with incidence rates of 18.7 (95% CI 14.8–23.3) per 100,000 children compared to 0.5 (95% CI 0.0–2.6) and 2.3 (95% CI 0.6–5.8) for the youngest and middle age groups in 2008. The overall incidence of CD was significantly higher than UC with the exception of the years 1998, 1999, and 2002 (Supplemental Figure 1A). Neither the overall incidence of CD nor UC significantly changed throughout the study period.
Table 2 Summary of Incidence and Prevalence of Pediatric IBD in BC in Children Aged 1 to 5, 6 to 9, and 10 to 17 Found in Health Administrative and BCCH Clinical Registry Cohorts
Prevalence trends by age group for the administrative data cohort are shown in Table 2 and Figure 2B. In the decade from 1999 to 2008, the overall prevalence of IBD increased significantly from 29.5 (95% CI 26.1–33.2) per 100,000 children to 44.0 (95% CI 39.8–48.6, p = 0.03). Prevalence did not increase over the decade for children aged 1–5 or 6–9. In contrast to the younger age groups, the prevalence among children age 10 to 17 increased significantly over this decade from 51.9 (95% CI 45.3–59.1) per 100,000 children to 85.4 (95% CI 76.9–94.6).
Incidence and Prevalence of Pediatric IBD in BC Using BCCH Registry Data
In contrast to what was found in the administrative data cohort, the overall IBD incidence derived from the BCCH clinical registry significantly increased from 4.3 (95% CI 3.0–6.0) per 100,000 children in 1996 to 9.7 (95% CI 7.6–12.1) per 100,000 children in 2008 (p = 0.004) (Table 2 and Figure 3A). Incidence in children aged 1 to 5 years remained essentially unchanged and although incidence in children 6 to 9 years old increased from 3.5 (95% CI 1.4–7.2) per 100,000 children to 7.3 (95% CI 3.9–12.6), this was not statistically significant. Conversely, the 10- to 17-year-old age group significantly increased from 6.3 (95% CI 4.1–9.3) to 15.3 (95% CI 11.8–19.6) per 100,000 children (p = 0.01). The overall incidence of CD in the BCCH cohort was significantly higher than the incidence of UC throughout the study period with the exception for 1998 (Supplemental Figure 1B). Additionally, the incidence of CD in the BCCH cohort significantly increased from 3.5 per 100,000 children (95% CI 2.4–5.0) to 6.8 (95% CI 5.1–8.9) (p = 0.04). Moreover, the incidence of UC also significantly increased from 0.6 (95% CI 0.2–1.4) to 2.5 (95% CI 1.5–3.8) per 100,000 (p = 0.04).
Consistent with the administrative data, the overall prevalence estimates of the BCCH registry data also significantly increased from 21.4 (95% CI 18.4–24.7) per 100,000 to 47.1 (95% CI 42.5–52.0) over the decade from 1999 to 2008 while the prevalence of IBD in the younger and middle age groups in the BCCH cohort remained unchanged as shown in Table 2 and Figure 3B. However, the prevalence of the 10- to 17-year-old age group increased significantly from 32.1 (95% CI 27.0–38.0) per 100,000 to 78.3 (95% CI 70.0 to 87.2) between 1999 and 2008 (p < 0.01).
Comparing Incidence Rates of the Administrative Data and BCCH IBD Cohorts
Figure 4 shows the difference in incidence rates between the BCCH clinical registry and administrative data IBD cohorts which overall indicates no significant differences from 2000 to 2008. However, the earlier years of the BCCH data underestimate the incidence of IBD compared to the administrative data in 1996, 1997, and 1999 (p = 0.02, 0.01, and 0.04, respectively) largely due to the very low estimates of children with IBD in the 10 to 17 age group in the clinical registry data compared to administrative data. With the exception of the year 2000, incidence of the 1 to 5 age group was not significantly different between the administrative data and BCCH cohorts. On the other hand, the 6 to 9 age group showed significant differences in the later years with an approximately 3-fold higher IBD burden in the BCCH data compared to the administrative data in 2007 and 2008 (p = 0.01 and 0.05). Taken together, incidence rates between the two datasets deviated the most prior to the year 2000 particularly in the oldest age group and in the middle age group in later years.
Our results are the first longitudinal analysis of the epidemiology of pediatric IBD in BC using both health administrative data provided by the BC Ministry of Health and clinical registry data from BCCH. Algorithmic identification of children with IBD from health administrative data and direct incidence acquisition from clinical registry data allowed us to calculate and compare incidence and prevalence rates derived from the same population using two different data sources and methodologies.
While the overall incidence of IBD appears to have significantly increased from 1996 to 2008 according to clinical registry data, the increase was not significant when calculated using the cases acquired from administrative data using an Ontario-derived pediatric case definition. Notably, in earlier years (1996 to 1999) fewer cases of pediatric IBD were identified in the clinical registry data compared to administrative data, particularly in the oldest age group of children. Meanwhile, in later years (2007 to 2008), the clinical registry data performed better, identifying more cases in the 6 to 9 age group. Disease subtype classification was generally consistent between the two data sources. Overall, both data sources estimate the overall <18 pediatric IBD incidence to be close to 10 per 100,000 by 2008.
The only previously reported incidence rate of pediatric IBD in BC using administrative data used a Manitoba derived case definition which estimated incidence of 8.6 per 100,000 between 1998 and 2000,21 the lowest in Canada. Our reported incidence rate using administrative data during the same time period was 7.0 per 100,000, a lower estimate, though the previous study included patients up to 20 years old. This estimate was significantly greater than our incidence calculated using the BCCH clinical registry during this time period (5.4 per 100,000). Regardless of methodology, BC does represent the lowest reported incidence in Canada during these years; however, by 2008, we found the incidence to have risen to 9.7 to 10.3 per 100,000 children, close to the reported overall pooled incidence of Alberta, Manitoba, Nova Scotia, Ontario, and Quebec in 2008.22 Compared to international reported rates of pediatric IBD, BC falls within the highest reported rates during this time period which includes Europe (0.2 to 23/100,000) and North America (1.1 to 15.2/100,000).23 This is in contrast to regions with the lowest reported IBD incidence which include Oceania (2.9 to 7.2/100,000), Asia (0.5 to 11.4/100,000), Latin America (0.2 to 2.4/100,000) and Africa (0.0 to 0.9/100,000).
While both data sources estimated the overall incidence of pediatric IBD fairly closely, there were two subpopulations that differed significantly as a result of the data source. The first group was the 10- to 17-year-old group, particularly between 1996 and 1999. While the BCCH clinical registry did not explicitly exclude 17-year-olds, these children were encouraged to engage in adult care following their 17th birthday to facilitate transition. Though this was standard practice in pediatric care, the result was an under-representation of 17-year-olds at BCCH compared to what was found in the administrative data which captured all 17-year-olds regardless of whether they received pediatric or adult care. The BCCH data only included 3 children aged 17 while the administrative data included 237 children aged 17. We anticipate this is a limitation for not just our but other worldwide pediatric clinical registries since the transition from pediatric to adult care can range between 16 to 18 years of age.24
The second group was the children aged 6 to 9 where the administrative data cohort only estimated 1.1 and 2.3 children with IBD per 100,000 in 2007 and 2008 compared to the BCCH cohort at 7.3 per 100,000 over the same time period. While it cannot be ascertained why the administrative data and BCCH cohorts deviated in these later years and particularly in the middle-age group, we suspected differences in practice pattern between BC and Ontario may reduce the efficacy of the identification algorithm. Regardless, further characterization of this subpopulation is required.
As the only tertiary care children’s hospital in BC, we expected the incident rates derived from the BCCH clinical registry data would reflect the province-wide distribution. Notwithstanding having direct diagnosis data, the clinical registry was not without its limitations. For instance, there was no clear record of when patients migrate away from BC thus leading to a potential overestimation of prevalence rates though provincially reported interprovincial migration rates only ranged from 9.75% to 10.43% in children aged 5–19 in 2011.25 However, due to a limited look-back time (the BCCH cohort began in 1994) and smaller capacity to see patients in early years, the BCCH cohort likely underestimated the early prevalence and incidence of IBD in BC.
While registry and health administrative data are both considered forms of routinely collected health data, and therefore share many of their strengths and limitations,9 there are unique aspects of each type of dataset which may result in different sources of bias. Population-wide data, residency checks, a more stringent incidence criteria and subtype characterization all contribute to the benefit of using administrative data.26 However, the greatest limitation is the potential for misclassification bias which we sought to mitigate or at least determine its impact by utilizing a validated case definition algorithm, though, we could not fully determine the extent of discord due to the absence of a linked validation cohort.7,27 Benchimol et al reported the following diagnostic accuracies for the Ontario pediatric case algorithm when applied to Ontario administrative data: sensitivity 89.6% to 91.1%, specificity of 99.5% to 100%, positive predictive value 59.2%-76.0%, negative predictive value 99.9–100%.13 Even with congruent administrative databases within Canada, the province-to-province applicability of provincially derived case definitions could vary significantly as suggested by a systematic review of the algorithm validation literature,28 and recently in a national Canadian report.22 With the availability of individual-level pharmacy dispensation records in BC, we intend to propose potential new case definitions which include additional layers of identification capability. Benchimol et al7 included a single pharmacy claim for an IBD-related medication in elderly patients to improve the accuracy of the algorithm. Drug dispensation data have also been used to improve the sensitivity of case definition algorithms in diabetes.6 In future studies, we aim to validate a case definition using a linked validation cohort which we predict will better fit the BC pediatric IBD population and serve as an example for other administrative data research groups.
As the use of administrative data becomes increasingly popular, we set out to compare its utility in characterizing chronic disease to clinical registry data. Overall, this study highlights both strengths and weaknesses to each approach and the importance of validating case definitions for both the applicable population and healthcare setting. However, with appropriate consideration in utilizing routinely collected data, broad, population-wide assessments of chronic diseases can be made and our estimation of pediatric IBD incidence and prevalence in BC signifies an increasing trend. Epidemiology investigators should be aware of the strengths and weaknesses of their data sources and account for these when reporting incidence and prevalence of chronic diseases.
J.M.C is a PhD candidate supported by the BC Children’s Hospital Research Institute Studentship and the Lutsky Foundation. K.J. is a Senior Clinician Scientist supported by the Children with Intestinal and Liver Disorders (CHILD) Foundation and the BC Children’s Hospital Research Institute Clinician Scientists Award Program, University of British Columbia. We thank Dr. R. Prosser for his help with applying the case definition and providing statistical advice and analysis.
The authors report no conflicts of interest in this work.
1. Rocchi A, Benchimol EI, Bernstein CN, et al. Inflammatory bowel disease: a Canadian burden of illness review. Can J Gastroenterol Hepatol. 2012;26(11):811–817. doi:10.1155/2012/984575
2. Knowles SR, Graff LA, Wilding H, Hewitt C, Keefer L, Mikocka-Walus A. Quality of life in inflammatory bowel disease: a systematic review and meta-analyses-part I. Inflamm Bowel Dis. 2018;24(4):742–751. doi:10.1093/ibd/izx100
3. Ye Y, Manne S, Treem WR, Bennett D. Prevalence of inflammatory bowel disease in pediatric and adult populations: recent estimates from large national databases in the United States, 2007–2016. Inflamm Bowel Dis. 2020;26(4):619–625. doi:10.1093/ibd/izz182
4. Griffiths AM. Specificities of inflammatory bowel disease in childhood. Best Pract Res Clin Gastroenterol. 2004;18(3):509–523. doi:10.1016/j.bpg.2004.01.002
5. Sørensen HT, Baron JA. Registries and medical databases. Teach Epidemiol. 2010;3:455–466.
6. Dart AB, Martens PJ, Sellers EA, Brownell MD, Rigatto C, Dean HJ. Validation of a pediatric diabetes case definition using administrative health data in Manitoba, Canada. Diabetes Care. 2011;34(4):898–903. doi:10.2337/dc10-1572
7. Benchimol EI, Guttmann A, Mack DR, et al. Validation of international algorithms to identify adults with inflammatory bowel disease in health administrative data from Ontario, Canada. J Clin Epidemiol. 2014;67(8):887–896. doi:10.1016/j.jclinepi.2014.02.019
8. Hudson M, Tascilar K, Suissa S. Comparative effectiveness research with administrative health data in rheumatoid arthritis. Nat Rev Rheumatol. 2016;12(6):358–366. doi:10.1038/nrrheum.2016.34
9. Benchimol EI, Smeeth L, Guttmann A, et al. The reporting of studies conducted using observational routinely-collected health data (RECORD) statement. PLoS Med. 2015;12(10):e1001885. doi:10.1371/journal.pmed.1001885
10. Molodecky NA, Soon IS, Rabi DM, et al. Increasing incidence and prevalence of the inflammatory bowel diseases with time, based on systematic review. Gastroenterology. 2012;142(1):46–54.e42. doi:10.1053/j.gastro.2011.10.001
11. Jutte DP, Roos LL, Brownell MD. Administrative record linkage as a tool for public health research. Annu Rev Public Health. 2011;32(1):91–108. doi:10.1146/annurev-publhealth-031210-100700
12. Pinsk V, Lemberg DA, Grewal K, Barker CC, Schreiber RA, Jacobson K. Inflammatory bowel disease in the South Asian pediatric population of British Columbia. Am J Gastroenterol. 2007;102(5):1077–1083. doi:10.1111/j.1572-0241.2007.01124.x
13. Benchimol EI, Guttmann A, Griffiths AM, et al. Increasing incidence of paediatric inflammatory bowel disease in Ontario, Canada: evidence from health administrative data. Gut. 2009;58(11):1490–1497. doi:10.1136/gut.2009.188383
14. Government of Canada SC. Population estimates, quarterly; 2017. Available from: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000901.
15. Hu W. Diagnostic codes in MSP claim data. Summary Rep. 1996;39.
16. Canadian Institute for Health Information. CIHI data quality study of the 2005–2006 discharge abstract database. Canadian Institute for Health Information = Institut canadien d’information sur la santé; 2009. Available from: http://epe.lac-bac.gc.ca/100/200/300/cdn_institute_for_health/cihi_data_quality_study-e/DAD_DQ_Study_2005-2006_August_2009.pdf.
17. PharmaCare annual performance report 2005. 2005:69.
18. Levine A, Koletzko S, Turner D, et al. ESPGHAN revised porto criteria for the diagnosis of inflammatory bowel disease in children and adolescents. J Pediatr Gastroenterol Nutr. 2014;58(6):795–806. doi:10.1097/MPG.0000000000000239
19. Services M of C. Population estimates – Province of British Columbia. Available from: https://www2.gov.bc.ca/gov/content/data/statistics/people-population-community/population/population-estimates.
20. Fay MP, Feuer EJ. Confidence intervals for directly standardized rates: a method based on the gamma distribution. Stat Med. 1997;16(7):791–801. doi:10.1002/(sici)1097-0258(19970415)16:7<791::aid-sim500>3.0.co;2-#
21. Bernstein CN, Wajda A, Svenson LW, et al. The epidemiology of inflammatory bowel disease in Canada: a population-based study. Am J Gastroenterol. 2006;101(7):1559–1568. doi:10.1111/j.1572-0241.2006.00603.x
22. Benchimol EI, Bernstein CN, Bitton A, et al. Trends in epidemiology of pediatric inflammatory bowel disease in canada: distributed network analysis of multiple population-based provincial health administrative databases. Am J Gastroenterol. 2017;112(7):1120–1134. doi:10.1038/ajg.2017.97
23. Sýkora J, Pomahačová R, Kreslová M, Cvalínová D, Štych P, Schwarz J. Current global trends in the incidence of pediatric-onset inflammatory bowel disease. World J Gastroenterol. 2018;24(25):2741–2763. doi:10.3748/wjg.v24.i25.2741
24. Escher JC. Transition from pediatric to adult health care in inflammatory bowel disease. DDI. 2009;27(3):382–386. doi:10.1159/000228578
25. Government of Canada SC. 2011 National household survey profile – province/territory; 2013. Available from: https://www12.statcan.gc.ca/nhs-enm/2011/dp-pd/prof/details/page.cfm?Lang=E&Geo1=PR&Code1=59&Data=Count&SearchText=british%20columbia&SearchType=Begins&SearchPR=01&A1=All&B1=All&Custom=&TABID=1.
26. Harbaugh CM, Cooper JN. Administrative databases. Semin Pediatr Surg. 2018;27(6):353–360. doi:10.1053/j.sempedsurg.2018.10.001
27. Haut ER, Pronovost PJ, Schneider EB. Limitations of administrative databases. JAMA. 2012;307(24). doi:10.1001/jama.2012.6626
28. Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol. 2011;64(8):821–829. doi:10.1016/j.jclinepi.2010.10.006