Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002345. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclaimer: Additionally, the views expressed in this article are those of the author only and should not be construed to represent in any way those of the United States Government, the United States Department of Defense (DoD), or the United States Department of the Army. The identification of specific products or scientific instrumentation is considered an integral part of the scientific endeavor and does not constitute endorsement or implied endorsement on the part of the author, DoD, or any component agency.

<ol><li>Concato, J., Shah, N., Horwitz, R. I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Randomized%2C+controlled+trials%2C+observational+studies%2C+and+the+hierarchy+of+research+designs%5BTitle%5D)+AND+%22New+England+Journal+of+Medicine%22%5BJournal%5D)">Randomized, controlled trials, observational studies, and the hierarchy of research designs.</a> New England Journal of Medicine. 342 (25), 1887-1892 (2000).</li>
<li>Austin, P. C., Platt, R. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Survivor+treatment+bias%2C+treatment+selection+bias%2C+and+propensity+scores+in+observational+research%5BTitle%5D)+AND+%22Journal+of+Clinical+Epidemiology%22%5BJournal%5D)">Survivor treatment bias, treatment selection bias, and propensity scores in observational research.</a> Journal of Clinical Epidemiology. 63 (2), 136-138 (2010).</li>
<li>Sturmer, T., Wyss, R., Glynn, R. J., Brookhart, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Propensity+scores+for+confounder+adjustment+when+assessing+the+effects+of+medical+interventions+using+nonexperimental+study+designs%5BTitle%5D)+AND+%22Journal+of+Internal+Medicine%22%5BJournal%5D)">Propensity scores for confounder adjustment when assessing the effects of medical interventions using nonexperimental study designs.</a> Journal of Internal Medicine. 275 (6), 570-580 (2014).</li>
<li>Schermerhorn, M. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Long-Term+Outcomes+of+Abdominal+Aortic+Aneurysm+in+the+Medicare+Population%5BTitle%5D)+AND+%22New+England+Journal+of+Medicine%22%5BJournal%5D)">Long-Term Outcomes of Abdominal Aortic Aneurysm in the Medicare Population.</a> New England Journal of Medicine. 373 (4), 328-338 (2015).</li>
<li>Williams, R. J. II, et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+Propensity-Matched+Analysis+Between+Standard+Versus+Tapered+Oral+Vancomycin+Courses+for+the+Management+of+Recurrent+Clostridium+difficile+Infection%5BTitle%5D)+AND+%22Open+Forum+Infectious+Diseases%22%5BJournal%5D)">A Propensity-Matched Analysis Between Standard Versus Tapered Oral Vancomycin Courses for the Management of Recurrent Clostridium difficile Infection.</a> Open Forum Infectious Diseases. 4 (4), (2017).</li>
<li>Mitchell, J. D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Impact+of+Statins+on+Cardiovascular+Outcomes+Following+Coronary+Artery+Calcium+Scoring%5BTitle%5D)+AND+%22Journal+of+the+American+College+of+Cardiology%22%5BJournal%5D)">Impact of Statins on Cardiovascular Outcomes Following Coronary Artery Calcium Scoring.</a> Journal of the American College of Cardiology. 72 (25), 3233-3242 (2018).</li>
<li>Rush, T., McGeary, M., Sicignano, N., Buryk, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+plateau+in+new+onset+type+1+diabetes%3A+Incidence+of+pediatric+diabetes+in+the+United+States+Military+Health+System%5BTitle%5D)+AND+%22Pediatric+Diabetes%22%5BJournal%5D)">A plateau in new onset type 1 diabetes: Incidence of pediatric diabetes in the United States Military Health System.</a> Pediatric Diabetes. 19 (5), 917-922 (2018).</li>
<li>Rhon, D. I., Greenlee, T. A., Marchant, B. G., Sissel, C. D., Cook, C. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Comorbidities+in+the+first+2+years+after+arthroscopic+hip+surgery%3A+substantial+increases+in+mental+health+disorders%2C+chronic+pain%2C+substance+abuse+and+cardiometabolic+conditions%5BTitle%5D)+AND+%22British+Journal+of+Sports+Medicine%22%5BJournal%5D)">Comorbidities in the first 2 years after arthroscopic hip surgery: substantial increases in mental health disorders, chronic pain, substance abuse and cardiometabolic conditions.</a> British Journal of Sports Medicine. , (2018).</li>
<li>Mitchell, J., Paisley, R., Moon, P., Novak, E., Villines, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Coronary+Artery+Calcium+Score+and+Long-term+Risk+of+Death%2C+Myocardial+Infarction+and+Stroke%3A+The+Walter+Reed+Cohort+Study%5BTitle%5D)+AND+%22Journal+of+the+American+College+of+Cardiology%3A+Cardiovascular+Imaging%22%5BJournal%5D)">Coronary Artery Calcium Score and Long-term Risk of Death, Myocardial Infarction and Stroke: The Walter Reed Cohort Study.</a> Journal of the American College of Cardiology: Cardiovascular Imaging. , (2017).</li>
<li>McCormick, N., Lacaille, D., Bhole, V., Avina-Zubieta, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Validity+of+myocardial+infarction+diagnoses+in+administrative+databases%3A+a+systematic+review%5BTitle%5D)+AND+%22PLoS+ONE%22%5BJournal%5D)">Validity of myocardial infarction diagnoses in administrative databases: a systematic review.</a> PLoS ONE. 9 (3), e92286(2014).</li>
<li>Huo, J., Yang, M., Tina Shih, Y. -C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Sensitivity+of+Claims-Based+Algorithms+to+Ascertain+Smoking+Status+More+Than+Doubled+with+Meaningful+Use%5BTitle%5D)+AND+%22Value+in+Health%22%5BJournal%5D)">Sensitivity of Claims-Based Algorithms to Ascertain Smoking Status More Than Doubled with Meaningful Use.</a> Value in Health. , Available from:
 <a target="_blank" href="https://doi-org.remotexs.ntu.edu.sg/10.1016/j.jval.2017.09.002">https://doi-org.remotexs.ntu.edu.sg/10.1016/j.jval.2017.09.002</a> (2017).</li>
<li>Nayan, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+value+of+complementing+administrative+data+with+abstracted+information+on+smoking+and+obesity%3A+A+study+in+kidney+cancer%5BTitle%5D)+AND+%22Canadian+Urological+Association+Journal%22%5BJournal%5D)">The value of complementing administrative data with abstracted information on smoking and obesity: A study in kidney cancer.</a> Canadian Urological Association Journal. 11 (6), 167-171 (2017).</li>
<li>Birman-Deych, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Accuracy+of+ICD-9-CM+codes+for+identifying+cardiovascular+and+stroke+risk+factors%5BTitle%5D)+AND+%22Medical+Care%22%5BJournal%5D)">Accuracy of ICD-9-CM codes for identifying cardiovascular and stroke risk factors.</a> Medical Care. 43 (5), 480-485 (2005).</li>
<li>Preen, D. B., Holman, C. D., Spilsbury, K., Semmens, J. B., Brameld, K. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Length+of+comorbidity+lookback+period+affected+regression+model+performance+of+administrative+health+data%5BTitle%5D)+AND+%22Journal+of+Clinical+Epidemiology%22%5BJournal%5D)">Length of comorbidity lookback period affected regression model performance of administrative health data.</a> Journal of Clinical Epidemiology. 59 (9), 940-946 (2006).</li>
<li>Rector, T. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Specificity+and+sensitivity+of+claims-based+algorithms+for+identifying+members+of+Medicare%2BChoice+health+plans+that+have+chronic+medical+conditions%5BTitle%5D)+AND+%22Health+Services+Research%22%5BJournal%5D)">Specificity and sensitivity of claims-based algorithms for identifying members of Medicare+Choice health plans that have chronic medical conditions.</a> Health Services Research. 39 (6 Pt 1), 1839-1857 (2004).</li>
<li>Hernán, M. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Observational+studies+analyzed+like+randomized+experiments%3A+an+application+to+postmenopausal+hormone+therapy+and+coronary+heart+disease%5BTitle%5D)+AND+%22Epidemiology+%28Cambridge%2C+Mass.%29%22%5BJournal%5D)">Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease.</a> Epidemiology (Cambridge, Mass.). 19 (6), 766-779 (2008).</li>
<li>Austin, P. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+relative+ability+of+different+propensity+score+methods+to+balance+measured+covariates+between+treated+and+untreated+subjects+in+observational+studies%5BTitle%5D)+AND+%22Medical+Decision+Making%22%5BJournal%5D)">The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies.</a> Medical Decision Making. 29 (6), 661-677 (2009).</li>
<li>Robins, J. M., Hernan, M. A., Brumback, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Marginal+structural+models+and+causal+inference+in+epidemiology%5BTitle%5D)+AND+%22Epidemiology%22%5BJournal%5D)">Marginal structural models and causal inference in epidemiology.</a> Epidemiology. 11 (5), 550-560 (2000).</li>
<li>Robins, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Marginal+structural+models%5BTitle%5D)+AND+%221997+Proceedings+of+the+American+Statistical+Association%2C+section+on+Bayesian+statistical+science%22%5BJournal%5D)">Marginal structural models.</a> 1997 Proceedings of the American Statistical Association, section on Bayesian statistical science. , 1-10 (1998).</li>
<li>Thoemmes, F., Ong, A. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+Primer+on+Inverse+Probability+of+Treatment+Weighting+and+Marginal+Structural+Models%5BTitle%5D)+AND+%22Emerging+Adulthood%22%5BJournal%5D)">A Primer on Inverse Probability of Treatment Weighting and Marginal Structural Models.</a> Emerging Adulthood. 4 (1), 40-59 (2016).</li>
<li>Xu, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Use+of+stabilized+inverse+propensity+scores+as+weights+to+directly+estimate+relative+risk+and+its+confidence+intervals%5BTitle%5D)+AND+%22Value+in+Health%3A+the+Journal+of+the+International+Society+for+Pharmacoeconomics+and+Outcomes+Research%22%5BJournal%5D)">Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals.</a> Value in Health: the Journal of the International Society for Pharmacoeconomics and Outcomes Research. 13 (2), 273-277 (2010).</li>
<li>Cowper, D. C., Kubal, J. D., Maynard, C., Hynes, D. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+primer+and+comparative+review+of+major+US+mortality+databases%5BTitle%5D)+AND+%22Annals+of+Epidemiology%22%5BJournal%5D)">A primer and comparative review of major US mortality databases.</a> Annals of Epidemiology. 12 (7), 462-468 (2002).</li>
<li>Skopp, N. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Evaluation+of+a+methodology+to+validate+National+Death+Index+retrieval+results+among+a+cohort+of+U.S.+service+members%5BTitle%5D)+AND+%22Annals+of+epidemiology%22%5BJournal%5D)">Evaluation of a methodology to validate National Death Index retrieval results among a cohort of U.S. service members.</a> Annals of epidemiology. 27 (6), 397-400 (2017).</li>
<li>Buck, C. J. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aBuck%2C+CJ+%222015+ICD-9-CM+for+Hospitals%2C+Volumes+1%2C+2%2C+%26+3%2C+Professional+Edition%22">2015 ICD-9-CM for Hospitals, Volumes 1, 2, & 3, Professional Edition</a>. , Elsevier Saunders. (2015).</li>
<li>Buck, C. J. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aBuck%2C+CJ+%222018+ICD-10-CM+for+Hospitals%2C+Professional+Edition%22">2018 ICD-10-CM for Hospitals, Professional Edition</a>. , Elsevier Saunders. (2018).</li>
<li>Guo, S., Fraser, W. M. <a target="_blank" href="http://scholar.google.com/scholar?hl=en&safe=off&q=author%3aGuo%2C+S+%22Propensity+Score+Analysis%3A+Statistical+Methods+and+Applications%2C+Second+Edition%22">Propensity Score Analysis: Statistical Methods and Applications, Second Edition</a>. , Sage Publications. (2015).</li>
<li>Elixhauser, A., Steiner, C., Harris, D. R., Coffey, R. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Comorbidity+measures+for+use+with+administrative+data%5BTitle%5D)+AND+%22Medical+Care%22%5BJournal%5D)">Comorbidity measures for use with administrative data.</a> Medical Care. 36 (1), 8-27 (1998).</li>
<li>Charlson, M. E., Pompei, P., Ales, K. L., MacKenzie, C. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+new+method+of+classifying+prognostic+comorbidity+in+longitudinal+studies%3A+development+and+validation%5BTitle%5D)+AND+%22Journal+of+Chronic+Diseases%22%5BJournal%5D)">A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.</a> Journal of Chronic Diseases. 40 (5), 373-383 (1987).</li>
<li>Deyo, R. A., Cherkin, D. C., Ciol, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Adapting+a+clinical+comorbidity+index+for+use+with+ICD-9-CM+administrative+databases%5BTitle%5D)+AND+%22Journal+of+Clinical+Epidemiology%22%5BJournal%5D)">Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.</a> Journal of Clinical Epidemiology. 45 (6), 613-619 (1992).</li>
<li>Austin, P. C., Stuart, E. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+performance+of+inverse+probability+of+treatment+weighting+and+full+matching+on+the+propensity+score+in+the+presence+of+model+misspecification+when+estimating+the+effect+of+treatment+on+survival+outcomes%5BTitle%5D)+AND+%22Statistical+Methods+in+Medical+Research%22%5BJournal%5D)">The performance of inverse probability of treatment weighting and full matching on the propensity score in the presence of model misspecification when estimating the effect of treatment on survival outcomes.</a> Statistical Methods in Medical Research. 26 (4), 1654-1670 (2017).</li>
<li>Austin, P. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Balance+diagnostics+for+comparing+the+distribution+of+baseline+covariates+between+treatment+groups+in+propensity-score+matched+samples%5BTitle%5D)+AND+%22Statistics+in+Medicine%22%5BJournal%5D)">Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples.</a> Statistics in Medicine. 28 (25), 3083-3107 (2009).</li>
</ol>

The authors have nothing to disclose.

Retrospective analyses using large administrative datasets provide an efficient and cost-effective alternative when randomized controlled trials are not feasible. The appropriate data set will depend on the population and variables of interest, but the MDR is an attractive option that does not have the age restrictions seen with Medicare data. With any data set, it is important to be intimately familiar with its layout and data dictionary. Care should be taken along the way to ensure that complete data are captured, and data are accurately matched and merged.
Codes for diagnoses should be defined using existing literature and a thorough understanding of the ICD-9-CM and ICD-10-CM coding system to maximize the value of the assigned diagnoses. Existing sets of comorbidity codes, including the Elixhauser27 or refined Charlson comorbidity index28,29, can be used to define comorbid conditions that may influence the outcome of interest. Likewise, validated coding algorithms in administrative data and should be leveraged. Validation should remain an area of active research, as there is continued learning on the optimal use of ICD-9-CM and ICD-10-CM coding algorithms to maximize accurate classification of a wide-range of diseases.
Propensity scores can be used to address the bias inherent in any retrospective analysis. Effective propensity score weighting or matching should reduce the absolute standardize difference (ASD) below the desired threshold, generally set at 0.1. Appropriate balancing helps ensure comparability of the treatment groups with regard to known confounders, and appropriately employed propensity score techniques have been used to successfully replicate randomized trial results. Once properly balanced, the treatment groups can be compared with univariate time-to-event or other analysis.
Even with appropriate balancing, there is potential for residual confounding3, so the investigative team should limit the effect of unmeasured confounders. Additionally, if the effects of the covariates on treatment selection are strong, bias may still remain30. In small cohorts, the propensity scores are unlikely to fully reduce the ASD below 0.1 for all variables and regression adjustment can be employed to help remove residual imbalance31. Regression adjustment can also be used in subgroup analysis when appropriate balance is no longer assured.
When done correctly, research with administrative data provides timely answers to important clinical questions in the absence of randomized clinical trials. While it is impossible to remove all bias from observational studies, bias can be limited by using propensity scores and remaining meticulous analyses.

Randomized, placebo-controlled trials are the strongest study design to quantify efficacy of treatment, but they are not always feasible due to cost and time requirements or a lack of equipoise between treatment groups1. In these instances, a retrospective cohort design using large-scale administrative data (&#34;big data&#34;) often provides an efficient and cost-effective alternative, though the lack of randomization introduces treatment selection bias2. Treatment selection bias occurs in non-randomized studies when the treatment decision is dependent on pre-treatment characteristics that are associated with the outcome of interest. These characteristics are known as confounding factors.
Because propensity scores minimize this bias by balancing the known confounders between treatment groups, they have become increasingly popular3. Propensity scores have been used to compare surgical approaches4 and medical regimens5. Recently, we have used a propensity analysis of data from the United States Military Health System Data Repository (MDR) to assess the effect of statins in primary prevention of cardiovascular outcomes based on the presence and severity of coronary artery calcium6.
The MDR, utilized less frequently than the Medicare and VA data sets for research purposes, contains comprehensive administrative and medical claims information from inpatient and outpatient services provided for active duty military, retirees, and other Department of Defense (DoD) healthcare beneficiaries and their dependents. The database includes services provided worldwide at US military treatment facilities or at civilian facilities billed to the DoD. The database includes complete pharmacy data since October 1, 2001. Laboratory data is available from 2009 but is only limited to military treatment facilities. Within the MDR, cohorts have been defined with methods including use of diagnoses codes (e.g., diabetes mellitus7) or procedure codes (e.g., arthroscopic surgery8). Alternatively, an externally defined cohort of eligible beneficiaries, such as a registry, can be matched to the MDR to obtain baseline and follow-up data9. Unlike Medicare, the MDR includes patients of all ages. It is also less biased towards males than the VA database since it includes dependents. Access to the MDR is limited, however. Generally, only investigators that are members of the Military Health System can request access, analogous to requirements for use of the VA database. Non-government researchers seeking access to Military Health Systems data must do so through a data sharing agreement under the supervision of a government sponsor.
When using any administrative data set, it is important to bear in mind the limitations as well as strengths of administrative coding. The sensitivity and specificity of the code can vary based on the related diagnosis, whether it is a primary or secondary diagnosis, or whether it is an inpatient or outpatient file. Inpatient codes for acute myocardial infarction are generally accurately reported with positive predictive values over 90%10, but tobacco use is often undercoded11. Such undercoding may or may not have a meaningful effect on a study&#39;s results12. Additionally, several codes for a given condition may exist with varying levels of correlation to the disease in question13. An investigative team should perform a comprehensive literature search and review of the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) and/or ICD-10-CM coding manuals to ensure that the appropriate codes are included in the study.
Several methods can be employed to improve the sensitivity and accuracy of the diagnostic codes to define comorbid conditions. An appropriate &#34;look-back&#34; period should be included to establish baseline comorbidities. The look-back period includes the inpatient and outpatient services provided prior to study entry. A period of one year may be optimal14. Additionally, requiring two separate claims instead of a single claim can increase specificity, while supplementing coding data with pharmaceutical data can improve sensitivity15. Select manual chart audits on a portion of the data can be used to verify accuracy of the coding strategy.
Once comorbidities have been defined and assessed for the cohort in question, a propensity score may be used to balance differences in covariates between treatment groups. The propensity score is derived from the probability that a patient is assigned to a treatment based on known covariates. Accounting for this propensity treatment reduces the effect that the covariates have on treatment assignment and helps generate a truer estimate of the treatment effect on the outcome. While propensity scores do not necessarily provide superior results to multivariate models, they do allow for assessment of whether the treated and untreated groups are comparable after applying the propensity score3. Study investigators can analyze the absolute standardized differences in covariates before and after propensity matching or inverse probability of treatment weighting (IPTW) to ensure known confounders have been balanced between groups. Importantly, unknown confounders may not be balanced, and one should be aware of the potential for residual confounding.
When executed properly, though, propensity scores are a powerful tool that can predict and replicate results of randomized controlled trials16. Of the available propensity-score techniques, matching and IPTW are generally preferred17. Within IPTW, patients are weighted by their propensity or probability for treatment. Stabilizing weights are generally recommended over raw weights, while trimming of the weights can also be considered18,19,20,21.
Once study groups are balanced, they may be followed until the outcome of interest. Studies utilizing administrative data may be interested in outcomes such as readmission rates and time-to-event analyses. In studies interested in mortality, the Military Health System Data Repository includes a field for vital status that can be further augmented using the national death index (NDI)22,23. The NDI is a centralized database of death record information from state offices that is managed by the Center for Disease Control. Investigators can request basic vital status and/or specific cause of death based on the death certificate.
The following protocol details the process of conducting an administrative database study using the MDR augmented with mortality information from the NDI. It details the use of IPTW to balance baseline differences between two treatment groups including SAS code and example output.

<table><tbody><tr><td>CD Burner (for NDI Request)</td><td></td><td></td><td></td></tr><tr><td>Computer</td><td></td><td></td><td></td></tr><tr><td>Putty.exe</td><td>Putty.org</td><td></td><td></td></tr><tr><td>SAS 9.4</td><td>SAS Institute Cary, NC</td><td></td><td></td></tr><tr><td>WinSCP or other FTP software</td><td>&lt;a href=&quot;https://winscp.net/eng/index.php&quot; target=&quot;_blank&quot;&gt;https://winscp.net/eng/index.php&lt;/a&gt;</td><td></td><td></td></tr></tbody></table>

inverse probability of treatment weighting (propensity score) using the military health system data repository and national death index

When randomized controlled trials are not feasible, retrospective studies using big data provide an efficient and cost-effective alternative, though they are at risk for treatment selection bias. Treatment selection bias occurs in a non-randomized study when treatment selection is based on pre-treatment characteristics that are also associated with the outcome. These pre-treatment characteristics, or confounders, can influence evaluation of a treatment&#39;s effect on the outcome. Propensity scores minimize this bias by balancing the known confounders between treatment groups. There are a few approaches to performing propensity score analyses, including stratifying by the propensity score, propensity matching, and inverse probability of treatment weighting (IPTW). Described here is the use of IPTW to balance baseline comorbidities in a cohort of patients within the US Military Health System Data Repository (MDR). The MDR is a relatively optimal data source, as it provides a contained cohort in which nearly complete information on inpatient and outpatient services is available for eligible beneficiaries. Outlined below is the use of the MDR supplemented with information from the national death index to provide robust mortality data. Also provided are suggestions for using administrative data. Finally, the protocol shares an SAS code for using IPTW to balance known confounders and plot the cumulative incidence function for the outcome of interest.

Upon completion of IPTW, tables or plots of the absolute standardized differences can be generated using the stddiff macro code or the asdplot macro code, respectively. Figure 1 shows an example of appropriate balancing in a large cohort of 10,000 participants using the asdplot macro. After application of the propensity score, the absolute standardized differences were reduced significantly. The cutoff used for the absolute standardized difference is somewhat arbitrary, though 0.1 is often used and denotes negligible difference between the two groups. In a small cohort, proper balancing is more difficult to achieve. Figure 2 shows the unsuccessful results of attempting to balance covariates in a cohort of 100 participants.
Once the standardized propensity score is generated, the study team can proceed with outcome analysis. Survival analysis is often employed due to the need to censor participants with uneven follow-up information, and Figure 3 depicts an example of the use of proc phreg with standardized propensity score weights to generate a cumulative incidence function (CIF) plot. The CIF plot depicts the increasing number of events over time. In this case, the untreated, or control, group (No Rx) has a larger number of events and is comparatively worse than the treated group (Rx).
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/59825/59825fig1.jpg" /> 
Figure 1: Example of successful balancing. In a large cohort (n = 10,000), IPTW achieved balancing of the covariates with all absolute standardized differences reducing to less than 0.1. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59825/59825fig1large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/59825/59825fig2.jpg" /> 
Figure 2: Example of unsuccessful balancing. In a small cohort (n = 100), IPTW was unable to achieve balancing of the covariates with many absolute standardized differences remaining greater than 0.1. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59825/59825fig2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/59825/59825fig3.jpg" /> 
Figure 3: Example of cumulative incidence function plot comparing treatment groups. Over time, the cumulative incidence of mortality increases in both groups, though it is higher in the untreated group (No Rx). Thus, in this example, the treated group has improved survival. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59825/59825fig3large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
Supplementary Materials. <a href="https://www-jove-com.remotexs.ntu.edu.sg/files/ftp_upload/59825/supplementary.pdf" target="_blank">Please click here to view this file (Right click to download).</a>

Watch this Scientific Journal Video about Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index at JoVE.com

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

When randomized controlled trials are not feasible, a comprehensive health care data source like the Military Health System Data Repository provides an attractive alternative for retrospective analyses. Incorporating mortality data from the national death index and balancing differences between groups using propensity weighting helps reduce biases inherent in retrospective designs.

Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index

When randomized controlled trials are not feasible, retrospective studies using big data provide an efficient and cost-effective alternative, though they ...

inverse-probability-treatment-weighting-propensity-score-using

Nanyang Technological University

Research

JoVE Journal

Medicine

14.4K Views.  Washington University School of Medicine. In het verleden hebben we de militaire gezondheidszorg database gebruikt om te helpen bepalen welke patiënten het meest zouden profiteren van statine behandeling te helpen verminderen hun kans op een toekomstige hartaanval. Ons protocol biedt een roadmap aan elke medische aanbieder die geïnteresseerd is in het gebruik van big data om hun eigen klinische vragen te beantwoorden en kan worden toegepast op een veel variërend en bijna onbeperkt aantal onderwerpen. Na goedkeuring van de institut...

Video: Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

Expedited Radiation Biodosimetry by Automated Dicentric Chromosome Identification (ADCI) and Dose Estimation

Biological radiation dose can be estimated from dicentric chromosome frequencies in metaphase cells. Performing these cytogenetic dicentric chromosome assays is traditionally a manual, labor-intensive process not well suited to handle the volume of samples which may require examination in the wake of a mass casualty event. Automated Dicentric Chromosome Identifier and Dose Estimator (ADCI) software automates this process by examining sets of metaphase images using machine learning-based image processing techniques. The software selects appropriate images for analysis by removing unsuitable images, classifies each object as either a centromere-containing chromosome or non-chromosome, further distinguishes chromosomes as monocentric chromosomes (MCs) or dicentric chromosomes (DCs), determines DC frequency within a sample, and estimates biological radiation dose by comparing sample DC frequency with calibration curves computed using calibration samples. This protocol describes the usage of ADCI software. Typically, both calibration (known dose) and test (unknown dose) sets of metaphase images are imported to perform accurate dose estimation. Optimal images for analysis can be found automatically using preset image filters or can also be filtered through manual inspection. The software processes images within each sample and DC frequencies are computed at different levels of stringency for calling DCs, using a machine learning approach. Linear-quadratic calibration curves are generated based on DC frequencies in calibration samples exposed to known physical doses. Doses of test samples exposed to uncertain radiation levels are estimated from their DC frequencies using these calibration curves. Reports can be generated upon request and provide summary of results of one or more samples, of one or more calibration curves, or of dose estimation.

Expedited Radiation Biodosimetry by Automated Dicentric Chromosome Identification ADCI and Dose Estimation

The cytogenetic dicentric chromosome (DC) assay quantifies exposure to ionizing radiation. The Automated Dicentric Chromosome Identifier and Dose Estimator software accurately and rapidly estimates biological dose from DCs in metaphase cells. It distinguishes monocentric chromosomes and other objects from DCs, and estimates biological radiation dose from the frequency of DCs.

Biological radiation dose can be estimated from dicentric chromosome frequencies in metaphase cells. Performing these cytogenetic dicentric chromosome ...

Genetics

Establishing a Competing Risk Regression Nomogram Model for Survival Data

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively easy to apply and interpret and can be depicted visually. However, when competing events (e.g., cardiovascular and cerebrovascular accidents, treatment-related deaths, traffic accidents) are present, the standard survival methods should be applied with caution, and real-world data cannot be correctly interpreted. It may be desirable to distinguish different kinds of events that may lead to the failure and treat them differently in the analysis. Here, the methods focus on using the competing regression model to identify significant prognostic factors or risk factors when competing events are present. Additionally, nomograms based on a proportional hazard regression model and a competing regression model are established to help clinicians make individual assessments and risk stratifications in order to explain the impact of controversial factors on prognosis.

Presented here is a protocol to build nomograms based on the Cox proportional hazards regression model and competing risk regression model. The competing method is a more rational method to apply when competing events are present in the survival analysis.

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively ...

Cancer Research

Forecasting Hepatocellular Carcinoma Mortality using a Weighted Regression Model to Estimate Cohort Effects in Taiwan

To eliminate the influence of age and period in age cycle contingency table data, a multistage method was adopted to evaluate the cohort effect. The most general primary malignant tumor of the liver is hepatocellular carcinoma (HCC). HCC is associated with liver cirrhosis with alcohol and viral etiologies. In epidemiology, long-term trends in HCC mortality were delineated (or forecasted) by using an age-period-cohort (APC) model. The HCC deaths were determined for each cohort with its weighted influence. The confidence interval (CI) of the weighted mean is fairly narrow (compared to the equally weighted estimates). Due to the fairly narrow CI with less uncertainty, the weighted mean estimation was used as a means for forecasting. With the multistage method, it is recommended to use weighted mean estimation based on a regression model to evaluate the cohort effect in the age-period contingency table data.

We depict a multistage method to measure a cohort effect with age data, thereby allowing data to be eliminated in many situations without sacrificing data quality. The protocol demonstrates the strategy and provides a weighted regression model for analyzing the hepatocellular carcinoma data.

To eliminate the influence of age and period in age cycle contingency table data, a multistage method was adopted to evaluate the cohort effect. The most ...

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common MCI tests are time-consuming such that indiscriminate massive screening would not be cost-effective. Here, we describe a protocol that uses machine learning techniques to rapidly select candidates for further screening via a question-based MCI test. This minimizes the number of resources required for screening because only patients who are potentially MCI positive are tested further.
This methodology was applied in an initial MCI research study that formed the starting point for the design of a selective screening decision tree. The initial study collected many demographic and lifestyle variables as well as details about patient medications. The Short Portable Mental Status Questionnaire (SPMSQ) and the Mini-Mental State Examination (MMSE) were used to detect possible cases of MCI. Finally, we used this method to design an efficient process for classifying individuals at risk of MCI. This work also provides insights into lifestyle-related factors associated with MCI that could be leveraged in the prevention and early detection of MCI among elderly populations.

This methodology produces decision trees that target population groups more prone to suffering from mild cognitive impairment and are useful for cost-effective selective screening of the disease.

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common ...

Behavior

An R-Based Landscape Validation of a Competing Risk Model

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival outcomes. Different from the traditional Cox proportional hazard model, competing risk models consider the presence of competing events and their combination with a nomogram, a graphical calculating device, which is a useful tool for clinicians to conduct a precise prognostic prediction. In this study, we report a method for establishing the competing risk nomogram, that is, the evaluation of its discrimination (i.e., concordance index and area under the curve) and calibration (i.e., calibration curves) abilities, as well as the net benefit (i.e., decision curve analysis). In addition, internal validation using bootstrap resamples of the original dataset and external validation using an external dataset of the established competing risk nomogram were also performed to demonstrate its extrapolation ability. The competing risk nomogram should serve as a useful tool for clinicians to predict prognosis with the consideration of competing risks.

The present protocol describes codes in R for evaluating the discrimination and calibration abilities of a competing risk model, as well as codes for the internal and external validation of it.

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival ...

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful system for supporting the early detection of individuals at-risk of psychosis at a large scale, by leveraging electronic health records (EHRs). This risk calculator has been externally validated twice and is undergoing feasibility testing for clinical implementation. Integration of this risk calculator in clinical routine should be facilitated by prospective feasibility studies, which are required to address pragmatic challenges, such as missing data, and the usability of this risk calculator in a real-world and routine clinical setting. Here, we present an approach for a prospective implementation of a real-time psychosis risk detection and alerting service in a real-world EHR system. This method leverages the CogStack platform, which is an open-source, lightweight, and distributed information retrieval and text extraction system. The CogStack platform incorporates a set of services that allow for full-text search of clinical data, lifespan-inclusive, real-time calculation of psychosis risk, early risk-alerting to clinicians, and the visual monitoring of patients over time. Our method includes: 1) ingestion and synchronization of data from multiple sources into the CogStack platform, 2) implementation of a risk calculator, whose algorithm was previously developed and validated, for timely computation of a patient&#39;s risk of psychosis, 3) creation of interactive visualizations and dashboards to monitor patients&#39; health status over time, and 4) building automated alerting systems to ensure that clinicians are notified of patients at-risk, so that appropriate actions can be pursued. This is the first ever study that has developed and implemented a similar detection and alerting system in clinical routine for early detection of psychosis.

We demonstrate how to deploy a real-time psychosis risk calculation and alerting system based on CogStack, an information retrieval and extraction platform for electronic health records.

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful ...

Nomogram Development for Survival Prediction in MPCC Patients

Competing-Risk Nomogram for Predicting Cancer-Specific Survival in Multiple Primary Colorectal Cancer Patients after Surgery

Cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients is competitively affected by death from other causes. This study aimed to investigate CSS and associated risk factors by competing risk analysis in MPCC patients. The data of this study is from the SEER database. Using univariable and multivariable analysis of the competitive risk model to weaken the impact of competitive events, it explores the risk factors of CSS and develops a nomogram model. Then, the performance of the model is verified by the ROC curve, calibration curve, and DCA. The study encompasses a total of 8931 patients, with 6255 in the training cohort and 2676 in the validation cohort. Univariable and multivariable analyses showed that sex, Tumor Node Metastasis (TNM) stage, and tumor grade are independent risk factors for cancer-specific survival in MPCC patients. Based on the risk factors, we developed a diagram model to predict CSS. ROC curve, calibration curve, and DCA also show good results. In conclusion, a nomogram is developed that serves as a valuable tool for predicting CSS in MPCC patients, providing clinicians with crucial insights for personalized treatment planning.

The study found that male gender, poor tumor grade, and advanced Tumor Node Metastasis stage were associated with poorer cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients after surgery. We developed a nomogram to predict the CSS of MPCC patients and contribute to clinical treatment decision-making.

Cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients is competitively affected by death from other causes. This study ...

A Method of Trigonometric Modelling of Seasonal Variation Demonstrated with Multiple Sclerosis Relapse Data

This report describes a novel Stata-based application of trigonometric regression modelling to 55 years of multiple sclerosis relapse data from 46 clinical centers across 20 countries located in both hemispheres. Central to the success of this method was the strategic use of plot analysis to guide and corroborate the statistical regression modelling. Initial plot analysis was necessary for establishing realistic hypotheses regarding the presence and structural form of seasonal and latitudinal influences on relapse probability and then testing the performance of the resultant models. Trigonometric regression was then necessary to quantify these relationships, adjust for important confounders and provide a measure of certainty as to how plausible these associations were. Synchronization of graphing techniques with regression modelling permitted a systematic refinement of models until best-fit convergence was achieved, enabling novel inferences to be made regarding the independent influence of both season and latitude in predicting relapse onset timing in MS. These methods have the potential for application across other complex disease and epidemiological phenomena suspected or known to vary systematically with season and/or geographic location.

Combining plot analysis with trigonometric regression is a robust method for exploring complex, cyclical phenomena such as relapse onset timing in multiple sclerosis (MS). This method enabled unbiased characterisation of seasonal trends in relapse onset permitting novel inferences around the influence of seasonal variation, ultraviolet radiation (UVR) and latitude.

This report describes a novel Stata-based application of trigonometric regression modelling to  55 years of multiple sclerosis relapse data from 46 ...

Systems Analysis of the Neuroinflammatory and Hemodynamic Response to Traumatic Brain Injury

Mild traumatic brain injuries (mTBIs) are a significant public health problem. Repeated exposure to mTBI can lead to cumulative, long-lasting functional deficits. Numerous studies by our group and others have shown that mTBI stimulates cytokine expression and activates microglia, decreases cerebral blood flow and metabolism, and impairs cerebrovascular reactivity. Moreover, several works have reported an association between derangements in these neuroinflammatory and hemodynamic markers and cognitive impairments. Herein we detail methods to characterize the neuroinflammatory and hemodynamic tissue response to mTBI in mice. Specifically, we describe how to perform a weight-drop model of mTBI, how to longitudinally measure cerebral blood flow using a non-invasive optical technique called diffuse correlation spectroscopy, and how to perform a Luminex multiplexed immunoassay on brain tissue samples to quantify cytokines and immunomodulatory phospho-proteins (e.g., within the MAPK and NF&#954;B pathways) that respond to and regulate activity of microglia and other neural immune cells. Finally, we detail how to integrate these data using a multivariate systems analysis approach to understand the relationships between all of these variables. Understanding the relationships between these physiologic and molecular variables will ultimately enable us to identify mechanisms responsible for mTBI.

This protocol presents methods to characterize the neuroinflammatory and hemodynamic response to mild traumatic brain injury and to integrate these data as part of a multivariate systems analysis using partial least squares regression.

Mild traumatic brain injuries (mTBIs) are a significant public health problem. Repeated exposure to mTBI can lead to cumulative, long-lasting functional ...

Neuroscience

Shocktube Method to Model Highly Repetitive Low-Level Blast in Mice

Modeling Highly Repetitive Low-level Blast Exposure in Mice

Exposure to explosive blasts is a significant risk factor for brain trauma among exposed persons. Although the effects of large blasts on the brain are well understood, the effects of smaller blasts such as those that occur during military training are less understood. This small, low-level blast exposure also varies highly according to military occupation and training tempo, with some units experiencing few exposures over the course of several years whereas others experience hundreds within a few weeks. Animal models are an important tool in identifying both the injury mechanisms and long-term clinical health risks following low-level blast exposure. Models capable of recapitulating this wide range of exposures are necessary to inform acute and chronic injury outcomes across these disparate risk profiles.
Although outcomes following a few low-level blast exposures are easily modeled for mechanistic study, chronic exposures that occur over a career may be better modeled by blast injury paradigms with repeated exposures that occur frequently over weeks and months. Shown here are methods for modeling highly repetitive low-level blast exposure in mice. The procedures are based on established and widely used pneumatic shocktube models of open-field blast exposure that can be scaled to adjust the overpressure parameters and the number or interval of the exposures. These methods can then be used to either enable mechanistic investigations or recapitulate the routine blast exposures of clinical groups under study.

Author Spotlight: Deciphering the Long-Term Effects of Low-Level Blast Exposures in Mice

Presented here are methods for producing repeated low-intensity blast exposures using mice.

Exposure to explosive blasts is a significant risk factor for brain trauma among exposed persons. Although the effects of large blasts on the brain are ...

Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index

Samenvatting

Meer video's verkennen