20% increase in injuries reported with the change to an electronic system
Electronic Health Records (EHRs) are systematically collected health records electronically stored in a digital format. EHRs can be linked with sports-related data to create a foundation for “real-world health research.” This means that the database can be used for research inquiries – as long as the data is collected and maintained appropriately (i.e. data curation). The National Football League (NFL) has such an “evidence platform” using an EHR that is linked with other sport-related data to study health and safety in the league. A recent study describes the NFL’s data collection, data curation, quality improvement, and analytic processes.
The NFL’s Evidence Platform
The NFL has systematically surveyed and collected data on injuries for more than 30 years; there is data on 8000 players and more than 42,000 injuries just from the past decade (1). Initially, data came from the voluntary reporting of injuries but this injury reporting system eventually became mandatory. In 2014, the NFL, in partnership with the NFL player’s association (NFLPA) launched an initiative to capture injury and treatment information through an EHR system adapted for use in sport in order to comprehensively examine injury occurrence. Athletic trainers and team physicians alongside independent neurotrauma-specific physicians and the NFL Game Day Surface Task Force enter data into this system. Data collected includes: anatomic location of injury, physical findings, how much time missed from injury, play type, player position, contact type, impact source, football activity, game location, stadium and field surface type, and play counts.
An important component of an EHR system is not only the data collection part of the process but also the data curation. Data curation is the organisation of data to ensure quality control. Amongst many processes, the NFL has implemented reviews to check for completeness of reporting, trainings to standardize data entry as well as highlight areas of importance when completing data entry, and feedback collection processes on data entry challenges.
How is the Data Used?
With more than 3000 tables of unlinked raw data, there is an abundance of information that can be relevant to a research question. Players, owners, general managers, athletic trainers, team physicians, medical staff, alongside medical professionals working with the NFL pose questions that can be answered with the use of the collected data. There also are committees for a variety of medical areas which help to facilitate cross-specialisation discussion on research questions. Specific research projects include examining the unaffiliated neurotrauma consultant and spotter programs for concussion detection alongside looking at injuries on artificial turf versus natural grass (2,3). Interestingly, not only does proper data curation help facilitate research but research projects can help direct how curation occurs.
Challenges for EHR-Enabled Research
Despite the potential and real benefits of EHR systems for research and creating evidence-based policies and procedures, there are challenges with using EHRs for research which ultimately impact the reliability of the data for a given research project (4,5). Data may be:
- recorded in an unstructured format.
- located in areas that are inaccessible to researchers.
- inconsistently recorded.
- missing (6-8).
These challenges arise due to the nature of how data is collected in this professional sport environment. For instance, a normal research initiative would have a small and highly trained team entering data with a common understanding of operational definitions (in the NFL, it is currently challenging to quantify injury severity in a standard manner across the league) and the data would be collected in a controlled setting with an EHR system tailored to the specific area being examined.
Another challenge to consider is that when interpreting data one must also consider system-related factors that may impact the data. For instance, technical system changes can impact completeness of data entry and even accessibility to the system while changes in awareness of reporting can lead to increased reporting.
Yet, there are solutions that can be implemented to help with these challenges such as improving the technical interface to reduce data entry burden; additional guidance/training for those entering data; more comprehensive data collection; and implementing external quality control to ensure data accuracy. Furthermore, one must not forget the strengths of an EHR system specifically within the NFL environment which include the ability to study a complete and clearly defined population as well as having a strong research tool for researching and understanding “explanatory and predictive studies” that not only will benefit the NFL but can contribute to the broader sports medicine field.
Conclusions
The study ultimately found that there was a 20% increase in injuries with the change to an EHR system. This finding does not necessarily mean the sport is more dangerous, but that the reporting has gotten better. Furthermore, most injuries were accurately reported with less than 2% of entries requiring corrections.
In saying that, it remains important to continue data curation by means of reviewing data, collecting feedback, completing regular training, and developing guidance documents in order to maintain a high-quality system that is “robust enough to support decision making.” Correspondingly, examining how effectively this data informs player health and safety policy and practices and of course, how effectively these policies and practices are implemented are similarly important initiatives to pursue.
Ultimately, the NFL’s EHR system and any lessons learned from its implementation and development can possibly benefit other sport programs and the healthcare of athletes overall especially in a time where society demands evidence-based decision making.
Works Cited
- Powell JW, Schootman M. A multivariate risk analysis of selected playing surfaces in the National Football League: 1980 to 1989. An epidemiologic study of knee injuries. Am J Sports Med. 1992;20:686-694.
- Hershman EB, Anderson R, Bergfeld JA, et al. An analysis of specific lower extremity injury rates on grass and FieldTurf playing surfaces in National Football League Games: 2000-2009 seasons. Am J Sports Med. 2012;40:2200-2205.
- Mack CD, Hershman EB, Anderson RB, et al. Higher rates of lower extremity injury on synthetic turf compared with natural turf among National Football League athletes: epidemiologic confirmation of a biomechanical hypothesis. Am J Sports Med. 2019;47:189-196.\
- Daniel G, Silcox C, Bryan J, et al. White paper: Characterizing RWD Quality and Relevancy for Regulatory Purposes. Published October 1, 2018. https://healthpolicy.duke.edu/sites/default/files/atoms/files/characterizing_rwd.pdf. Accessed June 16, 2019.
- Miksad RA, Abernethy AP. Harnessing the power of real-world evidence (RWE): a checklist to ensure regulatory-grade data quality. Clin Pharmacol Ther. 2017;103:202-205.
- Dreyer NA. Advancing a framework for regulatory use of real-world evidence: when real is reliable. Ther Innov Regul Sci. 2018;52:362-368.
- Gliklich RE, Dreyer NA, Leavy MB. Registries for Evaluating Patient Outcomes: A User’s Guide. 3rd. ed. Rockville, MD: Agency for Healthcare Research and Quality; 2014.
- Gliklich RE, Dreyer NA, Leavy MB, Christian JB. 21st Century Patient Registries: Registries for Evaluating Patient Outcomes: A User’s Guide: Addendum. 3rd ed. Rockville, MD: Agency for Healthcare Research and Quality; 2018.
- Mental Health in Competitive Athletes - October 5, 2020
- Concussions & Race: Does Race Impact Care - September 28, 2020
- Even 20 Years Ago, Females Were Suffering from More Concussions - August 18, 2020
Resources