A recent article in the New York Times entitled “N.F.L.-Backed Youth Program Says It Reduced Concussions. The Data Disagrees” has exposed USA Football / Heads Up Football for publicizing misleading data and conclusions about the findings in the Datalys study. In their blogs, USAFB crowed about the effectiveness of the Heads Up Football program in reducing concussions; however, the actual data from the study do not support this.
Sadler Sports Insurance realized that misleading statements were being made several months ago after an independent review of the Datalys Study and mentioned this in our blog entitled AYF Study: 2005-15 Concussion Trends in Youth Tackle Football. We are in favor of getting the head out of the tackle, but have serious concerns about any program that publishes misleading conclusions in its blogs, especially when making the claim that their program is the standard in youth tackle football.
USAFB hired Datalys to study effectiveness of HUF and practice contact restrictions
USAB hired and funded Datalys Center, an independent research firm specializing in sports injury research and prevention, to study player safety in youth football on a national level. They did so in an effort to promote safer playing standards and lessen injury risk, including concussions. Datalys completed a study in 2014 entitled Comprehensive Coach Education and Practice Contact Restriction Guidelines Result in Lower Injury Rates in Youth American Football. The study compared overall injury rates, lost time injury rates, and concussion rates between three groups:
- Heads Up Football Leagues with Practice Restrictions (Combined HUF | Practice restriction)
- Heads Up Football Leagues without Practice Restrictions (HUF Only)
- Non Heads Up Football Leagues (Non HUF)
The study has been widely cited by USAFB and the NFL as evidence that Heads Up Football leagues have significantly fewer injuries and fewer concussions than Non HUF leagues.
The unstated reason for the study was to prove that the HUF Only group significantly reduced concussion rates vs the Non HUF group.
It’s my opinion that the primary reason for the study was to find out if the HUF initiative reduces concussion rates in youth tackle football. Youth tackle football is under attack by the media and various researchers and activists, not because there are too many overall injuries, but instead because there are too many concussions and subconcussive impacts.
However, the statistics generated from the study were not favorable for the HUF Only leagues as regards to reduction of concussion rates. The concussion rates for the Non HUF leagues were actually a little lower than the HUF Only group, as will be explained.
USAFB made misleading claims about Datalys study
In various online publications such as its blog, USAFB made the following statement as regards the Datalys study (but recently taken down):
“Compared to non-Heads Up Football leagues, leagues that adopted Heads Up Football had a 34 percent reduction in concussions in practices and a 29 percent reduction of concussions in games.”
To the extent this statement appears to compare HUF Only leagues vs. Non HUF leagues, this is just not true according to Table 2 in the 2014 Datalys study.
What the math really says about concussion rates in HUF Only leagues vs Non HUF leagues
Table 2 of the Datalys study indicated the following about concussion rates in Combined HUF | Practice restriction leagues vs. HUF Only leagues vs. Non HUF leagues:
Practice concussion rate
Game concussion rate
Combined HUF|Practice restriction
Concussion rate is the rate per 1000 “athlete-exposures,” defined as one athlete’s participation in one practice or one competition.
It’s clear that the concussion rates are actually slightly greater for the HUF Only leagues as compared to the Non HUF leagues in both practices and in games, although these slight differences were statistically insignificant.
Practice restrictions drove the reduction in concussion rates in the study
The Combined HUF | Practice Restriction Group produced significantly lower concussion rates as compared to both the HUF Only leagues and the Non HUF leagues. Why is this the case? Is it the combined synergistic effect of HUF + practice restrictions or is it primarily due to practice restrictions?
The researchers lamented not having a group of leagues to study that consisted of leagues with practice restrictions, but not HUF. That is the only true way to isolate the practice restriction variable. Regardless, instituting HUF without practice restrictions did not reduce concussions.
Was a new group formed after the fact to rescue HUF?
It is possible that the misleading statistics referenced in the USAFB blogs differ from the Datalys study due to an attempt to rehabilitate the poor HUF Only performance? A new group may have been formed, called “Leagues that implemented Heads Up Football,” consisting of all leagues that used HUF, regardless of whether or not they also used practice restrictions. Data from this new group is consistent with the statistics that were referenced in the USAFB blogs, which USAFB has since taken down. Because the injury rates were much lower in HUF leagues that used practice restrictions, the Datalys report separated out these groups to show the difference, whereas the USAFB blogs may have combined these groups to hide these differences. The statistics that were created in the blogs were misleading because they implied that HUF was the driving force behind lower injury rates and don’t give the practice restrictions the proper credit.
Our conclusions and what this means for brain injury / concussion risk management
We still believe that removing the head from the tackle is an important element of a youth tackle football brain injury / concussion risk management program. However, it is clearly not the primary solution as touted by USAFB and the NFL. USAFB has hurt its credibility with this misleading marketing campaign.