How Virtual Reality Could Help Improve NFL Officiating

There's no replacement for on-field practice, but NFL officials could start utilizing virtual reality to help put them in the center of the field—without actually being in the center of the field.

NFL officials—much like players—need to practice in order to be at the top of their game, and watching football on a screen is no substitute for standing on the field in the middle of the action. To get that practice, many attend team practices and camps, making unneeded calls while players run through drills and throwing flags that result in no penalties. Away from the field they review game film, study the rules book and take quizzes, but nothing beats reality.

That might be about to change, though, thanks to the virtual equivalent. STRIVR is a Silicon Valley-based virtual reality firm that records practice plays using a 360-degree camera positioned where a player would stand on the field, allowing players to watch that play later using a VR headset, getting extra, virtual, reps. The NFL has been keeping a close eye on what STRIVR has been doing with sports teams since 2015, and a year and a half ago, the league reached out with a question: Can virtual reality help our officials perform better?

Starting in the 2016 preseason, STRIVR began building a library of clips that officials can watch, repositioning its cameras to capture the view from each of the seven officiating locations. “The perspective that a fan might want in virtual reality is different from an official, as is the perspective of a particular position, a quarterback or linebacker,” says Damani Leech, VP of football strategy and business development at the NFL. The database is now up to 500 clips, and features footage of half a dozen teams, including the 49ers, Cardinals, Rams and Broncos.

STRIVR captures video from these three positions on the field, mimicking where NFL officials stand / Courtesy of STRIVR

The NFL hasn’t yet opened up its library to its 124 game officials, but instead it has been piloting the use of STRIVR among members of its Official Development Program class—a mixture of college and CFL officials “who are on the cusp of becoming NFL game officials,” according to Leech. These prospective NFL staffers had the chance to try STRIVR at off-season clinics this past May and June.

Eight ODP members were named to the NFL’s roster in May and June, raising the distinct possibility that some existing officials have VR experience, even if they might not currently be using STRIVR. “No active game official is training with VR right now,” Belch says, “[but] some will. I would say by halfway through the year we’ll actually have some that are using it.”

STRIVR seems to be particularly well-suited to capturing an officials-eye view. The camera can remain in a fixed location because officials don’t move around as much as players and are usually under no threat of getting sacked. In comparison, says Derek Belch, STRIVR’s founder and CEO, “if you have to do it live you can’t really put the camera in the [quarterback] pocket. With an official, we can literally put the camera right where they would be.”

Some of the clips are filmed from the referee’s perspective, back behind the play on the quarterback’s throwing side, while others are from the umpire’s view, on the other side of the quarterback pocket. There is video looking straight across the line of scrimmage, the down and line judges’ positions, and also from the backfield, recreating what the side, field and back judges see. Because players in practice will drill everything from full team plays to seven-on-seven to one-on-one, officials get all of those situations too. “We’ve got a lot of different scenarios,” Belch, a former placer kicker and graduate assistant coach at Stanford, says, “we have almost everything that a referee would ever see.”

The videos also start long before a play does and run on after it has finished, simulating the full responsibilities of officials. “Before the ball’s even snapped, these guys have so much to document and detail and process,” says Omar Ahmad, STRIVR’s director of NFL training partnerships. When the huddle breaks, officials count the players to ensure neither team lined up with an extra man. Before the play they look for false starts or encroachment. When the ball goes dead they watch for any unsportsmanlike conduct.

The MMQB Mailbag: Long View on Lions

And they need to do all of that without significantly disrupting the flow of a game. “People don’t realize,” Belch says, “they love to hate on refs. [But] they do a really good job considering they’re making all these calls in a fraction of a second.”

After every game, the NFL Officiating Department reviews every single play from seven different angles, one for each of the different positions. According to the league, officials got 95.9% of calls right last year, out of more than 40,000 total plays. While that statistic represents a high level of accuracy, it also means that there were half a dozen bad calls in every single game. Some of those mistakes may have cost a team a game it could have won, a 31-34 playoff loss for the Cowboys against the Packers last January, for example. In the first quarter Dallas receiver Brice Butler was assessed a 15-yard penalty for unsportsmanlike conduct that the NFL later admitted was a mistake. The Cowboys would end up punting on that series.

STRIVR might help the league fine-tune officiating accuracy a couple more percentage points, and not simply because it offers officials a chance to practice. Head-tracking data from the Oculus VR headset allows STRIVR and the NFL to see where officials are looking at every point throughout a play. Unlike fans and commentators, officials can’t watch the ball. Ask one about a breathtaking play they oversaw, and you might find he or she missed out on the excitement. “The value of this has been more about are the officials looking at the right things at the right time that puts [them] in the position to make the call,” Leech says.

Cameras, like these mounted at U.S. Bank Stadium, capture the action on the field for freeD / Courtesy of Intel

A big plus for virtual reality is that it can help officials train for rare events, like onside kicks. “There are very few special teams plays in a game,” Leech says, “so we don’t get a lot of practice with those plays.” An un-quantified minus is whether practice plays are sufficiently similar to real game action to be truly useful. There is no way to capture footage directly from the field without carrying a camera into the melee.

At least for right now, there’s not. Along with improvements in virtual reality, there’s another emerging video technology—Intel’s freeD—that may in fact allow the NFL to capture a true official’s perspective.

FreeD uses a ring of cameras located around the stadium to recreate for fans what a player would see on the field. There is no reason it couldn’t do the same for an official’s perspective. The problem with freeD right now is the processing and storage of data. Thirty-eight 5K-resolution cameras capturing 30 frames per second creates so much data that Intel’s freeD installations at NFL stadiums are constantly dumping most of it, and processing just one frame of a player’s view can take a bank of 40 servers 90 seconds or more. Processing an entire play would take days.

But processing time is rapidly decreasing and storage ability is rapidly increasing. Soon, freeD might start to approach real time, and could feed into systems like STRIVR’s. One day that might replace the need for officials to even stand on the field.

• We have a newsletter, and you can subscribe, and it’s free. Get “The Morning Huddle” delivered to your inbox first thing each weekday, by going here and checking The MMQB newsletter box. Start your day with the best of the NFL, from The MMQB.

• Question or comment? Story idea? Email us at talkback@themmqb.com.