The NBA's Tantalizing and Terrifying Future in Virtual Reality

The NBA has big plans when it comes to virtual reality. How will fans, referees and players be impacted? The Crossover breaks down the NBA's rapidly-approaching future.

The year is 2023. LeBron James Jr., the NBA’s newly-minted No. 1 draft pick, attacks the basket from the wing as LeBron James Sr. slides across the paint to take a charge. The younger James lifts off for a tomahawk dunk over his father, causing the crowd to rise as one and prompting TNT commentator Kevin Harlan to scream, “Who’s your daddy now!?”

Within seconds, a fan in Ohio, wearing a virtual reality headset connected to a smartphone, adjusts the camera angle of the replay so that he can view the highlight from James Jr.’s perspective. Halfway across the globe, a Chinese fan dons his VR headset and tilts the camera so that he can watch the same play from the poster victim’s angle, trying to determine whether James Jr. committed a foul.

As both fans rush to share their clips on social media, TNT’s producers craft a rotating replay that spins around James Jr. and James Sr. at the moment of impact. During the next dead ball, TNT airs an extended highlight reel featuring the dunker’s view, the defender’s view, the sideline referee’s view to determine whether there was an offensive foul, and then a fan-centric view from within the cheering mob.

At halftime, James Jr. puts on a VR headset in the locker room to relive his coming of age moment frame by frame. Across the hallway, James Sr. puts on a VR headset, queues up a top-down angle, and wonders aloud why the referees didn’t call his son for a charge. Meanwhile, back in TNT’s Atlanta studio, Charles Barkley and Shaquille O’Neal watch a side-by-side breakdown of James Jr.’s highlight next to a similar James Sr. play from earlier in the season, both viewed from the dunker’s perspective. “Like father, like son,” they marvel.

The TNT broadcast then throws to the NBA’s video review command center in New Jersey, where an off-site official had studied dozens of alternate angles to isolate whether the game officials had missed a foul call on the dunk. Later that night, the three referees in the arena conduct a VR-enhanced post-game review, donning headsets to re-watch James Jr.’s dunk and other hotly-debated judgment calls from carefully-chosen angles.

This Brave New NBA World—as tantalizing or terrifying as it might sound—is rapidly approaching.

“Imagine that a virtual camera is like a bumblebee that can fly anywhere on the court,” NBA Vice President of Global Media Distribution Jeff Marsilio told The Crossover by phone on Monday. “You could even fly the bumblebee right on the nose of your favorite player, and its video output could be incorporated into the television broadcast or sent straight to a VR headset, so the viewer is looking out of the bumblebee’s eyes. We are still a couple of years away, but this science–fiction reality isn’t as far off as you might think.”

While VR is currently viewed as the domain of early adopters, NBA executives are betting that the technology will play a major role in growing the league’s global audience by enhancing its broadcasts, providing new fan experiences, and possibly even improving its officiating.

On Tuesday, the NBA and Turner Sports announced deals that make Intel the league’s exclusive provider of VR for the NBA on TNT. The new partnerships will tip off during 2018 All-Star Weekend in Los Angeles. Although specifics of the coverage have not yet been revealed, fans should expect VR coverage from Sunday’s All-Star Game, Saturday’s Slam Dunk Contest, and the other marquee events.

Intel’s “volumetric video”—which was previously deployed at the 2017 NCAA basketball tournament—allows users to select from multiple viewing angles, mimicking the courtside seating experience. Going forward, the crown jewel of Intel’s VR capabilities will be its “freeD technology,” the bumblebee-like 360-degree virtual camera that will capture action from any angle on or above the court, including from a player’s perspective. While a traditional TNT broadcast uses between 10 and 15 cameras—as well as a producer and a director to determine what a viewer sees—the immersive “freeD” experience will theoretically allow a VR-equipped user the freedom to hand-select from a seemingly endless number of angles.

Courtesy of the NBA

​​

According to James Carwana, VP and GM of Intel Sports, the building block of the company’s VR capture system is a pod that features 12 cameras with stereoscopic lenses. By placing numerous pods around the court, Intel can capture the action from enough vantage points that it is able to stitch together a 3-dimensional video, much like a cell phone’s camera stitches together a panoramic photo. To create the bumblebee-like virtual camera, Intel will utilize as many as 36 ultra-high definition 5K video cameras around the arena.

“We flock to opportunities like immersive media where there are vast amounts of data,” Carwana said. “Just think about the number of cameras and the amount of data they put out. We’re talking about terabytes per minute. There are vast amounts of data coming off these systems, and Intel believes strongly that it is the best-suited organization to help process this data and make it come to life.”

While the NFL, MLB, NCAA and the Olympics have all broached VR experiences for their viewers, the NBA is arguably the most natural fit for this as-close-as-possible technology. NBA players don’t wear helmets or hats, gameplay is in (near) constant motion, and the highlight-play culture is well-established. The league has benefited greatly from increased access to its players through social media, thanks to a relatively young and technologically-savvy fanbase that overlaps with the VR demographic and grew up with lifelike video games such as NBA 2K.

The NBA still sees its foray into VR as a long-term experiment, acknowledging that immersive media has yet to fully unfold across the market to casual users. But Marsilio, who calls himself an “enthusiastic” VR advocate, strongly believes the technology will be key in the league’s global push.

“More than 99% of our fans are not within practical reach of our arenas,” he said. “VR is the next best thing to sitting courtside. To offer that experience to fans wherever they might be in the world is tremendously valuable. We believe VR has the promise of bringing the world closer to our game than any technology has before.”

Careful viewers will have noticed that TNT has already incorporated Intel’s 360-degree replays—through swinging cameras that can zoom in or out during replays—into its studio shows. Intel also showed off its technology in an ad starring James last March.

[youtube:https://youtu.be/ZzrLFHd3q3w]

At present, there are hurdles to widespread adoption. For starters, Intel’s full “freeD” technology will be rolled out gradually across the league, beginning with Dallas’s American Airlines Center and Cleveland’s Quicken Loans Arena. And while TNT’s VR offerings will run through the 2018 postseason, including the conference finals, ESPN and ABC broadcasts will not be VR-enabled. In other words, don’t expect to watch James in the 2018 NBA Finals on a VR headset.

Ezra Shaw/Getty Images

​

Not every interested fan will be able to view the NBA’s VR offerings this season. To experience VR content, fans need a cable or satellite subscription, a Samsung Gear VR or Google Daydream headset, and a corresponding smart phone. VR-ready users will then be able to download the forthcoming “NBA on TNT VR” app, although Apple phones are not supported.

Nevertheless, potential VR applications extend well beyond fan-focused entertainment. The league’s instant replay video review system has undergone numerous technological refinements in recent years, including the establishment of a centralized New Jersey command center. Currently, the off-site command center referees have access to every broadcast camera angle to make their determinations. But what if they could use the bumblebee-like virtual camera to pivot around a controversial play in three dimensions for the exact angle needed to make a definitive ruling?  

“We’re always striving towards perfection around officiating,” Marsilio said. “To the extent that technology can help us do that, we’re very forward-thinking. We have a ref center that has every angle available live, providing real-time feedback to refs. You could go further than that and pinpoint the exact perfect angle with a virtual camera and provide that in real time to a referee. That would be something like the Holy Grail. That’s still a way's away, but it’s something that we’ve talked about and fantasized about.”

Maybe that fantasy will be realized before the 32-year-old James calls it a career. Or, maybe VR winds up shaping LeBron Jr.’s generation like Facebook and Twitter have impacted the NBA over the past decade. Either way, basketball fans will soon be able to experience the sport’s most memorable moments from unprecedented angles, all thanks to a bumblebee camera and its endless terabytes of data.