Dozens of iPhone 14 (and newer) phones will replace the vexing VAR system in the English Premier League

It’s been exactly a month since the grand finale of the Euro 2024 football championship, but it’s time once again to talk about football. Yeah, I mean the sport where you dribble the ball with your right (or left) foot.

I hope US readers won’t scoff at this intro, because this story is really about… the iPhone!

See, the English Premier League will ditch the controversial VAR (Video Assistant Referee) system for the upcoming 2024/25 season. Instead, “a fleet of iPhones” will be used for the League’s new offside-detection system: the Apple handsets should be able “to spot a player’s position on the field, and call them offside, with more accuracy than ever”, as reads a lengthy exclusive by Wired.

The new system is to be developed and implemented by Genius Sports and subsidiary Second Spectrum (which have years of experience with optical tracking and data-based work in NBA). The new iPhone-based system is known internally as “Dragon”.

Dragon centers around using iPhones – in particular, iPhone 14 and newer models – which will be used to capture high-frame-rate video from various angles during the football matches. By employing a multitude of iPhones, Dragon leverages the cameras to gather detailed visual data. This data is then processed with custom machine intelligence software designed to coordinate the iPhones and handle the information effectively.

Of course, Dragon could be used for advancing motion-capture and AI technologies across a bunch of other sports. Early motion-capture systems were simple and needed minimal computing power, able to answer basic questions like how far an athlete ran or their average speed using just a few cameras. However, as queries became more complex, the technology had to evolve.

– Mike D’Auria from Genius

Sports often present challenging situations like players clustering together, leading to “occlusion” where certain angles are blocked from view. Earlier systems used educated guesses to fill in these gaps, but for critical moments, such as offside decisions, higher accuracy is required. Traditional semi-automated systems used in events like the FIFA World Cup struggled with occlusion and accuracy in offside calls.That’s where Dragon comes into play: it aims to improve this by using at least 28 iPhone cameras at Premier League stadiums. The system utilizes iPhone 14 units housed in custom waterproof cases with cooling fans and power sources.

The iPhones capture continuous video from multiple perspectives, allowing Dragon to track between 7,000 and 10,000 points on each player. This high level of detail is expected to enhance precision in offside calls and other aspects of game analysis. That’s pure sci-fi, as far as I’m concerned: again, your iPhone will be used to track up to 10,000 body points on a player. Now this is phone innovation!

The Dragon/iPhones system can capture up to 200 frames per second, significantly higher than the 50 or 60 frames per second typical of broadcast video. This capability helps reduce gaps between frames, improving the system’s ability to pinpoint exact moments like a ball’s kick.

Dragon’s machine intelligence system, known as “object semantic mesh”, is trained on years of football data to provide contextual understanding of movements, enhancing real-time analysis. While this technology is advanced, human referees will still make final decisions with the aid of Dragon’s AI tools.

Despite the high cost of implementing traditional optical-tracking systems, iPhones offer a more affordable solution. Genius plans to adjust the number of cameras used based on the problem at hand, with flexibility to scale up or down as needed. This adaptability also supports the creation of “digital twins” – virtual replicas of players that can be used for detailed analysis and new broadcast possibilities.

Although exact improvements over previous technologies are not disclosed, the upcoming Premier League season will showcase Dragon’s capabilities.