StreamOn recap l Preparing for large-scale events

00:00:07:32 – 00:06:36

Really excited to host this panel at our inaugural StreamOn Summit. I’ve known Sergey, Mayur, and Jerónimo for many years. And I’ve always wanted to bring the Olympics, Fox and ITV together to hear perspectives from a global US and UK European perspective.

All three of their organizations help broadcast some of the largest events in the world, like the Olympics, the World Cup, and the Super Bowl. So they are well positioned for a session on preparing for these large scale events.

As in the last year, Fox just did the last Super Bowl. We had the Olympics and the Euros last year, and are preparing for the World Cup next summer.

Before we start, let’s do a quick round of introductions.

From Fox, we have Mayur Srinivasan, based out of LA, SVP of the Digital Video Platform team and leading all of Fox’s digital video efforts. Mayur has been in the industry for over 25 years, from being a researcher at ASU over devex to now leading the Fox teams for the last eight years.

Those that were present during his presentation at the last NAB know that Mayur has a wealth of knowledge about end to end video workflows in place for some of the largest events in the world and the preparation that goes into it. So we thought he’d be the perfect guest for this session. Mayur, feel free to introduce yourself.

I know that is a very kind introduction. Thanks, William. Yeah, my name’s Mayur Srinavasan or Mayur. I basically head up the digital video platform team at the Fox. We at Fox are responsible for all of the end to end delivery of our video streaming content, starting from signal or origination to encoding to hosting, to multi-screen delivery and all of on the back end side of things.

We also manage the player SDKs across the Roku vertical separate app of Roku and web. So internally we are the platform team. As William alluded to, we’re responsible for a lot of large scale sporting events. We did the Super Bowl this year. Yeah. Good stuff there. Thank you. Thanks for having me.

Thank you, Mayur, for being here. I know it was really tough to get these three gentlemen together on one call. So being here, it’s a real privilege, to have them all together.

From the Olympics, we have Sergey Podlesski, based in Madrid, New Media technical manager at OBS, the Olympic Broadcasting Services. But truly the main man for all of OBS efforts in the video streaming space. We’ve been working together since 2018, preparing for the Tokyo Olympics.

And next to knowing about everything from ingest to play out, from codecs and containers to smart TV SDK, Sergey is also one of the kindest people you’ll ever meet. A man with nerves of steel who always remains calm and composed in difficult situations. Sergey as well has been over 25 years in the industry, from RTL to NTV plus over Panorama to HBS and now OBS, so I’m very happy to be able to host him here for our panel.

Hello everyone! Willem, thank you very much for this introduction. Also, it’s a pleasure and an honor to be in this group here on this stream.

Yes, you mentioned the Olympics as one of the good examples of the big events. I just wanted to make a note that Olympic Broadcast Services has a much broader scope than just streaming. OBS is the host broadcaster of the Olympics. First and foremost, our organization is in charge of producing the live coverage of the games. But, in our other capacity, we also help media rights holders to distribute the Olympic content.

And this is the area where I’m working in and where I’m in charge of the umbrella of products called Olympic Video Player, where we help rights holders to build their OTT streaming experiences and video experiences around live coverage of Olympics.

Thank you, Sergey. That’s a great explanation of what OBS does because I think it’s sometimes still a bit unclear where that sits.

Last but not least, from ITV, we have Jerónimo Siahinolfi, based in Lisbon, video streaming engineering manager at ITV. Jerónimo has been in the industry for 15 years, from Portugal Telecom to Vodafone and now leading the video engineering team at ITV.

We’ve been working together for many years and it’s been a pleasure to see him and Mark Eisen leading the way of more innovation and streaming efficiencies at a large org like ITV. They are in full preparation mode for the World Cup next year, so I thought he’d be the perfect guest for this Cross Continents panel.

Thank you so much for this amazing introduction. Yes, it’s been a pleasure to work together and it’s very important to have this type of event so we can share the painful parts and also the lessons learned. We have across different events how we are building amazing partnerships together. Thank you!

And myself, I will be your host. My name is Willem De Saegher, based in New York City, responsible for the strategic accounts within Dolby OptiView. Close to my ten year anniversary in the industry, so a rookie compared to these heavy hitters, but getting there.

I came over from the THEO Technology side, which got acquired by Adobe last summer. It’s been an exciting journey, especially working with the large sports leagues and broadcasters and streamers.

The session today will be about the preparation that goes into large scale events from technical challenges over playback quality, scalability, monetization, latency to fan engagements.

So without further ado, let’s kick it off.

00:06:36 – 00:10:52

We’ll start with large scale events, the operational scale and readiness. Can you give us a sense of the scale of the largest sports events you’ve streamed recently in terms of audience reach or geographic footprint?

Maybe, Mayur, you’ve done a very recent big event.

Yeah, I can kick this off. So on February 9th of this year, Fox TV with the help of our platform team at Fox streamed the largest Super Bowl to date. We served the Super Bowl to 24 million unique viewers, but when we peaked at about 15.5 peak concurrent viewers. That’s a pretty large scale event.

And we did this whole serving platforms. So actually, for the very first time, were able to beat broadcast latency, like the glass-to-glass latency that you see, OTT versus actually digital streaming on certain platforms. We’re able to beat that. And also 4K content to end users who are capable, at scale. So that’s something..

Amazing Jerónimo, Sergey. Anything, any event that comes to mind for you?

Yeah. So I think, on ITV over the obvious one, it’s we have every two years, very big event. Last year was the Eros that we hosted and we managed to deliver the semi-finals with England. So this means a lot of concurrent streams being played back.

We have reached in a single channel, 3.5 million concurrent streams. It was 3.7M across the entire platform. And so this was delivered only in the UK. So probably later on we can talk about some of challenge that we face with this.

Yeah. And during the final was, 4.5 million concurrent streams. And that was shared with BBC. Sergey back to you.

Yeah. So, our cycles are obviously also every two years we have Olympic Games, alternating winter and summer games. So we’re now on the final stages of preparations in Milan and Cortina, Winter Olympics.

But last year, very successful, very big and beautiful, Paris Olympics. So obviously the scale, you may imagine of this as a whole globe, but as I mentioned so on this part of the delivery on the distribution side, we just help some of the right holders.

So in the case of Paris, our coverage with the various major rights holders we’ve been acting as a service provider for them in the OTT domain, we were, covering Olympics on around 70 different countries all over the world and in different four continents, actually.

So for us, the scale is not just the audience, but also the scale of the content. So Olympics is not just one stream, is a big number of sports. So for summer games, it’s more than 30 sports, more than 40 disciplines.

And at the peak moments, this normally goes during the first weekend of the games. We can see up to 52 simultaneous streams running with different events on the first qualification stage. But, yeah.

So the variety of contents is another dimension of the scale we’re addressing with Olympics. Yes.

Are you still in Madrid Sergey or are you already in Milan?

In the last two weeks in Madrid. And in the beginning of November, I’m relocating to Milan already.

You know that?

I remember that from previous Olympics. You guys moved there very early on, so that’s maybe a good segue in the next question on the preparation.

00:10:52 – 00:15:57

Like how far in advance do you guys begin preparations for these events?

Sergey, let’s start with you.

Yeah, sure. So the preparation overall is, as I said, there are a lot of different tasks. So OBS itself starts immediately after the games are assigned, because there’s a lot of planning done. There’s liaison with organizing committee. I’m not talking now, the streaming side, but the overall the preparation for the games. So the broadcast side and production.

But when it comes to the real on site type of things, our construction teams who are in charge of building the International broadcast center normally land around a year before the games, already with the work, planning and meetings going much earlier, but the actual work on the ground starts then.

Now, when it comes to building the OTT flows and distribution flows, we normally start immediately after the previous Olympics finishes. So year and half, a bit more than that, with planning and designing the next games platform.

And then a year before is when we try to finalise the designs and freeze the architectures and enter into more like testing, load testing, and integration phase. And now as I just said, a couple of months before the games, we already hit the ground with building the on premises parts of the solution, because our flow covers also the on site contribution encodings.

So we get the baseband feeds from broadcast engineering and our OTT head-end is on premises. The encoders that do their ABR encoding are being built already now. So, we have games in February next year. November is when we start building the OTT platform.

Okay, cool!

Mayur, you’ve just done a major Super Bowl. I loved your session at NAB, can you give us a bit of insight on when you start and how you ramp up towards the event?

Yeah, I mean, this is our third Super Bowl and in between we’ve done multiple large scale events and stuff. So the platform—we started building this platform way back in 2019 for the 2020 Super Bowl, and it has matured over the years. Honestly, we started a little too late for the 2020 Super Bowl.

But typically, to give context for this particular Super Bowl, we started our planning as early as April of the previous year at NAB. So I was holding discussions with CDN vendors to kind of preserve capacity and ensure that we’re able to handle this large scale event.

Just to give you some context, in 2020 we had about 15 terabits, for 2023 around 40, and for this one we planned pretty well and hit about 135TB. I had actually reserved a little bit more than that and made sure we started early preparation back in April.

Then we got a little bit of a curveball with the news that Tubi had to stream the Super Bowl, back in June or July of the previous year. So we started all of our integrations, player SDK integrations with client teams, and wrapped that up by September or October.

After that, we did a bunch of rehearsals, load testing, and validations through October, November, December. We locked things down in January and used that time to fine-tune, running multiple load tests and scaling tests to ensure readiness for the day of the event.

Amazing!

That topic of reserving CDN capacity—Jerónimo, it really resonates with what I’ve seen you guys do. Can you give us some insight into how you prepare and when you start for these large scale events?

Yeah. So now it feels like we have a very similar challenge here. We started working with CDN vendors one year ago during IBC. It’s usually a long journey because CDN capacity is not something that can be built overnight. It requires ongoing discussions and planning.

I think we are in a good place right now. We are entering a phase where we have about nine months to go until the World Cup. We have already finalised some development, but we continue stabilising the platform to focus more on readiness.

So that’s my perspective—this is continuous development and improvement from the last event. Every time we have an event, we learn from previous lessons and implement improvements for the next one.

So yes, we are now working through the critical path for the World Cup, with a lot of interesting challenges ahead.

Yeah. It’s amazing to see the similarities between your organisations in terms of preparation.

00:15:57 – 00:25:27:40

I think the audience always is interested in like, what were the challenges, right. Like what were the things you didn’t prepare for? What’s an interesting technical challenge you faced when preparing for your large scale events over the past years?

Yeah. First we need to be ready for unexpected. And this can be the RME issue. This can be an ordinary issue. So we build a lot of redundancy across the entire stack as much as we can.

Then, of course, there’s points that are more difficult. And we never know when the entire region will go down. So it’s very important to have sometimes geographic redundancy, sometimes a lot—we have a lot of queue switch. We have a great good control panel that we can turn on and off features when we need.

And very important also to have good metrics in real time, to be able to see what is happening in the platform and to be able to act as soon as possible. So there’s a lot of preparation in terms of forecasting how we want, what we need to do to be able to accommodate this amount of traffic, but also have real time feedback to quickly act if something goes unexpected.

Yeah. Mayur, I think that’s a topic we discussed for a long time—is all the preparing for the unknown. Give us your insights. I’m very curious.

Yeah. As I was saying, this is a new integration for us, the business context was, we want to drive a lot more registered new users to Tubi. So it’s not just about scaling the video platform or the link CDN integration, CDN reservations and stuff. It’s also thinking about it from a holistic perspective.

How do we try collective operational excellence across the board, not just video, but say, registration services behind the scene and, like, other entry points into the app before the user even gets into the stream, right. So I think the biggest challenge was, like kind of, ensuring that we did it about 29 different, connected client integrations across all of these devices, across all of these platforms and across all of these back end services, not just video, but across like everything else.

How do we ensure that we’re scaled and ready, because live streaming is hard. At scale, it’s even harder. And then yeah, zero failures are tolerated. You cannot have any downtime. People are going to be top of you, various scenarios, like handling various scenarios and stuff.

How do you prep for all of that? Like bunch of users end up joining this team at the beginning of the game. Really, how do you ensure all of those thundering personalities matter your services are all scaled up across the board? I think that collective operational excellence and training people to react quickly at the cost of acceptance.

Yeah, I think that I found a very interesting talk, once talking to you was like training your people to think quickly. Right? Because things will go wrong anyway. But it’s the pace on how they can solve it.

Maybe one add-on question, Mayur, how do you deal on a technical level with all these spike joint sessions, like all these people obviously for a Super Bowl everybody joins at the same minute. Like, do you spread it out or how do you do it?

No. You have to ensure that all of your services are pre-warmed, pre-scaled, to ensure you are able to meet those specific spikes. So you got to anticipate what the spikes would be. I think we were kind of like, expecting my RS list to be around, like, minimally 200K requests per second, like 200K users joining on this per second basis.

They even load tested our services all the way up to 500K requests per second. We ensured that we were ready to hit manage any amount of scale, and we did encounter that. Like, in fact our, even before the game started, we’d already crossed the previous Super Bowl numbers and stuff.

I mean, it’s kind of been therein and just within that -20 to +20 is like a massive spike, like from 6 million to 1.30 million users kind of joined, within that 20 minutes. So it’s crazy to see that Spike.

It must have been exciting in those first minutes, seeing like, oh, we already crossed the last Super Bowl and we haven’t even started yet.

Yeah, I’m not exactly, here it was like, it’s interesting to see the audience grow. I think the thing which throws off, which we realised later on was, it was advertised as free. Tubi was, the Super Bowl was advertised as free, and that the traffic patterns and all that were different than what we’ve normally seen.

We’ve done a lot of data analysis to kind of see which are the hotspot supporting areas for CDNs. Where could traffic typically comes, it was adequately on Fox, whereas this was Tubi and a different ecosystem. The marketing was around free. So the traffic patterns were very different.

So we had to kind of dynamically adjust and make sure we’re serving traffic with good QE nodes.

Yeah. Interesting. Sergey, anything to add.

No, I just can add that most of the challenges in preparation are indeed not really technical, but operational. So it’s preparing for the unknown, as you said. Yes, things may go wrong.

So it’s not just being prepared to think fast, but also to have plan B, C, D, disaster recovery for most of scenarios. To have around you the people and the technical partners who know the system very well, and they can address things and can think out of the box, indeed, and react quickly.

I would add that on our end something maybe quite specific is that we operate every time in a new location and every games are unique. So there is as much as you can do on the testing side. Yes, we test in advance and way in advance, but the reality is that the system as a whole comes to life only during the live.

So we can see things that are really happening for the first time when everything came together. So we were able to test things individually at scale, yes, but things all connected together. So there is no way we can repeat or create the Test Olympics in advance.

So there is always something that is for the first time when we’re actually going live. And for us normally the first day of competition. So the opening ceremony is what attracts the whole world, but it’s still one stream. But when the real thing start to happen is the day after, when all the events happening together on different venues, signals coming from different venues, a lot of data streams, because we also dealing with the non video stuff.

So these things—and yeah you only can be there if you can address this, if you have the team that knows the system very well and you have all the scenarios prepared.

Another thing that is quite challenging on our end is, as I said, we’re dealing more as a system integrator or service provider. So we have to deal with the timelines and the technical teams of our clients, which also have different preferences, have different skills, and have different rhythm of preparation to Olympics.

So from the ones who do it for many games, and have the trained teams who know it to the ones who just came for the event for the first time, there are a lot of operational challenges related to integration with their platforms.

Yeah, very interesting. I think, Mayur, it’s a bit similar for you, right. Because you provide the technical platform for Tubi apps as well as other entities within Fox.

Yes, we do. So we basically when we are running live streaming operation also doing streaming also. Right. Like it’s a 24/7 ecosystem. But I think it’s interesting, like for large scale events, you almost need to put your platform into what I would call like a large scale event mode, where you’re trying to simplify and reduce the number of bells and whistles that your typical day to day platform has.

You want to make sure like you’re running a very simple platform with all the necessary knobs within that simple workflow, right. You don’t probably need necessarily complex things like DAI, or things, third party systems which could potentially feel not very good. You’re trying to reduce dependence on other third party vendors as much as possible.

That’s the key philosophy we follow.

That makes a lot of sense. Thank you all for sharing those insights. Like, I might start a big event soon, actually, having all that knowledge now.

00:25:50 – 00:35:01

Gonna make a bit of a switch. We obviously came, I came from the player world, THEOplayer, but now Dolby OptiView player. Playback quality is a big word in my dictionary.

How do you assess video playback quality, like startup times, rebuffering during your live events. Which metrics or tools do you rely on?

Yeah, we have a bunch of tools, standard QE metrics tools that the industry has. Its familiar, but we allow marks out to ourselves. Others have refused us to kind of look at the video QE quality.

Typical things we look at as videos standard times, videos rebuffering rates, playback videos, are there any spikes over there? But those are just from a player QE review.

We have metrics across each and every part of the ecosystem when we’ll have observability was a huge, thing for us. We wanted to make sure each and every part of our ecosystem but monitoring, like, for example, like our CDNs reserving about the like what’s the throughput the CDNs are having, like all that kind of highs.

But from a player perspective, main things were done. I think very close eye on rebuffering rates and, quality of experiences, like what bitrate have you. And of course, to be prepared. Yeah. Because I get it.

Do you use those quality metric tools also to like make real time decisions?

Yeah, I mean, in fact, from a hotspot in perspective, you can drill down and see which parts of which CDNs are, probably having some issues, in terms of constraints and stuff. Because if you see like, say, the buffering spikes are playback failures or any of those QE metrics being affected on a particular location on a particular CDN, you can track, take real time decisions to potentially either add more capacity to that by a written real-time requesting that or dropping more or routing it away and using the different parts that get those users.

Very interesting. Jerónimo. I saw your notes on the switching between the CDNs. Do you see that happening at ITV as well?

Yeah, yeah. So, we do all of this. We can use one of the most famous tools in the industry for the video quality. And we also have our own internal cox base tool to measure the video quality. Yeah. This is part of our runbook.

So we also need to make sure that these tools are also scaled up, warmed up properly. Because if not, the data will start arriving late with delay. So then we lose the visibility. And they’ve the possibility to react very fast. And this is something not we do. We have to it’s part of business as usual.

So if we see some problem on, and usually, it dont want my everything, if we fix CDNs, specifically, ESM maybe can be performing a little bit better or worse in or have some rebuffering, and the video start failure and so on. We have good tools to put the traffic, in another CDNs, for that this year. And for, it’s very important to have this real-time tools.

Also maybe a bit of ahead of the session. But we also need always to have visibility and scaling of or at system. So we need to prefetch the ads to not have problems at scale.

That’s a great segue to our next topic, indeed, Jerónimo. Like scalability, how do we touched upon it a little bit, but how do you ensure that your infrastructure can scale to meet, like peak demands during your events? And Mayur, you touched upon that you are preparing for a lot more and almost still a hit at the peak. Like, how do you know what is enough, is maybe the question.

When you look towards that sharply, you know, when you’ve got to figure out what capacity you need. And then, what could I backwards from there, we in fact, anticipated will probably at about 10 to 12 peak concurrents. We ended up getting what 15.5.

So I would say like get your estimates and then probably 1.5x it. Right. And see if you can, get to those numbers. And if you have the ability like just keep pushing the boundaries even beyond that 1.5x, maybe go up to 2x, and see where your system start failing. And I think that’s that’s probably a good lens to look at it from.

So as I said before, we were not trying to get to that 200K request per second, kind of thing from our backend services, we kind of went up to all the way up to 500K just to see how our systems react and stuff.

Interesting. Do you just fully for the amount of viewers coming in, it’s hard to simulate that. Like the best we can do is some particular tests at them. And then you kind of simulate these thoughts, hitting traffic from distributed parts of the country.

But even with these tools, I mean, you get like that’s the extent encoded, cannot truly really simulate that day off behavior. You certainly not simulate like say 15 million peak concurrence being hit that platform for that bit for doing these tests over and over.

So you have to be smart with these load tests and do it. Yeah. The real test is in production in a way. And for being that time, that’s a good point.

I mean, if you can, I would advise folks like test in production whatever system you’re planning on the day off, ensure that you’ve actually put it through production. Traffic can run, even though it’s not that scary.

And everyone it’s like a 10th or 100th or a 1000th of the scale that you are anticipating actually tested in production. And we’re fortunate enough to have multiple events leading up to the Super Bowl, when we were able to exercise this platform and ensure that things were working as expected.

Yeah, that’s a good advice. Thanks for sharing that.

Sergey with you, I think that’s a bit different because there’s obviously only one Olympics every two years. Like how do you deal with that issue.

Yeah. No it’s a challenge indeed. And we don’t have a luxury. Although, there are smaller events like Youth Olympics where, we’d potentially can, but the scale is completely different there.

So it’s more like testing the design ideas or more the flows, but the actual scale, there is no magic. So we need to press scale. We need to have automation in place. We need to benchmark. So that’s what we do during our load testing. So we just exactly know where is the limits of each node. And then if we start approaching it in production then we will have the means to scale.

So we discussed this. So we use the hybrid infrastructure. So there is the cloud element there. But there is a big on prem element which is hard to scale. So it has to be pre-scaled all the infra there and the bandwidth and the capacity on that.

One thing I can mention on our architecture when it comes to the liaison between origins and CDN we have we manage our own in-house caching layer. So it’s we don’t connect CDNs directly to the origin, so we have our own caching, which also scalable and obviously the geographical diversity and then and redundancy, all sorts of DRS.

And the last thing I wanted to say is also that there is as much as we can rely on automation. So monitoring tools are extremely important. So there should be physical eyes on what’s going on. And actually people’s personal decision on making certain switches or making certain actions because obviously in case of Olympics or Superbowl, others.

So we want our users first to see without interruptions and without degradation. So it’s like first we switch to something that works, then we deal with what’s wrong with the other side. But there should be an immediate way to restore the quality to restore the flow if something happens. So the switching and the scaling is essential there.

Interesting. I saw Mayur nodding that. Jerónimo, is that the same on your side, the caching that happens, so you don’t always hit the CDN directly?

We have very big concern on that to protect our origins in this sense. So of course, it depends. So if you have a single user join the edge, then the caching on the CDN works in a much better way.

And actually we have a very good origin shield in between our CDNs and our origin to protect and to be able to collapse the request to minimise the impact on the inside.

Okay. Very cool. Thanks for sharing all that. Like, so much effort goes into this scaling problem.

Making again, a move to monetization.

00:35:01 – 00:47:02

I’m curious. Obviously, we have an ad solution. How do each of your organizations approach monetization for these large scale events? Mayur, you said a lot about, like, keeping it simple for large scale events.

There is a big buzz about personalized server guided ad experiences now at scale. Like, are you guys already looking at DAI and how do you look at it combined with large events?

Yeah, I mean, it depends on the scale of the event. For extremely large scale events like Super Bowl, we tried to avoid that, doing any dynamic decisions. The whole idea there is to reduce the number of dynamic decisions that you do and keep everything as statically cached and big fat users as possible.

So for those typically we have done pass through ads, but then in other events like the World Cup and stuff, which are magnitude of scale slightly smaller than the Super Bowl. Some of the tactics we use for DAI or NMECADAN ad insertion is like prefetching and I think it was a little bit to in the past that.

So things like that, which will help ease the load of third party ad management systems. Right like, so that’s where you’ve to be smart about how do you kind of, the request going or transitioning systems are like, kind of, coming in waves that like just bursting and having every user make, like if you have like, say 10 million users, you know, 10 million requests going to a third party system seeing, and seeing that those systems failed.

All right. So where we careful about that end.

Yeah, the other question, we are looking at Server Guided Ad Insertion (SGAI). I think that’s very interesting piece of technology. And we’re investing in that, and looking at it, I think the challenge there, the promise is there I think in the device adoption pieces that critical challenge.

So we’re kind of, working with all various, partners in this ecosystem to ensure that we can, at least where applicable. We use such intersection or SGAI, to kind of, scale up platforms even more. So very exciting.

Jerónimo, how does ITV deal with the monetization aspect? Because as mostly commercial broadcaster, it’s your lifeline. So how do you deal with it during large scale events?

Yeah. So, I think there’s two parts here. One as Mayur saying, for these big events, and we do have the ads, just to get clear what is our current setup. We have DAI, yeah, it bring a lot of impact but we don’t have for all devices in these big events.

Sometimes you say it’s a commercial decision because the value of the ads. If every single person in the country you see the same ads then the part of the ad goes up. So then not the technical challenges, all that, because it’s what makes more sense in the commercial perspective.

Then even though we do have DAI, and that brings a quite significant challenge because of the personalization, every single, video session needs to be personalized. This means, very big piece of infrastructure.

And as Mayur was saying, yes, DAI brings a lot of promise. But also we bring challenges in terms of device integration. But that’s why Dolby’s here with that very good promise on that front.

I think when, not now the technology is getting more and more major. So it will be sooner time to look at that, to see because it’s a promise to scale this type of personalization for this amount of traffic.

Yes, we’re happy to support you in that journey, Jerónimo.

I see we’re at the last 5 to 10 minutes, so I want to make sure we touch upon a few other topics as well.

Latency. That’s obviously a world where I came from as well. We with our HESPprotocol. And now or web RTC solutions as well. What kind of latencies are you currently achieving for live sports, and how has it evolved over time and maybe what is the target latency that you are working towards?

Yeah. Yeah, I can touch upon that. Like as I was mentioning previously, I think latency is interesting as a problem. Right. Because typically latency is pretty close to the leverage you have more chances of rebuffering seem to be very cautious over that.

At the end of the day, you want to deliver a really good experience at least. But that being said, I think good latency can be achieved with good architecture. Right?

And we actually as humans, kind of evolve over the years and we were able to for this particular Super Bowl on certain platforms actually beat over their broadcast latency. And I think that’s the gold standard, if you think be better than broadcast, and deliver to end users that’s probably what you’re aiming for.

But it becomes a challenge on certain, it’s more of a device constraint, right like that, certain devices, when you don’t have control over the entire player stack, you can make these optimizations to ensure that you’re hitting those broadcast latencies to where you can and where it’s optimized.

I think that those can be achieved with good architecture.

I do have a question on that. Mayur, like you beat the broadcast with your streaming solution. Was that ever an issue? Was there ever like, hey, the streaming services should be on par with broadcast. They cannot beat the broadcast latency.

No, I mean traditionally, I mean, if even to this day, if you see a lot of the major, like, MVPDs partners out there are typically 30 to 40s behind, the broadcast, beats. Right like, we’re watching over more cable or antenna versus like streaming.

Why are these digital properties, it’s typically around 30 to 40s behind. And that’s driven by a lot of factors, including DAI which we all have. And we have the server ads. And that adds another dimension to latency.

But, I think for things like Super Bowl and when, you want to be as close as possible to the live, instead of like having a neighbor show it. Hop and say heyyy! the stream one or whatever. Right.

Like you want to make sure that you’re seeing that also performing well enough.

Hey, exactly. And that’s where my question came from, like you beating the broadcast latency with your streaming, did that give you did you get questions from leadership like, hey, like in the future. Like, if we cannot beat it, like, we need to be on part or that’s not an issue to be faster.

I mean, that’s the goal. And I think, people in the industry do understand now what are some of the constraints which increase latency. And if you need bells and whistles like DAI and similar features, the latency is going to be slightly worse compared to broadcast latency.

But I think there are further improvements like LL-HLS and similar approaches that could help achieve even better performance.

Sergey, how is that work in the Olympics world? Latency.

Yeah, I agree there was one important point made by Mayur is that, it’s other factors that kick in when you start improving latency. So for large scale events, like Olympics, quality is something we need to prioritize.

From games to games there are improvements. For Paris Games, standard HLS latency was around 20 seconds, while low latency HLS achieved around 12 seconds. And we’re aiming for below 10 seconds for Milan and Cortina.

However, we are not aiming to beat broadcast yet. We listen to what our clients prioritize, and latency is usually not the main requirement.

We need to deliver smooth and high-quality experiences first. That is the priority for large scale events.

I think it’s maybe for events like 100-meter sprints where latency matters more. But for overall Olympic coverage, quality remains the main focus.

Exactly the same applies for us. Usually, it’s not the latency that is the top priority, especially for public service broadcasters.

Fans watching frequently may expect low latency and advanced features. But when delivering large-scale national events, the priority is ensuring that every user can watch reliably and with good quality.

Are any of your orgs looking at integrations with interactive features or betting, which would require lower latency?

I can say so. As for betting is a forbidden thing for Olympics. So we don’t mention it. But we do have professional use cases where latency is relevant. We provide low latency streams for specific users who require near real-time access.

Yeah, typically for us, we’ve shied away from adding interactive elements for very large scale events. But we have experimented with features like multi-view or limited interactive experiences.

Those use cases exist, but they also introduce latency and scalability challenges. Ultimately, it’s about balancing innovation with stability.

As a viewer, you might want these interactive features, like betting on a penalty in real time. These are emerging capabilities in streaming. But they must always be delivered with stability and high quality.