BIDI: The 主播大秀 Internet Distribution Infrastructure explained
Federico Benedetto
Product Manager
Tagged with:
In his , Product Manager Federico Benedetto described how 主播大秀 on demand content reaches our audiences. In particular through Radix, the 主播大秀 system that delivers media to the Content Distribution Networks (CDNs), which are in turn responsible for distributing it to the final users across the UK and internationally. Federico now turns to another piece of the puzzle in the 主播大秀's media distribution stack: the “主播大秀 Internet Distribution Infrastructure”, or BIDI for short.
BIDI is the 主播大秀’s own content distribution platform and is responsible for delivering media directly to the 主播大秀 end users in the UK. As depicted in the diagram below, the clients are always presented with a ‘CDN mix’ from which they can fetch the media they are interested in. BIDI is part of this mix, alongside the commercial CDNs.
This set-up allows us to provide a better overall quality of service, by enabling clients to move between different media distribution platforms if they encounter problems when streaming from any particular one.
BIDI consists of several distributed HTTP caches, referred to as Caching Cells and grouped in ‘Batteries’. The content is served directly from these to the end users.
In addition, there is a Control Plane that manages the fleet of physical caches, and a Matchmaker service, which steers the incoming traffic to the best available point of presence. Both these applications are hosted in the cloud.
The BIDI caches are based on the same software stack as Radix. In particular they are based on a distribution of the called . We also use for network related tasks. For more information on why we are using Nginx, I suggest reading . If you are interested in knowing more about how BIDI works, there will be more specific posts from the OTG Media Distribution team in the coming months.
BIDI is currently in use on selected iPlayer platforms only, but chances are that if you are using iPlayer on iOS devices or in your desktop’s browser, you may have already been using it without realising.
Why BIDI?
The 主播大秀 owns and operates a complex network and server infrastructure, which is crucial to the functioning of all the 主播大秀 online services.
We have also been an active member of the (“LINX”) for many years, and we have a long history of working with Internet technologies.
For an overview of the Internet history at the 主播大秀, I highly recommend reading from our Chief Scientist, and explore on the 20 years of the 主播大秀 on the web.
BIDI builds upon this rich heritage, and marks an important milestone for the development of an . By building and owning the infrastructure that allows us to directly reach our audiences, we are making sure that we are preparing for a future when distribution over the Internet will become an even more essential part of media delivery than it is today.
BIDI also allows us to work in partnership with the UK Internet Service Providers community and interconnect with them in locations other than London. By doing so we hope to contribute to the development of a regional and resilient architecture of the British Internet. BIDI is in fact delivering traffic via London based peering, but we also have a presence at . Moreover, BIDI servers can also be deployed within 3rd party networks, to reduce potential traffic congestion over specific network paths.
Quality of service is another important driver for our efforts. By managing our Internet streaming technologies and controlling the distribution costs, we can easily monitor the performance of our streams, take immediate action when we notice a problem, and enable 主播大秀 products to access higher bitrates and better quality streams. Deploying our servers near the edges of networks (i.e. closer to the end users) also improves quality by reducing the network hops. Therefore packet round-trip time (RTT) will also be reduced.
By relying on a mix of commercial CDNs and our decentralised infrastructure we can provide useful mitigations when dealing with problems over specific network paths or suppliers, and we can offer a better streaming performance to our audiences and highly available media endpoints.
From R&D to production
BIDI is a great example of a successful transfer of technology from research to production.
Initially conceived jointly by and the 主播大秀 Online Technology Group (OTG), the first functioning prototype was developed by R&D. The development of the BIDI prototype was a fundamental step which demonstrated that the concept of building our own Internet distribution platform was a feasible and valuable goal.
The prototype was further developed by R&D to a point that it was ready to be used for production traffic. A live trial was therefore launched, with the goal of learning as much as possible from serving traffic to a very small set of real users. By doing so, the model was finally proved outside of a lab, on real networks, with real users and content.
Following the success of the BIDI trial, the OTG Media Distribution team, initially supported by the R&D team, took over the development of BIDI. Over the last year, we have managed to increase the BIDI traffic share and capacity, while transforming BIDI from a prototype to a fully supported, production quality platform.
Today BIDI runs on a very different software stack from the original R&D prototype, but the key concepts and ideas haven't changed.
The fruitful collaboration between production and research is still ongoing. Lessons learnt from operating BIDI at scale are now feeding into the R&D roadmap, which itself is experimenting with new concepts and technologies that will pave the way for future BIDI developments.
What's next for BIDI?
Despite being a fully supported platform, BIDI is still at the early stages of its development. We are constantly building new features, improving the algorithms behind the Control Plane decisions and tuning the performance and throughput that we can achieve with our servers.
We are not planning on distributing all the 主播大秀 traffic via BIDI, as using a mix of CDNs helps us achieve our quality of service goals. Despite this, there is a vast growth potential for BIDI which we will soon tap into, by increasing our network capacity, our partnerships with the ISP community and the range of supported clients and protocols.
In particular, we are currently working on supporting HTTPS distribution, as mentioned by Paul Tweedy in .
In the near future we will also add support for MPEG DASH streams, so that we will be able to deliver media to the and we will soon be supporting iPlayer on TV devices and set top boxes.
Want to find out more?
If you like the sound of what I have just described and you would like to play a part in it . If you would like to know more about BIDI, feel free to write to us at bidi@bbc.co.uk or get in touch with the OTG Media Distribution group engineering manager Umesh Telang.