主播大秀

The 主播大秀’s VR Hub is a small team at New Broadcasting House in London, working to demonstrate how virtual reality could become a valid part of the 主播大秀’s output. To do that we’ll be creating a number of ambitious VR projects over the next twelve months.

In February, we finished our first 360 video documentary, . The project uses a ‘headset first’ philosophy. By that we mean that it was written, shot, and edited to work best when viewed on a VR headset.

It’s a two part series, of 12 and 16 minutes duration, taking users on a trip down the Nile with accompanying them. With Alastair presenting, there’s a real heart to the story, which is about a dispute over control of the Nile’s waters. We wrap that story in an immersive 360 video journey down the river.

While we were working on the production, the VR Hub brought out a app, which is available on Gear VR and soon Oculus Go. It allows us to deliver high quality 360 video content to the mobile VR platform made by Oculus and Samsung. Damming the Nile is available in the 主播大秀 VR app, along with some other experiences which the 主播大秀 have made, on bbc.co.uk/virtualreality.

The camera

The 主播大秀 has been making 360 videos for a while but usually using small cameras like Thetas, GoPros, or Samsung’s Gear 360. We chose instead the Insta360 Pro camera and a Sennheiser Ambeo to record 360 sound. (If you’re in the 主播大秀, an improved version of the kit we built is now available for hire from Production Services.)

The Insta360 Pro is essentially six cameras in one. Each camera fires at the same time and the image is captured through a super wide angle lens. Its total field of view is so huge that everything around the camera is actually seen and recorded twice - from two different perspectives. Using a clever algorithm, the images are then stitched together to create a spherical ball of footage. It uses the same ‘rectilinear’ format that maps of the earth use, basically a spherical format mapped onto a flat surface. The net effect on a VR headset is that you can look in any direction and see what the camera could see at that moment - and in 3D.

Strictly speaking, this is 360 video, not virtual reality. In true VR, the aim is to simulate as much of the human condition as possible: in real life we can pick things up, move around, prod things and watch as they react. We can look wherever we want and can move through our world. To achieve that in VR, you have to build your content in a . Everything will look, in some capacity, like a computer game, but what you lose in realism you gain in immersion and interaction. We’re also building this kind of experience at the VR Hub but Damming the Nile isn’t that: it’s 360 video. The viewer can look anywhere they want but they can’t move unless we moved the camera while we were filming. They’re fixed to the point where the camera was. 360 video allows us to capture a scene for real then build experiences which add to the illusion of being there.

A couple of years ago this was already possible but it wasn’t easy enough to bother with on a news production. It was just too difficult, cumbersome and risky. With new cameras, creating 3D 360 video is still difficult but it’s a more realistic possibility for small and nimble productions.

So what did we learn?

If there was one take home message from this entire trip, it’s that going the extra mile to use the Insta360 Pro for each and every shot is almost certainly worth it. Smaller ‘action cameras’ are an exciting tool for 2D filmmakers, but in 360 they cause an immersion dip that just isn’t worth it, no matter what vision you had for a nimble and interesting shot.

I also confirmed my belief that that 3D is definitely worth pursuing. It creates something believable and beautiful - something truly immersive. So since we can get it, we really should get it.

Having your subject and your scene close to the camera is the best way to go. The shots I like the most in the film are the ones which made me feel most nervous as we shot them.

“The subject is too close to the camera,” I’d think as the contributors wander clumsily around the rig (not you Alastair!). That’s because in the past, 360 cameras had a significant ‘dead zone’ around the camera: you’d get stitching errors, parallax issues, and a bad viewing experience if your subject strayed into this area. So you’d want to keep the subject away from the dead zone, but close enough to appear more than an incomprehensible smudge of pixels in the VR headset. A blind tightrope walk between stitching disaster and useless sludge ...360 film-making was fun!

But with the Insta, having the subject in close proximity creates pleasantly surprising results. Our shot on the train feels just right. You could really believe that you’re there and yet on the shoot, the camera felt way too close for comfort. The same is true for the Egyptian temples: they feel reachable, touchable. At times you may even forget you’ve got a phone strapped to your face!

But once we’ve achieved the illusion of being there, are we done? Is that enough? We could immerse you in a wonderful rainforest waterfall, but for how long could we hold your attention there?

We need to keep the story moving along, which may mean dragging the viewer away from whatever scene they’re currently engrossed in. And that can be frustrating for them. Are we taking them away from a place they’d only just got to know? Could we slow the pace right down so they can just enjoy the moment? An immersive scene can overwhelm the senses to the point that the story is briefly forgotten about: instead the viewer is content to just be in that particular moment and looking at some aspect of the shot that you, the storyteller, never paid much attention to. So are we aiming for immersion or story? And if we’re trying for both, how can we bring the two together? (Alastair also talks about this in our ).

These are the considerations you have to make as you’re writing the script and unpacking the story. We structured the documentary as predominantly image-led, with Alistair narrating from scene to scene. The relationship between the camera and the production crew is an interesting problem. We treated the camera as the fifth member of the crew, even naming him Marvin to really humanise him. At key points in the story, Alastair would do a piece to camera, which is a very different thing from a traditional PTC for news. It’s more like talking to one of your friends - that’s certainly how it feels when you watch it back in a headset - so if Alastair had used the conventional tone of a news piece, it would have just been odd.

For the most part, we hid the crew out shot, but this was only to create the best view for Marvin. We didn’t make it a rule that we should stay hidden. We actually wanted the audience to understand there was a crew taking them from place to place. And from a practical point of view, it’s impossible to hide on every single shot, so we made a feature of the crew being a part of that journey where it made sense.

Thinking about all of these problems inevitably opens a can of worms, and there’s an infinite supply of worms in the VR/360 can. Each could be diligently inspected but then there’d be a danger we’d overthink it to the point that we’d never make a 360 video again.

The exciting part about finishing a project like this, is that through inspecting the can of VR worms, you’re left thinking of many other ideas that might work well. For example, now that we can shoot 3D 360 video in close proximity to the subject, what subjects might work best? Since the subject can get inside your ‘personal space’, does that open up opportunities in fictional storytelling? Since immersive 3D video is achievable now, can we create experiences which exploit immersion more than story?

Practicalities

I should probably share at least some practical learnings. First, you probably won’t create quite as much data as you might expect if you’re shooting on an Insta - especially if you’re used to shooting on a hacked together GoPro rig. Data management is much more streamlined when you’re shooting on just one high speed SD card. As a result, the edit is certainly quicker, but here be dragons.

The new tools that allow you to quickly get ‘a stitch’ out of your 360 shots (integrating the output of the different cameras in the Insta), mean you can review footage as you go. Inevitably this leads to an assembly being created, even while you are on location. But there’s a danger is in using this assembly as the basis of your edit - for reasons that are way too dull to go into here. I’d recommend getting back to your edit suite and stitching everything you’ve shot into 4K. Don't make selects and render individual parts of your shots, just stitch the whole thing on the most powerful machine you can find, then import everything into Premiere Pro. From there, create proxies of your renders and edit with these. This requires a lot of computational power, but it will put you a good position to get the post production done with fewer (but still some) headaches.

(If you want more on of why I think the above workflow works best, I’ll be happy to share. Or if you think I’m wrong, please shout and we can discuss the alternatives.)

Update

I only wrote this blog a week ago and it’s already out of date. Insta have announced they’ll be releasing a plugin for Premiere Pro which would have saved our production an enormous amount of time. Details of the new workflow are If it works as well as they claim, then I highly recommend that you create your edit in the way they describe. I’ll be investigating this workflow properly soon.

In conclusion, if you liked this torrent of thoughts on VR/360 and are inspired to try your hand, come and talk with us at the VR Hub and maybe we’ll make something awesome together.