Welcome, and an Exploration of Split-Screen 360 Film
David Vincent
Welcome to our blog!
As fellows at the CRI this year, we will be exploring Virtual Reality filmmaking, diving into the process of creating 360 films from conceptualisation, writing, directing, to analysis. Every month we will be releasing a short film that experiments with 360 concepts in filmmaking. To complement this, we will blog here about our workflow, analysis, discussions with filmmakers and scholars, and even tutorials.
Our first experiment is a short 360 film (currently on YouTube) that explores the concept of spatial editing using split-screen as a storytelling tool. Watch below, then read through our conceptualisation and analysis:
Concept
Virtual reality (VR) exists in an omnidirectional environment. Cinema on a rectangular screen is often composed using the “rule of thirds,” however no such rule exists for the composition of a shot that completely encompasses the camera. This creates a unique challenge for the 360 degree cinematographer for which there are very few documented solutions. Our first experiment is an exploration of editing within a spatial dimension.
In theatre, which is not necessarily bound to two dimensions or a restricted frame, light is an essential tool in directing the attention of the viewer. Stage-lighting techniques can enhance our ability to do the same within a VR headset.
Objective
In editing film, there is a physical seam between lengths of film that the editor uses to his or her advantage by exaggerating or hiding the cut. In either case, the goal is to make the edit seamless, to avoid breaking the suspension of the viewer’s disbelief. Edits in VR cinema are more difficult to make seamless.
We have speculated that by editing 360 degree video spatially, rather than temporally, we can create edits that are more seamless when viewed in the VR headset. Splitting the encompassing screen by hemispheres is a possible solution to edit between two shots.
Experiments
We created our first short content piece, which took the form of a narrative video. Two characters, occupying opposite hemispheres of the screen, spoke on their phones with friends about an encounter they had had on the train with one another.
Expected Results
We expected that using a hemispherical environment would present the viewer with two persistent worlds that seem both as separate environments and as a consistent experience. The audience’s head motion determines which world they wish to follow.
Narrative
We collaborated with a screenplay writer to adapt a short story by Haruki Murakami into a short VR film. We wanted to tell a story that was personal, emotional and in some way romantic - rather than rely on the audience’s awe of the technology to create a convincing experience (a debate that will come up often on this blog).
Methods
Our experiment took the form of a short film. It was filmed, blocked, and timed to blend two shots which would persist for the full length of the film. High contrast and limited/minimal on-set lighting were used to direct the viewer’s attention from one hemisphere to the other. Audio was mixed in mono to avoid a directional bias.
Spatial Analysis
Our spatial edits are directly linked to the staging and composition of the shot. We played with four specific features:
Hiding vs Showing characters - Our characters appear and disappear in order to shift your focus to the other hemisphere.
Motion matching - we matched the horizontal motion of characters on each side. When our characters leaves the frame, she walks from the left to the right, creating a matching directional motion within the other sphere (Figure 1).
Light zones - What areas to light helped move characters forward in the narrative, and also added to the visual continuum. We wanted the user to be visually stimulated, so attempted to build a dynamic image through darkness.
Space within a space - The living room hemisphere had a strategically placed mirror that reflected the characters off-stage actions (in the kitchen). When the character leaves the key linear narrative for a moment (“I’ll call you back”), their presence is a minor detail that can be spotted or ignored in the overall composition (Figure 2).
Lighting is a critical feature of our storytelling toolkit in VR. Consider the sequence of light below from beginning to end of the film:
The character's motion is being told through the lights in the staging. We move between spaces - both existing and hidden (i.e. the mirror and window reflection) - by following the light itself.
Conclusion
For editing, we do believe that the segmentation of the screen is a useful way of blending at least two shots. However, overlapping action should be choreographed so that the viewer can continuously explore content, even though it may not be the intended focus. We can’t assume that people will follow our intended motion - since their head motion moves the field of view, its their choice what to look at. We need to make sure that there is enough content to fill the entire sphere. Maybe in the form of hidden content that can be discovered as details, or an exciting visual environment to discover. The list will grow over the next half year.
Upcoming Blog Posts
Basics of operating a multi-GoPro 360 rig
Visual Composition and Staging in 360 degrees