Visual Concept

For the visual concept we were inspired by biological and organic images of membrane, alien tissues and embryos. The concept of birth is also very inspiring and prominent in our previous work that we want to iterate on. We were also inspired by our friend malgavochka and her nails :)

After conducting several discussions, we made the decision to create a sack made of fabric with an actress inside it. We were fortunate to discover a high-quality, transparent and stretchable fabric at a local store. Our team member, malgavochka, made the sack using this fabric. To test its look, we did a test shooting by placing Marina inside the cloth and spraying water over the fabric. As we lacked appropriate lighting options, we incorporated bike lights into our setup. The combination of the wet fabric surface and the light emanating from within the sack showed promising results.

Approach and tools

Making visual effects on live action footage is a hard process that requires several steps such as camera tracking and compositing.

Camera tracking is used to estimate camera motion in a video using software like PFTrack, SynthEye, or 3DEqualizer. These programs track high contrast points on footage to estimate camera movements, producing a virtual 3D camera motion. This process is essential for adding computer generated elements to videos. The 3D camera's motion is then applied to synthetic elements making them appear to be part of the scene.

Compositing is another important step when making visual effects with live action footage. It is a process of overlaying synthetic elements on live action video in a coherent fashion. Compositing tasks might include green screen removal, camera tracking, rotoscoping, color correction, object removal and more. The full scope of work depends on the particular task.

We have done several experiments in the past where we incorporated some synthetic imagery over live action footage. It is a fun and interesting process but it takes a lot of time to make something that looks professional and real.

For the Cocoon project we wanted to take a different approach where we separate synthetic and live action imagery. This approach simplifies the work by eliminating the camera tracking and compositing steps.

By keeping things separate we were able to make full 3D environment in Unreal Engine without worrying about live action. In fact, we finished the first version of Cocoon fully in 3D before we filmed the live action.

We choose Unreal Engine for this task because we knew we wanted to render a couple minutes of full 3D environment. A couple minutes in VFX terms is a lot! 2 min of video will result in rendering 3000 frames! Unreal's ability to render beautiful looking light and shadows real time is game changer. Let's say we choose Blender Cycles for the rendering. I've done test with similar scene that took one minute to render a single frame which equates to 50 hours of rendering time in Blender! This exact fact hinders ones ability to iterate on the work, which has a big impact on creativity. On the contrary we can render the entire 3000 frames of animation in a matter of minutes.

For all of the geometries in the scene we use HoudiniFX. At that time Snay was experimenting with FEM (Final Element Analysis) and Vellum simulations which fit perfectly with the organic look we were going for. For transferring simulated meshes to UE, at first we tried to use VAT (Vertex Animation Textures) from SideFX Lab tool set, but it was difficult to work with. At the end we settled on Alembic caches since performance was not a big concern for us.

We put a lot of work into texturing and shader look development for all the assets because it was challenging to achieve that soft tissue look in real time rendering engine. We were excited to try generating textures in Houdini but that was harder then we first thought. After several failed attempts we ended up using Substance Painter and making PBR textures by hand drawing over a few procedural mask such as thickness and curvature.

Filming

For shooting we needed a place where actress can lie on the ground, while positioning the camera above her. Behind our house, we discovered a small creek with a tree that the camera person can climb and shoot a subject on the ground. However, this location had a few disadvantages. The ground was somewhat muddy, and the presence of numerous mosquitoes posed additional challenges. Prior to the scout, we collected various dead branches, which we believed could contribute to the scene. We had the idea of constructing a nest of sorts for the actress to lie in within the sack.

To address the issue of the actress coming into contact with the muddy ground, we used a transparent cover from our kitchen table, which proved to be an ideal fit.

On the day of the shoot I equipped with a saw and an axe went and built a nest out of the prepared branches. I also climbed the tree and did a few test shots. Shortly, Marina came prepared with tools and supplies for face make up. She even brought her own mirror. After her make up was applied, Snay and I did a few test shots of her face to test the camera setting and bike lights. Snay experimented placing different color plastic bags over the light, some of them looking very interesting and similar to moon light. However, the power of the bike light was not enough to light our large scene.

Around 6:30pm (about 40 min before sunset), Snay and I descended to the creek, equipped with Lumix S5, bottle of water, fabric for cocoon, couple of bike lights and glass of beer Marina.

It was already dark in the creek even though the sun wasn't fully down. Since everything was ready we started shooting immediately. I climbed the tree with the camera. Marina wrapped herself in the cocoon on the ground directly below me, and Snay was on the side wrangling the light and directing. We ended up placing one cold blue light on a side tree branch, pointing approximately 45 degrees down on the cocoon. We set another light to red mode and gave it to Marina inside the cocoon.

The scene looked gorgeous on camera. We experimented filming various poses for about 40 minutes until it came time for Marina to rip the cocoon and escape into the darkness. We knew we will only have 2 opportunities at most to shoot the escape, since we only had one piece of fabric. After completing a few short rehearsals we decided to go for the final fabric rip and escape sequence. I stood over Marina with the camera and shot the whole sequence in one take. Marina did a great job performing and we were all happy with this single take result. We called it a wrap and went back to the house for hot tea and review.

While the girls were drinking tea, I loaded all the footage to DaVinci resolve for us to review on the TV panel we had previously set up in a dark room. I also played some ambient music to the the mood right while we were watching raw footage. Everybody was excited about the result. We called it a day and Marina went home.

Putting it all together

The next day, we embarked on a two-week trip to Turkey from Tbilisi. I only brought my MacBook Air, while Snay brought her Windows machine. During these two weeks, we not only engaged in "parents" and tourist activities but also managed to incorporate our previously filmed materials with the 3D clip we made earlier.

One of the challenges we faced was ensuring a consistent look between the real footage and the 3D elements. While exploring Istanbul, we took a local ferry where I captured footage of the sea water swirling behind the ship's engine turbine. We then overlaid this footage onto the cocoon video using a "subtract" blending mode. The result was quite astonishing, prompting us to stick with this look regardless of any obstacles we might encounter. However, we soon discovered that this overlay approach presented some challenges. It did not produce favorable results for clips with significant motion, and the "subtract" blend mode made the areas outside the cocoon appear pitch black. After some trial and error, we were able to achieve the desired color balance and create a mask specifically for the cocoon portion of the video. This cocoon mask was created using the luma keyer tool in DaVinci Fusion, and remarkably, it worked well across all the shots with only minor adjustments needed.

For the sound track we sampled the Half-Life Alyx official sound track, from a 4 hour compilation on Youtube made by @SalimMuller. It worked very well creating proper suspense and emphasizing the fabric tear.

Vlog

Credits

Marina - Actress, Makeup.

Kif - Camera, Layout, Light

Snay - Director, HoudiniFX Simulation, Edit, Color