Milica Zec Milica Zec is a NYC-based virtual reality and film director. Raised in war-torn Serbia, Zec roots her work deeply in issues of conflict, identity and the human struggle. In addition to her own creative and client work, Zec has parlayed her artistic experience into a 9-year collaboration with performance artist Marina Abramovic. Highlights of her directorial work include the short film, “Christina” (Cannes Film Festival 2012) and “Marina Abramovic Presents MACCOC” (Venice Biennale 2011). Zec's work has appeared in The New York Times, MSNBC, Art Forum, The Huffington Post and many other media outlets.
Winslow Turner Porter III Winslow Turner Porter III is a Brooklyn based director, creative technologist and producer who has always been fascinated with the possibilities of how the intersection of art and technology can elevate storytelling. With over five years of experiential and digital agency experience, he has created experiences for TED, Google, Delta, Diesel and Merrell as well as worked with the who’s who of creative coders, having produced Tribeca Film Festival Transmedia award-winning virtual reality documentary CLOUDS among other internationally acclaimed projects.
Author's Note: This article uses Google VR Views to present immersive 360 images. Explore the images by placing your phone in a Google Cardboard viewer or by simply tilting your phone to change angles. On a computer, just click the full screen button and use your mouse to drag around. It's an experiment—give us your feedback in the comments below or @lisajamhoury.
A 'Giant' Year for VR
It’s July Fourth weekend. Like many New Yorkers, I’m at an upstate barbecue enjoying poolside corn on the cob. I’ve made a new friend, and we’re discussing her dissertation on the future of technology when a siren goes off.
Loud. Almost piercing. I think to cover my ears, then get a feeling—not quite panic or fear, maybe worry. It’s a memory. I’m uncomfortable. Something’s familiar, but I can’t quite place it.
Then it hits me, and I immediately think, “I need to call Milica.”
Earlier that week I had been at NEW INC, the New Museum’s art, technology and design incubator, to experience “Giant,” a six-minute virtual reality experience created by director Milica Zec and producer Winslow Turner Porter III.
“Giant,” born of Zec’s experiences in 1990s war-torn Serbia, places the viewer in a basement with a young, Western family during a bombing campaign.
Playing at 90 frames per second in the first Oculus consumer version (CV1), the visuals are crystal clear and the head tracking is perfect. Spatial sound makes the sirens all too real. Bass transducers in the seats allow the bombings to shake even the limbs.
Indeed, Zec and Porter have gone to great lengths to make the experience much more than visual. My visceral reaction to a siren days later proves to me that what they are doing works, and I’m not the only one who’s noticed.
Zec and Porter have had a whirlwind year. Started in April 2015, “Giant” premiered at Sundance nine months later. The partners have since toured the world’s most prestigious film festivals, including Cannes. They’ve made headlines across tech and film trades and spoken on numerous VR expert panels.
Following their accelerated path from incubator to red carpet and back to incubator, it’s easy to see the duo’s experience as a microcosm of the VR industry. They got a great idea, worked countless hours to take big steps forward in technology and storytelling, and received great buzz as a result.
Still, their real challenge—like the industry’s—is what lies ahead. Three years after the Oculus Rift DK1 release, and with billions of dollars in investment from nearly every major player in tech and entertainment, VR continues to be the realm of technologists, gamers and festivalgoers. Revenue streams and distribution models for content are embryonic. Consumer-version headsets are only beginning to make their way to market. Although 2015 headlines promised 2016 as the “year for VR,” as Q4 approaches, it’s clear that the marathon is just beginning.
So, what’s it like on the starting line? Zec and Porter shared their stories and forecasts in an interview with Ochre. What follows has been edited for length and clarity.
Ochre: Did you have any idea how successful “Giant” would be?
Zec: I don't think we even thought about success while we were making it. We just were preoccupied with how to make it: "Is this going to move the viewer? Is this going to reach its emotional goal?" The success part. ... It was a total surprise for us, completely.
Ochre: Winslow, you were quoted in The New Yorker saying, “If you’ve been a sculptor for three months, people are not inviting you to speak on panels. … In this art form, you’re an expert.” This VR industry phenomenon has contributed to your success. Tell us what it’s like from the inside.
Porter: The technology is changing faster than anything I've ever seen before. We could compare it to the app boom when smartphones came out, or before that when the internet first went public. But those things took a much longer time to gain buzz and to have compelling experiences.
There's so little [content] out there that all the rules are being made and then broken the next day. We're still in such an early phase with this nascent technology that anybody could be an expert and anybody could also make a compelling experience. You could have a major studio putting in $4 million for something, or you could have somebody creating something on their laptop.
So, the bar right now, because it's all so new, is relatively low, and the barrier for entry is relatively low if you have the technology. The problem is that computers are expensive, headsets are expensive and they're not that easy to get. But every year, graphics cards are getting faster, computers are getting cheaper and software is getting easier to use.
Augmented reality (AR) overlays virtual content over the user's real environment. Think Google Glass or Microsoft HoloLens.
Mixed reality (MR) combines virtual objects with objects in the user's real environment. The secretive startup Magic Leap uses MR.
Virtual reality (VR) places users in an entirely different environment than their current surroundings. Oculus Rift, HTC Vive, and Samsung Gear all are VR headsets.
Ochre: So, VR has come very far, but still not far enough. Are you afraid that VR won’t catch on?
Porter: It's a question of getting into people's houses and making it more affordable, but that's all going to happen. It's a question of, I think, not “if” but “when.”
Ochre: So, when?
Porter: I think when the Sony PlayStation VR comes out. The headset is a lot cheaper than the other ones, and also the VR community is already so rich with collaboration. They already have, I think, 50-plus titles that they're going to be releasing when it comes out. They're going to be reaching, I think, 20-plus million households. … If that works, then I think it's a done deal, VR is definitely not going away.
Ochre: Will the revenue follow?
Porter: There already are marketplaces out there ... that are viable places for commerce. We've seen, through platforms like Steam VR, there are titles selling at $10, $30, $50.
We're talking about two different things, too. There's interactive, more like video game stuff [SteamVR], and then there's 360 video experiences, and people like Netflix, Hulu, HBO and Amazon are all investing in that, too. They are investing enough that I think it will make a mark. Whether all of the content will stick, we're going to find out.
Ochre: “Giant” falls somewhere in between the two. Tell us about it.
Zec: “Giant” is a short virtual reality experience. It's about a family hiding in a basement from bombs. They have a 6-year-old daughter, and they can't tell her it's a war going on outside so they start inventing a story of a giant that is approaching their building and wants to play with the little girl. So, as the bomb gets closer and closer, the play intensifies.
“Giant” is actually an installation. We built a 15-by-15 foot dark room and positioned three chairs [with] bass transducers attached to them, so whenever there is a bomb blast inside the experience, your chair shakes and then your body physically shakes as well. [The room] resembles the darkness and claustrophobic feeling of the basement inside the experience.
Ochre: How did you come to make the experience together?
Zec: First I met the script writer, Lizzie Donohue, and then she wrote a first draft. She came up with the whole fictional story. It's loosely inspired by what I went through [in Serbia], but it's actually an entirely new story happening to an American family, happening in the future.
It was supposed to be a short film, but then as soon as I read it, I loved it, and I thought this should be made into virtual reality, because I wanted people to actually feel what it means to be in a conflict zone as opposed to watching it on the screen across from them. That's when I called Winslow, my long-time collaborator and friend.
Ochre: Milica, you’ve said that you never wanted to make a film about war, and now your career is defined by one. How has that experience been for you?
Zec: As a filmmaker, I didn't want to speak about war. But then I really loved Lizzie's script and it definitely changed my mind.
Now that's all I'm talking about for the last six months. Before … not even my friends knew that I lived during a bombing. You really try to lead a normal life and pretend it didn't happen. But it actually did happen, so I'm glad that “Giant” brought this out of me, to accept that it was part of my life. It feels normal for me to talk about it, because it did happen.
Porter: It was also interesting, not just for me, but for other people on the team, to learn more about the conflict. We went to Serbia to see different sites to contextualize it for us. That meant a lot because it showed that these places are real. … Seeing the building where you would go into the basement, and meeting [Milica’s] father, it helps bring the piece alive.
Zec: My father kind of doesn’t want to see “Giant.” He’s afraid to watch “Giant.” I don’t know if I’ll ever convince him to see it. He’s like, “I don’t want it, I lived through it, I don’t need to see it.”
Ochre: You are both personally invested in Serbia. Why use fiction?
Live Action is videography that typically uses live actors, rather than computer graphics—often referred to as CG or CGI.
Game engines, like Unreal and Unity, are used to render CG environments and characters.
Depth information brings the third dimension to 3D imagery. The DepthKit uses the depth sensor in XBox Kinect to add depth data to a video camera's 2D image.
A frame rate of 60 frames per second (fps) or higher is considered a necessity by many in the VR community. Lower frame rates are thought to contribute to motion sickness in the headset.
Porter: One of the purposes of having an American family, or Western family, we should say, is that they don't understand that conflict is more than a headline. The word empathy gets used a lot in VR, but for us, that was our main objective—to be able to place [audiences] in this so they're not over intellectualizing it, but they're feeling what it's like. If it was just a 16-by-9 frame they would be seeing and hearing what it's like, but we want them to go past the understanding and have it as an experience.
Ochre: You used a combination of 360 live action video and a game engine environment to create the experience. Why?
Porter: There was a big crossroads between 360 video or the game engine.
We didn't want to do everything purely CG. We wanted to bring the strengths of having actors, live-action video, but also the immersive qualities of a video game engine. So, with the help of our technical producer Juan Salvo we created our pipeline, which was shooting actors on a green screen using a Red 5k sensor, the Red Dragon, and also the Kinect 2 using DepthKit.
Shooting in 5K allowed us to future-proof it, so if new headsets come out we can use higher quality video.
Ochre: Other major hurdles?
Porter: When we started it, the [Oculus Rift] DK2 was the only headset available, so we couldn't get a full sense of what our project looked like until we were given the Vive and the Oculus CV1. … It was like putting on glasses, it really brought the performance to life.
[The game engine] Unreal, at the time, only supported a 720p video player. We wanted to get up to 1080p to show more detail to bring more emotion, so we worked with Framestore and a developer named Omer Shapira to develop our custom video player, which would allow us to have depth information, an alpha channel, and the original video all together in one.
Also, because of the depth kit, we were able to create something called the disparity shader, which Omer Shapira helped work on as well. You see something slightly different with each eye, and [the disparity shader] really helps [the video] fit better inside the game engine.
Zec: There was nothing easy in creating “Giant.” For example, the difference with doing a regular film, where you can cut to a different shot and close-ups, we couldn't do it. This is all one take and it's a wide shot. What we could do, we could manipulate the environment, the basement, move the objects around, adjust the lighting. That's really all we could do to enhance the performance of the actors.
We were lucky to work with Jack Caron, lead technical artist, and Todd Bryant, director of creative technology, who helped create the world of "Giant" inside Unreal Engine.
Ochre: There’s an ongoing debate about the ethical implications of placing audience members in VR environments that could be traumatic, as you’ve done with “Giant.” How have your viewers reacted?
Porter: We do give a trigger warning at the beginning so that people know. At Cannes, somebody who was from Israel … opted out.
[Most] people who have experienced similar scenarios in their own life actually thanked us, because they said, “As much as I've tried to explain this to other people, this was the closest we could get to having somebody else know what it was like for me growing up in Israel, in Lebanon in Syria or in Sri Lanka.” What was interesting was, all different conflicts, but everyone had this same story: They hid in their basement and their parents were there with them, or they were there with their families.
Zec: Out of probably 4,000 people that saw it so far, two people opted out.
Ochre: Compared to numbers for a successful film, 4,000 viewers is not much. How do you feel about the relative inaccessibility of VR?
Porter: There’s a lot of talk about how it's the 1 percent [who experience VR], and that is very true. But if we look back at … making traditional movies, that bar was much higher. If you wanted to have a Steenbeck or an Avid, we're talking hundreds and thousands of dollars. That's compared to being able to have a computer that has a graphics card, we're talking $1,200 dollars. So it is more accessible than when film was first coming out.
But I think, pretty soon, we're going to be able to create content on the phone to be seen inside the phone.
I think that as [VR] becomes more part of culture, or when the units are no longer just at someone's neighbor's place, when they're actually in their own home and everyone has it, and smartphones can do really impressive stuff, and when there's somebody at the other end of the experience too, when it's not just in a headset, when we're able to have a communal experience, a collaborative experience, I think that's when things are going to get really interesting.
Ochre: Do you think VR will replace film?
Zec: I think that virtual reality is just a new medium, that it's not going to replace film. There is theater, film, and there is VR now, and there is AR emerging. So, I don't see it as replacing any medium, just coexisting.
Porter: At Cannes, that was probably the question we got the most. I think some people feel threatened by it, because a lot of people have never tried it. We've even had volunteers at different festivals where [we've said] "if you're going to be showing the experience to somebody you should try it," and they're like, "No, I'm good. I don't like 3-D movies, I think that's a gimmick and this is just the next gimmick." ... It's funny that people sort of have these preconceived notions.
Ochre: What’s next for you two?
Zec: We want to create a trilogy. “Giant” is Part One, atrocities that humans do to each other. Part Two, atrocities we do to Mother Nature, is going to be about climate change. Part Three, there is still hope and something bigger than us. Now we are in the very early stages of creating Part Two, the climate change piece.
Ochre: What are the current must-see VR experiences (aside from “Giant”)?