It’s fair to say that Naughty Dog haven’t made an unsuccessful game since 1996. Starting with the “Crash Bandicoot” franchise, which sold over 40 million copies, and continuing in the 2000s with “Jak & Daxter“, which sold over 12 million, and now with “The Last Of Us“, which topped many “Best Of 2013″ lists and sold over 6 million to date. I reached out to their team in Santa Monica to talk about their handling of audio duties for the that title. This led to a round-table chat with the whole team, consisting of Jonathan Lanier, Phillip Kovats, James Barker, Erick OCampo , Derrick Espino and Scott Hanau.
– Hi guys. Thanks a lot for taking the time out of your schedules to speak with me. Could each of you tell me about your positions at Naughty Dog?
Derrick : I’m a Senior Sound Designer who works for Sony Computer Entertainment America, though I’m currently located at Naughty Dog.
James Barker: I’m the Dialogue Supervisor at Naughty Dog.
Phillip Kovats: I’m the Audio Lead for “The Last Of Us“.
Erick Ocampo: I’m also with Sony, and am located at Naughty Dog as a Sound Designer.
Jonathan Lanier: I’m the Audio Programmer at Naughty Dog.
Scott Hanau: I work for the Music Department of Sony’s Service Groups, which services most of Sony’s first-party games. For “The Last Of Us“, I was the associate music supervisor who scripted the music into the game, and worked with Neil Druckmann to decide where the music plays. We also produced 90 minutes worth of in-house music for the game’s soundtrack CD.
– What kind of interplay exists between the different people on Naughty Dog’s audio team?
Jonathan: Phil, James and myself are employed 100% by Naughty Dog, whilst Erick and Derrick are employees of Sony. Naughty Dog is a wholly owned subsidiary of Sony, so we have a direct relationship with them in a corporate sense. Sony has an internal Product Development (PD) staff who are loaned out to different projects as the need arises, so when Naughty Dog is in-between projects, people like Derrick and Erick move on to work on other Sony projects. When we need them, they come back here.
Phillip: We’re very much a team here at Naughty Dog and Sony. For example, at the end of the project, Bruce Swanson, our Audio Director, got involved to help Derrick and Erick with the opening level of the game. We also had another ten people from Sony’s PD sound teams in Santa Monica, San Diego and Foster City help out. So we lean heavily on them when we need help.
Jonathan: But guys like Derrick and Erick aren’t just contractors that we use to fill space. Even though they’re not Naughty Dog employees, we kind of consider them honorary Naughty Dog staff. For example, Derrick has been here since 2009, which is a while.
Scott: I’ve known Phil a long time before “The Last Of Us“. We worked on “God of War 2“, which was before either of us worked with Naughty Dog. So thankfully, there’s no ego that gets in our way, and it’s the same with other team members. We all know how to recognize good ideas and know what the deadlines are for our projects, so we don’t have to beat each other up over things like that.
– Is a there hierarchy at Naughty Dog, where staff members play specific roles and answer to their supervisors? Or are you guys allowed free reign to be as creative as you want?
Phillip: There is a loose hierarchy. Naughty Dog is a pretty flat company, and we don’t have game producers per se. We have Evan Wells and Christophe Balestra, who are the co-Presidents. Underneath them are the creative, art and game directors. Following that, the studio has Leads for every discipline, such as animation, cinematics, audio and environment art. Under the Leads are teams of people that we trust and work with to get things done.
As a Lead, I don’t really sit around and say, “You do this and he does that “. For “The Last Of Us“, I needed to be able to delegate duties to people I could trust, so Derrick was my Senior Sound Designer, and we would work on the sound effects in the game together. James worked with me on the dialogue, Scott worked on the music part, and Jonathan handled the technology aspect. We naturally let things flow based on what the project was calling for at the time. Derrick and I worked on The Infected sounds early on, and then when Erick was brought on later, he took over that and helped create the sounds for the Stalkers and Bloaters.
– What was the highlight of making “The Last Of Us“ for each of you?
Derrick: We knew that this would be a rare opportunity for the audio team because “The Last Of Us” was designed to need a lot of attention to sound. Given the use of not only sound, but also silence, the game had to have a pretty strong sonic signature. That was a bit scary because it’s common for sound designers to struggle to get their sounds heard in the final mix. But thankfully, that wasn’t the case for us. Consequently, the sound quality had to be very good, so we focused on everything, from characters fighting to dust falling off a door.
The ultimate highlight for me was inventing the Clicker sounds with Phillip and Neil Druckmann. Once it finally came together, we knew it would be powerful.
James: Since my job was to oversee all the dialogue, I got to be a part of the game from the very beginning, during all of the Motion Capture and Automated Dialogue Replacement (ADR) sessions, when all of the story content was being created by our actors. I also worked with a lot of programmers, who wanted very intensive dialogue systems for the Artificial Intelligence (AI), and we fleshed that out together. We also built breathing systems for all the characters and tried to make it as realistic as possible. It was nice to see it all come together.
Phillip: I was here almost from the beginning, when the initial pitch was given by Bruce and Neil. My favorite part to work on was the audio mix of the game. One of the things we were mindful of was the inspiration that Bruce and Neil took from the music of movies like “No Country For Old Men” and “The Road“, and other simplistic scores that use things like negative space as part of the audio experience. Usually, games have a lot of sonic information, like explosions, gun-fire, magic, etc. So to create something which was unexpected from that point of view was different.
Erick: I was the last one to join the project, and the team had already spoken with me about working on The Infected before I came on. The biggest highlight for me was getting the rest of The Infected to sound cohesive as a family of creatures. I also had to create convincing sounds of evolution for these creatures, as they morphed from Runners to Stalkers and Bloaters. They had to play back intelligently, rather than just fire off sound effects left and right.
Jonathan: I’m sort of the odd one out here, because I don’t do creative sound design like these guys. I’m a programmer, but Naughty Dog has allowed me to pursue my passion in both programming and audio. Many companies don’t even have the role of an Audio Programmer, but we have that position so I can provide my team with the technological assistance they need. With the exception of the dialogue manager, which was written by Jason Gregory, most of the other audio-related stuff was done by me. The biggest thing I did was the Environmental Audio System; I’m most proud of that. We knew that this game would need a wide dynamic range in order to convey a sense of fear and dread. Luckily, Bruce and Neil knew that, and were involved in pushing the game’s technology from an early stage.
We did a lot of things in “The Last Of Us“ that we’d never done before. For example, we updated the Environmental Audio to support Reverb Portals, so now you can hear sounds that are being reflected down the hallway from an adjoining room. So if you’re playing the game with a Surround Sound setup, and a Clicker is in an adjacent room, you hear it before you get there. We also spent a lot of time working to dynamically alter the mix of the game, which meant that the overall mix was determined by the state of the game. So whether you’re in combat or exploratory mode, the mix changes, and we needed to create technology that would allow the game to drive itself in this way. So I worked a lot with Phillip on making that work, and I’m very proud of how it turned out.
Phillip: One of the things that made this project fun for us was that we were very involved in the emotional arc of the characters. One of things I focused on a lot while directing my team was that things were to be told from Ellie’s point of view, and not Joel’s. He’s not necessarily scared of what’s happening around him, but Ellie is. So the soundscape mirrors that.
– One of the things I’ve heard Naughty Dog say in the past was that you were striving to achieve a sense of minimalism with “The Last Of Us“ music. Can you explain how you achieved that?
Scott: We chose to not start off with the big, emotional moments in the story, and worked backwards from that by adding in ambience, textures and tension music. We made sure there was a lot of space between everything, and only had a few sounds playing at one time, and there are only two or three points in the game where we use an orchestra.
Another thing me and my manager did was play the game endlessly in the weeks before it shipped, and constantly took out some pieces of music. We were able to play the game from start to end, rather than in 20 minute increments, as we had to do whilst it was still in development. Having too much music in a game can make it feel like wallpaper, so we tried to get away with 20-30 minutes of having no music, which creates a more profound effect when it finally does come in.
– What were some of the problems you guys encountered in the making of this game, and how were you able to solve them?
Scott: My biggest challenge is always dealing with transitions between different music pieces. There are about 1200 segments of music in this game, and each of those require transitions, whether it’s overlapping music or varying levels of intensity during a fight scene. I’ve played a lot of games where the transitions are very jarring, so I wanted to avoid that in “The Last Of Us”.
Phillip: Bruce and Neil did a great job of directing this game, but at the same time, there was a lot of trial-and-error involved in making it. For example, we went through a lot of iterations with the Clickers. During the viral campaign for the game, we uploaded a video to Youtube which had Clicker sounds that me and Neil had made in my office, recorded on an iPhone. It was a silly clicking sound that I had made with my voice. But we evolved the sounds from that and it became much more. We also worked collaboratively with other departments. When one of the guys from the art department drew a Clicker for the first time and it had no eyes, so we figured out that they use echo-location. So everything we do informs itself down the line, and we thrive on that challenge.
Derrick: We tried a lot of different things for the Clickers. Sound Deluxe DMG sent us some samples of ideas they had, which inspired me to try different things. We took some of their ideas along with what we were already working on and played around with it. When we finally decided to bring in some voice-actors, I thought of Misty Lee, whom I had worked with on “Soul Sacrifice” for the Playstation Vita. She had a great scream and was very emotive. She should get a lot of credit for the female clicker sound. All the weird dolphin-like sounds are her, mixed with some sounds me and Phillip made. The real challenge came when we had created all of this, and then played it for Neil, and he said “No, it’s too much “. We were like, ”We think it sounds scary. Why don’t you like it? “. But if Neil doesn’t like it, it’s back to the drawing board, which is actually something I agree with. We end up doing better work this way because we constantly refined our process. We ended up sitting with Neil through-out one of the opening Clicker scenes, and he would give me pointers like “Mute this, solo that, give it more space here “, until finally he said “That’s it. Now it’s minimal”. The right amount of activity and negative space is what gave us the signature Clicker sound.
(Listen to Phillip talk about working on the Clicker sounds below.)
Erick: There had already been so much work done with the Clickers that the language for the Stalkers and the Bloaters was set by the time I joined. But for the Runners, we had to cast more actors because the sounds we had were a little campy. We needed more desperation in their voices, and not just a zombie-like rasp.
Phillip: One of the things that Bruce stated early on was that the people in this game weren’t evil. It’s the Cordyceps fungus that’s driving them nuts. So we were always trying to achieve this sonic dichotomy of “I don’t want to eat you, but I have to “.
Erick: That’s why the sound of the Runners is almost psychotic, like people losing their minds, and not just random noise.
Derrick: Yeah. During some of the voice-over sessions, we told the actors to imagine things like petting a dead cat and singing to it. It was the kind of visualization we thought would help for getting certain emotions out of them. I remember saying to one of the actors, Otep Shamaya, “Remember that scene in Texas Chainsaw Massacre where they find the dead grandparents in the attic? “. She says “No, but go ahead “, and I was like “Well, imagine you’re in the attic and you’re singing to them as you rearrange their clothes“. And she goes, “Ok, I got it “.
Jonathan: It would be great to say we have some secret recipe for how we make games, but that would be a lie. The reason our games are good is because we work really hard, and only hire like-minded people. No matter how harsh our critics are on the Internet, they’re never going to be more harsh than our co-workers. If you do something that isn’t five-star awesome, you don’t have to worry about what online critics will say; we’ll call you out on it first. But even that doesn’t guarantee that we won’t make mistakes, because we do. If you played “The Last Of Us“ a year before it launched, you wouldn’t recognize it, so it’s worth noting that we fail a lot at certain things, but our talented team makes us not afraid to break things, because we can fix them later. So it’s better to go ahead and try stuff and then keep iterating until it finally works.
I’ve been here for ten years, and I don’t think that we’ve ever missed a ship date for a game, but we actually slipped for the first time with “The Last Of Us“. From a corporate point of view, we don’t want to slip our dates, but we don’t want to ship a game unless it’s awesome. So when we realized that the game wouldn’t be polished in time, we had to push it.
One of the problems we had with not only “The Last Of Us“, but the “Uncharted” games also is that these games are heavily story-centric, and a lot of the exposition of the story happens in the dialogue. This creates nightmares for us because game designers don’t always give proper consideration to the dialogue, and yet dialogue is very tricky to create. If you and I are sitting across from each other and talking, the projection of my voice will be completely different than if I’m eight feet away. This kind of depth can’t be created in a game by just adjusting volume levels. But then there are additional problems created by the level layout and the Environmental Audio System, where characters have conversations whilst they’re moving around. If they move into certain areas and get obstructed, then you wouldn’t be able to hear them talking. A lot of games don’t bother with addressing things like that. They just play all the audio in the center channel and don’t pan it, which is sort of a cop-out. If you do things like that, you might get around the problem of obstructions of sound in the environment, but it pulls players out of the reality of the space, and we didn’t want to do that. To be honest though, we got to a point where we almost took the cop-out route of pulling the dialogue out of the game’s environment, but at the last minute I came up with a plan. We used a psycho-acoustic technique to trick the listener. We created a separate reverb unit that had a very short delay on it and dedicated it to dialogue in cases where characters were obstructed. We called it the “Dialogue Sweetener”. This allowed us to create a tiny bit of reflection that was enough to trick your brain into thinking that the sound was bouncing around in the room. So if Ellie walks around a corner, you can still hear her as if she’s close by.
A lot of games emulate how sound works in the real world but that’s actually easier than what we did for this game. Game audio consists of dialogue, music and effects, but the human brain can only latch on to one or two at a time, as they compete for the same space and frequency spectrum. In the film and music world, those elements are fixed, since they’re cut to a picture or a single audio recording. But in a game, you can change the camera angle or just move the character anywhere you want. That’s way harder to work with. So “The Last Of Us“ isn’t a perfect game. But thankfully, our bar is so high that most people don’t notice the flaws.
– What kind of inter-dependency exists between the audio team and other departments? Did your work ever get delayed up-stream because you had to wait for other departments to finish their work?
Phillip: Oh, yeah. Audio is inherently a post-production process. When games are in pre-production, there are a lot of moving parts. Any self-respecting audio professional would say, “I’d like to see the media first and then put my sound to it afterwards “. But in an iterative project like “The Last Of Us“ we didn’t have the luxury to wait on other departments, who needed our audio for their play tests so that they could see what was working and what wasn’t, so there were times when we’d give them an audio-sketch of sorts to work with.
James: For dialogue, we grabbed some random people from around the studio and used the first draft script to record everyone, just to get the dialogue in. We essentially recorded the entire game with dummy actors so that the designers had audio to work with.
Phillip: I’d like to point out something that not everyone might know about. In games, you have a lot of artists, animators and designers. A level designer might only have one or two levels in total to work on, and an animator might only make one character or part of the game. But the audio team touches every single aspect of the game, from pre-production to the final mix. That’s a level of work that might seem daunting to grasp unless you work in this field.
Derrick: Like Phillip said, we try to front-load the game with dummy audio, but that’s hard when we haven’t seen any of the actual game elements yet. But we do try to gather audio assets by doing sound and field recordings. If I had seen the initial storyboards for “The Last Of Us“ I would know that we’d need audio files for things like debris, blood, metal sounds, environmental sounds, insects, etc. We try to prepare such things beforehand, because at some point the other departments will come to us and say “We need audio to test what we’re doing “, and we might say “But we don’t have any right now “, and they’ll say “Just put anything in there that’s close enough “.
– How did you guys divide up the work that stays in-house and the work that you contract to freelancers?
Phillip: The internal Naughty Dog team for “The Last Of Us“ consisted of me, Bruce Swanson, Jonathan, James on dialogue and a scripter for programming; that’s it. For sound designers, we mainly turned to Sony’s PD team; Derrick and Erick come from there. I actually used to work there myself before coming here, at the Santa Monica studios. So we have a group of around 26 sound designers that we can call on to help out. But sometimes we will ask for outside help beyond that. For example, we brought in Neil Uchitel,who’s a commercial sound designer and foley artist with great ideas. He records a lot of physical sounds like doors slamming and rooms shaking. There’s a scene in the beginning of the game where an explosion goes of in Texas, and the sound of the bedroom shaking is actually Neil jumping up and down and recording the sound of windows rattling.
Jonathan: When you’re making a game that’s as dynamic as “The Last Of Us“, the technology has to inform the sound design. If you’re told to, “make a sound for something“, without knowing how it’s going to be implemented in the game, you can’t possibly do that. We like all of our sound designers to be on site and invested in the project.
Erick: If we do contract people, it’s usually for things like cinematic audio.
Phillip: Right. We hired Sound Deluxe DMG to do the cinematic work and certain foley. They also provided the facility that we used for recording dialogue.
– What made you decide to ask Gustavo Santaolalla to score the game?
Phillip: Early on in the project, Bruce Swanson was helping Neil with audio ideas, and music was a big part of it. They were looking at a lot of different composers, such as Carter Burwell who worked on “No Country For Old Men”. But Gustavo’s music was so inspirational. We initially thought he wouldn’t be interested in working with us because he hadn’t done games before, but one of our colleagues from the Sony PD Music Group said “Just try calling his agent “. So we did, and he was interested. He sat down with Neil, got the pitch and said “When do I start? “.
Gustavo never really liked to work to video for “The Last Of Us”. He created musical themes and atmospheres on his own, and then Scott would go in and edit the pieces with Neil and pick what they wanted. He started sculpting various musical themes based on the imagery and ideas he was given by Neil, and it was amazing to see how he solidified our partnership with the very first piece of music that he delivered, which became the main theme of the game. That’s the kind of artist he is.
Scott: Gustavo delivered us a CD of 70 minutes of music before there was hardly any gameplay. Then Neil would go through it afterwards and say “This piece of music would work in this and that scene “. We also collaborated a lot with Gustavo on the instrumental soundtrack music. We spent a few days with an ensemble in Nashville, and paired him with a symphonic sound designer called Tim Davies, so that they could orchestrate some music.
– How did the voice-over process work?
James: We did a lot of the principal character auditioning on the motion capture stage, where try-outs would be recorded and brought back the studio for review. Ashley Johnson, who plays Ellie, was picked first. Troy Baker, who plays Joel, came after that, after which the remaining cast was brought in to play off of them. Once the bulk of the principal dialogue between the main characters was done, Liam O’Brien helped to direct enemy interactions. At that point, we would have people suggest certain actors to us, and we’d bring them in for auditions. Liam also had his own network of actors that he knew, whom he’d pitch to us.
– Did Naughty Dog license any music for this game?
Scott: Yeah, we licensed two Hank William tracks, “Alone and Forsaken” and “I’ll Never Get Out Of This World Alive”. For the “Left Behind” extension, we also licensed Etta James’ cover of Sonny & Cher’s “I Got You Babe“.
– What kind of audio resources and facilities are available to Naughty Dog?
Phillip: We have six audio rooms at our Santa Monica offices which were built in collaboration with Chris Pelonis, as well as our theater. We mostly work in our own audio rooms, with a standard Pro Tools setup and plugins.
Scott: We all use Pro Tools and share the sessions amongst ourselves. We also have proprietary music scripting technology that Naughty Dog developed over the years.
Derrick: We used a lot of microphones and recording gear from Sony’s PD sound department too.
Jonathan: The main thing we need in our offices is a clean signal path, good speakers and a good room when recording. We use a really nice Dynaudio surround speakers setup, with a completely digital system that has no noise floor. We don’t need much more than a Pro Tools setup and our interfaces, since most of our audio recordings aren’t done at our offices. Our sound designers will do field recording on their own gear. Sony PD Music has an incredible recording facility up north in Foster City, and their gear is phenomenal. So they record the music there.
Phillip: When working on this game, one of the most important things was to have everything sound as natural as possible, which makes the game feel immersive. So we weren’t too experimental on the sound design. We did do some pitching and editing of certain Infected sounds, but that’s it.
Derrick: One of the most valuable pieces of gear I have is Soundminer, which is a unique search engine program that comes with pitch shift functionality that was very useful to me. It enabled me to manipulate sound very quickly via the built-in VST rack. Having my own equipment, I can travel around and record things whenever I want. So if I think, “We might need some rock sounds one day if we ever work on “Uncharted” again “, then I can record it.
Jonathan: We don’t generally use canned sounds and sample packs for our audio. It’s all designed specifically for the game, which is something I think our sound designers like Derrick can brag about.
Derrick: If you’re a sound designer, your sound library is a part of your identify. It’s what you bring to the table in addition to your talents.
Erick: The main sound of the Bloater actually came from my son. When he was an infant, I’d sit next to him and record him crying and making baby noises. So I slowed one of those sounds down, and that became a groan sound that we used. It’s interesting how you end up using sounds in unexpected ways. At best, I thought I would use that sound if I worked on a game with babies in it, but it ended up being used completely differently.
– Can you take me through the mix-down process for the audio on “The Last Of Us“?
Phillip: When Naughty Dog made the switch from the PS2 to the PS3, they made a very intelligent decision to pretty much start over on audio. Bruce and Jonathan started a new audio engine from scratch. We’re still using Sony’s proprietary SCREAM tool for sound, but our unique metadata system is responsible for how it gets played. With this, we can tell the sound how to behave when it gets triggered in a game. This includes manipulating things like fall-off, reverb amount, filter cutoff and atmospherics. So the mix actually starts from when we put the first sound into the game, and our sound designers are tasked with having to intelligently place sound in this game.
Jonathan: The mix of the game isn’t driven from the output of the audio, because we’re not trying to replicate the way the human ear hears things. Rather, we’re trying to do something stylized, like Hollywood does. Anyone who knows anything about Hollywood knows that there’s nothing remotely realistic about the way that audio is mixed for a movie. But we’re working with a game, not a movie. So we not only need to set the mix knobs for the stems, but we have to set other knobs that teach the game how to mix itself depending on in-game changes. You can’t mix game audio by just spreading everything out on a mixing board and setting individual levels. By the time you realize that different foley sounds don’t match in volume between different parts of the game, you’d be screwed.
– Was there anything you feel that you should have done better on this game?
Phillip: I think we’re our own harshest critics. We’re getting some new technology for the Playstation 4 which will allow us to do new things.
Jonathan: Things will get better though. We’re still in the early phases of exploring what can be done with the PS4. But you can rest assured that the bar has been set with this game, and we’re not shipping anything inferior to that.
Derrick: From a sound design point, every game represents where I’m at with a certain style, and then I build on that for the next game. Ben Burtt, who did the sound design for the Star Wars movies, always says that he’s looking for the perfect explosion. I’m looking for the perfect punch, and I think I got pretty close on this game. Not to say that the “perfect punch” is a sound that can be used for every punch, but I think I got pretty close using this particular punch sound to tell a story in this particular game. But I won’t be using “The Last Of Us“ punch sounds for the next game. That’ll be a whole new creation process.
James: You can break down dialogue into spoken words and effort sounds like climbing and punching. Within efforts, we have to build a different system to make them sound more natural. So if you’re pushing an object, you’ll hear the starting sound, followed by some loops like grunting and huffing, followed by a final sound when you stop pushing. As a part of fleshing out that system, we create probability systems that ensure you don’t grunt on every step. So we intend to keep working on systems like that to make it sound more natural. As we develop more games like this, we have to create complex dialogue systems to try and model realty. But each time we do that, it has inherent problems, and we have to work hard to make it sound more believable.
Jonathan: Touching on that, it’s worth mentioning that the conversation system in our dialogue manager can be a double-edged sword, because the more you allow characters to interact verbally, the more telling it is when they break form or do something unrealistic. We call it the “uncanny valley”, which means that we’ve made things look so realistic that anything which doesn’t look real will stand out. When your dialogue manager is so sophisticated that you can feel the character’s moods, the in-game AI can’t afford to make mistakes. If the AI makes a dumb move, it’ll be telegraphed through the dialogue. So one of our biggest problems with raising the bar for dialogue is to create a smarter AI, because it can easily do things that are wrong. Not that the AI is malfunctioning, but we depend on it to think like a real creature. So if the AI makes a bad decision, then sound is the first indication of that. For example, a character might fire a weapon randomly or someone might call out an alert to a character that he shouldn’t know the name of, plot-wise.
The dialogue system in “The Last If Us” is amazing, and James should get a lot of credit for working with Jason Gregory on it. We never had anything remotely as good for the “Uncharted” series. The next “Uncharted” will benefit heavily from this.
James: We shipped this game with close to 70,000 lines of dialogue, which is a huge amount for a non-RPG game of this size. We also built a context awareness system so that enemies can call out your location when you’re fighting. In order for that to work, you have to record every single object in the different environments. Someone might shout, “He’s by the van! ” or “She’s by the bar! ” or “She’s across the street!“. So once the levels were fleshed out, we’d have to go through each area and take note of the fact that we might have a store, an office, a street and an alleyway with vans and trucks parked outside. We’d also have to take into account one car parked over here and multiple cars over there. Also, when the enemy spotted you, they have to call out that you were standing next to a car, and if they can’t see you, they say you’re standing behind it. Not only did all of that dialogue need to be recorded, but it had to be recorded multiple times with variations, so that it didn’t become repetitive and unnatural. Our poor voice actors were screaming “He’s over there” and “She’s by the car“, for four hours over the course of five recording sessions. I’m sure they all hate me (laughs). But that’s what it takes to model this kind of thing, and as games continue to model reality even more, we’ll get into this kind of territory even more. Sorry voice-actors….
– What made “The Last Of Us“ different from past Naughty Dog games, like “Crash Bandicoot” and “Uncharted” , and how will its success change the future of the games you make?
Phillip: Instead of us being reactive, we participated more than you might have expected. Prior to this, I did cinematic work for “Jak & Daxter“, and then I worked on the first “Uncharted” and “Uncharted 2“. So I’ve seen how Naughty Dog has progressed. We started off with “Crash Bandicoot“, which was very cartoony and platformy. Then we took it to another level with “Uncharted”, which was aimed at being realistic, though still a bit pulp and platformy. But taking it to a serious level and making people cry is what we did with “The Last Of Us“. The result of that has changed how we approach making games.
Derrick: I think this game sets a precedent for improving how we work together at Naughty Dog. We don’t have to deal with other team members saying, “We’re done with our work. Now you guys go make some sounds for this “. We kind of get to design things alongside other departments, which we appreciate.
Phillip: When the director comes up to us and says, “We’re depending on you guys to make this work “, and he isn’t just talking about fixing a random problem, but actually creating some emotional context, like with the Clickers, it’s special for us.
Erick: After having worked at a bunch of different studios, I can say that Naughty Dog is the most collaborative one that I’ve worked at.
Phillip: There’s a part in “The Last Of Us: Left Behind” where Ellie and Riley come across a broken arcade game, and Ellie plays the fight sequence in her mind. So the health bars and character names appear on screen, but when you think about it, that entire scene is based on sound. Erick worked very closely with one of our designers to make it work.
Erick: Yeah, that was an iterative process that was very audio-driven. The whole thing is her imagining how the game happens, so we had to use all kinds of Mortal Combat-like sounds to sell the moment. It worked out well.
Derrick: There were moments like that on “Uncharted 2” as well. In fact, one of those lines was initially cut out of the game. During the museum heist, when Drake’s climbing up the wall, Harry Flynn talks into his receiver and says “There’s a man above you! There’s a man above you!“. So Drake grabs the guy and throws him off the building, and Flynn says, “There’s a man below you! There’s a man below you! “. I love that line (laughs). But then one day it was taken out of the game by someone. So when one of the animators was walking by, I said “I can’t believe they took out Flynn’s line!“, and he replied, “You want that line to be there? “, and I was like, “Yeah, it was a funny line! “. So he goes, “I’ll be right back “. He leaves and comes back a minute later and says with a grin, “It’s back in “. So everybody values each other’s opinion here, and I think that sort of thing will continue to be a part of how we work.
Jonathan: I’ve been here since “Jak 3“, and it’s been really interesting to watch the evolution of audio that’s taken place at Naughty Dog. I might have a job title today that’s called “audio programmer”, but I was never hired as that. My job interview was grueling, since they’re very picky about who they hire here. But we got around to talking about audio somehow, and I remember mentioning that I was very passionate about it. At the time, no-one around here wanted to work with audio and I was like, “I love doing that! “. even though there wasn’t a lot of audio technology at our disposal and audio memory was a precious resource. Also, the games that we made back then were more cartoonish and fantasy-based, with beep-boop-boink sounds. Things took a turn with the “Jak & Daxter” games, but Jak didn’t even talk in the first installment because we didn’t know how to stream audio at that time. So one of the first things I did when I started working here was to re-write the streaming system. I knew we weren’t streaming anywhere near what we could, and I sped up the streaming format so that we could move from using MIDI music banks to actually streaming audio in the games. In fact, one of my first meetings was a talk about how we were running out of memory, and Bruce Swanson was like “We need to save memory for the MIDI banks “, and I said, “But we can free up all the memory banks and just stream the music “. He replied, “No we can’t do that. The streaming system isn’t fast enough “, and I’m like, “But I can make it fast enough “. Everyone looked at me and said, “Are you sure? The guy who worked on it previously said it was impossible “. “I can do it“, was my answer. So one of my first tests was to re-write the system by a certain deadline, and when we shipped “Jak 3”, we were able to do so with real streaming music.
It’s been a slow progression, but I think Naughty Dog realized that the game’s are cooler when the audio sounds better. So when they reset the company, discarded our old code and switched to C++, the company came to me and said “You’re our only audio guy and Bruce really needs you right now, so we want you to work with him as long as he needs, and then we’ll pull you back onto the game team when he’s done “. Seven years later, I’m still on the audio team (laughs).
Like we said before, sound has a history of being treated as something that comes in at the end of a game, when the programmers hand over their work to the audio team and say, “Here, add your sounds to it “. Bruce doesn’t work that way. He said, “If you want a sound, you come and talk to me and I’ll give you a sound that’s designed for your particular thing “. So as audio became more important, we grew more collaborative with the other disciplines, and now they don’t think of us as guys at the end of the pipeline. Sadly, there are still companies out there who don’t work this way. Erick has probably dealt a lot with that (laughs).
Erick: Yeah, it’s not fun….
– I hear you. Hey guys, it’s been quite an extensive talk about your work at Naughty Dog and I’m happy you took the time to speak with me. I’ve learned tons about how your audio team works and I’m sure many others will too.
