How the guitar arc was developed in the game The Last Of Us-Part 2

“Swiping on the Six Strings”: Crafting an Interactive Guitar in ‘The Last of Us: Part II’

— — Naughty Dog Game Designers: Mark Burroughs & Grant Hoechst

From the very beginning of the game development, we knew that the guitar was going to be a key part of the game’s narrative.

Joel’s love for the guitar was introduced in the first Last Of Us including a promise to teach Ellie how to play.

Even in the first story pitch for Part Two, this game was booked with the instrument, beginning with Joel playing Guitar for Ellie, and gifting her with her guitar, and ending with Ellie struggling to play on her own journey, her missing fingers embodying the cost.

The guitar symbolizes the bond between Joel and Ellie. It’s a symbol of life tranquility and hope. In that same vein, it was important to our creative director Neil from the very beginning to use future days by pearl jam as our musical through-line. Alongside that, the first cinematic we shot for the game was one of Ashley Johnson who played Ellie, singing a contemplative cover of A-ha’s take on me. And beyond that, the acoustic guitar is important tonally to the world of the Last of Us, from Gustavo’s evocative guitar-infused score to Joel’s Texan country singer roots. The guitar then was a narrative and tonal linchpin from the beginning.

So the question was, can we put this on the stick? It’s sort of a mantra that we used at naughty dog especially by one of our game directors Kurt Marginal, we could have simply left every instance of the guitar in the non-interactable cinematic, this would let us control the emotional impact pacing and focus of these moments since the player cannot interrupt them with incongruous behavior, after all , if the player can’t smash random chords at a forensic pace, they can’t ruin a beautiful moment of Joel playing a deeply personal rendition of future days to Ellie.

Grant — — But again, the guitar is fun. We knew from video games like guitar hero that playing the guitar in a video game can be a really engaging experience. We could make a mini-game that was at least enjoyable to play, so the question for us is: could we make an interactive guitar in our game that fit our needs for this story. Well, we do have an intriguing narrative arc that we could play along with. Joel plays a song at the start of the game to introduce Ellie to the guitar, then Ellie slowly masters the guitar over the course of the game. But her missing finger at the end of the game undercut the ability, and sever this connection with Joel. This teaches master subvert structure reminded us of the parallel design formula in games like Super Mario 3D world where it’s used for level-specific mechanics. If we set up our system to align with this progression, there’d be a direct link between the concept and the narrative journey of Ellie. Furthermore, putting the player in control of the music just as Ellie is in control in game would immerse the players more deeply than just watching and listening. And when Ellie is playing, she is recalling one of her fondest memories of Joel. If done right, an interactive guitar could be used to bring the emotional experience of Elliie and players in line, by having the player to remember the song that they originally played as Joel.

Mark — — So how does the tone of Last of Us affect this minigame? Our world is realistic and grounded, so our guitar game probably can’t have fireworks and smoke machines. These are also going to be the most important and character driven moments in the game. We need the players to feel nostalgic and wistful, so this can’t be sex drugs and rock and roll. We also need players to feel they are practicing, we are not providing the fantasy of having the mastery of rock god, so we can’t just plop in guitar hero as is.

Constraints & Vision

Grant — — At this point, we think it’s worth trying but before we started prototyping, we need to define some of our constraints. The first and obvious thing to point out is that we will not be shipping this game with a peripheral controller. Much of the fun of guitar hero was holding something that felt like a guitar and having the feeling of playing one in a rock band, a lot of the joyment disappears when you put that functionality under the buttons and analog sticks of a controller. It’s much harder to rock out in the squirrel stance. So we know that we won’t be simulating the feeling of holding a real guitar, so what do we have to work with? Well, a PS controller has four face buttons four shoulder buttons, a D-pad, two analog sticks, and three buttons we can’t touch, cz’ they are used for metagame functions like pausing, sharing, and leaving the game. None of these necessarily have a great correspondence or affordance to play guitar.

Mark — — But, there is a touchpad, this is where I need to stop in and say I’ve an obsession with trying to make something of the touchpad in one of our games. I’ve done prototypes of painting, scrolling around UI maps, choosing dialogue options, none of these have seen the light of the day but there have been attempts to utilize the most maligned input on the PS4 controller. Now as anyone listening is no doubt familiar within their use of smartphones, the touchpad has universal affordance with swiping. We can use that swiping action to correlate to the strumming of the guitar, it feels like there is an immediate affordance there.

Prototype Constraints

Grant — -Before trying out strumming options in our prototype, we should determine some pitfalls and mechanics we want to avoid, we should steer clear of anything that goes against our narrative and tonal needs, like which mechanics might conflict with our game. This will help us focus our prototyping and not waste time exploring dead ends.

Mark — — first of all, no quick-time events. QTEs is not the best option although it provides a cool unique result with simple player interaction, press square and see your characters skewer the eyes out of a big baddie. However, players, they can tell that their participation is not commensurate with the results. It can feel like you are unpausing a pre-canned animation rather than performing the action yourself as a player. This can result in the reduction of immersion as they watch their character perform some interesting maneuver they feel they have no control over. Instead of bringing the players into the action, they’re further removed and made aware of the gap between the characters and their own actions.

Grant — -So we should strive for a one-to-one correspondence between the player’s input and the output of the guitar. No QTEs, no playing complicated series and notes from one strum. The other issue with QTEs is that they are usually testing the players’ reaction time and timing in this case is not that important for Ellie and her guitar playing so we probably want to avoid making any kind of rhythm game. Ellie is just learning how to play the guitar, these are introspective moments both Ellie and the player should be able to play at their own pace. Additionally, the final moments of the game are about Ellie’s inability to play the notes due to her missing fingers, not that she can’t keep in rhythm. If we want that moment to land, we need the players to feel that they can’t meet the same metric they’ve repeatedly met throughout the game so that game-side metric should be about notes not about rhythm.

Mark — — What else we should avoid? The guitar is meant to be a personal connection between Ellie and Joel, who is dead once Ellie has the guitar. So no Dina on the drums, but neither should that mean having a metronome the player hears of even Ellie humming. Maybe at most, we can have Ellie’s head bop up and down to the beat but there should be nothing audible that the player might feel forced to play in time with. This goes back to our avoidance of time pressure and sense of practicing, but it also limits our options on how to tell the player what they should be doing since there is nothing to play to. We should also let me layer noodles around if they can, if a player is meant to play a B major and switch to G sharp minor, we shouldn’t fail them because they’re strumming on the B chord for a little longer. Again, it should feel you are practicing, you are allowed to make mistakes, so in that vein, let’s not have any incredibly strict fail states.

So we know the boundaries of the space, in terms of the controller, technical, narrative, and tonal restrictions. We know vaguely, what we want the emotional experience to be since we know the beats are meant to convey in the cinematic we’ve already recorded, that featured the guitar.

And we have some goals set out:

1. Try using the touchpad for strumming
2. Feel like practicing
3. Don’t make it a rhythm game

Currently, we don’t have a way t play different notes and chords, we still have these buttons and analog sticks available, we should try to find something that is comfortable and easy to use alongside the touchpad. We want to tell the players when transitions between two chords is right or wrong, so we need to explore to convey what chord they should play next and how to tell them when they mess up. This should feel similar to playing a real guitar and easy for anyone to play.

Working with Constraints

Let’s start with the one song we have actually playing take on me from a recorded cinematic, so we have something that will show up in the final game that we can use as a target. We’ll place those notes and divide the touchpad into six-string regions. The player can strum through those to play a chord or touch one region at a time in order to play one note. At this point, just playing the desired note when this region is pressed will be enough, we can worry about how it sounds later on. Now, intake on me there is five chords that are played in the verse and chorus.

Players need to choose to play from a list of five chords. If we put them in a list connected to face buttons, however, players either have to check or remember how the buttons are paired to the chords, there is a mental leap player have to overcome. Even when the buttons are placed on-screen with their corresponding chords for people unfamiliar with the layout, they have to check where a button is, creating diversions of that player’s attention, so the simplest affordance is probably going to be a wheel menu on the analog stick, the players can deflect in a corresponding direction without having to look down and in fact, the only way to properly confirm an input is on the screen, meaning their attention remains in one place. They can also highlight a selection without committing to it like a button press would, let’s start there.

When learning guitar, you first learn the chord progression and then you speed up faster and faster until you can play in tempo. Let’s focus on that first half where the chord progression is key. For now, I will place a yellow square near the edge of the next chord to tell the player what to play next. When the player strums with a chord selected, we’ll either flash green for correct or red for wrong, this feedback let the players know when they are making progress.

Testing it out:

As a player, am conscious of choosing a chord even though I don’t necessarily know what the connections are between the chords. I can follow the yellow square if I want to make progress and red tell me if I am making an error. But even I make an error, I can hear the chord that I chose and even dance around and play chords in any order I like.

Take a look at our main song in the game Future Days. This one is going to be a little different as future days is in a different key and has different needs. We’re probably going to focus on playing six notes in the melody, in the key of C. Maybe we can show the notes instead of the chords this time for this song,

The Free Play Prototype

Since we are early on in the project and we want to pre-approve the system, so we may let the players to play any song? Very exciting, but to make it work though, we need some music theory.

What Makes Up a Wheel?

Before we could answer how many wheels we’d need to let a player play anything, we need to think through the basic structure of a wheel itself. Through the iteration on future days and take on me we settled on a wheel with six slices that felt good for our interface. If we wanted to translate that to a free play or practice mode though, that would beg a question of which cord should make up a wheel.

A different wheel concept from western music theory: the circle of fifths. This might look familiar to you if you’ve got a music background. In short, the circles of fifths is a musical organization tool that places closely related musical keys near to one another. Major keys are around the outer ring and minor keys are around the inner and the more notes the key have in common, the closer they are on the wheel. Adjacent slices have only a single note different in their major scales and each major key is directly outside of its relative minor key, which has the same key signature.

It wounds technical but one thing this basically boils down to is that chords close together on this wheel sound good together, generally a little more easily consonant and used together in more songs than chords that are further away. That means when we are picking our six chords we could take three adjacent major chords and their three relative minor chords and we’d get a wheel that generally sounds great with itself, and would allow the players to play a lot of songs just based on the chord relation of one wheel. If we generalize and pick the chord at the top of the key of the wheel as the key in this case C, that gives us a template of chords that look like this. We have one chord at the top of the key of the wheel and the other chords that fill out the wheel are the major four and five chords and the minor two, three, and six chords. There is a lot more we could dig into but effectively what we get from constructing one wheel this way is two major benefits:

1. Anyone familiar with guitar or the musical theory, in general, has a relatively easy-to-understand structure for the layout of the chords. Whether they know the circles fit themselves or are familiar with one chord, four chords, five chords, etc.
2. For anyone who doesn’t know anything about the music theory, we have a wheel that sounds internally pleasant and consistent with the sorts of pop or rock songs that Ellie has been playing, learning, and even writing.

Small tangent, Ellie mentions to Dina early on in the game that she is learning some original music since learning the guitar.

Beyond being both fun for the player in creating this introspective tone, this free play wheel provided a wonderful and somewhat accidental opportunity for the player to role-play as Ellie, the songwriter experimented with chords and progressions at their own pace.

Now we know how one wheel works and how many wheels do we need to cover everything?

So we finally have five wheels of six chords. Players can play every minor and major cord and the theme songs “Future Days” and “Take on Me” in free play if they want.

Since the players’ thumbs are occupied by selecting a chord and strumming on the touchpad, we grabbed the l1 and R1 on the controller as sensible options to cycle through the five wheels with their index fingers.

As we may expect though, this did have a few side effects:

1. Any chord is maximum of two taps away from any other chord (wheel selection cycles).

Since the selection loops around from end to beginning, that would also be the case if we thinned it to just four wheels, so in that respect, our five was just as good as four.

2. Some missing keys: Many guitarists might say why’d you give me a weird key like D-flat and not a normal guitar key like G,D or E. Since there are classic guitar keys that have a lot of overlap with one another, we decided to sacrifice those keys having their own wheel in favor of minimizing the number of wheels in total and trusting the players that really wanted to play in those keys would at least be able to cycle easily enough to find the chords they needed.

3. No crunchy chords. Finally, we elected to include just major and minor chords that mean no crunchy chords, seventh chords, diminished chords, anything complicated. This was on purpose while it technically restricts the flexibility of what songs you can play, those chords for this genre are more niche and risky for novices. We always try to think the balance, flexibility for the experienced but forgiving accessible, and encouraging for the novice.

There is even a secret mode where you can play by the fret rather than by the chord. You can access that by pressing L3 and R3 with photo mode turned off.

FEEL , LOOK, AND SOUND — The whole experience

1. Feel: How does the player play the guitar? The issues we had to solve on the touchpad are small and awkwardly placed on the controller. It can only register two inputs on the surface at a time. There are no clear delineations between the strings where the strings are. So it can be hard to predict which string or strings you’ll play when you strum. The sides of the touch pad especially the top curve off and have unexpected detection on them since they don’t lie flat plain with the rest of the touchpad. And we can only detect a finger position once per frame.
2. Swiping directions — 1)affordance of the controller to the guitar suggests vertical strum. 2)Fidelity increased with horizontal strum
3. Limited Fidelity — 1) Strumming too quickly 2) Tapping too quickly

4. Add dead space around edges

LOOK

How does Ellie play the guitar ?

— 24 basic gestures+ hand of extra access (missing fingers etc.)

— Blending vs Transition animations — can’t support all transitions

Strumming animation VS. picking animation

— Whole arm strum, fingers pluck

Animation on strings themselves

— Both physically string and UI element

SOUND

How does the guitar play?

4 different guitars in the game (music/audio teams)

30 different notes, plus messy buzzed notes for finale

Multiple takes of each note(round robins)

— crossfading

Muting and slide sounds

Takeaways

By Mark Burroughs & Grant Hoechst 2017 GDC Talk

Dictated and translated by Gigi Geng 2021

Any questions, please reach out to yanzigengus@yahoo.com