The coming age of conversational NPCs

News & General Discussion
1

Executive summary: game NPCs will soon be able to carry on real, spoken conversations with the player. I’m interested in how this will impact game design.

So, let me set the stage by pointing out a couple of recent developments:

  • Apps like Siri, Amazon Echo, and Google Talk have already got us talking to inanimate objects. We have an Echo in our kitchen, and the whole family uses it on a regular basis. These aren’t really conversations; it’s a voice command interface. But… all of us have occasionally caught ourselves saying “please” or “thank you” to it.
  • A fully conversational Barbie is coming soon. It’s not perfect, but kids don’t (generally) care.

I believe these are the tip of the iceberg, and it’s going to be in toys first where this really takes off. But video games are just expensive toys for grown-ups. It won’t be long before the same technologies start to appear there. And it will probably be small indie studios, rather than the big AAA studios, who innovate here first.

So! Imagine you could just speak (you know, with your voice) to NPCs in a video game. And they would understand you and respond by voice. They’re not geniuses (though they may be savants), but they can understand any common questions or commands and respond appropriately.

How would this impact the design of your game? People already get attached to NPCs sometimes, despite their clearly 2-dimensional character. How would this change when characters know you, remember things you’ve seen and done together, and reminisce with you later? What about subordinates that can follow verbal orders? And what about enemies that can engage in a bit of repartee before (or during) the big duel?

Any thoughts?

2

I don’t like to talk out loud while I play games. It disrupts my entire family and keeps them awake if I am gaming late at night. Currently, I have to listen to my 15 year old play Minecraft with her friends, talking, making weird noises, for hours. I am very close to tossing her mic in the garbage can. :slight_smile: Having her play in her bedroom on her laptop only makes her talk louder. lol If I could type to the NPC, I would be much happier.

So…very cool concept and I am sure we will see this sort of thing in game someday, but I am not looking forward to it. We need more human interaction, not more electronic interaction. :slight_smile:

3 Likes
3

I’m with @Teila … I guess I’m too old school. I don’t even like dealing with things like teamspeak when I’m gaming. For me, it takes away from the immersion. I can see where spoken dialog with NPCs could increase immersion, but could also lower family/roommate faction. I prefer old school where I can pick a choice during interactions or way old school where I can type responses - especially if NPCs are smarter and can formulate understanding and not look for a 100% specific response.

4

It’s going to be a long time before people are interested in actually speaking to characters in games versus playing then add they are now.

Voice driven games are usually flops. It’s a cool idea but not sustainable right now.

5

Meh, I say you’re all a bunch of old-fashioned fuddy-duddies. :slight_smile:

No, seriously though, I do appreciate the feedback. I just think that, looking back on it 10 years from now, it’s going to be in the same bin as “Why do we need to have moving pictures in our games? I much prefer text-based games, because the pictures I make in my head are so much richer than anything you can display on a screen.” There’s some truth to it, but it doesn’t stop progress from marching on.

And, kids who grow up confiding in Barbie are going to simply expect to be able to speak to videogame characters at least as well. Show your granddaughter (in the year 2025) an old game where the characters can’t talk, and she’s going to be like “Hello… HELLO?!? Why won’t he talk to me?!?”

And @LaneFox , voice driven games are usually flops now because they suck. Decent voice interfaces are very new and getting perceptibly better every year; it’s only in the last few years that they’ve gotten barely competent, and the real bend in the progress curve hasn’t even hit yet. That will happen when millions of talking Barbies, and other conversational toys, start hitting the market. This will provide the combination of market forces, forgiving audience, and massive amounts of real-world data needed to make this really take off.

So. For the sake of argument, can we just suppose for now that I’m right and this technology is coming? We can either stay firmly in the familiar and let others innovate with it, following grudgingly along after the fact; or we can think about how we would innovate with it, and make others follow. I’d like to get the latter discussion going. If you don’t believe it’s going to happen, then consider it just a bit of science-fiction fun.

Thanks,

  • Joe
6

I don’t think either of us said it was going to happen and in fact, I did say it would. :wink:

Just not something I plan to put into our games. Our goal is to encourage cooperation and interaction between players so it doesn’t fit into our plan. We will leave that to you “younger folks” with your new fangled ideas.

Oddly enough, my teens prefer the older games to the new ones. They spend MUCH more money on those. So not sure about “kids these days” since I think they are more discriminating than we are in some ways. They have the ability to compare the older games to the newer ones and find the newer ones lacking.

I would not assume that just because some new cool fad comes out that you will get the next generation of gamers.

I remember Furbies…my kids threw them away after a few months and now remember them with distaste. I can name dozens of other such “next amazing electronic thing” that never caught on. I lived in the days of “intelligent houses”, that now people hate because of all the maintenance. lol

And oddly enough, books stay popular, maybe on Kindle now, but people buy them like crazy. My daughter has a stack she wants for Christmas. I have faith in today’s kids. I spent two hours in the car with 5 teens this past weekend and their conversations were enlightening. You know what? They were talking about some hand drawn artistic game on Kickstarter that they love and is so much better than the new games…no action, more story…but so cool. Three of these teens were boys. Then they talked about genetic mutation among insects in Japan. And then about how violent and bad so many movies have gotten…unnecessary violence that didn’t add to the story or to the development of the character.

Wow…at there age, I wasn’t talking about that stuff. lol

2 Likes
7

lol! That is almost the exact statement I kept making while sticking with Simultronics’ DragonRealm before moving into graphics based MMOs. I avoided graphics based MMOs for a couple of years stuck in the mindset that I didn’t want some artists painting pictures and showing my imagination where I was.

2 Likes
8

Conversations with NPCs aren’t naturally enhanced. In fact they can be downright rubbish. A player who’s a bit boring (lets face it, a lot of people are) wouldn’t have a particularly brilliant conversation and might end up being bored by the responses as well. It’s a novelty.

When you speak to a shopkeeper in real life, how much do you really want to drag that out? Also, it lacks creative writing. When the hero says something cool in a scripted event, people can laugh or appreciate it.

By using AI and real conversations you would end up laughing at your own jokes essentially.

So while there probably is a place for this, it isn’t naturally a better thing.

4 Likes
9

@JoeStrout - How far are you willing to extend the speculation? There’s a world of difference between a Teddy Ruxpin that remembers your favorite color and Daniel Day-Lewis in-character on improv night.

I do think knowledge awareness (or at least much better appearance of such) is near. But generating compelling natural speech will take longer. Once that’s in place, though: VR dating sims and the end of the human race.

How times have changed! Seems like not long ago that video games were “only for kids.”

3 Likes
10

It’s not hard to imagine scenarios where having to speak to NPC’s would actually lower immersion. If the protagonist is Lara Croft or a gruff space marine, and I had to speak to characters in my own voice, that’s kinda funky. Even if the protagonist was very similar to myself, I’d be a terrible “voice actor.”

3 Likes
11

My six-year old niece and three-year old nephew are precisely the reason I wear noise isolating headphones practically all day long. I even have a fan running in the background for some extra white noise. I still hear them on occasion but I’d imagine they’re louder than your daughter.

Why not buy her an affordable headset (or just a microphone) that is capable of picking her up so she isn’t trying to talk over the fans in her laptop? I know that can’t be very fun for those listening to her either.

12

@JoeStrout I didn’t say it wouldn’t happen, I said its going to be a long time - like probably 10 years.

@hippocoder makes a good point too, it really dials back the quality you would be getting from scripted narratives.

1 Like
13

We’re decades before it’s even remotely feasible in games. The major catch to siri and google is that these are grafted onto big data. Their ability to parse language is second to the massive databases fueling them. Even ignoring the computational expense, the operating a database is nowhere near possible, never mind the labor involved in filling that database.

The more fundamental issue is: what value does it bring? Unless you’re working on a wiafu simulator and want desperate japanese guys to marry your game characters, will it actually make the game better? Maybe you can say it will give games a greater sense of world building (ignoring the massive amount of labor involved with feeding in information), but what happens when the player asks an NPC to do something? Will it even be enabled to do anything other than talk? Will it have a personality and tell the player to fuck off? Will there be multiple personalities?

There is no way an indie could make this. Even if it was a third party library/service (nearly unlikely at this point), just implementing it in any reasonable manner (that actually had a reason to be in a game and wasn’t horribly out of place) would probably take a team the size of most AAA studios.

1 Like
14

Do you know the TV show “black mirror”? I get a feeling you might like it.

I think there could be something interesting made with this technology that works well with a 3D VR experience. I could see a bigger indie studio make an interesting game where you play a police negotiator that needs to talk to crazy people that have hostages or want to jump from a bridge. I’d imagine such a setting would excuse a few not so natural npc responses that I have no doubt will occur.

I bought a game on steam where you play a mage and have to say magic words into your mic to make spells happen. It is an interesting gimmick but ultimately I barely played that game. In general I prefer not to talk if I don’t have to. Even when I play Insurgency online, have my headset on and I’m home alone I usually use text chat instead of voice chat. One of the reasons for that is that I’m not a native English speaker and I don’t speak it nearly as well as I can write it. I could see how that would be a problem with speech driven games in general if the game ends up misunderstanding people. I often have big trouble understanding other people that use voice chat myself, either because of their shitty recording setup or their heavy accent. If it was working correctly I could see great value in using such technology to transfer voice to text clientside, transmit that data instead of the voicechat stream and then turn it into speech again with speech synthesis on the client side. If working perfectly and properly tuned that could boost immersion a lot I think. Also it could apply a bit of content filtering to auto-block voice spamming or insults about your mom.

Considering how this could be implemented in games like GTA, Skyrim or Fallout I would prefer to not have it in the game at all and focus on properly voice acted content with great writing instead.

1 Like
15

It is not her fan. lol She is in her room alone and now she can be noisy. At the desktop, which is more powerful so her preferred computer, she is in the family room with the rest of us. :slight_smile:

16

Indeed, but there’s no reason games won’t hook into exactly the same big data. When you use the Nuance SDK, the actual audio recording flies through the intertubes to their servers, where the actual speech recognition is done, and the parsed speech (including several alternatives) is sent back. When you speak to Siri or OK Google, the natural language processing is done in the cloud too.

As an engineer it strikes me as a horribly inefficient way to do it, but it works — most of the time. Reliability will go up, and eventually, I expect there will be offline options too, but in the meantime, I fully expect games to work this way.

Yes, thank you, that is the discussion I’m trying to start! :slight_smile:

I should certainly hope so. I was trying not to interfere with the creative process by throwing out my own ideas too soon, but what the hey:

  • To my wingman or cohort in a fighter jet/spaceship sim:

  • “Let him go, stay on the leader!”

  • “Red Three, draw their fire; Red Four, you’re with me!”

  • “Get out of here, Biggs, you can’t do any good like that!”

  • To my companion in an RPG:

  • “Hang back and keep me healed, I’m wading in!”

  • “Wait here, but if I’m not back in three minutes, come find me.”

  • “Take out the spellcaster!”

  • “You help Furg, I’ll hold them off!”

  • “Have you seen the Count anywhere recently?”

  • To a companion in an action game:

  • “Hide on that ledge and get your Plasma Lance ready; I’ll lead 'em in!”

  • “I’m going in, cover me!”

  • “Quit using that pea shooter. What you need is one of these.”

Those are mostly commands, and yes, in most cases I would expect the NPC to do what it’s told (given the usual status of the player as the hero/leader of the story). This sort of thing can actually change the gameplay, or at least make it a lot easier to achieve what would otherwise require a tedius UI.

Of course there should be non-command dialogs too, particularly for players that enjoy role playing:

  • “Why do you hate orcs so much?”
  • “Whew, we almost bought the farm there, didn’t we?”
  • “Nice shooting, Ray!”

and so forth, all with appropriate responses from the NPCs. This sort of interaction doesn’t really change the gameplay, but then again, neither does a high poly count or a rich backstory.

Sometimes, if that’s in character.

You mean in one character? Or just different personalities for different characters? If the latter, then yes, of course.

This year, yeah. 10 years from now? I don’t think so. There will be third party libraries/services, and they will be no harder to use than 3D modeling and rendering tools.

17

Honestly, if it’s that level of engagement, I really want to play with my fellow human beings.

3 Likes
18

For a positive, though, I would expect a VR game to be much more immersive with this technology. It would be a case of “is it right for the game” rather than it being right for all kinds of games (it really wouldn’t IMHO).

2 Likes
19

It could also help with games like Elite Dangerous that require many functions to be managed by the player. You can map everything to a controller but then you have to memorize dozens of button combinations to do stuff that you don’t use often but need anyway. Things like “computer - activate landing gear” would make sense in the context of the game, make the input scheme more accessible and might even boost immersion a tiny bit. The “computer” keyword could be used to filter the sentence out from voice chat so that other players don’t have to hear those things.
Thinking about it, that game possibly would be a good early candidate for an actual implementation since it is online-only anyway. So not really a problem of having the voice processing in the cloud (even though I’d never want to put something like that in a game that doesn’t run clientside).

1 Like
20

At least then you can hurt someone’s feelings if they play like shit.

Let’s just assume player stupidity for a minute, and say they leave Rick and Morty on in the background. How should an agent interpret getting schwifty?
https://www.youtube.com/watch?v=hm9gF5Gi9x4

Half of the time you should probably get a quizzical look, while the other half should be a slap in the face.

Being able to use natural language for commands is a great idea in theory, but it comes down to the whether the commands are complex enough to warrant doing without a UI. For simple commands, normal interfaces are faster. This is also the area where most of the current solutions fail miserably, and they have been for decades.

Somehow, I can’t imagine creating personalities will even be remotely as easy as modeling a complicated character. God forbid it ends up being impossible to map input variables to the type of personality you get out, so it requires constant testing to figure out if it’s the type you want.

Now if you actually want these to be characters in a world, you would then need them to build relationships. Then there is the world’s knowledge base which has to be created by somebody. Unless every game that uses a conversation system is also running as a front-end for Dwarf Fortress, I can’t see any indie project being capable of making something out of it.