Chris Butcher talks about Destiny's matching making 'mesh of servers'

New Zealand is a damn awesome country. We have Peter Jackson and the Hobbits. We have Martin Crowe. And we have lent Chris Butcher to Bungie to help them make Halo. The trouble is the brilliant game engineer never came back and is still with Bungie making their new Destiny game. Here's an interview he did with Game Informer about the stuff that needs to go into makings sure players can find each other in the Destiny world. 

How did the big idea for Destiny’s multiplayer philosophy start?

You have all of these examples of people who are doing big server cluster things like World of Warcraft or something like that. But we didn’t really want to do that, because if you think about those kinds of games, you’ve got a centralized server that’s simulating everything in the world, but that can only scale up to some number of players. Maybe it’s 1,000. Maybe it’s 5,000. Maybe it’s 20,000. You compare that to the population of a console game and it’s tiny. 

So what that means is that you have to have dozens or hundreds of these separate servers. So we started out by thinking, “We want to have a single world that everybody can be in.” 

We took this mesh-based networking that we’ve been developing for years and years with Halo and adapted that networking to work in a seamless interconnected world full of other players and AIs. So when you’re playing a destination you’re moving from area to area and every one of those areas has got this mesh networking with a group of players that are in it at this one time. And then it has its own servers for that particular area so you’re continuously moving around between these groups of both consoles and also dedicated servers that are hosting it. 

That’s the thing that I’m really excited that we’ve been able to do because I think it’s been really hard for us. And I don’t really think anybody else is going to be able to pull it off in the timeframe we’re talking about on consoles. 

How does this mesh-based network play out in the game?

What happens is everybody in the world can play together. There aren’t these barriers that are in place. You’re all playing in one connected online world. When you’re moving from location to location you’re always going to have people to play with because there’s this huge population. You never have to go to an area of the world that’s deserted because there happens to be no one here on the server at this time. 

There are these artifacts that you get when you have, when you do a simulation that runs on a single server on a big mainframe type thing, there are two problems that you get. One problem is all of the 5,000 people on your server nobody is playing in Old Russia at the moment so it’s just empty when you go there. Or the other problem is all 5,000 people logged in and tried to go to the same place at the same time and the server crashes or it gets so full up that it’s totally lagged out. And if it does get full then there are some of those games that will start having different instances, but typically they don’t handle that very well because it’s not a core part of the game design. 

For us we’ve kind of said we want this game world to be able to work with millions of players online at once. And that means playing to the strengths of the consoles. Being able to use these very powerful machines to run a lot of the simulation. Being able to use the servers in a seamless fashion so that as you’re moving from place to place you’re switching networks with all of the different people that are around you. You’ve got a very high quality fast action gameplay experience. If you have all of these calculations taking place in a central server that’s one place in the world you can’t really have a fast action experience. 

What if you’re playing from Brazil or the West Coast of America? When we have this big pool of global players to matchmake with we’re able to use all of our technology to make sure you’re playing with people who are physically located near you so you have good ping to everybody else. So you’re able to have this great action experience that you’re used to in these action games. But at the same time you can do it in this global fashion.

We’re always storing data about how good your connection is to everybody else. We have all of this logic that’s going on about who’s got the fastest connections and therefore who the best person is to be coordinating the activity in that particular region. 

It’s as if you’re the host right there. There’s a ton of technology that goes into it. When it works it feels like you’re playing a single player game because everything that’s being simulated near you is running on your local machine and there’s no lag and no latency or anything like that. But at the same time you’re in this world full of lots and lots of players.

Will there be any limitations to the current-gen versions of the game in terms of networking?

The current generation of consoles, Xbox 360 and PlayStation 3, they have a lot of headroom still for us. We have the most optimized game engine we’ve ever produced for these consoles. We’re getting at the point now where we are wringing pretty much every ounce of juice out of the Xbox 360 and the PlayStation 3. We’re running higher character counts and higher player counts than we thought would even be possible on these things because we’re using every ounce of the CPUs and the SPUs and we’re offloading stuff to the graphics processor. You’re basically eating the whole buffalo at this point in the current generation. And that’s been our target right from the start is to make sure that we can provide that game experience that is consistent for everybody. 

Will there be individualized instances of these predetermined multiplayer zones in the world?

Yeah, there are millions of people playing all at once so that might mean that you have a couple dozen or maybe a hundred instances of one particular area of the world. But the players don’t see that because in all of our game design we’ve made sure that the way that the gameplay and missions are structured is invisible to players. They’re able to just traverse the world in this seamless fashion. They don’t ever need to see any details about where they’re switching from one game world to another, like how we’re handing off the network entities. 

In competitive multiplayer, how do you account for different player skill levels and powers in matchmaking?

We play multiplayer a ton here in the studio, and we play a lot of multiplayer games. We know how much it sucks to be in an unfair match. It is really not very rewarding at all. 

[There are] two things that we do for that. [First,] we have a lot of best-of-class practices to make sure that the matches are as fair as possible. The other thing that we do is make sure that when you are playing and you are experiencing the competitive multiplayer in Destiny you’re always improving and upgrading and receiving rewards. Obviously you want to win the game, but if there’s no way you can win this particular game, we’ve all had that experience in different games, you don’t come away from it with nothing. You come away with “I was really happy there because I managed to get kills on that guy even though he’s got this exotic weapon.” It’s really satisfying. So we reinforce that as well. 

If you’re kind of in an underdog type of situation, then we make sure that we give you both the investment rewards, but also call out that you’re doing a really good job in this particular match. For example, when we play in the studio playtests there is always the guy that has the sniper rifle and likes to sit up high. So he’s getting a lot of kills and that’s really satisfying for you to take him down. Maybe you get three kills on him over the course of the match but it’s satisfying to you and the game rewards you for doing it because you’re an underdog in that situation.