Few video games have a following as dedicated and engaged as League of Legends. League is an anomaly of endurance in the world of gaming, where newer and shinier releases are always a click away. At more than a decade old, the game still hosts almost eight million daily concurrent players around the world.
While the main objective — and we’re vastly oversimplifying here — of choosing champions, forming teams and destroying opponents’ Nexuses remains unchanged, the game continues to evolve. Riot Games, the company behind LoL, released an auto battler game mode called Teamfight Tactics last year, which, at last count, hosts 33 million monthly players. Meanwhile, a stream of updates flows from Riot’s West LA headquarters, many of which remain out of sight and mind for legions of League players.
We wanted to learn what it takes to build, maintain and evolve a game whose influence permeates online culture. The mythical backstories of League’s champions are well-known to players, but where do they really come from? Who’s behind the code? How do Riot Games’ technologists — known internally as “Rioters” — find and fix bugs? And who keeps the networks humming to handle millions of players each and every day?
We met five Rioters working across the technical spectrum and asked them to unpack these questions and more. Their insights — on network security, gameplay engineering and champion power rankings — shed light on what it’s like to work on one of the most premier titles in gaming.
What do you do for a living?
I’m on the Champions team on League of Legends, which works on all the new champions that come out in the game. As a gameplay engineer, my job is to build the tools or systems that allow other content creators to create champions and game play. I don’t necessarily create the fun, but I build technology that allows people to create fun in the game.
Take us through a project you recently enjoyed working on.
Last year, I worked on a champion called Senna. I’ve shipped about five champions here, but she’s the one I’m proudest of. For starters, she’s the first African American female in League, and it was important for me that we show diversity within the game.
Learning an engine is a lot like detective work at first.”
From a technical perspective, Senna does something that no other champion does: she can make every champion on her team invisible. This means we had to understand what it means to give all 148 champions stealth abilities — not just the handful who already have them. A lot of it involved getting in a room with the designer and a QA person and saying, “Okay, what are our interactions? What do we want to happen if I’m a tank like Alistar and I’m casting spells after Senna has made me invisible? If the spell does damage, do we want that spell to reveal me? What if it’s a heal, or a shield?” So we took spells and extrapolated them to this category level, and that began to inform the system.
We generalized like this for 80 percent of interactions. The other 20 percent were more bespoke things, which are important because they fit within player expectations. However, you don’t need to live entirely in the bespoke world, because that means you’re going to do more than 130 different solutions, including a new solution for every new champion.
Is it difficult to get a handle on Riot’s proprietary game engine?
Every game I’ve shipped has been on a different engine, so I’m fairly used to jumping into a new set of code and learning it. The way I start with anything is to figure out how the input works, because the player normally interacts with the game through input — in League, that’s with mouse and keyboard. Then I ask: How does this spell work? When I press the cue button, how does that turn into a spell? What is the path it goes on? You follow that path, and you learn.
Learning an engine is a lot like detective work at first. Once you have a full picture, you can start to change things within the system. But first you have to know what the foundation is — you have to know the why and the how before you can change anything.
What style of LoL player are you?
I used to be a micro-intensive player, which means I was all about styling on people and trying to show off. But I’ve gotten older and I can’t keep up with the kids anymore. I can think the game through, though, so I tend to play support because I can tell people what to do. The champions I like the most are the assassins that are going to style on you, but I mostly play supportive champions because that’s what wins me games.
What does a normal work day look like for you?
I’m primarily focused on quality assurance as craft, product problems and making process improvements. A good chunk of my time is spent on educating individuals and teams on QA craft and quality-minded best practices. I also specialize in dropping onto time-sensitive projects to perform RAID (risks, assumptions, issues and dependencies) analyses and to provide help within my given area of expertise, which is end-to-end content delivery.
My main goal is to ask a lot of questions that can help teams better understand what they are doing, why they are doing it and how to go about doing it in ways that are less risky and of a higher quality.
We’ve heard game QA is unique at Riot Games. How so?
Our analysts are directly embedded onto teams. They work alongside developers from early in the development process to ensure we are thinking about quality at every step of the way — not just toward the end, when it can be hard to change what we’re doing. We test and then analyze the results to help our teams make both technical and process improvements so that we can get better and constantly push the quality bar in a positive direction. Why did an issue happen? Was it a problem with the tech? With the process we used? With training? We help answer those questions, and then help to come up with solutions.
What are some common League of Legends-related issues you have encountered as a QA?
Oh boy, what don’t I encounter? With my past work on Skins, I got to see a lot of fun issues happening with characters, from small animation breaks that were mere earflicks all the way up to said characters breaking the game when they would try to load in. One of the most satisfying things about my job is that we help to make common issues become uncommon.
I watch for much broader patterns that might indicate deeper issues under the hood.”
For instance, way back in the day we used to get visual effects on characters dropping to their feet when they performed certain animations, like dancing. That’s because VFX was attached to a bone per animation manually, and the animator occasionally missed one or two. We identified the pattern and overhauled how our buff bone system works, so that the animator did not have to attach VFX per animation manually. As a result, those bugs went from common to uncommon.
These days, I watch for much broader patterns that might indicate deeper issues under the hood that are impacting multiple systems or teams. League of Legends is definitely not a perfectly clean game, and we know we have quite a lot of work to do. But I will say that we have come a long way over the years in terms of quality, and we can do a lot of things today that we weren’t capable of doing at all a few years ago, from innovative champions like Sylas to releasing something as big as Teamfight Tactics.
What style of LoL player are you?
I like tanky engagers the most, like Nautilus and Leona. I enjoy being able to either wade into a fight and soak up a lot of damage while locking down as many opponents as possible for my team, or peel baddies off of a squishier teammate when things go sideways. Being the one to start an ace or getting a friend out of a no-win situation both feel very satisfying to me — even if I wind up with a gray screen for my troubles.
What are your team’s goals?
I focus on leading our security efforts across Riot from a product and tech perspective. We don’t want to be the team that says “no,” but we will block if we have to in order to protect our players, Rioters or Riot.
From a philosophical perspective, we believe that security should be a guardian of the company’s culture — we are driven by that culture, not the drivers of it. We always try to remain feedback-driven and audience-focused, and are careful to provide options — not roadblocks.
We have tried to make security as self-service as possible.”
Are there any LoL-related security challenges your team has recently encountered?
We recently collaborated pretty heavily with the L10 team, where we proactively engaged and collaborated to review communication plans; understood what was acceptable risk to both L10 and various product teams; explained what should or shouldn’t be done and emphasized the impact of these risks coming true; advised on how to securely protect our planned announcement and the intellectual property that was to be announced; and reviewed the actual content of the announcement with the L10 team lead.
How do you shift a security team away from a reactive mindset and toward a focus on preventing and mitigating threats?
Traditionally, security teams have been the team in the corner that no one wanted to work with because they nearly always said “no,” didn’t have alternative solutions and ultimately didn’t add value. There will always be a reactive element because you can’t predict attacks and the business needs frequent change. To be honest, it’s more important to be able to pivot rather than simply being proactive.
We have tried to make security as self-service as possible and the process of working with security relatively predictable. We also try to talk to our audience in their language — for example, using the RFC library with engineers. We now have a high-level checklist we share with game teams from pre-production so that there’s mutual understanding of what our expectations and requirements are for the game team to move through the various stages of production. The checklist is a “live” document and is already onto its second version, which has only been possible through strong collaboration and two-way feedback as we learned what worked and what didn’t.
What style of LoL player are you?
Bad! I prefer tankier champions that can be a little more forgiving of my poor mechanics and bad strategic choices. After trying attack damage carry and the middle lane, I ended up in the top lane. Though it can often literally be playing on an island, I do enjoy the solitude.
I’ve been really enjoying TFT since it came out and find myself playing it much more than Summoner’s Rift, especially as it gives me a chance to play with my son in a relaxed environment where he can contribute to the game — though he currently has a heavy bias toward Ocean Mages.
What is your role within engineering?
During my tenure at Riot, I’ve been on many different teams, but my focus has always been building technology that underlies our games. I work as an individual contributor, collaborating with other engineers to bring whichever feature or system we’re working on to players. I spend most of my day writing code, but I also do a lot of code reviews, discuss system design with other engineers and answer a lot of questions on Slack. I feel fortunate to have been part of some pretty valuable projects, like bringing Teamfight Tactics to mobile.
My main goal is to unlock capabilities we didn’t have before. I love working on fundamental technology that allows us to do new things, and some of the work I’m proudest of are systems that give other developers tools to do awesome new things.
Tell us about an interesting project you recently worked on.
The most interesting work I’ve done recently is the new League patcher. This is the part of the League client that downloads game updates, which happen every two weeks.
While this may seem like a simple problem on the surface — how hard could it be to download some files? — there is a significant amount of complexity. The updater has to deal with an inherently unreliable internet, players who haven’t updated the game for months and game installations that are in myriad states of disarray — all while doing its job invisibly so players can get in the game quickly.
We develop, publish and operate the game ourselves.”
We felt it was unacceptable to make all our players download the game again from scratch with the new tech, so we had to carefully manage a massive migration of files in preparation for the new patcher. We spent months slowly rolling it out to increasing numbers of players, all while monitoring the process to ensure it was going smoothly. We managed to complete the rollout with very few problems, and we have now been able to retire the old tech entirely. All of our new games will be using this new tech as well.
As a technical operation, how does League of Legends compare with other games you’ve worked on?
I think the biggest difference between League and other games I’ve worked on in the past is how vertically integrated we are. We develop, publish and operate the game ourselves. For me, the most interesting part is that I get to work directly with people who have very different backgrounds and expertise — and I can learn a lot from them. For example, a few years ago I spent some time on the Riot Direct team, which operates our internet network backbone optimized for games. I got to rack some network gear, configure some routers and run fiber optic cabling in our test lab, which was really fun and something I had never done before. In my career, this kind of cross-pollination is pretty unique.
What style of LoL player are you?
I’m not very good at League, so I like playing with others who can call the shots and carry me to victory. I like playing support, but sometimes I try other roles and attempt high-risk, high-reward plays. They rarely pan out.
What are your main responsibilities?
I fill the roles of team captain and product lead for Riot Direct. My teams are responsible for the health and security of Riot’s production network backbone — the global network that connects us to our data centers, cloud partners and players. My job specifically is to ensure that we are appropriately scaling our network-based capacity, capabilities and costs up and down with our current and future requirements. In simpler terms, we try to keep pings low, DDOS attacks out and costs down.
Give us a sense of the scale at which the Riot Direct team operates. What does it take to build and maintain specialized network infrastructure?
The majority of our team comes from network engineering backgrounds with a specific focus on telecommunications or internet service provider skills. The fundamentals of networking remain the same for us — moving packets of data in and out of our environment — but the scale and toolkit are very different. Rather than call up our ISP to report problems or issues and beg them for a fix — like we would in the old days — we are empowered by our own infrastructure and footprint to come up with ways to circumvent performance bottlenecks by going around them completely.
Tell us about an interesting project you worked on.
We have struggled with network automation for a long time, mainly due to an older Riot mindset of “build vs. buy” and a strategy we had around multiple vendors. We had to cater to every scenario we designed into our automation in very different ways, as our different hardware partners worked very differently to one another. In order for us to successfully support the launch of Riot’s new games and scale up our network capacity, we knew we would have to do a complete 180 on the old strategy.
We are empowered by our own infrastructure and footprint to come up with ways to circumvent performance bottlenecks by going around them completely.”
Our team tested several options and then adopted a single hardware vendor. They built an automated deployment system to support the refresh of our various sites using open-source tools on top of Riot’s existing build system. This vendor consolidation and automation will allow us to rapidly deploy new points of presence, and give the capacity needed to support our upcoming games.
What style of LoL player are you?
I used to play top lane a lot — Malphite was my go-to, heavy on the hard engage. Becoming a parent in the last few years has caused me to re-prioritize how I spend my spare time, and you’ll more than likely find me playing ARAMs or grinding out single-player role-playing games that I can pause at any time to deal with sudden emergencies. TFT has been good for juggling a kid at the same time as well!