a place to jot

While talking to a friend about how PvP was going in Ashes of Creation, he mentioned some bugs that they were experiencing with PvP mechanics, in particular players disconnecting while in combat and being immediately removed from the server. This gave me an idea to write this blog post outlining all of the various problems an MMORPG, and in particular a PvP MMORPG, has to contend with.

So, let’s do a threat model of a PvP MMORPG. This list is trying to be as exhaustive as possible, if you have others to add, please let me know either on mastodon or bluesky!

Network & Server Level

Network level bugs and attacks are where attackers target either the game server or it’s dependent services (such as login services) to cause availability problems. This is usually in the form of DoS/DDoS attacks.

It’s not uncommon for attackers to target MMORPG or Game servers directly with denial of service attacks. These usually come in the form of a Distributed Denial of Service (DDoS) attack where they attempt to flood the game or various external services with massive amounts of data.

Ways to mitigate:

• Cloudflare I guess?
• With UDP based MMORPGs, use encryption and validate all connections so it’s easier to drop obviously unknown/malformed packets.

Another issue to worry about is packet/message reply or forging. This is less of a worry in TCP based games as the connections are validated from the TCP socket.

Ways to mitigate:

• For UDP based games in particular, you should be using encryption of all messages and use sequence ids to validate that things are coming in the expected order.
• Be sure to tie each user with their own encryption key so users can’t forge packets for other players.

Similar to above, packet sniffing / interception & modification. You should assume that all attackers will be able to read and intercept your game traffic and modify it anyway they want.

Ways to mitigate:

• Encrypt all messages with player specific keys, this will allow you to easily ban people if you notice them sending obviously malformed packets
• Set packet sizes that are reasonably small (<16k) and validate all message sizes appropriately
• Set rate limits on a per connection basis. (e.g. a client should never send more than 25 packets per second)
• Be sure that there are no obvious amplification of responses. Meaning an attacker only has to send a single message of 10 bytes, but forces the server to send back 64k bytes in a single message
• Only send what’s necessary to clients, this will somewhat help protect against radar hacks (where attackers can be forewarned of other players positions/locations.)
• Be super careful with integers, integer overflows are a common class of bugs that affect games where negative values are interpreted as large positive values

Information Leaks

Personally Identifiable Information (PII)

It should go without saying but client IP addresses should never be sent to other players. The last thing you want is your players getting SWAT’d due to some silly MMORPG PvP beef.

Ways to mitigate:

• Never send player specific identifiers that can be easily tied back to players. Obviously you’ll need to send player ids, but any other information (IP addresses, customer names) should never be anywhere near a game server

Random Number Generator (RNG) leaks

This is usually a local/single player only type of hack but recently it has been exploited against servers. If any sort of RNG oracle is accessible, you can expect that it will be abused by people to get an advantage. It really depends on what the RNG is used for. The most famous (and only one I know of) is the Minecraft Randar hack. Honestly though, this attack is pretty far out there.

Ways to mitigate:

• Use different RNG seeds for different purposes. Never use the same RNG for the same thing. For example if someone does /roll to get a random number, this should not be the same RNG that is used for calculating the rarity of crafted items
• Only send necessary data to clients

Straight Up Hacking

If you run an MMORPG, you probably have tens to hundreds of various servers and services running. Login servers, community forums, databases, redis caches, k8s deployments and so forth. Each and everyone of these could be targeted. Assume any API calls that are baked into your client binary are visible and will be tried out by curious reverse engineers.

Ways to mitigate:

• Know your assets, ensure you know where all of your servers are and what software / dependencies they are running
• Be super careful with leaking debug APIs in binaries, especially during testing/alpha/beta phases
• Do the usual security hygiene, run SAST/DAST/SCA tools to identify vulnerabilities
• Hire consultants to do penetration tests/network assessments of your assets

Client Level

Just like any other multiplayer game, you always have to worry about client level cheating. This could be anything from players using cheatengine to wall hacks and aim bots. Mitigating hacking of the client locally is by far the biggest challenge for any competitive PvP game. I guess I could suggest using a service like Easy anti-cheat but just be aware people are adept at bypassing it.

Aim bots

These aren’t as much of an issue for tab targeting games, but definitely are for more skill-shot based games. Note that there are some completely undetectable aim bots that use external systems (think OpenCV sending data to a mouse USB HID with head shot coordinates).

Ways to mitigate aim bots:

• You can’t really. Best you can do is save last N inputs from players and build in a reporting system (/report <cheater>) that will analyze all players in the vicinities and their last N inputs to determine if they have inhuman aiming ability. A good sign for this is if a player is able to 180 degrees spin and headshot someone in less than 5-10 frames. Bonus points if you can replay these inputs like how Deadlock does it

Camera hacks

If the game is top-down, you may have to worry about camera/zoom hacks. This is where players modify the client to zoom the view out so they can see more of the world. This was exploited in games like Albion online.

Ways to mitigate:

• Have a small Area of Interest (AoI) view, and only send data about other players that fall within that AoI.
• You could also send fake player data that only shows up in this ‘zoomed out view’ to trick cheaters to target them. If the player targets this phantom player, you know they are cheating and can report/ban.
  • Activision calls this “Hallucinations” and you should absolutely read up on it as it is a pretty awesome technique

Teleportation/Speed hacks

This is where a malicious player modifies their x, y, z coordinates to either teleport immediately somewhere, or increase their velocity. This is usually due to an incorrect implementation on the server as it should never even accept positional or velocity data from a player.

Ways to mitigate:

• TLDR: THIS SHOULD GO WITHOUT SAYING, DON’T TRUST CLIENT INPUT
• Only accept player inputs (w, a, s, d, space bar, mouse clicks etc.) from the player. Position or velocity data should never be sent by clients, all calculations must be done server side
• Ensure any fast travel mechanics have proper cooldowns and only jump to known locations that can be referenced by identifiers and not exact X, Y, Z coordinates
• Game logic must only really be calculated by the server and the result sent to clients

Client exploitation

Game clients at the end of the day are just another application that’s receiving untrusted data. Other players can influence what type of commands or data is sent to other players, whether it’s world state updates, or RPCs such as trade windows, or even the chat system. If you are not validating the incoming data or are processing the server data in an insecure manner, people will exploit it. A good example of this was an issue with New World where players realized they could send HTML to the chat system and it would be rendered, in some cases it would crash clients, others, a Giant Sausage.

Ways to mitigate:

• Validate server traffic, treat it as untrusted
• If any of your systems use HTML, for example CEF embedded, be extra careful. CEF can easily become outdated and exploited, be sure to disallow any client rendering from user input
• Be very careful with chat systems as players will try to send all sorts of malicious data through this channel

Game specific

There’s really no way to tell what players will try to exploit locally, as some of the exploits may focus on game specific logic and functionality. The best you can do is limit what clients send, and limit what the server sends back to clients. This means if the player shouldn’t be able to see an opposing character, don’t send that data. This may get expensive with line of site calculations, but at the very least consider having a small area of interest field to limit cheaters creating outlines/radar systems from sniffed network traffic or memory captures.

Combat Level

PvP is notoriously hard to balance. The vast range of builds, items, player skill levels and group compositions, it is a combinatorial explosion of possible ways to get an advantage in a fight. Even assuming your game is relatively well balanced with gear and player abilities, there still are other things to worry about.

Unfair distractions or interactions with the environment

Obtrusive UI windows

Ways to mitigate:

One PvP specific exploit for this is causing UI windows to pop up during PvP combat. For example while being attacked, a bystander (who is working with the enemy) will open a trade window with you, blocking your view or causing you to have to click a close button.

• Set an in combat PvP tag that disallows any sort of trade or pop-up windows while the tag is applied to a player
• Have user controllable settings for disallowing trades by unknown players (and set it to disallow by default)

Pulling Mobs, Guards or Charmed NPCs

This is a common method of abuse where a player will attack a high level Mob or guard which will attack on sight. They will then pull them to the player. Once within range the player the instigator either goes stealth or “de-aggros” themselves so that the mobs focus on the target player.

Ways to mitigate:

• A mob which is pulled to a certain range should no longer aggro to players who did not engage with the mob. (e.g., they should reset their aggro levels and return to their original position)

Damage Shields

Buffing a target player with a damage shield that is always triggered when attacked. This attack depends on how the combat system works. In some cases an attack missing will still cause the attacker to be hit with damage. In this case, if a guard observes the damage event, but not the instigator missing, the guard will kill the victim player instead. (This was exploited in EverQuest). Another variation of this is if the game allows Charming. You can charm another player and have them hit a guard/high level mob.

Ways to mitigate:

• Consider miss attack events as instigator attacks when calculating aggro levels of guards
• Disallow charmed players from attacking mobs/guards

Exploiting pulls/pushes/knockbacks in environments

Abusing pulls/pushes/knockbacks to get players stuck in environments. This type of problem usually exploits some sort of collision/physics system where players can get stuck. By pulling or pushing players into places they can’t get out of, they can get easy kills or just be a jerk.

Ways to mitigate:

• Require all pulls or pushes to require line of sight
• Don’t do instantaneous or very fast pulls/pushes as these have a higher chance of accidentally penetrating through a collision box in a single simulation step

Exploiting collision/physics environments

Similar to above, the instigator gets themselves stuck in a position where they are able attack, but players can not attack back (usually due to mesh normal pointing outwards, meaning the attacker can see other players from the “inside”, but invisible / not targetable from the outside.)

Ways to mitigate:

• Similar to above, try not to add instantaneous/fast movements as it could penetrate through a collision box in a single simulation step
• All you can really do is try to find any gaps in between meshes and test to make sure players can’t pop through them. Depends on the collision/physics engine, but be wary of complex geometry that intersects in various places

Allowing players to add objects/collision to environment

This one is also similar, but the game allows players to dynamically add objects to the environment either permanently or temporarily. Obviously, permanently adding objects to the environment is a reciepe for disaster. This can cause players to get stuck in collisions, blocked in, or popped through the world. It may also be used to allow players to jump on them and get to places that are unexpected or not as well tested for collision gaps.

Ways to mitigate:

• Be REALLY sure you want to add this feature to your game. Not only is it risky from an abuse standpoint, it will add a lot of additional network traffic
• Limit it to player owned ‘lots’ or places where other players can’t easily or accidentally enter
• Thoroughly test any spells or abilities that create temporary collision boxes to see if they can be abused

Easy escape(s)

This one may seem easy on the surface but there is a bit of nuance. Your combat system’s TTK (Time To Kill) will play an important factor in how hard or easy it is to escape a PvP encounter.

There’s nothing more frustrating than engaging in a PvP battle only for a target to instantly escape. There’s a few ways they can do this, some are fair and expected, others are not:

Stealth – Player can stealth and is not visible to enemies. Usually this can be done in a fair manner.

Ways to mitigate or counter:

• Allow stealth players to be hit by Area of Effects (AoEs)
• Have debuffs which disallow players from stealthing to begin with
• Set stealth time limits to a level in which a player can get far away, but would wear off and still be within viewable distance(s)

Players purposely pull the plug/disconnect themselves. I remember a fun bug in EverQuest where you would make the game windowed then click and hold the window to make the event loop stop processing messages, causing the player to “freeze” and not take damage. Apparently this same stupid bug existed in New World 20 years later, but you did it when you dodge as you were invulnerable when dodging.

Ways to mitigate:

• Set an in combat PvP tag that only expires after X number of seconds before allowing the player to disconnect, or leave a zone/area
• The server should recieve any sort of dodge or ‘invulnerable’ command and then set an internal timer, irrespective of what the client is doing

Teleport(s) – A player can either teleport themselves or request to be teleported.

Ways to mitigate:

• Set an in combat PvP tag that only expires after X number of seconds before allowing the player to be teleported
• Teleportation spells are long (1 second or more) and can be interrupted by being hit

Animation timing attacks (and probably trusting the client)

Older games (such as shadowbane) used animation timing as a method to have “cooldowns” on melee attacks. They were most likely using this as a way of only sending inputs to the server after the animations completed. By interrupting the animation, you could attack faster, resulting in more damage.

Ways to mitigate:

• Never trust the client for input or blocking sending messages based on user actions. The server needs to tag and set timers for when a player sent an attack input and disallow any further attack inputs until that timer expires or the tag is removed on the server side and not client

Imbalanced Conversion, Transfers, Stacking

This is an ability mechanic to be extremely careful with. The idea is for a player to convert mana to health, or transfer health in a way that allows them to dish out more than they have. Usually the conversion is where you’d lose X percent of mana to gain a fraction of that percentages health. One way this could happen is if there’s no cooldown on mana potions and the player constantly drinks mana potions while transferring it to health, meaning they could live forever (or they run out of potions). Another very common bug is when damage multipliers stack to ridiculous levels. Sometimes these are just obvious bugs, other times they are when separate mechanics interact with each other.

Ways to mitigate:

• Require cooldowns on all potions/consumables and abilities
• Be careful of items / gear that gives percentages that could cause an imbalance between values taken and values given
• All abilities that are stackable must have hard limits.
• Be careful of integer overflows where negative values are treated as extremely large positive values

Straight up imbalance

Again, TTK plays an important role, if it’s too short of a window, you have a high chance of introducing chances for players to discover ways of one-shotting particular types of characters. Too long of a window and you’ll have never ending healer battles, or people will straight up walk away from a fight.

Ways to mitigate:

• Automate or build spreadsheets of combinations of abilities and ensure in no way is it possible to immediately kill a player
  • Iterate, iterate, and iterate different builds and group compositions to find any obvious issues
• Ensure there are plenty of Crowd Control (CCs) to allow interrupts to healers, or debuffs (reduce healing %) to counter excessive healing. Also consider stacking or having timers to not allow healing of a percentage to exceed a threshold
• Play close attention to any ability that uses a percentage for calculations. Players who min/max will focus on these values as they can sometimes be combined to give greater than expected results
• This seems like a great use case for training AI models to iteratively fight each other to identify any easy “winning combinations”

Damage exploits and bugs

These are most likely going to be game specific. For example in WoW there was a bug where legacy damage (older zones) scaling was able to be stacked for a particular attack and used in PvP.

Ways to mitigate:

• This was ultimately a stacking bug, while the stacks were limited, the percentage calculations were not. Be careful with percentages as they are easy to abuse or calculate incorrectly
• Be careful of integer overflows where negative values are treated as extremely large positive values

Deleveling: An exploit in EverQuest allowed players to purposely “de-level” themselves, but keep their skill levels of the higher level, this allowed them to hit for excessive damage to players of the same level because their defensive skill levels were not as high. Probably kept some modifiers in place and used the higher skill values.

Economy Level

These types of bugs can legitimately ruin a MMORPG. If your game allows a small number of characters to influence the economy it will lead to players not trusting the game any more.

Gold / Item / Resource duping

This is one that has plagued almost every MMORPG I’ve ever played in some form or another. Usually this occurs due to the inability for a game to know who “owns” the item. But other times it’s just poor implementations of game mechanics and rewards.

The quintessential dupe bug

These dupe bugs occur frequently due to an important fact, items that stack are not like normal items. In most cases, normal items such as weapons and armor that “drop” have an identifier. This identifier is usually tracked in a database. Stacks of items however, usually aren’t. You just can’t have a unique ID for every single piece of gold or platinum or food, it’s just not feasible.

Currency is usually targeted here, too much of it exists to assign individual IDs, so games rely on transactions (hopefully!) to ensure stacks of items/gold are safely copied from one place to another. If anything fails, the transaction is rolled back. Unfortunately, MMORPGs are just giant distributed state machines, it’s easy for a problem to occur where there’s no clear delineation of who should own the state.

Back in the old days, lots of these dupe bugs were exploited by initiating trades then pulling the plug or causing your game to lag or just attempt to desynchronize who “owns” the stack. In some cases this ended up with two different parts of the application or servers thinking they owned the stack, and all of a sudden you have a dupe exploit.

Ways to mitigate:

• Run all trades within transactions, if any error occurs roll everything back, if this is done with in the game server itself, use extra care to do proper locking with mutexes/guards to ensure race conditions are harder to occur
• Be careful of integer overflows where negative values are treated as extremely large positive values

Other dupe bugs

They can really come from anywhere, broken quest mechanics, servers accepting client input without validation, etc.

Rewards and Conversion bugs

This happened to eve online with their forex/loyalty points where players were able exploit a conversion of currencies for rewards for killing ships. If these ships were laden with particular types of cargo, you’d get a bigger payout. So they filled up ships with this cargo and just repeatedly killed it, reaping the rewards.

Eve also suffered from another bug, where trading items amongst themselves caused the game to determine the value of an object was worth more than it was. Classic market inflation aided by an algorithm that doesn’t take in to account possible adversaries.

Ways to mitigate:

• Be extremely careful of any sort of currency conversions. Ensure the differentials can not be inflated or multiplied.
• Do not use player behavior to determine market prices algorithmically, it WILL be abused.
• Be careful of integer overflows (this seems to be a common theme…)

Overall, duping/economy bugs occurs so much so that I’m going to list a few of the well known/common exploits:

• Everquest TradeSkill Depot dupe. This apparently caused a lots of drama. Looks like a bugged vendor was multiplying outputs incorrectly leading to ridiculous amounts of items being generated/created
• EverQuest recent “plat dupe” bug. This podcast tries to go into the technical details with non-technical people but it seems like this particular bug was caused by modifying packets. Apparently the game server did not validate that clients actually had copper to trade and people could change it the client side packet data. I suspect this may be an integer overflow as EQ currency would automatically translate from copper -> gold -> plat
• Rift plat dupe bug this one seems to be the ol’ lag your connection trick. In this case it was turning in quests repeatedly
• ArcheAge gold dupe looks like a hacked game client or packet modification. I saw mentions of speed hacks / teleport hacks as well so most likely they were just trusting client input
• WoW gold dupe bug looks to be exploiting some quest mechanic that allows you to turn in multiple times

Instant/Fast mob spawning

This ties into economy bugs because usually these fast spawning mobs drop items which are collected by players. This type of issue can occur due to some algorithmic logic that says if X players are in an area, spawn Y mobs. More players, more mobs. However sometimes this gets bugged, or players could exploit this fact to cause mobs to spawn at unrealistic rates.

Ways to mitigate:

• This really comes down to having timers on spawners and any logic that tries to use population as a determining factor heavily tested.

Bots

The bane of every live service developer, and gamer to be honest. Bots ruin economies, they ruin immersion, they ruin groups, they ruin games. Unfortunately, they are also extremely lucrative for Real Money Trading (RMT). Bots enable players to amass wealth which they can then sell for real money. All “gold for sale” messages you see for a game usually comes from dupes, hacked accounts or most likely, bots.

Ways to mitigate:

• Much like aim bots, they are pretty hard to protect against unless you are monitoring all transactions and player inputs. Look for signs that particular players are playing long hours, or doing the same actions over and over. Additionally you can put in hallucination data for, say, resource nodes or mobs and see if these bots try to interact with them. A real player would never see them so you know they are sniffing network traffic or reading memory. Keep in mind there are completely undetectable bots that use external systems (opencv/mouse hid usb data)
• Try feeding game transactions and inputs into standard statistical models and machine learning models to identify outliers.

Wow that was a much longer post than I anticipated. If you have any other attacks, exploits or general security concerns for PvP games please let me know and I’ll update this page.

Resources & References

• Twenty Years of MMORPG Hacking: Better Graphics, Same Exploits – Epic talk at DEFCON about someone who used pretty much the same (integer overflow) tricks over and over through out 20 years of various MMORPGs
• # 13 Biggest Exploits in EverQuest History? – The negative hp tunic instantly killing newbs was the most funny bug.
• # Top 10 World of Warcraft Exploits That Had To Be Fixed
• # Warhammer Online Exploits + Hacks – Looks like they read position data from the client as it was susceptible to teleport / speed hacks
• # 5 Exploits That Broke EVE Online – Percentage / stacking bug
• New World bugs shortly after release – I remember playing during these, hilarious