2

Very large games on the scale of a Runescape or a World of Warcraft often are frequently updated. In some of these updates, a small but significant change might accidentally be made and not detected in time for the patch release, which due to the size of the playerbase tends to be almost immediately found and exploited by the players. Or they could break things in an entirely unrelated part of the game.

  • Runescape recently had an incident where they accidentally added a spawn point for an extremely rare and valuable item to an easily reached location, and last year they had a bug where under certain conditions killing a player could cause them to drop absurdly high amounts of currency.
  • A Blizzard CS representative once said World of Warcraft had a bug about a decade ago (though I can't find a source anymore due to this being from 2 forums ago) where they fixed a bug with the spawn rate of a named quest target somewhere in a starter zone and broke a high level boss encounter.
  • Destiny had an incident where a random cave somewhere was spawning massive amounts of high level enemies and dropping large amounts of item rewards.

The problem is that due to the way that this type of game is built, this type of bug often stems from faulty data rather than faulty code, which I think might make it a lot harder to automate tests like this. You'd effectively have to test that your underlying data is correct, and for games of this scale testing all of the data at a glance seems wildly implausible due to how much it really is and how volatile this data can be. However, due to the unexpected effects that a change can have (like the Warcraft bug), you do need some form of automated testing to ensure that changes don't impact something on the other side of the world. I don't think that code reviews and the like can catch bugs of this type.

In cases like this, are there any best practices, methodologies or patterns that can help find these bugs before they go live?

Improve this question

1 Answer 1

Reset to default
1

which I think might make it a lot harder to automate tests like this. You'd effectively have to test that your underlying data is correct, and for games of this scale testing all of the data at a glance seems wildly implausible due to how much it really is and how volatile this data can be.

But so is the code base- it's huge and contains a lot of logic.

To some degree testing of data might be even easier since it's static- it doesn't go anywhere, you don't need long deployments and executions before you begin your testing, it parallelize easily and practically since timing is more relaxed you have more tools to help you test it.

I don't have experience testing games data but some of the things you will want to check are

  • statistical tools- check all kinds of indicators about your data. like everything else in testing don't expect to cover everything from day one, start from something and add as you go.

  • big data analysis- this adds up to statistical analysis

  • machine learning- here it gets interesting, it's not going to be easy but at least you don't have the hard task of extracting the data. You can teach an AI how a normal game behave based on some data and hope for it to find anomalies

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.