22

I was reading this paper about the differences between software development in general and game development and the authors made some good points regarding software testing, pointing out, for instance, that "game developers are hesitant to use automated testing because of these tests 'rapid obsolescence in the face of shifting creative desires of game designers'".

So, this reading made me think, what other aspects in software testing should we consider as different or particular when we are dealing with a game / testing a game? Anyone has experience with this or heard something else about it?

  • 1
    here is very good blog about software testing and game testing gamasutra.com/blogs/JohanHoberg/20140721/221444/… – Nitin Rastogi Jan 11 '18 at 4:55
  • 3
    You can auto-test things like file format loaders but how will you write a unit test that taking damage from a bomb exploding while simultaneously trying to grab the bomb and put your shield up doesn't cause you to glide backwards indefinitely at warp factor 4? And if you do write that test what will you do tomorrow when they decide bombs should be half the size they are today? (P.S. At 0:55 in that video is a quintessential example of unintentional behaviour) – user253751 Jan 11 '18 at 7:10
  • 3
    A big difference between testing 'regular' software vs games is the intent of the end user and its impact on your tests. For regular software, it can be assumed that most (legitimate) users will attempt to use the software as designed, attempting to find the happy path. With game testing, players will often attempt to break some aspects of the game in an attempt to gain an advantage while playing (see things like duplication glitches, for example). This is pretty important to keep in mind as a missed bug or issue can both break a game and render it less fun as a result of player action. – Valthek Jan 11 '18 at 13:27
  • 4
    You can make people do it for free by calling it a beta. – Studoku Jan 11 '18 at 16:41
  • Factorio has a neat video on their unit tests, there was a more recent version, but I can't find it – phflack Jan 11 '18 at 17:03
12

In game testing, a tester focuses mainly on the following points (from Johan Hoberg's blog):

  • Fun Factor Testing
  • Balance Testing
  • Game Level/World Testing
  • AI Testing
  • Multiplayer/Network Testing
  • Audio Testing
  • Physics Testing
  • Realism Testing
  • Modification API Testing

I personally believe that testing a game required very sharp judgment capability, fast decision making, attention to details, efficiency, focus on the application, layman approach, flexibility with the situation, greater observation on the specific area, these all instance happen in at the same time & tester require a rhythm to handle all these factors. Here is the very good blog with better explanation about software testing and game testing

https://www.gamasutra.com/blogs/JohanHoberg/20140721/221444/Differences_between_Software_Testing_and_Game_Testing.php

  • 3
    Please do not copy-paste content without attribution. The Gamasutra article's license does not allow unattributed usage, and fair use requires you to refer to the owner. – Kroltan Jan 11 '18 at 12:02
  • 1
    I have given reference of the blog in Question. – Nitin Rastogi Jan 11 '18 at 12:05
  • 1
    You should edit the answer to reference the post directly, instead of at a comment. Comments are ephemeral. – Kroltan Jan 11 '18 at 12:06
  • 4
    I formatted the material you quoted as a quote so it's clearer – Kate Paulk Jan 11 '18 at 12:44
6
  • Software testing against Games is potentially cheaper as you can ask players to do it for you for free during closed beta, open beta or even early access stage, when games actually test your game for you after they pay you. (Assuming your game is not free).
  • Software testing against Games is rarely life-threatening, unlikely when you are testing a life-supporting piece of software.
  • Games are more immersive and interactive than other software, gamers will try almost everything beyond your imagination; it is more than likely for them to cheat, modifying key parameters to crash your game. You cannot brute-test everything, even with test automation, as test automation is not creative enough to simulate a gamer.
  • Test automation is used to mainly test simpler, non-interactive game aspects, such as making sure there is no gap on this map, all trees are taller than 3 meters and etc.
  • Are gamers really better at breaking things than typical hackers for any other system. – Qwertie Jan 12 '18 at 4:38
  • @Qwertie, I do not think they are comparable. Gamers and Hackers are not mutually exclusive. – Yu Zhang Jan 12 '18 at 5:46
  • @Qwertie Not one-on-one, but there are so many many more of them. – Stig Hemmer Jan 12 '18 at 9:20
  • Why do you assume only games are tested by users for free? – Jakub Kania Jan 12 '18 at 9:30
  • @JakubKania, I am sorry I do not follow your question, can you please elaborate? – Yu Zhang Jan 12 '18 at 9:37
3

As an ex-video game tester, some of the article is fairly inaccurate.

Whilst we were encouraged to raise defects (we actually had weekly targets) we weren't allowed to comment on the 'fun factor'. I remember suggesting that one game we tested shouldn't have a physical release because there was a lack of content and fun... just to be shunned and reminded that we're testers, not developers or publishers.

So we would execute Game Level / World testing, Multiplayer / Network testing, Audio testing and Physics testing... never Fun Factor testing (that's not a thing), Balance testing, AI testing, Realism testing, etc.

Testing video games is very similar to testing software in terms of functional and non-functional testing. A tester's job is to find faults in what they've been given and gain confidence in the product on the stakeholder's behalf - no matter if that's in banking, aviation, gaming, finance, and so on.

  • "never Fun Factor testing (that's not a thing)" Maybe it wasn't your job but it is definitely a thing. – Confuzing Jan 11 '18 at 16:04
  • 1
    we're going off on a tangent here... but unfortunately, 'fun' isn't a tangible requirement so, in testing terms, isn't a thing (this is SQA, after all). If you can find 'how to test for fun' in ISQTB / ISEB then I'll be happy to amend my answer :) – trashpanda Jan 11 '18 at 16:13
  • 1
    I agree that it doesn't have defined metrics, but you also include things like "Balance testing,.., Realism testing"? I just thought it was worth mentioning because of your absolute statement "(that's not a thing)". – Confuzing Jan 11 '18 at 16:24
  • 2
    Surely this depends on the culture at your employer? I personally believe testers can be some of the best user advocates in a company, speaking up for how users will see a product. It's a delicate situation when testers raise concerns about things that are working as designed, but pose more subjective quality concerns. Good working cultures will value such input (meaning encouraging it and honestly listening), while many employers will unfortunately stifle such views. If your employer finds such feedback unwelcome, and it's clear the culture isn't changing, then sadly there's your answer. – Zach Lipton Jan 11 '18 at 21:55
1

There is a distinction to be made between whether the software does what the developers intended (the most basic type of testing for bugs), and whether the software does something desired by the user.

With games, the former almost doesn't matter so long as the user enjoys playing the result. So yes, that's a major difference.

I say almost doesn't matter, but in practice, the game is unlikely to be a huge hit without the developers' and game designers' creative intentions actually being carried out. Nevertheless, some games are quite famous for glitches deliberately left in after being discovered.

When the entire game is riddled with glitches that prevent playing the game at all, though, game testing will fail for lack of prior software testing.

0

Below is a list of Methodologies that I tried to compile of different types that make the game testing especially.

  • Fun Factor Testing
  • Balance Testing
  • Game Level/World Testing
  • AI Testing
  • Multiplayer/Network Testing
  • Audio Testing
  • Physics Testing
  • Realism Testing
  • Modification API Testing

Game tester should the same general knowledge base as an sofware tester but with a special focus of what makes games unique.

Of course Game testing is every bit complex as any other testing. though it should be treated in the same way. There is a place for live user test or alpa and beta test though its much bigger larger puzzle.

For instance Fun factor testing is something unique to games. since they are an entertainment product. Games are not only supposed to work intuitively and provide a good user experience.

two other big things I think differ is: In Software testing greatly utilizes automation scripts and it is the matter of testing a software application.

Also the scripts can be created before or after producing written test cases, either can be used to supplement the testing. A number of tools and frameworks have been developed to make automation quicker as setting up automation scripts cant take more time.

In Software testing makes greater of test-cases and test scripts. this generally makes the software testing job more technical.
Software testing is an engineering discipline wheter it is games applications or other types of software.

the things I mention I base the knowledge base partly from article belowThings are partly taken from gamasutra Regards

  • 3
    Please do not copy-paste content without attribution. The Gamasutra article's license does not allow unattributed usage, and fair use requires you to refer to the owner. – Kroltan Jan 11 '18 at 12:02
0

To answer your title question, I wouldn’t really say there are differences between testing software and games as what is a game but a piece of delightfully visual software?

Components of a game still have to work properly together (functionality) just as the components of a program like Microsoft Word or Adobe Photoshop. The rest depends on what where you’re working (company-wise and their specific values), what they specifically expect, and your leverage. At the company where I work at now, they’re very open to feedback for the games we produce because our core value to the quality of the game that we put out for the player. There are other places where I worked where my voice didn’t carry much weight so they much rather have functionality over anything else even if it was boggled with things that didn't make sense (there’s also places that care about bug quota).

For example, things I mostly test for (in no particular order):

  • Functionality
  • Coherent/Consistent UX/UI design
  • Core Mechanic/secondary mechanics (in the case of software, primary functions/secondary functions)
  • Bugs/defects
  • Interactivity (how things interact with each other and the user)

The list can get very long on specifics depending on the project but all of these things, I test no matter what the project is (game or software).

For background, I’m currently a QA Lead at a mobile game company (I still actively test every game for every release) and a former QA tester, both freelance (software and hardware) and through agency, where we often took on many different projects to QA (Anywhere from games to websites).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.