7

I had this question in a job interview:

Imagine that you have got a new application. You have never seen it before, it is absolutely new. What types of testing will you do first?

As far as I understood they mentioned in general functional and non-functional testing but when I said functional (using documentation) they said that I am wrong.

  • 1
    Has the application got a UI? Is it Web application? Desktop application? – dzieciou Jun 16 at 16:56
  • web application – joel4 Jun 16 at 18:20

10 Answers 10

8

Naturally, it depends.

What types of testing you will do first?

If you will be working with this product for a long time, you probably will want to focus on understanding its functions.

The goal here is not to find problems, but to learn in order to improve your mental model of the components of the product.

There is where Testing Tours, more specifically, feature tours:

The first one he suggested was the feature tour. In the feature tour, you simply move through the application, getting familiar with all the controls and features you come across. You ask simple questions like, "What's this and what does it do?" This tour works effectively with the one factor at a time (or OFAAT) heuristic.

With this initial information, you can generate mind-maps, checklists, user stories, test cases, etc.

From there, your mental model can be used to do risk assessment; which will guide your future testing.

However, if you will be working for a shorter time, you will probably be guided more by the circumstances.

E.g., I participated in the Software Testing World Cup 2015, where my team had only 3 hours to interact with a mobile application.

We naturally did some testing tours, but our strategy was more focused on usability heuristics - according to the instructions from the client - and compatibility problems, since we know about fragmentation problem in Android and that the iPhone app was native, developed in parallel with the Android.

  • This answer seems to me the most appropriate,thanks! – joel4 Jun 16 at 18:21
  • It's probably fair to say that 3 hours within a competition will probably change the priorities of the testing, though: that being said, the perogatives are probably the same - find as many issues as possible as quickly as possible (I guess). – AndyW Jun 21 at 7:22
  • The goal of testing tours is not to find problems, but to do an initial charting of the product, in order to guide further testing. – João Farias Jun 21 at 7:27
5

As mentioned in other comments the first step involved would be to analyze the product and the requirement, once you understand what the product is expected to do and the scope of testing, the process usually evolves as follows:

Exploratory testing:

Where we navigate and use the product to understand more about the 'system under test'. Here we learn about the product and also identifies possible test scenarios that could be applicable to the test context ("As testing is context dependent,all possible scenarios won't be applicable to all products")

Monkey testing:

A test approach were tester tries to break the system using random actions without much knowledge about the product. Monkey testing can be dumb, smart or brilliant, were each of this defines the product and domain knowledge level of the tester.

Adhoc testing:

A focused approach to break the system using the product design and requirements that were learned from the above mentioned approaches.

API Functional and non-functional testing (Integration testing):

This is the systematic approach were you develop testcases for functional testing of the APIs. The test cases are usually made for manual and then automated, or could be automated straight away as in TDD approach.

GUI testing (System testing)

A systematic approach to test the system in order to ensure that the system works as a whole after integrating all systems together. UI automation, manual test cases all come under this.

Non-Functional testing:

Testing approaches to ensure system robustness through Security, Stress, and Load testing

Note: Non-functional testing can be incorporated in any test levels. Not necessarily after system test

We use smoke testing and sanity testing mostly for making sure the build we get for testing is qualified enough for further rigorous testing. It is not done for an application that we don't have any idea of.

Smoke Testing:

A test approach which tests only the critical functionality of the system to make sure these critical functionalities are stable and does not break.

Sanity testing:

it usually carried out as an entry criteria validation ( a criterion that defines whether further testing should be done). Here all the common functions like page navigation, login, etc are tested to ensure that the product works fine and can be used for further rigorous testing

  • I like this answer, but have to take issue with the term Monkey Testing. I ask myself - Why do we need to invent terminology for what is a standard test case for existing testing types? I find this creep particularly relevant here as there exists two subtypes: Smart Monkey testing and Dumb Monkey testing. I mean really? Anyway, i'm just ranting. Ignore me. – SlightlyKosumi Jun 17 at 9:51
  • 1
    istqb.org/downloads/category/20-istqb-glossary.html , it's defined in istqb glossary . Ya even I wonder why we have so much terminologies that at the end defines the same thing, it just makes all thing more complicated – PDHide Jun 17 at 10:01
  • I work in test automation - I can see the difference between smart monkey testing and dumb monkey testing. I've always thought these terms had more applicability to automation - with dumb monkeys just navigating wildly and pressing keys randomly to try to elicit issues, and smart monkeys knowing the layout of the application and targeting specific controls or areas of the application. – AndyW Jun 21 at 7:25
2

The very first test we perform BVT (Build Verification Test) or Smoke testing when a build released for Testing. Sometimes BVT breaks, There are some reasons to build fail like test case coding error, automation suite error, infrastructure error, hardware failures, connection failure etc.

2

In the interview, were you allowed to ask questions?
Before jumping in to answer I might have asked a few things.
Verify, "it is absolutely new" means new development never released to customers? Or is it new code added to an existing product?
What type of application (work product, game, etc) is it?
Who is the target audience for the application?
How many people will be getting the application? When is it set to be released?

Having done a few interviews, a candidate that is thoughtful and methodical enough to ask questions & glean more information would stand out to me over others who just jumped in to answer.

1

If it's totally new to you, it is a great time to do usability testing.

Until you know how it works, what it is supposed to do, what the use cases are ... well, what else can you do but play around and find bugs?

Thinking about it, the question probably is geared towards identifying how many types of testing you know about. The different types of testing, and their utility changes from year to year. So this question is a good one to keep in your top drawer.

  • Usability testing is not about "what" it does, but how the user interacts with the application. For it, one needs to have deep understand of both the application and its users: Definitely not trivial nor that can quickly enhance the mental model of the product. – João Farias Jun 16 at 17:48
  • 1
    One of the golden rules of a UI is that it should be possible for anyone to work out how use it inside 1 minute in an emergency. As a first time user, a software tester isn't a bad person to have try to work out if they can use the product for its intended purpose. Sure, you're not managing a study, but this is playing with words. It's an open ended interview question anyway. They're just looking to see your opinions on what's important. – SlightlyKosumi Jun 16 at 20:00
  • 1
    Download any flight simulator and tell me if anyone can understand an airplane interface in under one minute. As usual, another "golden rule" that does not stand. – João Farias Jun 17 at 8:19
  • That rules have exceptions isn't really the issue here. Exploratory testing is essentially just a subset of Usability testing. You don't need a deep understanding of the product to do either. At the simplest level, you're looking to see if the application is intuitive, or if it is generating instances of people throwing their hands up in the air in frustration after getting stuck in their workflow. Of course at a subtler level a deeper understanding is good. – SlightlyKosumi Jun 17 at 9:37
1

You have never seen it before, it is absolutely new.

I would say the above line qualifies any ensuing testing as Exploratory testing. You are let into a wild, unknown territory without any kind of map or any knowledge about the terrain besides what your senses tell you and you are tasked with bringing back the most information possible. You are an explorer.

0

"As far as I understood they mentioned in general functional and non-functional testing but when I said functional (using documentation) they said that I am wrong."

Rather than thrash through the list of test approaches as others have, I'd like to concentrate on their assertion that your answer was wrong, and what I assume are their two choices, functional and non-functional.

If their question had included some form of compressed time schedule - or any time schedule at all, notwithstanding the usual compressed testing timescales at the end of any project thanks to the usual dev creep - then there would be at least some context to choose between these two.

The biggest problem is that (as ever) their technical question didn't frame the terms of the answer (if you see what I mean), giving only two choices. Assuming this is a valid assertion on my part, they asked a fairly free-form question to try to find out what your thinking was, and this could only be a valid scenario (or of use in the work environment you were interviewing for) if the application had a compressed testing schedule and you were brought in cold.

I guess I'm saying that I don't think this was a great way to interview.

If they'd have said there were compressed testing schedules I'd have said I would dedicate a proportion of the time to non-functional testing (finding obvious UI issues and usability stuff), and a proportion to targeted functional testing after reviewing documentation which helped point out the most important functions of the application under test, to ensure the basic functionality worked.

0

For me, the approach would be two- fold.

First, is to find out what is important

/critical/fundamental functionality is in place in the AUT.( By any means, be it exploratory testing or by using documentation if it's in place).

Second, to make sure it works

by smoke testing.

That would be my initial step , after confirming it works on basic fundamental level , we can dive deeper and deeper to test on different levels.

-1

Assuming this application (or its deployment - where it runs) will change in the future, you need to know if changes or deployments work or break the application. You usually want to know very quickly and comprehensively. Therefore you need automated tests so I would start by writing automated tests. I would write to them in this order:

  • Unit tests. Verify they exist as the application developer should have written them and they can also help provide documentation for writing other, higher level, tests.
  • Smoke tests. Quick and easy to write. Often the most value. Page Loads. The function exists, etc.

  • Happy path tests. Make sure the app works and continues to work after changes

  • Sad path tests. Make sure that error paths and cases are accounted for.

  • Performance and (for UI) End User Usability (non-functional)

  • Firstly, to automate any check, one will need to know what to automate - which means that one would have to test the application before the automation activity. Secondly, any automation of checking naturally demands a big initial effort, which means that the information from the initial effort will come after a longer time: Definitely not the best way to create the initial mental model of a product. – João Farias Jun 16 at 17:51
  • Fair enough. I get your point. I actually still disagree. "visit home page" is pretty simple to implement in automation. Although it might be a few hours or even days for a newbie compared to a few seconds for a manual 'visit URL' test. Therein lies the danger I see too often. Automated testing is not seen as 'do today' it's frequently 'do tomorrow'. Therefore making the ambitious goal of automated from the outset is a good way to frame it in my opinion. Yes automated UI tests require a manual run through first most times. – Michael Durrant Jun 16 at 21:02
-1

Well, it's a tad ambiguous when you say no documentation or reference available. This is where testing becomes heuristic in nature. You can start by ensuring things like:

  • No links should be broken
  • No server errors on accessing any part of the application
  • Ensuring proper re-directions from URL links
  • Ensuring proper industry standard validations
  • Ensuring understandable error messages across application
  • Ensuring no spelling mistakes across application
  • Ensuring support across major browsers

These are just a few examples to start testing an entirely new application. Please add more to these aforementioned. Thanks!

  • 1
    Noone said this a Web application :-) – dzieciou Jun 16 at 16:44
  • First rule of testing, never assume anything :) – Vishal Aggarwal Jun 25 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.