Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I've worked for companies that have little documentation for their applications, and most of it is probably comments in code.

How is a person, who is placed in the quality assurance department, suppose to verify that an application is doing what was intended to do; if there are no requirements documents and they are just told to "play around" or "read the user manual"?

share|improve this question
3  
You have comments in your code? You lucky dog! More than I can say. :( –  corsiKa May 11 '11 at 2:41

11 Answers 11

"Playing around" and documentation like user manuals can definitely provide a start to developing tests, but they should just be a beginning. Properly testing that an application is doing what it is supposed to requires more domain knowledge. End users who have been using the application and know what it is supposed to do can be a good source for creating tests. Requirements or early specifications for the application can also be helpful in generating tests. In order to make sure that the stakeholders are actually getting what they want (and to make sure you are not testing the wrong assumptions) from testing, once the tests are planned out, the requesting manager/stakeholders should approve and sign off on the tests.

share|improve this answer
2  
+1, although it does assume the stakeholders know what they want. I always assume they don't, even when they think they do. –  corsiKa May 11 '11 at 2:42
1  
+1 to glowcoder for the reminder that stakeholders might not know what they want. The number of times I've seen stakeholders who know the problem they want solved but don't realize that the implementation they're asking for won't necessarily solve their problem is... better not listed :) –  Kate Paulk Aug 17 '11 at 10:44

A strategy that I commonly use is that I will put my "end user" hat on and then test the system like it was an application that has been deployed to the general public.

I will make a number of assumptions as to why things are the way they are, and document them. I will then take those assumptions and have them checked by whoever I can, ideally subject matter experts, stakeholders and end-users.

So esentially I document assumptions, which are used in leiu of requrements and align my testing to them.

Whilst this is far from ideal, at least I can get some of the job done.

share|improve this answer
1  
+1 for the point on assumptions. We make many assumptions unknowingly. It is very important to be aware of our assumptions, list it and validate with the subject matter experts –  Aruna Dec 14 '11 at 6:01

Approaching the application methodically and with intent can yield a significant amount of the information that you need. Start with simple questions - any application should/ almost certainly will have the most commonly used functions on obvious display - follow every option in the menus for a while. Start to map what features you know the application has. Only the very worst designed software will hide away its most significant features (I've never seen this happen).

Draw information by speaking the software stakeholders - ask the developer of a feature for a quick demo, discuss with the project manager what success for this project should look like, if you have contact with the end users/ project sponsors discuss what is important to them. Take every opportunity to learn from each project stakeholder you work with.

Obtain domain knowledge perhaps by drawing on your own experience or use Google to find out what the application's or project sponsor's direct competitors are.

Ask a question of the application that you feel it should be able to answer. Take note of the response. Follow this with another question. Then another question. Build better questions from the answers that the application gives you.

If there really is nothing to create tests from but your intuition, you may find that things are missed. Learn from these experiences too - make sure your tests learn from these experiences. Encourage the development teams you work with the learn from these experiences.

Communicate what resources you need to be able to test effectively, over time and over successive projects you should start to find that the things you need to test effectively will start to be provided. Keep offering constructive feedback on the support, information and applications that you receive. Never stop learning and never stop be willing to teach.

Also, I can't agree with the statements above about Agile projects providing not enough documentation. In my experience Agile project provided better documentation as it is focused, to the point and relevant. If you find your being given an application to test on Agile project and you don't know what it is meant to do, realise that you're probably not on an Agile project.

share|improve this answer
2  
+1 for Communication, that is often lacking in places. Especially those without documents. –  MichaelF May 13 '11 at 20:05
1  
and as you're drawing information, obtaining domain knowledge, asking questions etc, be sure to take the opportunity to create documentation on a Wiki, Sharepoint server or other place available to all in your org - sharing is important. You're doing it for yourself but also as an example for others. Once you've proven how useful it is, perhaps others will follow suit. –  StevenV Aug 17 '11 at 19:56

This is an extremely common problem in software development. There are many testing resources in an organization: support team, end users, documentation writers, and the bug database. Utilize these resources to help determine potential problem areas and common use cases to focus your testing.

share|improve this answer
    
in addition talk to developer, they know much much more than usually assumed, and ya +1 to you –  Tarun May 11 '11 at 3:01
    
I like the support team, documentation writers, culling through prior bugs & asking the developer - basically this is what I do. The only thing I would add is: in this type of testing/engineering environment, it is even more crucial to document what you have tested thoroughly to cover your tail. This includes the test scenarios by platform & version, pass/fail results w/date, bugs found & resolution, hallway conversations & email responses to questions. –  Laura Hensley May 11 '11 at 13:49

Our team started dealing w/ this issue by having testers either create a wiki page online or create a Q&A section on the existing wiki spec that the developers created. All questions about vague areas in the spec are asked there, and all answers are documented there. If you get an answer "offline", you would document it there, then email the person who gave you that information and possibly CC other interested parties (e.g., the dev, the business owner) asking them to verify that you got the information recorded correctly. Every so often, (every day or two at least, maybe more frequently depending on company culture), the testers email out the link to the Q&A page to the business owner, developers, appropriate management, etc., until all the questions are filled in. Call out key questions to answer in the email.

I found that specs started to improve as this process was more heavily used, as people anticipated the kinds issues the testers were asking about. The developers started getting better specs (or any spec at all!) from the business owners, and delays in testing were better understood, since no one will reasonably expect you to test without good information.

share|improve this answer
    
+1 Excellent suggestion! –  testerab May 11 '11 at 20:52

First, look for general requirements and work to document them. Some of these requirements come from previous versions of the application, some come from generally accepted usage. For example:

  • Must run on platforms x,y,z (perhaps because those platforms have always been supported)
  • Must use abc database
  • Must be able to process n records in m seconds
  • Must be at least as fast as release n - 1
  • Must not consume more memory (or other resources) than release n - 1
  • Must not crash
  • Must not corrupt data
  • Must use standards relevant to the platform (standard Windows UI, for example)
  • Must be consistent with relevant laws, regulations or business practices
  • Must not have any misspellings
  • Must be grammatically correct
  • Must incorporate the company's usual look-and-feel
  • Must be internally consistent
  • Must work in particular locales
  • Must be complete when expected by the stakeholders (perhaps for some event, such as a Beta)

If it's a web site or application, some additional requirements might include:

  • Must not be missing any images
  • Must not have any broken links
  • Must bascially work the same in all browsers which are officially supported by the company

Then, interview the project manager or developers and find out what they intend to do with this release. Document the intentions and use them as requirements.

Solicit input from anyone who is a stakeholder in the project. Share everything you find with everyone and revise it as needed.

Does the product have a Help file or a User Guide? If so, that's a good source of requirements.

Do Sales materials exist for the product? Certainly the product should do what these materials say that it does.

Sometimes, writing all of this up as assumptions can go a long way toward gaining a consensus as to the "real requirements" you can use to test against.

Once the system is at all testable, do some exploratory testing. As you find "undocumented features", add them to the list of topics to be discussed.

Find out if the product is internally consistent. (This is an area I find to be very useful) Even if I know nothing at all about a product, I assume it must be consistent within itself, and within the environment in which it must operate.

Look for external standards within which the product must operate. If it is a tax or accounting program - tax law must prevail and generally accepted accounting principles must apply.

Ideally, all of these issues have already been considered and written into the formal Requirements documentation which is handed to you. But if not, don't give up. Dig in and discover!

http://strazzere.blogspot.com/2010/04/there-are-always-requirements.html

share|improve this answer
    
+1 for writing up assumptions and exploratory testing for "undocumented features" –  Ethel Evans May 12 '11 at 4:58

The proper ways are, Do a Exploratory testing, understand the business needs and the impacts, interact with programmers, clients, use your own knowledge to get the things right. Eg. for a shopping cart website you could check how amazon.com works for various commonly accepted functions like product search, display etc.

share|improve this answer

It depends on how your brain is wired. Personally, I start by drawing pictures. After reading the documentation and playing around with the site, I like to diagram what I understand about the product. I grab the biggest sheet of paper I can find and draw a graph of entities managed by the product, relationships between entities, user roles, actions performed by those users. As I draw the diagram, I will encounter issues I don't understand. I keep a list of those issues, and once the list is long enough, I find an expert who can help me with them. (If you interrupt someone every time you have a question, you risk annoying them.)

If the expert has time, show them the diagram. They'll probably be impressed that you invested the time the understand the product, and it may tell them that it's worth their time to answer your questions. They may be able to spot mistakes or omissions too. (They may even ask for a copy.)

Of course, if the product is big, you may not have time to diagram that whole thing. That's ok; you can start with the piece your boss has asked you to focus on.

Once you have the diagram, it's a lot easier to put together a high-level test plan, followed by individual test cases.

share|improve this answer

Communication and exploration. I have begun working on each of my project with no or negligible documentation. My approach has been the following:

  1. Figure out (explore and initial training with someone in the team) what the product is supposed to do - the happy path
  2. Figure out the critical sections of the product
  3. Frequently talk to developers, product people and other team members
  4. Understand who the users of the product are and what the company is trying to provide the users building the given product.
  5. If the product has existed for a while, figure out the modules that have a greater sensitivity to breaking.

This helps creating a sanity test. Then take one module at a time, and expand the tests.

Document as you go, the Test Cases need not be verbose and you can skip detailed steps initially - the main goal being to document what is required to be verified. These initial TCs can serve as a platform for learning, a checklist, and a reference for documenting the undocumented specifications.

share|improve this answer

Yeah what the others wrote basically. Also, when you are on an AGILE project you will encounter as less documentation as it can be. So you have to get creative on how to test applications.

All the above are very good starts and ideas on how to begin from zero. Basically you have to talk to people and generally assume things.

BUT! Be weary! Testers NEVER assume anything and never believe in one thing. Always make sure that you have multiple perspectives to one feature.

share|improve this answer

The only way to verify that an application is doing what it is meant to do is by knowing what it was meant to do in the first place. Hopefully that means requirements/user stories, user acceptance tests exist for all the software in your company.

share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

    
The only way? I've never read the manual for the UK railway's train ticket booking site. But I can recognise that being unable to delete an invalid journey from your cart is a defect. –  testerab May 11 '11 at 19:19
    
I agree that the only way is by "knowing". But knowing doesn't imply documents. Smart people like testerab already know a lot without referring to any written documents. –  Joe Strazzere May 12 '11 at 11:46
2  
-1. This does not answer the question, which already presupposes that knowledge of a system is needed to test it effectively. –  user867 Jun 4 '12 at 0:20
    
@testerab really? What are you basing that assumption on? –  Peter Kelly Jun 5 '12 at 10:07
1  
Sounds more like a discussion for chat than the (length limited) comments section! Do you want to ask again over in chat? Happy to discuss, but don't think the comments is the right place for an extended discussion. –  testerab Jun 5 '12 at 11:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.