17

I'd take a look at Google QualityBots. It's generally used for comparing websites on multiple versions of Chrome, but looks very similar to what you are trying to do. I personally haven't had time to try it out myself mainly because of its use of EC2 machines. Other than that, it is open source. Here is an article about it on Google's testing Blog And the ...


13

It depends What matters is that you are able to effectively communicate the current state of the software in test to your leads and management. How you communicate that information will depend on your workplace culture and any regulatory requirements your employer needs to meet. The biggest potential issue I see with the way you are currently communicating ...


12

As dzieciou commented, this can vary pretty dramatically. I will give you some ideas about what I look for when interviewing an experienced QA Engineer. One of the most important things I look for - whether the role is primarily manual or automated testing - is the ability to perform root cause analysis. For instance, if a candidate has experience ...


11

For me it's a non-functional requirement, even if the key users have some requests regarding the instalation folder location (for better integration with other software packages). UPDATE: The tests for the available features detection are still placed in the functional requirements area, given that: the software can be bin-deployed in some cases, and some ...


11

Your application uses an API to interact with the database. It is possible to write your API in such a way that it presents correct results to the application and yet still uses the database in the wrong way. For example, imagine a database with an EMPLOYEE table and a MANAGER table. The tables are alike -- e.g. each contains a first name, last name, ...


10

I would agree with alexandrul's answer, with a few small caveats. First of all, it depends on what the installer does. If it's just a plain normal install, it would fall under non-functional. If there are options in the installer that greatly affect the functionality of the application (ie: add/don't add specific features), I would normally put it in my ...


10

Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product). The approach I have used with great success is parameterized test data generation from equivalent partitions....


9

Imagemagick, a cross-platform imaging library and command line tool, has functions that can be used to compare images. A team I worked on circa 2012 used it with pretty good success to determine if two images differ. We had built a GUI with the same library that could pull the two images up side by side and highlight the differences for a human to decide ...


7

Wouldn't any data inconsistencies expose themselves in the application itself? Maybe, maybe not. I've seen cases where applications lost some data after you've logged off. So while the UI looked fine, the database was actually incorrect after logoff. In addition, are you sure that every single element in the database is being displayed in the UI ...


7

Because there are no requirements, I would say go with your first option. You are relying on normal expectations based on the button text. If the bug is rejected, you can ask why. Better yet though, find someone who should know (PM/BA/Customer or maybe even the dev) and find out what it's supposed to do. The more information you have, the better bug ...


6

I don't want to repeat great answers other have provided, but I would like to share with one more lesson I learned about using database in tests. Combining feedback you have from both database assertions and UI assertions, often in one test, can be very useful for test case design, test execution performance and defect root cause isolation: If you spot an ...


5

As I indicated in this reply Accessibility Testing - Should it be considered functional testing or non functional testing? I like the following definitions from http://www.lessons-from-history.com/node/83 "a functional requirement specifies what the system should do" "a non-functional requirement specifies how the system should behave" We have always ...


5

Lyndon makes some good points, it does depend on what your installer does. When I have worked with them in the past, mostly I deal with SaaS and not so much with client apps anymore, but there were a few in my past jobs and we mostly had functional tests for them. The rationale was we were not just dealing with installs, although that was there, but we ...


5

In addition to user246's excellent example, some other cases where you'd want to validate the database storage in your scenario would include: You have a bulk update/insert function where it's impractical to validate the results via the front end, such as importing new user records from a CSV file. While you can go in through the front end and check that ...


5

No, absolutely not. One of the reasons for developers and testers to be different people is because it's very easy for someone who developed some function to know how it should work, and test the happy path, the one that they know should work. I'm not saying they'll do it intentionally, just that part of the reasons that QA exists is to go down the paths ...


5

As others have said, communicate the way the BA is most comfortable with. Some other suggestions: Start by confirming your understanding of the functional spec/feature behavior - I can't stress this enough. If you and the BA have a different idea about what the new feature is supposed to do, you will never be able to communicate effectively. This will ...


4

We categorize things because we believe we need to treat the categories differently. It is hard to express an opinion about why separating functional and non-functional tests is important without knowing how one would use the answer. Important for what purpose? Sometimes people ask such questions in these forums because they did not have an answer in a ...


4

If you are curious about the tradeoffs around mock objects, there are entire websites and books devoted to the subject. I will restrict my answer to the specific problem you described. I never replace a real system with a mock system unless the real system does not meet my needs; a mock system requires time to write and maintain, and when a test fails, you ...


4

Why seperating functional/non-functional aspects of a system is important for me as an tester I make such an distinction at the very begining of a project (either during drafting the strategy or implementing it). It helps in making non-functional or 'implicit' (another word for overlooked,forgotten or ignored completely) requirements sufficiently 'explict' ...


4

The answer to this question is obvious: it depends. As an example, the customer complained on that the report generation pages takes a lot of time ~ 4 minutes to generate the report. Our developers were not able to fix the performance without rewriting the legacy code from scratch. So they just added “ajax-like” loading indicator. Well… our customers were ...


4

I've noticed that the advantage to have a person that is focused on quality and testing is having a point of view different than the developers one. However, since you're new in the project someone (not necessarily a developer) should take some time to help you understand the functionality, also you can rely on previous test cases, if there some, and in ...


4

User is able to change profile picture. Dev's automated unit test: As its name suggests, unit test is normally a functional test of a single function, e.g. test against a unit to a bigger picture. User is able to change profile picture may consist of a number of functions (units), for examples: a function that receives a mouse click from a user and passes ...


4

Depending on where you look, you'll get slightly different answers. Functional Testing : Testing the features and operational behavior of a product to ensure that the application is developed correspond to its specifications and requirements. Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs ...


4

You can use functional programming for acceptance tests. The F# library canopy is one that builds on Selenium, and could probably work with .NET tools like SpecFlow. I suspect that there are similar tools for the languages you're planning to work with but I don't know enough about them to find them. The key to how well functional programming would work ...


4

If you don't have specific requirements you should design test cases based on best practices, best user experience and security. It's not a bug, because it does not behave differently from expectations (you don't have any) but allowing to create password that is weak may lead to account hijacking and information leakage which is an issue. You have to think ...


4

One of the disadvantages of using SaaS for regression testing is that you need to allow outside organization access to your pre-production test environment. In most organizations, it has significant part of real production information, which can be security risk. Even if you de-identify user data (scramble addresses, phones, names etc) in pre-production ...


4

In the ideal world you as a tester should say "The actual behavior of the application under test corresponds to the given requirements". However to make that ideal situation having the place you at lease need those "requirements" which have to be comprehensive enough and unambiguous. This is quite hard to achieve in the real world especially in "agile" ...


4

The one and only correct answer to this question do not exist. It depends on many factors. It depends on the software development life cycle model, on the team and developer to tester ratio and so on. There are 5 main testing levels that we can highlight: Unit testing - It is basically done by the developers to make sure that their code is working fine. ...


4

In general I would say it is a good practise to have at least one person (tester/qa) in the team who is in the lead with regards to functional testing. By having this person constantly focus on tests and testability of end user functionality, this forces the whole team to also think about this (even when the tester is not there). I've worked many years as a ...


4

It's not an easy question and I am not sure it belongs in SQA, but have you tried to set up meetings and ask the questions face to face ? Many times a small demo or a few minutes of screen time can resolve misunderstandings.


Only top voted, non community-wiki answers of a minimum length are eligible