Tag Info

Hot answers tagged

10

For me it's a non-functional requirement, even if the key users have some requests regarding the instalation folder location (for better integration with other software packages). UPDATE: The tests for the available features detection are still placed in the functional requirements area, given that: the software can be bin-deployed in some cases, and ...


9

I would agree with alexandrul's answer, with a few small caveats. First of all, it depends on what the installer does. If it's just a plain normal install, it would fall under non-functional. If there are options in the installer that greatly affect the functionality of the application (ie: add/don't add specific features), I would normally put it in my ...


8

Your application uses an API to interact with the database. It is possible to write your API in such a way that it presents correct results to the application and yet still uses the database in the wrong way. For example, imagine a database with an EMPLOYEE table and a MANAGER table. The tables are alike -- e.g. each contains a first name, last name, ...


7

Generating test data is a difficult problem because if you don't understand the symantics of the data then you are likely to generate test data that will throw false positives in your tests (test failure due to faulty data, not a bug in the product). The approach I have used with great success is parameterized test data generation from equivalent ...


6

There are two approaches that I have taken to testing a WCF service. I am working under the assumption that you are looking to simulate the configuration both hosting and access that an application would do from a production set up. Hosting it I would set it up in a similar maner to what you are planning to do when deployed. For me this meant deployng it ...


5

Wouldn't any data inconsistencies expose themselves in the application itself? Maybe, maybe not. I've seen cases where applications lost some data after you've logged off. So while the UI looked fine, the database was actually incorrect after logoff. In addition, are you sure that every single element in the database is being displayed in the UI ...


5

Lyndon makes some good points, it does depend on what your installer does. When I have worked with them in the past, mostly I deal with SaaS and not so much with client apps anymore, but there were a few in my past jobs and we mostly had functional tests for them. The rationale was we were not just dealing with installs, although that was there, but we ...


4

As I indicated in this reply Accessibility Testing - Should it be considered functional testing or non functional testing? I like the following definitions from http://www.lessons-from-history.com/node/83 "a functional requirement specifies what the system should do" "a non-functional requirement specifies how the system should behave" We have always ...


4

In addition to user246's excellent example, some other cases where you'd want to validate the database storage in your scenario would include: You have a bulk update/insert function where it's impractical to validate the results via the front end, such as importing new user records from a CSV file. While you can go in through the front end and check that ...


4

I don't want to repeat great answers other have provided, but I would like to share with one more lesson I learned about using database in tests. Combining feedback you have from both database assertions and UI assertions, often in one test, can be very useful for test case design, test execution performance and defect root cause isolation: If you spot ...


3

The answer to this question is obvious: it depends. As an example, the customer complained on that the report generation pages takes a lot of time ~ 4 minutes to generate the report. Our developers were not able to fix the performance without rewriting the legacy code from scratch. So they just added “ajax-like” loading indicator. Well… our customers were ...


3

If you are curious about the tradeoffs around mock objects, there are entire websites and books devoted to the subject. I will restrict my answer to the specific problem you described. I never replace a real system with a mock system unless the real system does not meet my needs; a mock system requires time to write and maintain, and when a test fails, you ...


3

Cel, As always, it depends. If there's a requirement that the link contain a label then obviously the test needs to check that a label exists (whether automated or not). Here are some of the possible things I'd consider with this scenario: Is the code dynamically generated or static? For a static "once and done" page, the mark 1 human eyeball might be ...


3

We categorize things because we believe we need to treat the categories differently. It is hard to express an opinion about why separating functional and non-functional tests is important without knowing how one would use the answer. Important for what purpose? Sometimes people ask such questions in these forums because they did not have an answer in a ...


3

I use SOAP UI for this. It automaticaly generates all the functional test cases, based on wsdl definition, it is possible to load test your service, it is free and open source (and there is commercial version), etc. http://www.soapui.org/


2

When I define my test cases, (Normally in Team Foundation Server) I cross reference them to the requirements. Then, as they pass or fail I use that as a way to determine if a requirement has been delivered or not. Once all the test cases have passed to, it is the testers, not the developers who set the requirement from 'Ready for test' to 'Built'. That is ...


2

I'm currently trying FlexMonkium and while it seemed promising at first, the fact it only works with Selenium-1 is a kind of a turn off... Here I am stuck again with the "unable to handle file upload dialog" problem that Selenium-2 finally solved. Have to go back to my old Sikuli/AutoIt workarounds. Other than that, can't get it to work well without ...


2

We tried Flexmonkium and found that it was burdensome to get started and prone to versionitis. We've been dramatically more successful with Flexpilot. We use it with Selenium2 beta3 (WebDriver). The SauceLabs team is helping to move Flexpilot forward, whereas Flexmonkium didn't seem to be getting much contributor love.


2

I would recommend you taking a look at Jasmine if you like BDD style testing/specifications. Seems to be growing in popularity as well. There are a couple of screencasts on using Jasmine available on the net so you should have no trouble getting started. Check it out at: http://pivotal.github.com/jasmine/


2

Below option is also one possible soultion Generate client proxy class for the WCF Web service Using VSTT - Data Driven Test Approach for running tests with different test data - http://msdn.microsoft.com/en-us/library/ms182527.aspx For .NET 2.0 testing i have used VSTT for web service testing. It is pretty simple Creating a Data-Driven Web Test against a ...


2

As with most other tests, let your Requirements drive your testing, without being tool-specific. Do you have Requirements for what your tool-under-test must do? If so use that. Don't let the specifics of how the tool fulfills the Requirement become part of your test cases. For example, if your tool must be able to open an existing HTML file, then don't ...


2

The best time to start any testing is before any code is received. Performance should be part of the Requirements and Architectural reviews/discussions that hopefully occur before coding begins. That's where you can have the most impact on performance, and can adequately plan for performance testing. In your case, it seems pretty clear that none of the ...


2

Categories can help. Categories can help with some things. You might use categories to help indicate: Who will write or run the tests. For example, unit tests are generally written by developers (and are sometimes called developer tests). Acceptance tests might be executed by testers or product owners. The characteristics being tested--e.g. functional ...


2

Why seperating functional/non-functional aspects of a system is important for me as an tester I make such an distinction at the very begining of a project (either during drafting the strategy or implementing it). It helps in making non-functional or 'implicit' (another word for overlooked,forgotten or ignored completely) requirements sufficiently 'explict' ...


2

Question 1 - Regression testing the product you aren't working on is essential because both products are using the same core code. This is where you'd want to have a good automated regression library so you can know if a change to the core engine for product A has negatively impacted product B. Ideally, you'd be running your regression on either a per-build ...


1

In addition to what Kate said, here's one thing I always consider when writing up scenarios: I am writing them up as an user of the application, so I need to provide all the details that the user would be aware of, and not much more than that. The rest can be handled by the code behind it.


1

Any time you group tests into a suite of tests you are likely grouping along some category. For example, we have several suites of functional tests that get ran against every daily build. We also have other tests for security, battery, reliability (non-functional categories). Some of these suites get ran less frequently. Although we don't necessarily ...


1

There are several reasons to organize related groups tests into different test suites. Purpose - the fundamental purpose of a functional test is different then the purpose of a performance test, or a stress test, etc., and thus provide us with different information about the product being tested. Scheduling - tests organized into suites enable the test ...


1

Yes, there is a point about separating performance from functional, "If it does not work for one, then it will never scale to many..." The skills needed to implement manual or automated functional tests and performance tests are very different from a tester perspective. Performance testers have a very different view of the system and its architectural ...


1

I'd see it as more of a tracking issue, I haven't done much non-functional testing (at least I wouldn't call it that) although I could look back and probably say we covered that in some other kind of testing. Mostly when working on a project I look at the Functional Requirements and start basing tasks/tests/checks off that, anything not in the requirements ...



Only top voted, non community-wiki answers of a minimum length are eligible