Tag Info

Hot answers tagged

31

I think it's difficult to make through-the-UI tests reliable. The challenge comes down to the difficulty of reliably controlling and observing the variables that matter to your tests. Whether this is worthwhile depends on your ability to make your test code more resilient, and on the value of being able to run the tests automatically. Asynchrony. For web ...


30

Should QA push for programmers running QA's automated tests? Yes, but I'd also suggest that if the programmers don't seem to, it's probably more productive to find out why before pushing harder. What's preventing them? Access to proprietary test tools? Difficult to setup easily on dev environment? Results not meaningful or easy to read? Too slow to ...


30

No. You can't automate everything. You can't automate people's reactions (emotions) to your software. You can't automate things you don't think of. (eh? eh?) You can't automate users' thought patterns going from feature to feature. You can't run a "fun factor" metric (applies to games more than enterprise software, but still.) And that's just in 2 ...


25

My real world experience of running test suites of 1000+ tests every day on large web systems is that your hunch is right and that they don't find that many bugs. But what they do do is two key things: They free up the valuable time that testers would otherwise have to spend regression testing to do exploratory testing which does find bugs. They give you ...


24

In addition to what everyone else has said, it's absolutely realistic. With respect to what the hiring agencies are telling you, here are some reasons you could find difficulty convincing a developer that they want to be part of the test team: if your automation specialists are paid significantly less than your developers, you'll be asking any developer ...


21

If test automation does the job it is intended to, when a defect is introduced, it should be found and fixed very quickly, hopefully within a single daily cycle. If this is how the automation suite is working, then automation should become a "barrier" that prevents new bugs from being created in code that is already working and tested. When you think of it ...


20

It's not actually a true "either this or that" question since you can actually use both together via watir-webdriver. Firstly: If we are talking about testing at the browser level, you can pretty much completely ignore the language used to run the server side. There's no C# remaining once the page is rendered to HTML+Javascript and sent over HTTP to the ...


19

I think attempting to automate everything without thinking about what you're actually going to test is a horrible (but unfortunately common) practice. My standard line is that you should automate 100% of the tests that should be automated. Figuring out which tests to automate (or not to automate) is tha hard part. Testers frequently waste time attempting ...


19

A hard coded sleep statement is generally supposed to represent some sort of mocked delay in the application that doesn't exist during testing. The harmful aspect of this is that a hard coded value can't represent the complexity that actually exists in what its mocking. Take for example a network delay. Your production system usually takes 4 seconds to go ...


18

To answer your question, yes, if someone has the skill then QA should code tests. Beyond the simple question is a bigger issue. As a professional test engineer you need to know when automated/coded tests are right to use on a project. Do you need full on automation, a quick script, some home-grown tool or do you need a manual test plan executed by a ...


16

We run unit tests prior to check-in and rerun them as part of a larger test suite on each daily build (feature branch) and weekly build (main branch; multiple feature branches aggregated). Some advantages we have found include: Fewer build breaks, and virtually zero build breaks in main Additional test coverage for component/integration level testing ...


16

I work as a Manual tester on a big project. My project follows Agile methodology. I test a Web-based application. I sometimes ask my self , that am i giving any value to the stakeholders? So, the value that you are giving to stakeholders is the reduction in the amount of defects that go to production. The cost of fixing a defect drastically increases ...


15

I would suggest that you go back and do those unit tests when you are required to refactor that area of legacy code. The approach for do this is described in this question. When you are working with legacy code with no unit tests, its the same principle and techniques to add them regardless if you are doing TDD or not. I highly recommend Working ...


15

Yes. Over the last few years I have built, re-built and evolved testing tools for a number of things: A Windows autumation library on top of UIAutomation A full C# based testing stack built on top of Watin and the UIAutomation library Control generators Test case management systems Defect tracking systems Various intergraion tools to work with TFS, JIRA or ...


15

I think you're selling yourself short, here - you've said that in a language and skillset with which you're familiar and current you do well, but you're struggling with a language and skillset you have worked with in the past but where you've become rather rusty. That's normal. Seriously. Automation and programming are complementary skills but they're also ...


14

I definitely feel your pain. As noted in a question I had (I'll link in a minute) I too work for a 10-digit revenue company, and our primary software has 0 automated tests for over a million lines of production code. It boils down to the same philosophy that a lot of developers have about using external libraries rather than rolling their own: we fear what ...


14

> The integration tests can sometimes take a long time, > thus discouraging users from running the entire test > suite prior to checking in For the checkin-runs you can mark the long-running tests with their own category and tell the test-runner to exclude those long-runners You may also look at Is there a way to separate long running (e.g. ...


14

Any ideas for reducing test time for GUI tests? One tip that many seem to overlook is to ensure that all your tests are actually useful. Many times, tests decay over time - their usefulness fades to the point that the tests themselves are a waste of time, no matter how quick they are. I once joined an organization that ran automated overnight tests ...


13

I don't like the statement because it assumes that the only thing you can automate are regression tests. If you rewrote the statment as "Regression tests don't find new bugs", then I think it's accurate. I often write automation (e.g. performance, data driven, or model-based tests) that find new bugs - and I think the automation effort is incomplete without ...


13

There are plenty of testers who have learned to develop and there are also many testers who began their careers as developers. It is absolutely possible to find these kinds of people. There may be other hindrances however, such as availability in your area who have all of the skill-sets that you require, especially when it comes to specialized skill-sets. ...


12

You mentioned abstracting your test code from the framework. The page object pattern is one way that this can be done and is fairly common. It is popular with Selenium but can be applied to any UI automation: http://code.google.com/p/selenium/wiki/PageObjects. You can avoid using xpath that contains the entire path (or even part of the path) to the UI ...


12

In my experience ... Execute, YES. Maintain NO. Context for my comment. I am assuming Programmers in your question = developers of application, not testers with programming skills I think that there is an inherent motivation that testers, (even dedicated software engineers in test) have that developers don't, and that is that writing, running and ...


12

The truth is for a lot of people, there isn't a difference. A regression test is something that is going to check your test results against some requirements. Good examples of this are unit tests and functional tests. They tell you whether or not your application has regressed in its functionality (hence, regression.) Of course that assumes it was properly ...


12

The best place to start is over at http://www.thebraidytester.com/ where Michael Hunter describes the stack he built for testing Microsoft Expression in number of articles, papers and blog posts. I have used this approach as the inspiration for my own watin based stack which is available at http://testingstax.codeplex.com Essentially the framework breaks ...


12

You need to understand what are the core issues with these repeated breakages before you decide how to fix the problem. Relying on (more, improved, better, faster, etc) testing may not be the most effective solution here. It might not even help at all. Do you have code merge issues? Are fixes becoming undone because of that? Do you have new bugs ...


12

For hacking WebDriver without real production purpose, you need to find some task that really motivates you. Don’t forget, WebDriver – is not only about test automation, people use the tool (not so widely) for many different purposes: crawling some data from websites, semi-automating real job-related tasks. For instance, a girlfriend of friend of mine ...


12

1) Are there a difference between those ones? The Australian IT industry still views testing as a job that anyone can perform with minimal technical skills. A quick search on a leading recruitment site has a number of advertisements for Testers with requirements similar to the following: At least 3 years experience in a Tester or Test Analyst role ...


11

For Test Automation purposes, it's always been my belief that once you know 2 scripting languages, you are in very good shape. Being comfortable with 2 languages means you know how to automate, and that you know how to learn a new scripting language. As a hiring manager, I will be able to assume that you could easily pick up another new language as needed. ...


11

Palindromes testing is very representative in terms of QA way of thinking. Moreover, writing tests dramatically increase the quality of task specification. When a QA engineer writes test cases, it well may happen that certain case is not covered in initial specs. This is a good reason for QA to come up with idea to improve those specs. So, when you are ...


11

An answer - as Phil implied, is that you take every record and playback tool you can find, and burn them in a fiery pit of despair. I probably got myself a down vote for that, so I'll try to earn it back. Good Agile teams test constantly - not just at the end. If you include test design as part of feature design (and consider how the feature will be tested ...



Only top voted, non community-wiki answers of a minimum length are eligible