Hot answers tagged

6

The short version: Don't sweat on automation or test cases. Chances are you won't get time to build it because there's too much that has to be manually tested now. Instead focus on building light-weight, easy to maintain documentation of the key elements of the software you're testing. The longer version: This is my world. I'm the sole tester in a team of ...


5

What do you think? Should you eat one meal at a time and 2 to 3 meals a day? Or should you eat sufficient meals for 30 days at once and enjoy the remaining 29 days? Keep things as simple as possible! Combining everything and testing it all at once may make things too complicated to handle...


4

What you want is called an continuous integration (CI) server. A very popular one is called Jenkins (jenkins-ci.org), though there are others both open source and commercial. You can configure it to watch for checkins, and to build a project. If the build succeeds, it can trigger tests to be run. If the tests pass, you can trigger other jobs such as ...


4

As a general rule, retesting in the staging environment is done more as an integration/sanity test than a full retest because the QA environment will likely have different code (due to other changes that aren't being pushed to the staging environment yet). The usual considerations are: The staging environment is kept as close to production as possible, ...


3

If you are talking about unit tests I would consider moving the long running tests to a different test suite. If they are taking a long time they probably aren't unit tests and should not be treated as such. These tests should be moved to the location where you run integration tests. If that's not what your looking for you could also look into running ...


3

Welcome to SQA, I think you've got a few questions going on, so I'll try and answer them in sections. Whats Travis/Continous Integration Travis-CI is one of a number of build automation tools that help developers understand if their application is working by building, testing and reporting on the build sequences the developer has configured. Frequently ...


2

Similar question posted on Software Testing Club Robotium looks like it help fulfill some of your needs? (props to Stephen Janaway)


2

We use Jenkins to orchestrate our CI and Continuous Delivery. This training course looks pretty good: http://www.cloudbees.com/training.cb


2

The most efficient tool for testing the appearance of a website is still the human eye and brain. That said, if you want or need to automate, image comparison is probably the least effective method because even with fuzzy logic it can generate far too many false positives. All it takes is a change in hardware (a different monitor or video card - this will ...


2

I think the Op is asking two questions: If, in the version control system, the code in QA is tagged (or branched) separately from code in development, which version should you use with the Selenium tests? Where should the Selenium tests run: on the developer's desktop or in the QA environment? I don't have enough organization/business/technology ...


2

When it comes time to complete a roll out we do the following. Its important to note this phase is ran on a previous version test template with no previous data. We are also in the process of pulling a portion of our regression suite to automation. Shallow exploratory. Test basic navigation and control functionality per page. Deep Exploratory. Similar to ...


2

Apologies in advance for a quick/simplistic answer - it's crunch time here! My current client have automated around 80% of regression testing, the rest is done manually with an offshore team, where the basic rule of thumb is: "If you change B, test A and C" i.e., everything around the change - there's no point in your regression team (or the automated ...


2

There is still an open ticket to get the enhancement you want. To elaborate on Adurbe's answer: You could drop a .sh file onto your server and then call it via Bamboo. Check out this StackOverflow question and look over the answers to find what will be best for you.


2

In recent versions of Windows, Microsoft decided to give services their own hidden desktop, and so even if you check "interact with desktop" and running as current user, the service still is referring to a hidden background desktop. There is no way to change this, as Microsoft did this on purpose. So, for example, to run Selenium tests from a executor, ...


2

Ouch. No, the old school sort of test plan doesn't make much sense in your context. The only reason I can see for doing them is if you had a contractual requirement with a client to deliver them. Will anyone read them except you? (If no - then stop doing them now!) If you re-read them, what do you use them for? To remind you of how to do something (is it ...


2

If you're looking for TestComplete training, you probably want to look at the SmartBear website (http://www.smartbear.com). They have a number of webinars available at any time. There's also more intense training available: http://smartbear.com/products/qa-tools/automated-testing-tools/features/training-certification/ I've never done any of the training ...


2

You want to know why they do this and or if it is justified. It is difficult to answer without knowing more about the project length, product, technology and deployment environment and the acceptable defect goals. For instance there could be a number of valid reasons. It may be the exact same code, but the deployment process may be unreliable. It may also ...


2

It should be hosted where everyone who needs to test can access it. Important is that everyone can test in isolation, you don't want different processes or users interfering with each others tests runs. As you tagged this question with "continuous integration" I would like to add that for automated tests I love to spin up a test environment from scratch so ...


2

It really comes down to overall reliability and speed. If you can guarantee that you have 24/7 access to your 'test server' via the Internet, with guaranteed > 10Mbps download/upload speeds, and you can protect 100% from external attacks (e.g. DoS) then the Internet might be the way to go. But, if you are like most of the world that experiences ...


2

Selenium Builder is the evolution of the Selenium IDE. Its tooling page has a video (11 minutes) about using it with Travis-CI. Although I haven't watched it I expect it will answer most of your questions. To simplify test building (for non developers) I would suggest using something like Cucumber or another BDD framework, you can then make a lot of ...


2

You could try TestComplete - I believe we use it to test installation here. Essentially you're just testing driving another windows application: if the environment can let you check registry entries and file content then so much the better (I don't work in TC at the moment and can't recall). AutoIT could do it (I have used AutoIT a lot).


2

I would only run the tests on a code change, better on each code change and not just in the night. You want to know which code change broke what test. In your case this would mean when you branch the code to a production version (which is a sort of code change) you run all the tests one more time and that is it. Only when you hot-fix the production code you ...


1

I asked a similar question awhile back. Perhaps the answers located here may also be of assistance as you evaluate what would be the best fit for your organization. Based on the feedback I received from stackexchange, I modified our process to a mini-regression suite at every release. It tests only the showstopper level features. If I don't find bugs, I can ...


1

It depends on the test framework you are using. The frameworks I know allow you to define your own assertions. With that, you could define a time based assertion like assertSpecifiedBefore(<specification>, <date>), which checks the last modification date of the specification is not newer than the date of test creation (<date>).


1

Take a look at Microsoft Test Manager. Although not dependent on .net, it is fully integrated with Team Foundation System - so if you're not using TFS for work item management / scrum process - it will not work for you. It does allow you to define standard and custom relationship and you can create your own transitions to invalidate tests if that is what ...


1

We have this same problem with Ranorex and Teamcity. Running on a build agent that runs as a service is ok, but screenshots wouldn't come through right. What we ended up doing was logging in at console and running the build agent at console and running tests on that build agent. It's not an ideal solution, but it's solved all of our problems, so it's ...


1

Solution is here test: override: - case $CIRCLE_NODE_INDEX in 0) [Your action in first container] ;; 1) [Your action in second container] ;; esac: parallel: true


1

Your question is valid only for the first integration, where you make the first build. You should make a build every time you add something new...integrate and test as often as you can and you will get information.


1

Yes, we do just that. We use Python and Coverage. I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path. OK, after comment from OP: yes, it is valid reason to ...


1

To comment on exactly how to write that...sure you can "code" whatever you want to code, but if you are looking for simplicity and repeatability with minimal work... I would recommend Sikuli as it will use image recognition so it won't matter what VM it is as long as it kicks it off the same way. It can also execute any test just like you would manually. ...



Only top voted, non community-wiki answers of a minimum length are eligible