Tag Info

Hot answers tagged

4

As a general rule, retesting in the staging environment is done more as an integration/sanity test than a full retest because the QA environment will likely have different code (due to other changes that aren't being pushed to the staging environment yet). The usual considerations are: The staging environment is kept as close to production as possible, ...


3

If you are talking about unit tests I would consider moving the long running tests to a different test suite. If they are taking a long time they probably aren't unit tests and should not be treated as such. These tests should be moved to the location where you run integration tests. If that's not what your looking for you could also look into running ...


3

What you want is called an continuous integration (CI) server. A very popular one is called Jenkins (jenkins-ci.org), though there are others both open source and commercial. You can configure it to watch for checkins, and to build a project. If the build succeeds, it can trigger tests to be run. If the tests pass, you can trigger other jobs such as ...


3

What do you think? Should you eat one meal at a time and 2 to 3 meals a day? Or should you eat sufficient meals for 30 days at once and enjoy the remaining 29 days? Keep things as simple as possible! Combining everything and testing it all at once may make things too complicated to handle...


2

Similar question posted on Software Testing Club Robotium looks like it help fulfill some of your needs? (props to Stephen Janaway)


2

In recent versions of Windows, Microsoft decided to give services their own hidden desktop, and so even if you check "interact with desktop" and running as current user, the service still is referring to a hidden background desktop. There is no way to change this, as Microsoft did this on purpose. So, for example, to run Selenium tests from a executor, ...


2

You want to know why they do this and or if it is justified. It is difficult to answer without knowing more about the project length, product, technology and deployment environment and the acceptable defect goals. For instance there could be a number of valid reasons. It may be the exact same code, but the deployment process may be unreliable. It may also ...


2

If you're looking for TestComplete training, you probably want to look at the SmartBear website (http://www.smartbear.com). They have a number of webinars available at any time. There's also more intense training available: http://smartbear.com/products/qa-tools/automated-testing-tools/features/training-certification/ I've never done any of the training ...


2

We use Jenkins to orchestrate our CI and Continuous Delivery. This training course looks pretty good: http://www.cloudbees.com/training.cb


2

Selenium Builder is the evolution of the Selenium IDE. Its tooling page has a video (11 minutes) about using it with Travis-CI. Although I haven't watched it I expect it will answer most of your questions. To simplify test building (for non developers) I would suggest using something like Cucumber or another BDD framework, you can then make a lot of ...


2

It really comes down to overall reliability and speed. If you can guarantee that you have 24/7 access to your 'test server' via the Internet, with guaranteed > 10Mbps download/upload speeds, and you can protect 100% from external attacks (e.g. DoS) then the Internet might be the way to go. But, if you are like most of the world that experiences ...


2

It should be hosted where everyone who needs to test can access it. Important is that everyone can test in isolation, you don't want different processes or users interfering with each others tests runs. As you tagged this question with "continuous integration" I would like to add that for automated tests I love to spin up a test environment from scratch so ...


2

The most efficient tool for testing the appearance of a website is still the human eye and brain. That said, if you want or need to automate, image comparison is probably the least effective method because even with fuzzy logic it can generate far too many false positives. All it takes is a change in hardware (a different monitor or video card - this will ...


1

To comment on exactly how to write that...sure you can "code" whatever you want to code, but if you are looking for simplicity and repeatability with minimal work... I would recommend Sikuli as it will use image recognition so it won't matter what VM it is as long as it kicks it off the same way. It can also execute any test just like you would manually. ...


1

Yes, we do just that. We use Python and Coverage. I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path. OK, after comment from OP: yes, it is valid reason to ...


1

Selenese scripts generated by IDE are HTML without parameters, unlike traditional scripting/programming. What programming language is main language in your organization? Python is very easy to learn and has good Selenium bindings. Also, learning general programming language (Python) could be interesting career-enhancing move for your testers (so they ...


1

If your team members don't have much development experience and you would like to use Selenium Webdriver for your automation tests I highly recommend taking a look at Robot Framework. Robot Framework is a generic framework to which you can easily plug in external libraries and script your automation tests using keywords. There is Selenium Library which ...


1

Repeating all bug fix verification in stage seems like over-kill to me. One question, have you ever seen a fix fail in stage that passed in the QA environment? If this is a common problem, you should look at improving your QA environment (assuming stage is pretty close to production configuration). One class of bug fixes that we do verify in both QA ...


1

It depends on the test framework you are using. The frameworks I know allow you to define your own assertions. With that, you could define a time based assertion like assertSpecifiedBefore(<specification>, <date>), which checks the last modification date of the specification is not newer than the date of test creation (<date>).


1

Take a look at Microsoft Test Manager. Although not dependent on .net, it is fully integrated with Team Foundation System - so if you're not using TFS for work item management / scrum process - it will not work for you. It does allow you to define standard and custom relationship and you can create your own transitions to invalidate tests if that is what ...


1

Figured it out. The host IP in the VM configuration window in Jenkins has to be the IPv4 address of the host-only adapter (in this case, 192.168.120.1). I also changed the host port in the port forwarding rule from 3022 to 22. For historical reference, the username and password fields (which appear when the "Advanced" button under the "Host" field in the ...


1

You might start with Jenkins. It is free, open source, and popular, and it has a lot of extension points. You can use it to launch arbitrary shell or script files. A moment ago, I Googled for "Jenkins .NET" and found several articles about using Jenkins to install/run .NET applications.


1

We have this same problem with Ranorex and Teamcity. Running on a build agent that runs as a service is ok, but screenshots wouldn't come through right. What we ended up doing was logging in at console and running the build agent at console and running tests on that build agent. It's not an ideal solution, but it's solved all of our problems, so it's ...


1

Have you considered using a free (or paid depending on your circumstances) Amazon EC2 account?


1

I am running Watir scripts from Jenkins CI on a couple of projects. I do plan to write about it at my blog soon. If you have specific questions, please ask. I have recently recorded a screencast about the entire test automation solution I have developed for a client: Test automation at HomeSwap.com


1

A quick Google shows plenty of people doing this, including This page from WatirMelon ( if you dont know of this site could be worth your while reading it, lots of good stuff on it ) and an example of using ci_reporter is here


1

According to this blog post it is not possible to run tests from xcodebuild but there is a work around. This problem exists for xcode 4.0 - 4.3, I'm not sure about 4.4


1

You can try Railsonfire (Full disclosure I am one of the founders) We provide Continuous integration and Deployment for Ruby apps as a Cloud Service. Simple setup in < 2 minutes. If you need any help setting it up write me an email to flo@railsonfire.com, a tweet to @Railsonfire or use the olark chat box at the bottom of our site. If you do want to use ...


1

I think ci-infrastructure is what you are looking for. "Getting a fresh Jenkins server set up is kind of a pain and because of that, fewer projects have one than should. This project is designed to make it easy for people to fork and get set up with their very-own Jenkins server in the cloud because CI is awesome. "


1

I recently looked for the same thing and after not finding anything wrote my own implementation. I took the approach outlined in this blog: http://www.testingwithkunal.com/automated-deployment-validation-using-net/ and from there I added the ability to test for some additional things outlined in my response to this question: Deployment Testing



Only top voted, non community-wiki answers of a minimum length are eligible