Tag Info

Hot answers tagged

5

What do you think? Should you eat one meal at a time and 2 to 3 meals a day? Or should you eat sufficient meals for 30 days at once and enjoy the remaining 29 days? Keep things as simple as possible! Combining everything and testing it all at once may make things too complicated to handle...


4

As a general rule, retesting in the staging environment is done more as an integration/sanity test than a full retest because the QA environment will likely have different code (due to other changes that aren't being pushed to the staging environment yet). The usual considerations are: The staging environment is kept as close to production as possible, ...


3

What you want is called an continuous integration (CI) server. A very popular one is called Jenkins (jenkins-ci.org), though there are others both open source and commercial. You can configure it to watch for checkins, and to build a project. If the build succeeds, it can trigger tests to be run. If the tests pass, you can trigger other jobs such as ...


3

If you are talking about unit tests I would consider moving the long running tests to a different test suite. If they are taking a long time they probably aren't unit tests and should not be treated as such. These tests should be moved to the location where you run integration tests. If that's not what your looking for you could also look into running ...


2

Similar question posted on Software Testing Club Robotium looks like it help fulfill some of your needs? (props to Stephen Janaway)


2

We use Jenkins to orchestrate our CI and Continuous Delivery. This training course looks pretty good: http://www.cloudbees.com/training.cb


2

In recent versions of Windows, Microsoft decided to give services their own hidden desktop, and so even if you check "interact with desktop" and running as current user, the service still is referring to a hidden background desktop. There is no way to change this, as Microsoft did this on purpose. So, for example, to run Selenium tests from a executor, ...


2

You want to know why they do this and or if it is justified. It is difficult to answer without knowing more about the project length, product, technology and deployment environment and the acceptable defect goals. For instance there could be a number of valid reasons. It may be the exact same code, but the deployment process may be unreliable. It may also ...


2

If you're looking for TestComplete training, you probably want to look at the SmartBear website (http://www.smartbear.com). They have a number of webinars available at any time. There's also more intense training available: http://smartbear.com/products/qa-tools/automated-testing-tools/features/training-certification/ I've never done any of the training ...


2

The most efficient tool for testing the appearance of a website is still the human eye and brain. That said, if you want or need to automate, image comparison is probably the least effective method because even with fuzzy logic it can generate far too many false positives. All it takes is a change in hardware (a different monitor or video card - this will ...


2

It really comes down to overall reliability and speed. If you can guarantee that you have 24/7 access to your 'test server' via the Internet, with guaranteed > 10Mbps download/upload speeds, and you can protect 100% from external attacks (e.g. DoS) then the Internet might be the way to go. But, if you are like most of the world that experiences ...


2

It should be hosted where everyone who needs to test can access it. Important is that everyone can test in isolation, you don't want different processes or users interfering with each others tests runs. As you tagged this question with "continuous integration" I would like to add that for automated tests I love to spin up a test environment from scratch so ...


2

Selenium Builder is the evolution of the Selenium IDE. Its tooling page has a video (11 minutes) about using it with Travis-CI. Although I haven't watched it I expect it will answer most of your questions. To simplify test building (for non developers) I would suggest using something like Cucumber or another BDD framework, you can then make a lot of ...


1

Repeating all bug fix verification in stage seems like over-kill to me. One question, have you ever seen a fix fail in stage that passed in the QA environment? If this is a common problem, you should look at improving your QA environment (assuming stage is pretty close to production configuration). One class of bug fixes that we do verify in both QA ...


1

Yes, we do just that. We use Python and Coverage. I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path. OK, after comment from OP: yes, it is valid reason to ...


1

We have this same problem with Ranorex and Teamcity. Running on a build agent that runs as a service is ok, but screenshots wouldn't come through right. What we ended up doing was logging in at console and running the build agent at console and running tests on that build agent. It's not an ideal solution, but it's solved all of our problems, so it's ...


1

It depends on the test framework you are using. The frameworks I know allow you to define your own assertions. With that, you could define a time based assertion like assertSpecifiedBefore(<specification>, <date>), which checks the last modification date of the specification is not newer than the date of test creation (<date>).


1

Take a look at Microsoft Test Manager. Although not dependent on .net, it is fully integrated with Team Foundation System - so if you're not using TFS for work item management / scrum process - it will not work for you. It does allow you to define standard and custom relationship and you can create your own transitions to invalidate tests if that is what ...


1

Your question is valid only for the first integration, where you make the first build. You should make a build every time you add something new...integrate and test as often as you can and you will get information.


1

To comment on exactly how to write that...sure you can "code" whatever you want to code, but if you are looking for simplicity and repeatability with minimal work... I would recommend Sikuli as it will use image recognition so it won't matter what VM it is as long as it kicks it off the same way. It can also execute any test just like you would manually. ...


1

Figured it out. The host IP in the VM configuration window in Jenkins has to be the IPv4 address of the host-only adapter (in this case, 192.168.120.1). I also changed the host port in the port forwarding rule from 3022 to 22. For historical reference, the username and password fields (which appear when the "Advanced" button under the "Host" field in the ...


1

You might start with Jenkins. It is free, open source, and popular, and it has a lot of extension points. You can use it to launch arbitrary shell or script files. A moment ago, I Googled for "Jenkins .NET" and found several articles about using Jenkins to install/run .NET applications.


1

The book How Google Tests Software (chapter 2, "Test Execution" section) describes how Google categorizes tests by size: small, medium, large, or enormous. The size takes into account not only how long the test takes but also the extent of its dependencies. For example, small tests should run into under 100ms and use extensive mocking to minimize ...


1

This is one reason why it is important to know how long each test takes, and to refactor if necessary. A lot of times folks will just continue to pile on functionality into a test, use unnecessary sleeps throughout their code instead of events or polling loops, or just add new (sometimes redundant tests) to an already over-bloated automated test suite. In ...


1

You can try Railsonfire (Full disclosure I am one of the founders) We provide Continuous integration and Deployment for Ruby apps as a Cloud Service. Simple setup in < 2 minutes. If you need any help setting it up write me an email to flo@railsonfire.com, a tweet to @Railsonfire or use the olark chat box at the bottom of our site. If you do want to use ...


1

According to this blog post it is not possible to run tests from xcodebuild but there is a work around. This problem exists for xcode 4.0 - 4.3, I'm not sure about 4.4


1

I think ci-infrastructure is what you are looking for. "Getting a fresh Jenkins server set up is kind of a pain and because of that, fewer projects have one than should. This project is designed to make it easy for people to fork and get set up with their very-own Jenkins server in the cloud because CI is awesome. "


1

I recently looked for the same thing and after not finding anything wrote my own implementation. I took the approach outlined in this blog: http://www.testingwithkunal.com/automated-deployment-validation-using-net/ and from there I added the ability to test for some additional things outlined in my response to this question: Deployment Testing


1

No, there are no general purpose open source tools that could perform validation for you. Yes, there are a lot of small tools that could enable you to automate that validation. The main question is what you need to validate. Do you have manual check list already? Using any popular scripting languages: Perl, Python, Ruby, Jscript you could automate the ...


1

Have you considered using a free (or paid depending on your circumstances) Amazon EC2 account?



Only top voted, non community-wiki answers of a minimum length are eligible