Tag Info

Hot answers tagged

5

What do you think? Should you eat one meal at a time and 2 to 3 meals a day? Or should you eat sufficient meals for 30 days at once and enjoy the remaining 29 days? Keep things as simple as possible! Combining everything and testing it all at once may make things too complicated to handle...


4

As a general rule, retesting in the staging environment is done more as an integration/sanity test than a full retest because the QA environment will likely have different code (due to other changes that aren't being pushed to the staging environment yet). The usual considerations are: The staging environment is kept as close to production as possible, ...


3

What you want is called an continuous integration (CI) server. A very popular one is called Jenkins (jenkins-ci.org), though there are others both open source and commercial. You can configure it to watch for checkins, and to build a project. If the build succeeds, it can trigger tests to be run. If the tests pass, you can trigger other jobs such as ...


3

If you are talking about unit tests I would consider moving the long running tests to a different test suite. If they are taking a long time they probably aren't unit tests and should not be treated as such. These tests should be moved to the location where you run integration tests. If that's not what your looking for you could also look into running ...


2

Similar question posted on Software Testing Club Robotium looks like it help fulfill some of your needs? (props to Stephen Janaway)


2

We use Jenkins to orchestrate our CI and Continuous Delivery. This training course looks pretty good: http://www.cloudbees.com/training.cb


2

The most efficient tool for testing the appearance of a website is still the human eye and brain. That said, if you want or need to automate, image comparison is probably the least effective method because even with fuzzy logic it can generate far too many false positives. All it takes is a change in hardware (a different monitor or video card - this will ...


2

I think the Op is asking two questions: If, in the version control system, the code in QA is tagged (or branched) separately from code in development, which version should you use with the Selenium tests? Where should the Selenium tests run: on the developer's desktop or in the QA environment? I don't have enough organization/business/technology ...


2

When it comes time to complete a roll out we do the following. Its important to note this phase is ran on a previous version test template with no previous data. We are also in the process of pulling a portion of our regression suite to automation. Shallow exploratory. Test basic navigation and control functionality per page. Deep Exploratory. Similar to ...


2

Apologies in advance for a quick/simplistic answer - it's crunch time here! My current client have automated around 80% of regression testing, the rest is done manually with an offshore team, where the basic rule of thumb is: "If you change B, test A and C" i.e., everything around the change - there's no point in your regression team (or the automated ...


2

There is still an open ticket to get the enhancement you want. To elaborate on Adurbe's answer: You could drop a .sh file onto your server and then call it via Bamboo. Check out this StackOverflow question and look over the answers to find what will be best for you.


2

In recent versions of Windows, Microsoft decided to give services their own hidden desktop, and so even if you check "interact with desktop" and running as current user, the service still is referring to a hidden background desktop. There is no way to change this, as Microsoft did this on purpose. So, for example, to run Selenium tests from a executor, ...


2

If you're looking for TestComplete training, you probably want to look at the SmartBear website (http://www.smartbear.com). They have a number of webinars available at any time. There's also more intense training available: http://smartbear.com/products/qa-tools/automated-testing-tools/features/training-certification/ I've never done any of the training ...


2

You want to know why they do this and or if it is justified. It is difficult to answer without knowing more about the project length, product, technology and deployment environment and the acceptable defect goals. For instance there could be a number of valid reasons. It may be the exact same code, but the deployment process may be unreliable. It may also ...


2

It should be hosted where everyone who needs to test can access it. Important is that everyone can test in isolation, you don't want different processes or users interfering with each others tests runs. As you tagged this question with "continuous integration" I would like to add that for automated tests I love to spin up a test environment from scratch so ...


2

It really comes down to overall reliability and speed. If you can guarantee that you have 24/7 access to your 'test server' via the Internet, with guaranteed > 10Mbps download/upload speeds, and you can protect 100% from external attacks (e.g. DoS) then the Internet might be the way to go. But, if you are like most of the world that experiences ...


2

Selenium Builder is the evolution of the Selenium IDE. Its tooling page has a video (11 minutes) about using it with Travis-CI. Although I haven't watched it I expect it will answer most of your questions. To simplify test building (for non developers) I would suggest using something like Cucumber or another BDD framework, you can then make a lot of ...


1

I asked a similar question awhile back. Perhaps the answers located here may also be of assistance as you evaluate what would be the best fit for your organization. Based on the feedback I received from stackexchange, I modified our process to a mini-regression suite at every release. It tests only the showstopper level features. If I don't find bugs, I can ...


1

It depends on the test framework you are using. The frameworks I know allow you to define your own assertions. With that, you could define a time based assertion like assertSpecifiedBefore(<specification>, <date>), which checks the last modification date of the specification is not newer than the date of test creation (<date>).


1

Take a look at Microsoft Test Manager. Although not dependent on .net, it is fully integrated with Team Foundation System - so if you're not using TFS for work item management / scrum process - it will not work for you. It does allow you to define standard and custom relationship and you can create your own transitions to invalidate tests if that is what ...


1

We have this same problem with Ranorex and Teamcity. Running on a build agent that runs as a service is ok, but screenshots wouldn't come through right. What we ended up doing was logging in at console and running the build agent at console and running tests on that build agent. It's not an ideal solution, but it's solved all of our problems, so it's ...


1

Solution is here test: override: - case $CIRCLE_NODE_INDEX in 0) [Your action in first container] ;; 1) [Your action in second container] ;; esac: parallel: true


1

Your question is valid only for the first integration, where you make the first build. You should make a build every time you add something new...integrate and test as often as you can and you will get information.


1

Yes, we do just that. We use Python and Coverage. I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path. OK, after comment from OP: yes, it is valid reason to ...


1

To comment on exactly how to write that...sure you can "code" whatever you want to code, but if you are looking for simplicity and repeatability with minimal work... I would recommend Sikuli as it will use image recognition so it won't matter what VM it is as long as it kicks it off the same way. It can also execute any test just like you would manually. ...


1

Figured it out. The host IP in the VM configuration window in Jenkins has to be the IPv4 address of the host-only adapter (in this case, 192.168.120.1). I also changed the host port in the port forwarding rule from 3022 to 22. For historical reference, the username and password fields (which appear when the "Advanced" button under the "Host" field in the ...


1

You might start with Jenkins. It is free, open source, and popular, and it has a lot of extension points. You can use it to launch arbitrary shell or script files. A moment ago, I Googled for "Jenkins .NET" and found several articles about using Jenkins to install/run .NET applications.


1

The book How Google Tests Software (chapter 2, "Test Execution" section) describes how Google categorizes tests by size: small, medium, large, or enormous. The size takes into account not only how long the test takes but also the extent of its dependencies. For example, small tests should run into under 100ms and use extensive mocking to minimize ...


1

This is one reason why it is important to know how long each test takes, and to refactor if necessary. A lot of times folks will just continue to pile on functionality into a test, use unnecessary sleeps throughout their code instead of events or polling loops, or just add new (sometimes redundant tests) to an already over-bloated automated test suite. In ...


1

You can try Railsonfire (Full disclosure I am one of the founders) We provide Continuous integration and Deployment for Ruby apps as a Cloud Service. Simple setup in < 2 minutes. If you need any help setting it up write me an email to flo@railsonfire.com, a tweet to @Railsonfire or use the olark chat box at the bottom of our site. If you do want to use ...



Only top voted, non community-wiki answers of a minimum length are eligible