Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I am currently working on planning a test strategy for test automation for our mobile apps. The problem is that all the automation frameworks I've seen utilize a slightly different context than what the application really uses.

For example, I am evaluating SeeTest. To run automation tests on an non-jailbreak device the developer needs to add some frameworks to the project and actually build a different app the release app. So, when testing an instrumented iOS app with SeeTest, the app under test is not the same as the released app, which means that a full regression should be made on the released build.

This is the same of every framework I have seen.

How did others solve this issues?

share|improve this question
add comment

4 Answers 4

I worked at a highly-respected mobile development shop for 6 years. We did not trust automation or emulators as acceptable tools to finalize a build. Each build was manually tested on the actual handset it was targeted for before the build was sent to the customer.

That's the problem with mobile development. Unless you find a framework that will run automation using the actual device, you won't have 100% confidence in your build, as you will never really test the deliverable build.

share|improve this answer
    
Both Android and iOS have automation frame work that runs on the actual device. –  Guy Dec 18 '12 at 8:06
add comment

If you are talking iOS, you should look into the automated testing support supplied by apple in their tool called instruments.

share|improve this answer
    
I know about instruments and iOS automation FW. Android also has such FW. The problem is that both tests in a slightly different context then the real world. it is like the schrodinger's cat paradox... –  Guy Dec 18 '12 at 8:10
add comment

Have you looked into Device Anywhere and Perfecto Mobile? They are both services for testing against real devices. A warning: These aren't cheap solutions.

share|improve this answer
add comment

We at pCloudy.com also offer access to real devices using just a browser and we do that at affordable rate. All our devices are never jailborker/rooted, we also support Robotium automation.

Disclaimer: I am co-founder of pCloudy.com

share|improve this answer
    
This doesn't answer Guy's question. How will this help him avoid instrumented apps? –  Sam Woods Jan 28 at 16:30
    
Just another adversting. Your answer could be more useful. –  amazpyel May 14 at 10:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.