we are currently focus on the Mobile Testing using Appium as a mobile testing tool (with our own framework based on Appium) and using Jenkins and Android/iOS emulators/simulators to launch the test using CI.
We prefer using open source tools (such as Jenkins instead of TravisCI for example) and we wanted to create/configure and maintenance our own environment.
The problem is that the Android emulators and iOS simulators are not working as good as we expect:
- We need to check the update of the Android and iOS images.
- We need to check, clean and maintain all the emulators created.
- We investigate how to create a new emulator using command line tools everytime that we need to execute the test from a project and detele it after, but create a clean image it's a long time, waiting to be ready also it's a huge amount of time, and close and remove it can failed sometimes.
- We had problems with the full reset/no reset capability from Appium mostly on iOS (because on Android and iOS, the capability is not the same).
- We create a custom image and use it with a different emulator everytime we start a test execution, but still, it's a huge amount of time.
- Create one emulator per job (only for an app) it's a huge amount of resources.
- the idea of using several real devices its expensive than solution like Saucelabs (but we prefer having our CI environment ourself and not in the cloud).
Anyone here is using a CI strategy on its company using Appium and Jenkins for mobile testing in Android and iOS? Can you provice any kind of feedback for me and help me decided which could be the best option to move forward?
Have you decided to create docker containers and introduce kubernetes in your solution or not? why or why not?
Thank you in advance.