The task is to measure changes in performance of a system during daytime execution.
Test system description
Tests are performed on java desktop application which is started with cmd command:
java -cp [AUT dependencies, .jars] [main app class]
Interaction with application GUI is done with jemmy library so test project is started like:
java -cp [AUT & testng dependencies/jars] org.testng.TestNG testng.xml
And application is launched in the code of tests:
new ClassReference("[main app class]").startApplication();
So test project is a part of application and the question is how much test project influence performance of AUT?
Test execution
Tests are executed every 2 minutes within a single run of testng command. In testng.xml file <test>
blocks a repeated so that execution would not restart test (as well as AUT). But after ~40 minutes of execution testing fails because of wait timeout expiration. Here is a processor & RAM load graph:
And heap usage dynamics:
------------------------------------------------------------------------------------
| || Exec time [R# - Run number] [S/F/C - at run start/finish/crash] |
------------------------------------------------------------------------------------
| [MB] || R#2 S | R#2 F || R#10 S | R#10 F || R#21 S | R#21 F || R#22 S | R#22 C ||
------------------------------------------------------------------------------------
| Max || 455 | 455 || 455 | 455 || 455 | 455 || 455 | 455 ||
------------------------------------------------------------------------------------
| Total || 286 | 374 || 373 | 374 || 454 | 454 || 454 | 454 ||
------------------------------------------------------------------------------------
| Free || 123 | 240 || 86 | 94 || 67 | 51 || 70 | 50 ||
------------------------------------------------------------------------------------
R#2 - 04:00
R#10 - 20:00
R#21 - 42:00
R#22 - 44:00
Update:
Started testing with minimal number of actions in AUT (just clicking menu buttons). After an hour of execution heap usage was:
- maximum - 455
- total ~430
- free ~200
After 02h:00m:00-02h:20m of execution heap usage was:
- maximum - 455
- total - 455
- free ~35-70
After 02h:32m test failed:
- maximum - 455
- total - 455
- free - 53