Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
Lately I keep seeing articles about how artificial intelligence will revolutionize how quality assurance is done (one simple example here. There are even online courses online for AI in Software Testing.
Is AI in software testing a real thing - in terms of creating test cases, creating the appropriate tests and executing them?
It is to be defined probably in such a way that the workflows build on each other and become so more and more learnable. It aims at test automation.
Bringing AI into Quality Assurance
AI-led cognitive automation solutions (Intelligent Automation) combine the best of automation approaches with AI and help bring superior results. The focus is three dimensional – to eliminate test coverage overlaps, optimize efforts with more predictable testing and lastly to move from defect detection to defect prevention. Today, organizations have better machine learning algorithms for pattern analysis and processing huge volumes of data that result in better run-time decisions. For instance, during a software upgrade, machine learning algorithms can traverse the code to detect key changes in functionality and link them to the requirements in order to identify test cases. This helps optimize testing and prevents the making of decisions on hot spots that could lead to failure. Infosys PANDIT is one such AI-based testing platform that is helping our clients improve agility and predictability while optimizing efforts in testing by integrating AI in testing.
further information https://www.infosys.com/insights/ai-automation/Pages/quality-assurance.aspx
In many cases today Qualit Assurance is used in many processes and of course in CI, TDD, BDD as an AI that is capable of learning, but draws its experience from the networked process flows.
Facebook has implemented a predictive test selection to estimate the probability of each test failing for a newly proposed code change. - https://code.fb.com/developer-tools/predictive-test-selection/?source=post_page---------------------------
Netflix is using reinforcement learning for test case prioritization and selection. - https://medium.com/netflix-techblog/lerner-using-rl-agents-for-test-case-scheduling-3e0686211198
Shown here using Ebay as an example
Ebay describes it that way in her article:
Deep Learning Technology
DL simulates the human way of finding errors or anomalies. Humans are driven by past experience and conditioning to make decisions. Machines with the proper application of training or conditioning can detect errors that surpass human precision.
We begin our understanding of DL as the subset of a broader class called as the supervised machine learning algorithm. The supervised learning algorithms take a set of training examples called as the training data. The learning algorithm is provided with the training data to learn a desired function. Further, we also validate our learning algorithm by a set of test data. This process of learning from training data and validating against test data is called modeling.
In the further course we explain among other things how an AI operated GUI test looks like:
Facebook describes their approach as follows:
Why using build dependencies is inefficient
A common approach to regression testing is to use information extracted from build metadata to determine which tests to run on a particular code change. By analyzing build dependencies between units of code, one can determine all tests that transitively depend on sources modified in that code change. For example, in the diagram below, circles represent tests; squares represent intermediate units of code, such as libraries; and diamonds represent individual source files in the repository. An arrow connects entities A → B if and only if B directly depends on A, which we interpret as A impacting B. The blue diamonds represent two files modified in a sample code change. All entities transitively dependent upon them are also shown in blue. In this scenario, a test selection strategy based on build dependencies would exercise tests 1, 2, 3, and 4. But tests 5 and 6 would not be exercised, as they do not depend on modified files.
Shown in this example
Indeed! AI in Quality Assurance is a real thing. However, the entire idea of using AI in quality assurance is limited to automation. This at present includes defining the test cases and test suites based on the past test projects for their results and test specifications.
Since Artificial Intelligence is limited to its initial stages and has not reached the concept of artificial superintelligence. The present use of AI in the quality assurance process is more about running functional, performance, and regression tests that run on a defined path.
In other words, the use of AI technology in the quality assurance process is meant to improve the development and test process, allowing faster market releases and, of course, a save on money and resources.
Above all, AI has greater capabilities to determine any changes in the code or functionality as it is free from any kind of lethargy or human error and feeds only on digital requirements.
I won't share my opinion on AI in general, but will restrict myself to the advantages machine learning can bring to the field of quality assurance. Given the predictive nature of machine learning systems/algorithms, QA may use it to predict potential bugs/inconsistencies from requirements of the application. These predictions can be based on different perspectives like functional, user-point-of-view (application domain), security, performance, etc. Another area where AI can be helpful is in automating exploratory testing. These are the few areas in which software testing companies are working on. You may find the below links helpful,
https://blog.qasource.com/how-ai-improves-automated-testing-services
An AI-powered constant testing stage can distinguish changed controls adequately than a human, and with stable and nonstop updates to its algorithms, even the small measure of changes can be watched. With automation testing, Artificial Intelligence is being widely used in item application categorization for each UI (UIs).
Here, recognized controls are characterized when you frame tools & testers can pre-train controls that are ordinarily observed in out of the box setups. When the chain of controls is observed, skilled AI software testers can make a technical map to such extent that the artificial intelligence is looking at the GUI (Graphical User Interface) to get labels for the particular controls.
What are the advantages of AI in Software Testing?
It is next to impossible for the most QA/ Software dept to implement a controlled web application test with 1,000+ clients. With AI software testing, one can reenact tens, 100, or 1000s of virtual sets of clients that can cooperate with a product, a system, or web-based applications.
To err is a human creature. Indeed, even the cautious software tester will undoubtedly make mistakes while executing monotonous manual testing. This is the time where AI automated testing helps by performing the very same process or steps splendidly each time and failing to miss out on recording thorough outcomes. Testers freed from tedious manual testing steps have additional time to make new automated software tests and cope with complex features.
Even the expert software testers sometimes make mistakes. This is the key reason AI software testing became so popular. Unlike humans, Artificial Intelligence always performs the significant tasks exactly as planned, completing the same recurring tasks effectively, time after time. While AI works on tedious tasks, software testers can easily focus on creating effective automation solutions and on exploratory tasks that only humans can complete.
Shared automated tests can be used by software developers and designers to catch troubles quick before going to Quality Assurance. Tests can run automatically whenever source code changes are checked in and alert the group or the developer if in the event that not succeed. Features like these spare the valuable time of the developers and increase their confidence.
With software testing being repeated each time source code is customized, manually repetition of those tests can be truly time-consuming plus expensive too. In contrary, AI automated tests can be implemented again and again, with low to zero additional cost at a speedy pace. The time span of the software testing can be lowered from days to mere hours, which translates directly into cost cuttings. Even AI testing tools have helped to make software product releases and update that happen once a month to occur on daily or weekly basis.
Pattern recognition and image recognition allow Artificial Intelligence to discover visual bugs by performing visual testing of apps and ensuring that all the visual elements look & function appropriately. AI can distinguish dynamic UI controls despite of their shape and size, evaluating them on a pixel level.
Increase in Overall Test Coverage– With AI automated testing; one can enhance the overall scope and depth of tests bringing about an absolute improvement of software quality. AI software testing can investigate into program states, memory and file contents and data tables to decide whether the software product is working as it is expected to. All in all test automation can perform 1000+ distinctive cases in each trial providing coverage that is beyond the realm of imagination with manual tests.
Thus, regardless of whether a property of a component changes, the tests don’t fail; rather Artificial Intelligence identifies this issue and goes to the next best location strategy to effectively spot the component in the page.
