How to test and ensure the quality of an Artificial Intelligence system?

Question

In deterministic (software) systems we have a set of business requirements and such a system can be fully defined. Meaning that given enough resources of time and relatively low complexity we can map all scenarios and inputs to expected outputs. Meaning that even usability testing, compliant testing, stress testing, endurance testing, robustness testing and all other kinds of testing can be fully defined to be part of requirements and thus be converted to functional testing. Meaning that we can be as strict as we want, since the system is deterministic.

So given that AI systems are probabilistic, that you have no real output, but only predictions that can be more or less useful (but never fully correct) and that the inputs of the model change over time because the world changes in general and the users are sending data which were out of the ordinary at the moment when the model would train.. And given the above and all other aspects of machine learning how does one test effectively and detect difference between required and actual behaviors of Artificial Intelligence systems ?

Answer 1

Attempting to answer my own question by splitting the lifecycle of an Artificial Intelligent system to its steps and trying to see how a QA who supports a data science / machine learning team can help to ensure the quality on every step.

Note that each of the bullets below is not a single scenario but rather a whole section of possible scenarios

• Context
  • Ensure that there are clear specifications and defined requirements before proceeding with any other testing
• Collecting training data
  • Ensure data has a variety of sources and necessary variety to avoid biases
  • Ensure that after cleaning enough large dataset has remained
  • Ensure features are sane and within the expected range after cleaning
  • View training data and sample them by eye to see if they make sense
  • Write rule based scripts to check if what is generally expected is found within training data
  • Ensure that training data represent the targets/outputs in as much the same portion as possible
• Testing data
  • Ensure that test data are not merely a sample of the training data but at least some of them reflect the business goals (defining expected outcomes as test oracles) and are characteristic examples
  • Ensure that testing data are used only once and then are thrown away otherwise they will be used for the next model
  • Ensure that testing data, even smaller in size, are still a representative portion of the training data
  • Ensure testing data represent the very latest samples that we expect and reflect at least the near future
  • Ensure that the system is tested against totally random inputs (noise) and it is returning outputs that are of low certainty
  • Ensure that using GAN-based metamorphic approaches ([18] PDF - arxiv.org ) will test the AI system using inputs from the same space as the original data
  • Ensure that QAs will have generated by hand a few new test cases and have manually set (using their brain) the expected output
  • Ensure that past scenarios executed in production by real users can be replicated fully to be used as test-input
  • Robustness: refers to the resilience of an AI component towards perturbations
    • Ensure that small variations, perturbations, in the testing sample will yield similar output to the original and will not yield highly different results (ensure non high variance)
• Model wise
  • Ensure that a baseline model is always there to compare against
  • Ensure that the proposed model performs better than the baseline model
  • Ensure that the new proposed model performs better than the latest proposed model
  • Ensure that easy to create dummy models using Naive bayes for classification or Linear Regression for regression will not perform better than the proposed model
  • Ensure that a low cost to create rule-based, non ai, model will not work better than the proposed model
  • Ensure that the model should also provide the probability of the certainty of the model that the output is a good/average/bad prediction
  • Ensure that the model is non polarized for a few parameters and therefore non prone to AI-attacks (where some inputs are being changed and change the entire output to our own wish)
  • Ensure that an ensemble model, is not overfitting and it works as good or better than any of the individual underlying models
  • Ensure that using a Teacher-Student model, that the Teacher is slower yet more accurate model than the Student which is expected to be less accurate but more efficient
  • Ensure that self-adaptive and self-learning systems (e.g. Reinforcement Learning) will be able to self-assess themselves to make sure that they are not making
  • Interpretability
    • Ensure that using the training data to build an interpretable model that fits the predictions of our large model, then the interpretation of the parameters make sense
    • Ensure that the model is making predictions based on parameters that the current theory supports and does not have any weird pattern which might lead wrong model
• Checking output qualitatively
  • Ensure that the output of the model for very high probability of certainty are truly delivering a good answer
  • Ensure that the bad answers of the model are handled in such a way that the user retains his/her trust to the overall system instead of being misled
  • Ensure that the model generates output that is aligned with the business goals and these answers are useful to the user
• Performance / Efficiency
  • Ensure that the model generates answers fast enough in order for the user experience to not be severely impacted by them
  • Ensure that the time to train the new model will not need so large time as to miss the deadlines
  • Ensure that minimal resources are provided to AI models which are being under development in comparison to the AI model which is in production and that these are separated without having one (test/staging environment) consuming resources from the other (production)
• Production monitoring
  • Ensure that a feedback system have been set in place in order for users to be able and report unwanted or misleading output of the AI
  • Ensure that the feedback reported by the users is significantly high
  • Ensure that the measured error of the system while in production is within the acceptable levels similar to the ones that were measured during the execution of the model to the testing data
  • Ensure that the measured error of the system remains steady as new inputs are being received and does not have a declining trend
• User output
  • Ensure that the output of the model and its certainty probability are reflected correctly in the app
  • Ensure the using as input an instance which is very far away from the current distribution of the model will not allow the user to proceed with using the AI system
  • Ensure that having as output a prediction that has a low certainty will provide the user manual or rule-based alternatives to accomplish his/her tasks
• Data privacy: refers to the ability of an AI component to preserve private data information
  • Example: Having a chatbot and having it accumulate knowledge for a certain user, asking this language model information regarding some other user, should not be delivered. Each language model should be agnostic of other language models
• Security: measures the resilience against potential harm, danger or loss made via manipulating or illegally accessing AI components
  • Ensure that process of AI model is transparent and that there is a history of the changes that have happened to the deployed AI model
• Fairness: Avoid problems in human rights, discrimination law and other ethical issues
  • Ensure that the model output will comply to some "values" which are coded in rule based scripts
  • Example: A Sentiment analysis to never produce that the output of a language model will be very negative

Answer 2

@George, All and all it would be like:

1. You have to actually verify that the training data does a good enough job of accurately classifying or regressing data with sufficient generalization without overfitting or underfitting the data
2. This is done using validation techniques and setting aside a portion of the training data to be used during the validation phase.
3. In essence, this is a sort of QA testing where you’re making sure that the algorithm and data together in a way that also takes into account hyperparameter configuration data and associated metadata all working together to provide the predictive results you’re looking for.
4. If you get it wrong in the validation phase, you’re supposed to go back, change the hyperparameters, and rebuild the model again
5. Need less to say,
   • This is continuous process
   • Can not be completed without verifying in and with production data

Answer 3

Many outsourced software testing companies strongly believe that the testing of Artificial Intelligence softwares is completely different from testing any other software. This is so because these Artificial Intelligence software systems are expected to assure the standard attributes such as performance, reliability, usability, and security, other than establishing ethical behavior. Only after fulfilling these criteria, they can be deployed.

Issues and Possible Complications:

Problem: Enormous volumes of data collected is tedious to eliminate and streneous to store. Solution: To automate the matching data within the Artificial Intelligence system so that data is categorized and can be stored more competently. We acne update the machine learning algorithm so that it can distinguish and incorporate all collected data through predictive analytics.

Problem: Data may be collected during sudden events thus making it difficult to collect and use for training purposes. Solution: Modify the programmed algorithm within the Artificial Intelligence machine to regroup the collected data during such sudden events so the Artificial Intelligence system recognizes this set of data as dissident behavior throughout training.

Problem: Human bias may happen in training and testing data sets. Solution: We can execute technical tools and operational exercises that are designed to examine the neutrality of data sets.

Main components in Artificial Intelligence Testing are: Validating data: The important component of successful AI testing is having a good data. Before testing AI systems, the data should be scrubbed, cleaned and validated. We need to be cautious for any human bias data and variety that could intensify the system’s illustration of the data.

Performance and Security Testing QA for AI systems requires intense performance and security testing as well as regulatory compliance testing.

Training Data and Testing Data QA for Artificial Intelligence testing relies on the given training data, the set of accurate data trains the system’s model. This is mainly prepared by composing data in a semi-automated way. Testing data is a subset of the training data, it is built to test all the possible amalgams and determine how well the model is trained.