Determining optimal testing times for Markov chain usage models [software testing]

Abstract

Statistical software testing presents two difficulties for the tester: (1) establishing accurate user profiles (i.e. usage probabilities), and (2) incurring lengthy test times. An algorithm, named the frequency count method (FCM), is developed which addresses both difficulties simultaneously. FCM finds usage probabilities within predetermined ranges and concurrently minimizes the amount of testing time. First, FCM randomly generates a large number of matrices for a given Markov chain with constrained usage probabilities. For each one-step transition matrix associated with the given Markov chain usage model, FCM simulates the steps of the chain. FCM flags the usage matrix which requires the minimum expected amount of testing time (assuming no failures) and ensures theoretical and calculated stationary probability values are within some preset precision. Thus, by generating test sequences from the usage probabilities of the flagged matrix, the expected minimum statistical testing time is achieved. This minimum time is optimal with respect to the transition probability ranges and the given execution times. Employing a 5-state usage model with numerical values for the transition probability bounds and code execution times, the FCM algorithm is illustrated and the expected minimum testing time is calculated.