Special issue on new generations of UI testing

Abstract

Market demands for faster delivery and higher software quality are progressively becoming more stringent. A key hindrance for software companies to meet those demands is how to test the software due to the intrinsic costs of development, maintenance and evolution of testware, especially since testware should be defined and aligned, with all layers of the system under test (SUT), including all user interface (UI) abstraction levels. UI-based test approaches are forms of end-to-end testing. The interaction with the system is carried out by mimicking the operations that a human user would perform. Regarding graphical user interfaces (i.e., GUIs), different GUI-based test approaches exist according to the layer of abstraction of the GUI that is considered for creating test locators and oracles: specifically, first generation, or coordinate-based, tests use the exact position on the screen to identify the elements to interact with; second generation, or layout-based, tests leverage GUI properties as locators; and third generation, or visual, tests make use of image recognition. The three approaches provide various benefits and drawbacks. They are seldom used together because of the costs mentioned above, despite growing academic evidence of the complimentary benefits. User interfaces are, however, not limited to GUIs, especially with the recent diffusion of innovative typologies of user interfaces (e.g., conversational, voice-recognition, gesture-based and textual UIs) that are still rarely tested by developers; testing techniques can also be distinguished based on the way the test scripts are generated, i.e., if they are written inside JUnit-like test scripts or obtained through the capture of interactions with the SUT, or automatically obtained traversing a model of the user interface, as modern model-based testing tools do it. Test automation is a well-rooted practice in the industrial environment. However, there are software development domains, e.g., web and mobile apps, where UI testing is still not adopted on a systematic basis. The results of many investigations in literature highlighted many reasons for this lack of penetration of the most evolved UI testing techniques among developers: 1 Scarce documentation of the available testing tools; 2 Significant maintenance effort when keeping the test scripts aligned with the evolution of the AUT, e.g., for performing regression testing; 3 Limited perception of the benefits that advanced UI testing techniques yield when confronted with traditional manual testing.