A GUI TESTING STRATEGY AND TOOL FOR ANDROID APPS

The increasing popularity of Android and the GUI-driven nature of its apps have motivated the need for applicable automated GUI testing techniques. This paper presents a proposed strategy and a supporting tool for GUI testing of Android apps. The strategy employs a model-based approach to capture the event-driven nature of Android apps. It includes two phases: Modeling Phase and Test Evaluation Phase. In the modeling phase, an event sequence diagram (ESD) is created for each activity in the app under test (AUT), which depicts its events and possible transitions between them, and used to generate event sequences (test cases). In the test evaluation phase, certain event-based coverage criteria are employed to measure the adequacy of the generated test cases. The proposed tool analyses the AUT, creates an ESD for each activity, and generates event sequences. It handles the event sequences explosion problem and ensures the event sequences feasibility. For each event sequence, the tool generates a test script and a corresponding Robotium test class, adds it to the AUT and executes it. The paper also presents a case study that illustrates the use of the proposed strategy and tool for testing a simple Android app.


INTRODUCTION
The number of mobile apps continues to grow at a rapid pace and the requirements for their performance grow as they become more advanced and are exposed to higher loads of users. This makes the subject of mobile app testing important and of continuing interest. Mobile apps heavily depend on graphical user interfaces (GUIs). Testing of these GUIs is very important, as it lets developers ensure that the app meets its functional requirements and achieves a high standard of quality such that it is more likely to be successfully adopted by users. Automating these tests is very useful since it saves a lot of time, but it is very difficult due to the complexity of mobile apps and the limited resources available in mobile devices.
Due to the widespread use of Android platform, the work in this paper focuses on testing the GUI of Android apps. The paper presents a proposed strategy for GUI testing of Android apps, and a supporting tool for analyzing the app under test (AUT), generating test cases based on certain eventbased coverage criteria, adapted for Android app, and executing these test cases. The proposed strategy employs a model-based approach to capture the event-driven nature of Android apps. The employed model is the event sequence diagram (ESD), which depicts the events for an app and the possible transitions between them. The proposed tool collects the IO/Clickable views in each activity of the AUT and their events. Then, it generates an ESD for each activity, and uses it to generate a set of event sequences according to the specified eventbased criteria. For each event sequence, the tool generates a test script, from which it generates a Robotium test class, adds it to the AUT and executes it. The paper also presents a case study that illustrates the use of the proposed GUI testing strategy and the supporting tool for testing a simple Android app.
The paper is organized as follows: Section 2 presents a review of related research in the area of model-based GUI testing of Android apps. Section 3 briefly describes the main components of Android app GUI. Section 4 gives an overview of Robotium Test Framework. Section 5 describes the proposed GUI testing strategy for Android apps. Section 6 describes the proposed testing tool, which implements the proposed strategy. Section 7 presents an example of using the proposed approach and tool for testing a simple Android app. Section 8 presents the conclusion of the work presented in this paper.
Since this paper focuses on model-based GUI testing of Android apps, a review of related research in this area is given below.
Amalfitano et al. [4] presented a technique for rapid crash testing and regression testing of Android apps. The technique is based on a crawler that automatically builds a model of the app GUI and obtains test cases that can be automatically executed. Amalfitano et al. [5] presented AndroidRipper, an automated technique that tests Android apps via their GUI. AndroidRipper is based on a userinterface driven ripper that automatically explores the app GUI with the aim of exercising the application in a structured manner. Yang et al. [6] presented a grey-box approach and a tool, for automatically extracting a model of a given mobile app. In this approach, static analysis extracts the set of events of the app GUI. Then, dynamic crawling reverse-engineers a model of the app, by exercising these events on the running app. Azim and Neamtiu [7] presented Android App Explorer (A 3 E), an approach and tool that allows Android apps to be explored while running on actual phones, yet without requiring access to the app source code. They construct a high-level control flow graph from the app bytecode that captures legal transitions among activities, and use it to develop an exploration strategy named Targeted Exploration that permits fast, direct exploration of activities, including activities that would be difficult to reach during normal use. They also developed a strategy named Depth-first Exploration that mimics user actions for exploring activities and their constituents. Choi et al. [8] proposed an automated technique, called SwiftHand, for generating sequences of test inputs for Android apps. The technique uses machine learning to learn a model of the app during testing, uses the learned model to generate user inputs that visit unexplored states of the app, and uses the execution of the app on the generated inputs to refine the model. Amalfitano et al. [9] presented MobiGUITAR for automated GUI-driven testing of Android apps, which is based on observation, extraction, and abstraction of the run-time state of GUI widgets. The abstraction is a scalable statemachine model that, together with test coverage criteria, provides a way to automatically generate test cases. Su et al. [10] introduced Stoat, a guided approach to perform stochastic model-based testing on Android apps. Stoat operates in two phases: (1) Given an app as input, it uses dynamic analysis enhanced by a weighted UI exploration strategy and static analysis to reverse engineering a stochastic model of the app GUI interactions; and (2) it adapts Gibbs sampling to iteratively mutate/refine the stochastic model and guides test generation from the mutated models toward achieving high code and model coverage and exhibiting diverse sequences.
The proposed approach differs from the reviewed approaches in the following aspects: (1) it creates a simple model, ESD, to represent the events in the UI of each activity and possible transitions between them, and uses it to generate test cases; (2) it employs event-based coverage criteria, adapted for Android app, to measure the adequacy of the generated test cases; (3) it significantly reduces the number of generated event sequences by identifying subsumption between different event sequences and discarding any sequence that is a subsequence of another one, and by checking event sequences feasibility, i.e. their ability to be executed, and discarding any sequence that includes any illegal event subsequences; and (4) it utilizes the features of the Robotium Test Framework to extract the AUT activities' views and related information, and to execute the generated test classes.

ANDROID APPS UI COMPONENTS
The main components of an Android app, which dictate the UI and handle the user interaction to the mobile device screen, are activities [21]. An activity represents a single screen with a UI. An app may have more than one activity. For example, an email app might have one activity for showing new emails, another activity for composing an email, and another activity for reading emails. In this case, these activities can interact with each other, and the one, which is presented when the app is launched, is called the main activity.
The UI for each component (activity) of an app is defined using a hierarchy of View and ViewGroup objects. Each view group is an invisible container that organizes child views, while the child views may be input controls or other widgets that draw some part of the UI. Input controls are the interactive components in the app UI. Android provides a wide variety of controls, such as TextView, EditText, Button, CheckBox, RadioButton, RadioGroup, and many more. UI inputs of an app include the input controls and their events (actions) for each activity. Events are a useful way to collect data about a user's interaction with interactive components of apps, such as button presses or screen touch, etc. When an event happens, the corresponding Event Handler, which is the method that actually handles the event, is called to perform any required task.

ROBOTIUM TEST FRAMEWORK
Robotium is an extension of the Android test framework and was created to make it easy to write UI test automation scripts for Android apps [22]. Robotium tests allow the tester to define test cases across Android activities. Robotium tests perceive the AUT as black box, i.e., it only interacts with the user interface and not via the internal code of the app.
The main class for testing with Robotium is Solo. Through a Solo object and its methods, we can set values in input fields, click on buttons and get results from other UI components. Methods of JUnits Assert class can then be used to check those results. Table 1 shows examples of the Solo methods.

THE PROPOSED ANDROID APPS UI TESTING STRATEGY
This section describes the proposed strategy for UI testing of Android apps. In this strategy, testing is conducted at two levels: activity level and app level. Firstly, all activities in the AUT are identified. This is a process to divide a large complex app into independent components that can be tested in isolation. Then, all views within each activity and their events are identified. In the activity level testing, each activity is tested in isolation to verify whether it works as expected. Then, in the app level testing, the app as a whole is tested to verify whether all of its activities can communicate with each other to complete the desired tasks. In this level, each activity will be treated as a trusted component because it has passed the activity level testing. An execution path of the app will be represented by a sequence of these trusted components.
Each testing level of the proposed strategy includes two phases: Modeling phase and Test Evaluation phase. In the modeling phase, a model is created for each activity/app that is used to generate test cases for testing the UI of the activity/app, and in the test evaluation phase, event-based coverage criteria are employed to determine whether the generated test cases have adequately tested the UI of the activity/app. In the next two subsections, the model used to represent each activity/app, and the UI test coverage criteria employed in the test evaluation phase, are described.

THE EVENT SEQUENCE DIAGRAM
All the possible execution paths in an app UI are represented by a model based on the Finite State Machine (FSM) model, called the Event Sequence Diagram (ESD) [23]. In an ESD, each node represents an event while a state transition is determined based on how the current node is responding to the inputs. An ESD is created for each app activity, then the ESDs of all app activities are grouped together to create an App ESD.
An Event Sequence Diagram D is a two-tuple <N, E> where: 1. N is a set of nodes representing all the events for an activity/app. Each node nN represents an event in D.
2. E  N x N is a set of directed edges between the nodes. Each edge eE represents transition from one event to the next. An event e2 is said to follow e1 if and only if e2 can be initiated after e1. The constructed ESDs are used in generating test cases (event sequences) for each activity, in the activity level testing, and then for the app as whole, in the app level testing, based on certain event-based coverage criteria.

THE UI TESTING COVERAGE CRITERIA
As described in the previous subsection, the events identified within each UI's activity are represented by an ESD, which is used to generate event sequences as test cases. In order to measure the test adequacy of test cases, Memon [24] has defined two kinds of event-based coverage criteria: (1) intra-component criteria for events within a component and (2) inter-component criteria for events across components. Intra-component criteria include: event coverage, event-interaction coverage, and length-n event-sequence coverage; and intercomponent criteria include: invocation coverage, invocation-termination coverage, and length-n event-sequence coverage. We adapted these criteria for Android apps. We called the adapted criteria Intra-activity criteria and Inter-activity criteria, respectively, and defined them as follows: Intra-activity criteria -Event Coverage: each event in the activity should be triggered at least once. -Event-Interaction Coverage: after an event e has been performed, all events that can interact with e should be executed at least once. -Length-n Event-sequence Coverage: all length-n event sequences within an activity should be executed at least once. These criteria are employed in the test evaluation phase of the activity level testing.
Inter-activity criteria -Invocation Coverage: each event that starts a new activity must be performed at least once. -Invocation-termination Coverage: all length 2 event sequences consisting of an event followed by one of the invoked activity's termination events has to be tested. -Length-n Event-sequence Coverage: all length-n event sequences that start with an event in an activity and end with an event in another activity must be tested. These criteria are employed in the test evaluation phase of the app level testing.
Having defined the ESD and the event-based coverage criteria for UI testing, the following steps are performed in order to apply the proposed UI testing strategy to test the UI of an Android app: -Identify the app activities and create the corresponding ESDs.

THE PROPOSED ANDROID APPS UI TESTING TOOL
This section describes the proposed tool that implements the proposed UI testing strategy for Android apps. Fig. 1 shows the steps that are followed by the tool to generate and execute test cases for each activity in the AUT. The tool utilizes the functionalities provided by the Robotium Test Framework in two of these steps: in analyzing the AUT activities to extract their views and related information, and in executing the generated test class for each event sequence.

Figure 2 -Generate_and_Run_Test_Cases Algorithm
The procedural details of the tool steps are described in the Generate_and_Run_Test_Cases algorithm, shown in Fig. 2. In this algorithm, three data structures are created: Event list L, which contains, for each activity, its IO/clickable views with their events; Event Index List IL, which contains for each view its index in L, type, text, and id; and Event Sequences Set S, which contains all possible legal event sequences.
The tool builds an ESD for each activity, and generates test cases based on the ESDs of the AUT and the coverage criteria, described in Sec. 5. The input to the tool is the AUT, and the outputs produced by the tool are: UI event sequences, Executable test cases, Criteria coverage report, and Test results report. Generate a possible event sequence s from e to other events in L; 5.
If s is a subsequence of another generated sequence in Se or it includes any illegal event subsequences Then 6.
Add s to list Se; 9.
End If 10. End While 11. Return Se; End.

Figure 3 -Generate_Event_Sequences Procedure
The tool works as follows: Firstly, it uses a Solo object to identify the current activity and detect its views. From the detected views and related information, which includes the view's type, event, text and id, the tool selects only IO/clickable views and saves the text of each view with its event in the Event List L, and generates the Event Index List IL.
Then, for each event e  L, the tool generates all possible legal sequences of e with all other events in L, using the procedure Generate_Event_Sequences, shown in Fig. 3, and stores them in the Event Sequences Set S. To overcome the event sequences explosion problem, the procedure identifies subsumption between different event sequences, and discards any sequence that is a subsequence of a previously generated sequence. Also, to ensure the feasibility of event sequences, i.e., their ability to be executed, the procedure discards any sequence that includes any illegal event subsequences. Next, for each event sequence s  S, the tool generates a test script by using IL and the procedure Generate_Test_Script, shown in Fig. 4. For each event in s, the test script includes a line that contains the type of the corresponding view, its text and id. From the generated test script, the tool generates a Robotium test class, using the procedure Create_Test_Class, shown in Fig. 5, and adds it to app. Finally, app with the Robotium test class is executed, and the tool produces test results and criteria coverage reports. These reports provide the tester with information about the detected errors, if any, and the fulfilment of the specified testing coverage criteria, to decide whether more tests are required or not. Get the view type that corresponds to event e, with its text and id, from IL 4.
Add a line representing the action of this view, which contains this information, to the test script. 5. End 6. Save the generated test script in testScriptFile End. Read a line ln from testScriptFile 5.
Insert the following instructions into testClassFile:  The AUT tests are executed using an Android emulator. The tool provides users with the GUI interface shown in Fig. 6. The main components of the tool interface are: four buttons, "Browse", "Load AUT and Get Views and Sequences", "Generate Test Script and Test Class", "Run test class"; one EditTextBox; and one ListBox. Firstly, the user selects an app for testing by clicking on "Browse" button. Then, when the user clicks "Load AUT and Get Views and Sequences" button, the selected app is loaded, list of all the clickable/IO views of each activity of this app and their events are extracted, and from this list the tool generates all possible legal event sequences of views. Next, a cycle starts: when the user clicks "Generate Test Script and Test Class" button, the tool selects an event sequence and generates a test script for it, then generates a Robotium test class for the generated test script, and shows its file name in the ListBox. This test class is added to the AUT. Each test class contains calls to Robotium functions through a Solo object that correspond to lines in the test script. When the user clicks "Run test class" button the test class is executed. Then, the tool asks the user whether he/she wants to continue, if the answer is no, the tool stops, otherwise, the tool allows the user to do more tests by clicking "Generate Test Script and Test Class" button, which repeats the above cycle, (see Fig. 1).

CASE STUDY
This section presents an example of using the proposed approach and tool for testing a simple Android app called Currency_Converter. Fig. 7 shows the window (main activity) of this app. Its UI includes 3 buttons ("Compute", "Clear", and "Exit"), 3 Radio buttons ("Egypt", "Canada", and "Japan"), one EditText control, 4 TextView controls.
When the app is launched, its initial window/activity, shown in Fig. 7, appears. The app takes as input an amount of money in United States Dollars (USD) and outputs the equivalent amount in Egyptian Pounds, Canadian Dollars, or Japanese Yen, according to the selected country: Egypt, Canada, or Japan, respectively. It produces an error message if no country is selected or no USD amount is entered before clicking the "Compute" button. At any time, the user can clear the window, by clicking the "Clear" button, which returns the AUT to its initial state, or close the window, by clicking the "Exit" button, which quits the app.

CONCLUSION
This paper presented a proposed strategy for testing the GUIs of Android apps. This strategy employs a model-based approach to capture the event-driven nature of Android apps. The employed model is the ESD, which depicts the events for an app and the possible transitions between them. The proposed strategy includes two phases: Modeling Phase and Test Evaluation Phase. In the modeling phase, an ESD is created for each activity in the AUT and used to generate test cases (event sequences). In the test evaluation phase, certain event-based coverage criteria, adapted for Android app, are employed to measure the adequacy of the generated test cases for testing the GUI of the AUT.
Then, the paper presented a supporting tool for analyzing the AUT, generating test cases, and executing these test cases. The proposed tool collects the IO/Clickable views in each activity of the AUT and the associated events. Then, it generates an ESD for each activity, and uses it to generate a set of event sequences according to the specified coverage criteria. The tool handles the event sequences explosion problem, by discarding any sequence that is a subsequence of another sequence; and ensures the feasibility of event sequences, by discarding any sequence that includes any illegal event subsequences. By considering these two situations, the number of generated sequences is significantly reduced.
For each event sequence, the tool generates a test script, then generates a corresponding Robotium test class, adds it to the AUT and executes it. The tool utilizes the functionalities provided by the Robotium framework for extracting information about the views in each activity in the AUT, and for executing the generated test class of each event sequence.
Finally, the paper presented a case study that illustrates the use of the proposed GUI testing strategy and the supporting tool in testing the UI of a simple Android app.