ApTest Manager - Quick Tour

Quick Tour

ApTest Manager is a tool for managing testing of hardware and software products - defining test requirements and specifications, establishing test plans, executing tests, and viewing and comparing test results.

Testing is performed manually but ApTest Manager automates the activities surrounding it, improving consistency, organization, and control throughout the test lifecycle.

A single installation of ApTest Manager can support any number of tests for any number of products. General-purpose, Web-based, and highly configurable, ApTest Manager provides well-structured facilities for configuring, managing, executing, and recording results of all your testing projects.

Screen Shots

A picture is worth a thousand words - here are some representative screen shots from parts of ApTest Manager. Remember that everything you are looking at here are examples - ApTest Manager is 100% customizable to match your companies processes.

And here are a few sample reports:

Concepts

The basic element managed by ApTest Manager is a Test Suite. An ApTest Manager Test Suite is a collection of test requirements, specifications, and procedures and sets of results from running those tests. ApTest Manager can manage an unlimited number of Test Suites for an unlimited number of products. Tests may be structured into as many or as few Test Suites as you like and run repeatedly over different product versions and configurations.

ApTest Manager is VERY configurable. Each Test Suite can be separately configured to use its own set of templates for defining and executing tests as well as formatting reports. By specifying these templates you define the structure and management of the testing process for each set of tests.

The tests which make up a Test Suite are called Test Cases. Each Test Case is defined by the templates for its Test Suite: the fields which make up the Test Case and the possible values for attributes such as results. A Test Case can include data such as:

the component of the product it applies to
the requirement to be verified
the type of test it is
the preconditions it requires
the process to follow for the test
how to verify the requirement
associated files or other data
cleanup to be performed after the test
possible results
other information or observations that should be captured

Test Requirements have a separate set of fields and templates that can be configured for them.

Test Cases can be linked to Requirements, in one-to-one, many-to-one, and one-to-many arrangements. The use of Requirements is optional. Alternatively Test Cases may simply be defined on their own, with requirements not specified within ApTest Manager, instead maintained externally or simply not used.

Two additional concepts are involved in executing tests with ApTest Manager. Test Sets are selections of the Test Cases within a Test Suite. Thought you may execute a full Test Suite, often a test run needs to be a subset of Test Cases in a suite. Such a Test Set can be defined to include just Test Cases defined with particular attributes, such as tests for performance, functionality, or compatibility. Once a Test Set is defined it remains associated with its Test Suite and can be executed repeatedly in the future.

The execution of a Test Set is known as a Test Session. An ApTest Manager Test Session consists of the Test Cases defined for a Test Set and the results produced by running them. Test Sessions can be viewed or compared in reports and rerun in whole or in part to reverify failed or unexecuted tests on new product versions or configurations.

Test Methodology

ApTest Manager is test methodology neutral. It does not impose a particular process for testing or tie you to particular test tools. Rather it aims to provide a flexible framework within which the process you want to use can be managed.

Test Requirements and Specifications can be developed either before test procedures or in parallel.
A Test Suite can be fully defined before it is executed or testing can start small and expand to include more tests later.
What comprises a Test Case can be fully customized.

The information required defining the Test Case
The information required when executing the Test Case
The possible results of the Test Case
The types of Test Cases that will be developed

Roles within the test and development team can be assigned to any number of individuals.
Reports can be fully customized to present appropriate information for roles within the team and the organization.

Operation

ApTest Manager is operated from a Web browser: Internet Explorer, Netscape, etc. The ApTest Manager User Interface offers a familiar Web site look and feel: a series of screens navigated by clicking on links between them. Learning to use ApTest Manager is thus pretty easy - if you've surfed the Web you'll be comfortable with ApTest Manager from the first time you see it.

Though it's Web based and you can access it from anywhere through the Internet ApTest Manager offers more sophisticated functionality by far than your average Web site.

ApTest Manager creates and stores a repository of information about requirements, tests, and the results of test execution. You use ApTest Manager's screens to enter, modify, examine, and compare information from this repository. ApTest Manager also shares this information between users across your organization. For example a test developer might use ApTest Manager to define a set of tests which a test engineer uses ApTest Manager to run in order that a QA manager can ask ApTest Manager to compare the results to tests run previously on a different product version.

All in all ApTest Manager lets you use a comfortable Web-based interface to a rich set of features for managing information about testing across your organization.

Using ApTest Manager

When using ApTest Manager you identify yourself by logging in with a Username and Password.

Once you have logged in to ApTest Manager you'll be asked to select a Test Suite to work with. As discussed above a Test Suite is the basic element managed with ApTest Manager, consisting of Requirements as well as Test Cases and the results of executing them. You work with one Test Suite at a time.

How many Test Suites you will have to choose from depends on how your organization is using ApTest Manager and may be one, many or anything in between. For each Test Suite, ApTest Manager will show you the name of the suite and a description that was entered when the suite was created. Click on a Test Suite name to work with that suite. You can then define tests, execute tests, and view test reports for the suite you select.

Reports

ApTest Manager provides instantaneous access to reports on test information. Both information about test definitions and the results of running tests can be retrieved and viewed from your browser. Many types of reports are available.

Test Requirements Report - Shows the definition of a set of requirements
Requirements Report Traceability and Coverage Reports - Show the relationships between requirements and test cases
Test Specification Report - Shows the complete definition of a set of tests
Test Results Report - Shows the results of executing a set of tests
Progress Report - Shows the status of a project comprised of multiple test runs
Regression Report - Compares the results of multiple executions of a set of tests
User Report - Shows the performance and productivity of members of a project team
Coverage Report - Shows the degree to which a test campaign has executed
Schedule Report - Shows the planned and actual start and end dates for test runs, showing those that were early, late, and on time

Reports are requested for specific Test Sessions, using a screen which also provides management information such as the sessions run by a particular user during the last month. The Select Report screen is shown in Figure 1 while sample reports are shown in Figures 2-5.

Edit Requirements

ApTest Manager organizes and displays Requirements in a tree composed of Folders which contain one or more Requirements. A Requirements tree is displayed and manipulated with a user interface employing two side-by-side frames. This interface is similar to the familiar "Explorer" view native to the Windows platform. A sample Requirements screen showing a Requirements tree in the left frame and a Folder in the right is shown in Figure 13.

Requirements trees are populated and managed by selecting one of the links in the right-hand frame for the current Folder or one of the Requirements within that Folder. Requirements are defined by a number of fields, which can be customized for each Test Suite, presented as a form on your Web browser.

Requirements data can be entered as HMTL or plain text. Test Steps can be automatically numbered for you.

Requirement names can be strings entered by the user or numbers assigned automatically by ApTest Manager.

Edit Tests

ApTest Manager organizes and displays tests in a tree composed of Folders which contain one or more Test Cases. Like the tree used for Requirements a Test Case tree is displayed and manipulated with a user interface employing two side-by-side frames. This interface is similar to the familiar "Explorer" view native to the Windows platform. A sample screen showing a Test Case tree in the left frame and a Folder in the right is shown in Figure 6.

Test Case trees are populated and managed by selecting one of the links in the right-hand frame for the current Folder or one of the Test Cases within that Folder. Test Cases are defined by a number of fields, which can be customized for each Test Suite, presented as a form on your Web browser. A sample Edit Test form is shown in Figure 7.

Tests data can be entered as HMTL or plain text. Test Steps can be automatically numbered for you.

Test Case names can be strings entered by the user or numbers assigned automatically by ApTest Manager.

Run Tests

Test execution with ApTest Manager is focused around Test Sets and Test Sessions. Test Sets and Test Sessions are manipulated with a user interface employing two side-by-side frames. An example Run Tests screen showing Test Sets in the left frame and Test Sessions for one of those Test Sets in the right is shown in Figure 8.

When you select a Test Session to be run, the Test Cases in that Session are presented so they can be executed and results provided to ApTest Manager. Tests can be presented one at a time or many at a time. Sample screens are shown in Figures 9 and 10.

Security

External security can be provided with standard web mechanisms: https, firewalls, proxies etc. as desired.

Access to the product is account based and controlled by password. There are two types of user accounts: normal and administrative. A default admin account is created at installation and may create other such privileged accounts. Administrative privilege allows a user to modify installation level configuration information. This includes restricting the access of normal users to security sensitive features such as deleting or renaming tests and creating or modifying user accounts. There are a dozen or so such selectable features.

Once logged in, a user's access to test information is determined by their per-suite access level.

Users can be denied access to specific suites, in which case they are not visible to that user. For each test suite they have access to a user is granted one of six access levels:

Reports only - a user can view reports but cannot view, edit, or run tests.
Browse and Reports - a user can view reports and Test Cases but cannot edit or run tests.
Run only - a user can view reports and execute tests but cannot edit tests or modify Test Sets or Sessions.
Execute - a user can view reports, execute tests, and modify Test Sets and Sessions but cannot edit tests.
Execute and Edit - a user can view reports, execute tests, modify Test Sets and Sessions, and edit Test Cases.
Suite Manager - a user can view reports, execute tests, modify Test Sets and Sessions, edit Test Cases, and modify the configuration of the Test Suite.

These access levels are intended to be appropriate to business roles: management, test designers, testers, external consultants, etc. There is a default access level assigned to users for newly created suites and admin users can modify this for each user.

Customization

A key feature of ApTest Manager is the ability to customize the information that defines a Test Suite and how this information is displayed. Test Case Fields can be added or deleted and Field characteristics such as title, format, style, and placement can be specified. As well, both reports and data entry screens may be customized to display information in unique styles, and the results that can be assigned to Test Cases can be changed.

This configurability allows ApTest Manager to be adapted to fit easily into an organization's QA process, and for different test specifications and procedures to be used for different products, all under the ApTest Manager umbrella. The configurable aspects of a Test Suite are:

Test Case Fields define the information which makes up each of a Test Suite's Test Cases. This includes both a description of the information and how it is displayed when edited, e.g. as a text field, menu, etc. Examples of the sort of information a Field may contain for a Test Case are the author, creation date, test procedure, required preconditions, etc.
Requirements Fields define the information which makes up each of a Test Suite's Requirements. This includes both a description of the information and how it is displayed when edited, e.g. as a text field, menu, etc. Examples of the sort of information a Field may contain for a Requirement are the author, creation date, source, type, etc.
Execution Fields are similar to the Fields for a Test Case but apply to Test Sessions rather than Test Cases. They allow custom information to be collected at execution time through Data Entry templates. This information is associated with a particular run of a Test Case rather than the Test Case itself. Examples of the sort of information an Execution Field may be used to record are Problem Reports submitted for the execution of a Test Case.
Templates define how Test Case and Execution Fields are formatted in ApTest Manager reports and screens: which Fields are displayed and how they are laid out.

Separate Templates are provided for the screens used to edit requirements and test and execute tests as well as for an unlimited number of different reports. This allows the user to alter the display and formatting of Requirement and Test Case and Session information separately for different areas of ApTest Manager.
Test Case results define the display name and display color for the possible results of a Test Case. ApTest Manager does not attach any particular significance to a specific result and any number of possible results can be defined by the user.
Test Session Variables define information about the test environment for a Test Session. Their configuration includes both a description of the information and how it is displayed, e.g. as a text field, menu, etc. Examples of the sort of information Session Variables may contain are the platform, OS, software, and hardware used to execute a Session.