Search
Related topics are listed below.
Creating a Test Plan
Administrators Guide » Test Plan and System Connections » Creating a Test Plan
Access Testimony Once logged into the Testimony central system, enter transaction /n/bti/aut. This will display the Testimony main menu. To create a new blank plan, select the create button You will be presented with a screen where you enter a test plan name…
Test Plan and System Connections
Administrators Guide » Test Plan and System Connections
Overview The Test Plan in Testimony is the “container” for all testing activities that take place within Testimony. Each Test Plan can be configured to represent a regression test cycle such as a Monthly Release, Project or SAP Upgrade undergoing testing. You can…
Test Plans
Product Screens / Functions » Configuration Functions » Test Plans
Purpose This screen provides a list of all Test Plans that have been setup in this controlling system, and enables the users to maintain their settings, create a new Test Plan or mark one as the current Test Plan. Audience / Users Testimony Administrators Possible…
Plan Status
Product Screens / Functions » Overview Functions » Plan Status
Purpose The Test Plan overview provides the current status information relating to the currently selected Test Plan. From here you can see key information including what testing stage or cycle the Test Plan is in, and what the next step of the process is. Audience /…
Display Plan
Product Screens / Functions » Overview Functions » Display Plan
Purpose This screen provides details of the current Test Plan. The information is presented on different tabs to show the header information, the source and target systems and their mapping, plus the users involved in this particular plan. Audience / Users All…
Change Plan
Product Screens / Functions » Overview Functions » Change Plan
Purpose This screen enables all facets of the current Test Plan to be changed. Changes can be made on the respective tab for the header information, the source and target systems and their mapping, the test phases/cycles plus users involved in this particular…
Test Script Library
Product Screens / Functions » Repository Functions » Test Script Library
Purpose The Test Script Library enables you to manage test scripts that exist across all test plans in the central system. In many ways, they are similar to Adhoc test scripts but instead can be re-used across as many test plans as required. Audience /…
Managing Test Plans
Administrators Guide » Test Plan and System Connections » Managing Test Plans
As well as maintaining Test Plans from the Overview -> Change Plan screen (as previously shown) it is also possible to maintain them without first having to select the individual plan. Navigate to Test Plans item from the Configuration section. You will see a list of…
Remediation of Test Scripts
Administrators Guide » Performing a Playback » Preparing Playback » Predictive Difference Analyzer » Results Phase » Remediation of Test Scripts
Overview Testimony currently only supports adding of replacing fields to containers (with default values) for Dialog Transactions, and Inbound RFC. to help repair scripts that may not operate correctly due to changes in the target system. Example 1 New mandatory…
User Preference and Plan Configuration
Administrators Guide » Filters, Suppressions & Plan Configuration » User Preference and Plan Configuration
!Important: All user preferences are configured in the Central system only. The two important tables are /BTI/AUT_C_USPA and /BTI/AUT_C_USPR. User Preferences The following user preference configuration has been added or amended to the configuration tables in order…
Filters, Suppressions & Plan Configuration
Administrators Guide » Filters, Suppressions & Plan Configuration
Overview Filter sets are the way to configure Testimony to limit the selection of what is recorded and played back for a test and/or to suppress error messages for certain objects. There are 4 points (Process Steps) where this can occur: Recording, Transfer to…
Building the Execution Queue
Administrators Guide » Building the Execution Queue
Overview The test scripts that have been created in the test repository must be transferred to the execution queue in preparation for the playback process. For a given Test Plan, only a single active execution queue can exist. Historical execution queues can also be…
Authorisations
Administrators Guide » Security » Authorisations
Testimony comes with three internal roles: /BTI/AUT_ADMIN: The test plan administrator /BTI/AUT_TEST_MANAGER: The test manager /BTI/AUT_TESTER: A tester role Each test plan must have at least one administrator and by default the user who creates a test plan is…
Testimony User Roles
Administrators Guide » Security » Testimony User Roles
*Note the user roles are assigned and maintained in Testimony rather than with standard SAP Roles, however, users will require the transaction /BTI/AUT in their standard SAP authorisations to access Testimony. Within Testimony, users are considered to be…
System Setup
Administrators Guide » Test Plan and System Connections » System Setup
Select Test Plan to change Use the drop down menu to select your newly created Test Plan and click the Tick. Now you can click Change Plan Systems In the main window, you will see the Test Plan header information. Select the Systems tab and click the Create…
General Parameters
Product Screens / Functions » Configuration Functions » General Parameters
Purpose From within this screen users can maintain the general parameters of Testimony which are not specific to a particular Test Plan. Test Plan specific parameters are maintained within the Test Plan itself. The Parameters are displayed as a list view and the…
Demo Dashboard
Product Screens / Functions » Reporting » Demo Dashboard
Purpose The dashboard functionality in Testimony allows you to configure extensive “at a glance” reports to gain a deep insight into the performance metrics of the currently selected Test Plan. Dashboards can be customized in Configuration. Audience /…
Administrators Guide
Administrators Guide
The quick start guide is to be used after installation and includes the key basic steps required to create a recording and a playback. Please check the installation guide here if that process is not complete. It details the process steps required to allow you to use…
Reviewing shared memory parameters
Administrators Guide » Performing a Recording » Recording Preparation » Reviewing shared memory parameters
Overview During the installation and initial configuration of Testimony, you will have set instance profile parameters on the source system. Once you have performed at least one large recordings in your production system, it is useful to review these settings to see…
Preparing Playback
Administrators Guide » Performing a Playback » Preparing Playback
Overview After the Execution Queue is built you are ready to run a a playback in Testimony, you need to first prepare and check the Target and Central systems. The following process steps set out the steps you need for a successful playback. If you are going to be…
Preparation Phase
Administrators Guide » Performing a Playback » Preparing Playback » Predictive Difference Analyzer » Preparation Phase
Overview The preparation program analyses all test scripts in the execution queue and creates a list of technical objects that will be compared between the source and the target. Execution Ensure the correct test plan has been selected and navigate to the Predictive…
Detailed Notification Configuration
Administrators Guide » Notifications » Detailed Notification Configuration
Overview This section provides a more detailed guide to the functionality available for notifications. Notification Types The Notification Types are displayed as below. Each one has the following fields: Area is the part of Testimony that this Notification Type…
Technical Overview
Product Overview » Technical Overview
The following steps provide an overview at a more technical level of how the Testimony product operates. Installation Process Testimony is installed as a third-party add-on solution to a central ABAP system (typically your SAP Solution Manager system). Transports…
Installation status of Source Systems
Product Screens / Functions » Recording Functions » Installation status of Source Systems
Purpose This area shows you the installation status of the recording agent in all source systems of the current Test Plan. You can install the latest version of the recording agent into any source if it has already been installed previously.
Agent Install Status
Product Screens / Functions » Recording Functions » Agent Install Status
Purpose This area shows you the installation status of the recording agent in all source systems of the current Test Plan. You can install the latest version of the recording agent into any source if it has already been installed previously.
Defect Management
Product Screens / Functions » Results » Defect Management
Purpose Defect management allows you to manage all business tasks (defects) that have been created as a part of the current test plan and test phase. Various functions are available such as assigning, completing and updating the relevant defects Audience / Users All…
Demo Summary Report
Product Screens / Functions » Reporting » Demo Summary Report
Purpose The dashboard functionality in Testimony allows you to configure extensive “at a glance” reports to gain a deep insight into the performance metrics of the currently selected Test Plan. Dashboards can be customized in Configuration. Audience /…
Recording Status
Product Screens / Functions » Recording Functions » Recording Status
Purpose This area shows the user the recording status of the current test plan across all recording agents in all source systems. A user can either start, pause or stop the current recording within the source systems of the current test plan. Audience / Users All…
User Roles
Product Screens / Functions » Configuration Functions » User Roles
Purpose Users are assigned user roles within a test plan. Here you can create or change existing user roles including the default menu setup and authorizations regarding which actions are possible. Audience / Users All Testimony users Possible…
Glossary
Glossary
Bot The bot is an executable program that resides on a windows machine (normally a virtual machine). During playback the bot machines log on as the recorded users and execute the recorded transactions. The requirements for bot setup can be found here. Central…
Version Info
Product Screens / Functions » Overview Functions » Version Info
Purpose This screen provides product version information, showing the current installed version of Testimony for the central system. Additionally, users are able to see the current versions of the software installed into the source and target systems of the current…
Standard Recordings
Product Screens / Functions » Recording Functions » Standard Recordings
Purpose Business transactions are created during the recording stage of a Test Plan phase and represent the individual “interactions” that have occurred. Please select a source system to view the transactions that have been created. Audience / Users All…
Review Recordings/Transfer to Repository
Administrators Guide » Review Recordings/Transfer to Repository
Verifying that recording has captured activity After the recording has been completed, the captured recordings will be moved from the source systems into the central system. You are then able to review what has been captured. To check that your recording has captured…
File System Monitoring
Product Screens / Functions » Recording Functions » File System Monitoring
Purpose From the screens displayed here you can configure directories upon which files that are accessed will be recorded. Those files accessed are then displayed in the “Monitoring” screens for the given source systems in the current test plan. The…
Performing a Recording
Administrators Guide » Performing a Recording
Overview The Recording phase can occur once a Test Plan has been created with the associated Source and Target systems identified. It’s important to have an understanding of the anticipated activity load on the source system to help guide the amount of timefor…
Execution Queue
Product Screens / Functions » Execution Functions » Execution Queue
Purpose For a given Test Plan, only a single active execution queue can exist. Historical execution queues can also be seen here. Please select an execution queue to view the items contained within it, along with their current technical status information. Audience /…
Run the screen size analysis
Administrators Guide » Building the Execution Queue » Run the screen size analysis
Introduction When recording online (dialog) transactions, one thing that Testimony captures is the number of lines that can be displayed on a user’s screen. Since it is very likely that users will have a variety of monitors, screen sizes and screen resolutions,…
Audit Logs
Product Screens / Functions » Overview Functions » Audit Logs
Purpose Audit logs are recorded for all actions that users perform within Testimony. From here you can see all actions that have been triggered along with the authorization checks and failures (if any) and any action parameters and messages. Audience /…
Shared Memory Limits
Product Screens / Functions » Configuration Functions » Shared Memory Limits
Purpose Shared memory is used extensively in Testimony to achieve near zero impact upon the production systems during the recording phase. The shared memory settings are specified here which be adjusted to acceptable levels across the entire production application…
Start Playback
Administrators Guide » Performing a Playback » Start Playback
Overview Once the execution queue is built from the repository, the target system is prepared and the Bots are started, the execution queue can be started Process Steps Select Test Plan to use Navigate to the Execution Queue Double check the Check Steps Click…
Coverage Analysis
Product Screens / Functions » Execution Functions » Coverage Analysis
Purpose The recording process has effectively generated an entire test script library covering over N% of the customers actual SAP system usage (where N would be expected to range from 70% to 90%). After the recording process is finished, the testing team can…
Recording Preparation
Administrators Guide » Performing a Recording » Recording Preparation
Overview When you are ready to run a test phase leveraging the recording / learning capability in Testimony, you need to first prepare the source systems. Process Steps Select Test Plan to use Generate Inbound RFC Capture – Prepares for inbound RFC calls to…
Execute Check Steps
Administrators Guide » Performing a Playback » Preparing Playback » Execute Check Steps
Overview Similar to the recording, the playback has a set of check steps that can be run against it in order to validate that the system and test plan are ready for playback. Run all check steps and evaluate the results to determine readiness for playback. Process…
Product Overview
Product Overview
Testimony is a one of a kind, digital regression test generation and execution product developed by Basis Technologies. It is installed on and operates within the SAP ABAP stack and is particularly used to reduce the duration and cost of regression testing SAP…
Standard Scripts
Product Screens / Functions » Repository Functions » Standard Scripts
Purpose The Test Script Repository displays all test scripts that have been created from business transactions or those that have been manually added. Please select a source system to view the test scripts that have been generated or added for that system. Audience /…
Playback Overview
Product Screens / Functions » Results » Playback Overview
Purpose The Playback Overview displayed here allows you to drill down and view the functional comparison result of all test scripts that have been played back via an execution queue. Drill down on the queue to view the individual results of each test script and to see…
Repository Functions
Product Screens / Functions » Repository Functions
This drawer menu presents the Test Script Repository and has a Test Script Library link. A detailed explanation of this menu is provided in the subsequent section of the user guide.
Filtered Scripts
Product Screens / Functions » Repository Functions » Filtered Scripts
Purpose Adhoc test scripts are used to ensure that critical business transactions that might be missed during the recording period are able to be tested. You can create and manage all adhoc test scripts for the various source systems within these screens. Audience /…
Securing Recordings and Results
Administrators Guide » Security » Securing Recordings and Results
It is also possible to secure access to potentailly sensitive information by enforcing an authority check for the Testimony user attemtping to view test results. This can be achieved in the General parameters section for the object “Authority check for data…
Example of a script-level suppression
Administrators Guide » Filters, Suppressions & Plan Configuration » Defect Suppression » Script-level suppression » Example of a script-level suppression
In the below defect, we can see that we have a data-related defect we received a “No stocks exist…” message on the selection screen. In this case, notice that the screens are different in the expected and actual parameters. In the recording, the user was…
Business Scenarios
Product Screens / Functions » Repository Functions » Business Scenarios
Business Scenarios Business scenarios are configured here allowing you to relate a string of business transactions that can then represent an end to end business process. They are configured here but specified for each test.
Script Transformation
Product Screens / Functions » Repository Functions » Script Transformation
Purpose The Test Script Transformation functionality is for use when systems that are recorded are not played back into systems that are identical. This is typically in scenarios such as when the target system is scrambled or consolidated. Audience /…
System Manager
Product Screens / Functions » Utilities » System Manager
Purpose This utility enables you to take snapshots of SAP systems as a part of the Testimony process (soon after activating a recording). You can also restore the snapshot into a pre-configured test system. Audience / Users Testimony…
Configuration Functions
Product Screens / Functions » Configuration Functions
This drawer menu presents the configuration elements of Testimony and has the following links: Test Plans Number Ranges General Parameters Shared Memory Limits Report Builder A detailed explanation of each of these functions are provided in the subsequent…
Results Phase
Administrators Guide » Performing a Playback » Preparing Playback » Predictive Difference Analyzer » Results Phase
Overview The object of this phase is to review all errors that could cause a critical issue on the target system during playback and either fix or remediate before starting playback. Warnings should also be reviewed as while these might not technically impact the…
Software Support
Software Support
After appropriate testing on your SAP test systems, you are now ready to use Testimony within your productive landscape. Remember that Testimony is dormant in your SAP system until activated for recording. You must ensure that Testimony recordings are deactivated after…
Filtered Recordings
Product Screens / Functions » Recording Functions » Filtered Recordings
Purpose Filtered recordings are generated when a source system is recorded in “filtered” mode. This allows you to restrict which users, types or objects are recorded so that they can then be used to generate “adhoc” test scripts. Audience /…
Review Playback Results
Administrators Guide » Review Playback Results
Overview When Testimony records activity in a system, it captures inputs, outputs and certain linkages that occur in-between. When the playback is executed in the test system, Testimony will automatically check that the output in the playback matches the output that…
Run the playback simulation
Administrators Guide » Building the Execution Queue » Run the playback simulation
The playback simulation can be useful for checking that you have enough bots defined for the playback, as well as enough background processes defined on the playback system. To run the simulation, go to Execution —> Execution Queues, select your execution queue…
Introduction
Introduction
Welcome to the Testimony Administrators Guide, a detailed overview of the product and processes to enable your organization to record and playback a successful automated regression test. The Administrator role within Testimony allows the user to configure and execute…
Exclusions
Administrators Guide » Filters, Suppressions & Plan Configuration » Exclusions
To exclude objects, double click on a Filter Set and select the tab for the exclusion type (in this case, Objects was selected). You’ll then click the “Create Filter Values” button seen below where you get a new window to create the rules. Note that…
Setup of RCA
Administrators Guide » Review Playback Results » Root Cause Analysis » Setup of RCA
Overview Before Root Cause Analysis (RCA) can be run successfully there are a set of steps that need to be followed. If Root Cause Analysis is already setup on your systems then you can proceed to the linkage creation step. Setup Steps ActiveControl 8.31 (as a…
Playback Overview
Administrators Guide » Review Playback Results » Playback Overview
Introduction The Playback Overview, accessed from within the Results tray, shows the results of the playback in various different ways, as explained below. Failures / Open Defects In this tab you can see the failures from the playback grouped according to their SAP…
Filter Sets
Product Screens / Functions » Configuration Functions » Filter Sets
Purpose From within this screen, you can configure filter sets for use in recordings and playbacks. Entries for individual objects or ranges of objects can be specified and included/excluded. You can activate filter sets by clicking on the “Active” column…
Prioritization
Administrators Guide » Analyzing Coverage » Prioritization
Introduction The next step is to prioritise the transactions, batch jobs, etc. that have been retrieved from the source system. Testimony prioritises the usage data as follows: First of all, it prioritises by frequency of execution in the source system. The most…
The Recording Performance Analysis
Administrators Guide » Review Recordings/Transfer to Repository » The Recording Performance Analysis
Introduction One of the keys to the success of any playback is ensuring that you only play back what you really need. This ensures not only that the playback results are relevant, but also that the duration of the playback is not unnecessarily elongated through having…
Usage Retrieval
Administrators Guide » Analyzing Coverage » Usage Retrieval
Introduction The first step in the coverage analysis process is to retrieve the usage data from the source system. In this step, Testimony will connect to the source system and retrieve workload statistics data (from the ST03N repository) for a period that you…
Analyzing Coverage
Administrators Guide » Analyzing Coverage
Overview A Testimony recording generates, from real-life activity, an entire test script library which covers a high percentage of your actual SAP system usage. How big a percentage this is (i.e., the coverage you have achieved) depends firstly on the length of the…
Predictive Difference Analyzer
Administrators Guide » Performing a Playback » Preparing Playback » Predictive Difference Analyzer
Overview The function of Predictive Difference Analyzer (PDA) is to allow the target system to be reviewed prior to a playback starting. It works by comparing the objects currently in the execution queue (from the recording of the source system) and comparing those…
Coverage Analysis
Administrators Guide » Analyzing Coverage » Coverage Analysis
Introduction Now that the usage information has been retrieved from the source system and the prioritisation run as completed, you can run the coverage analysis proper. The coverage analysis run will compare the transaction volumes captured in your recording (by…