Basis Technologies recommends that the Testimony setup is fully tested before operating in a production system.
When deploying to the system identified to act as the source system for QA testing ensure the source system setup is run through as defined in the technical setup here.
The QA Testing of Testimony needs to take place in a system where there is a good volume of system operation can be recorded, this could be existing QA testing that is running. At this point a target system is required where playback can occur with no users logging on to change data after the copy is taken until the playback is completed, users logging on and changing data could cause unexpected failures in the playback.
The QA Phase scope and approach will be based on multi levels of testing, running Testimony recordings and playbacks with incremental complexity and volumes as per the example below. We would strongly recommend completing at least 2 high volume recordings and playbacks.
| Level | Volume | Users | Filtered | Duration | No. of Bots | Dailog | Batch | RFC |
| 1 | Low | 1 | Yes | 5 minutes | 1 | Yes | ||
| 2 | Medium | <50 | Yes | 10 minutes | 1 | Yes | ||
| 3 | Medium | <50 | Yes | 30 minutes | 2+ | Yes | Yes | |
| 4 | Medium | All | No | 30 minutes | 2+ | Yes | Yes | |
| 5 | High | All | No | 1 hour | 2+ | Yes | Yes | Yes |
| 6 | High | All | No | 3 hours | 2+ | Yes | Yes | Yes |
| 7 | High | All | No | 5 hours + | 3+ | Yes | Yes | Yes |
| 8 | High | All | No | 8 hours + | 3+ | Yes | Yes | Yes |
Verify a minimum of the following as a part of your testing
- Impact on the Source system is within the KPIs agreed while recording
- Verify the expected transactions are recorded with the correct data
- Ensure the playback is executed correctly with minimal errors
- Confirm that the playback completes within expected timings
- Ensure that critical transactions are played back correctly
- Ensure coverage analysis operates as expected
- Check the results of the playback are as expected
Post your comment on this topic.