What Does Faster Payments Mean for Testing in a Bank?
Beware!!!!!
This part of the site is no longer accurate as regards dates. At the time of the last update (April 2008) the member banks should have finished their Faster Payments Testing activities. This page has been left here for use by Test planners in those banks considering joining the scheme after Faster Payments starts in May 2008. Such banks would need a testing plan like the one described below but the actual dates would depend on when the bank wanted to join the scheme.
__________________________________
The industry level approach to testing is still not fully clear yet but some ideas and principles have been established, albeit the timescales and details of contents have not. As such this answer is very much at a moment in time (May 2007).
The diagram below illustrates the overall idea, but the timescale is not certain.
The testing can be thought of as broken into two parts;internal to a bank and external involving The bank and other players (eg other banks, Bank of England, the central infrastructure).
It is anticipated that External testing for the member banks and Direct Agencies should have the following elements.
Connection Testing
This would be a test on a bilateral basis between a member or Direct Agency and the central infrastructure to ensure both parties can send and receive messages and/or files. Both parties would be sending/receiving from test environments.
External Integration Testing
This will consist of a group of member banks, the Central Infrastructure and the Bank of England. This will test that the system works as an integrated whole with payment messages being moved from a member through the infrastructure to another member and eventually settling at the Bank of England. This integration test group should expand over time to include all members taking part in the initial go-live as well as any Agency Banks or Third Party Beneficiary participants who anticipate going live in November 2007. These tests should be carried out in test environments in the members.
Non Functional Testing
In addition to the functional testing at a cross industry level, some forms of
Disaster Recovery Testing
Performance Testing
Volume Testing
will also be carried out at a cross industry level, i.e. involving multiple members and the Central Infrastructure.
User and Procedure Testing
This set of testing is as much about business processes as technology and is designed to ensure that the Central Infrastructure, Bank of England, members etc can operate the Faster Payment scheme, including error conditions and escalations.
Cut-Over Testing
These test the actual procedures for bringing the FPS scheme to life.
For Indirect Agency Banks and their sponsoring Clearing Bank there will be a different type of external test. Essentially both parties will have to test the interface that will exist between them. Since these interfaces will vary from Clearing Bank to Clearing Bank the exact nature cannot be described here. However, it is reasonable to suppose some of the following will be needed:
A test to confirm that indirect Agency payments make it all the way to the infrastructure.
The information on payments being successful or otherwise is accurately returned to the Indirect Agency.
Adequate volumes can be handled by the interface within the service levels.
Files of Faster Payments coming into the Indirect Agency have the correct information and can be processed by the indirect Agency.
Every member bank or Direct Agency will have its own QA processes but it will be required to perform “self certification” testing prior to being permitted to participate in external testing. This will involve demonstrably being able to send and receive messages to and from the APACS supplied simulators. (Simulators are devices that simulate the Central Infrastructure for the Bank). Indirect Agencies will be required to pass some sort of certification test with their Clearing Bank
Additionally a bank should reasonably have to plan;
Internal Integration Testing
Given the large number of systems affected by Faster payments some form of massive scale integration test will be required inside the member bank. This will be a major “test environment” build challenge. It would expect to be driven by simulators.. We place particular emphasis on this activity and discuss it in more detail below.
Internal System Tests and Internal UAT Tests
Many systems are affected by Faster payments. Many, such as internet banking channel systems, Credit decisioning systems, will involve business departments signing the changes off in a UAT. Others, such as payments switches will require simulators to perform system tests.
Internal testing for the Faster Payments Scheme has four key characteristics.
There will be a very high level of repetition caused by multiple levels of integration testing – (see Levels of Integration Testing below), and the need to repeat the tests at various levels externally.
The amount of testing coverage required in a particular area/channel is quite small.
The consequences of not picking up errors in testing are severe (it is a high volume payments system and the financial and reputational repercussions of production failures are potentially disastrous).
There are a very large number of components needed for a decent integration test.
The implications for a testing strategy are that
There will need to be a major investment in creating an integration test environment because of the number of components involved.
A low-cost test running capability will be needed because of the high level of repetition.
Because only a small part of the functionality of a channel (e.g. internet banking) is modified, the existing channel text scripts would mostly be wasted. Rather only a very small subset of a particular systems test scripts is needed. The issue is coordinating the scripts with all the other channels/systems.
Focusing on the internal integration testing we see four logical levels of integration testing.
1. Single channel end to End.
The diagram illustrates the main logical components of this test. The aim is to send a single payment from a payment generating channel (e.g. the internet banking system) all the way through the bank’s internal payment switches to the external gateway and then report the response back to the channel.
The BACS/FP switch will be needed to decide whether the payment can be sent as an FP or has to go via BACS.
The simulator would be the industry supplied tester which simulates the Central Infrastructure for a member or Direct Agency, something else for an Indirect Agency. This testing should also test various validation conditions such as valid/invalid sort codes and valid/invalid amounts as well as the channel’s ability to represent the various statuses of the payment correctly (e.g. pending, authorised, rejected (plus reason) etc). Finally, this test should also cover incoming faster payment credits (as emulated by the simulator).
2. Multiple Channels End to end
This level of integration testing essentially repeats the Single Channel testing for all the channels that can create Single Payments.
3. Standing Orders Processing and Diarised Payments Processing
This testing creates new levels of integration testing complexity because it involves overnight batch processing, particularly for standing orders. These items are particularly tightly integrated into the overnight accounting and credit processes. The diarised payments are usually initiated by a channel so they have to have been proven to work already.
4. Accounting and Outputs
This area of integration testing assumes the previous three are all working correctly and looks at the various output processes that follow the payment transactions. These are typically batch based processes such as charging, statements and customer information (e.g. electronic data delivery to corporate customers).
As can be seen, the integration test environment by the end includes a substantial part of the bank’s software base and the tests at the different levels will require constant re-submission of the initial payment instructions through the channels to trigger all the required events.
Testing with a Member’s Corporate Customers and Indirect Agency Bank
Firstly, the Banks are struggling to hit the (already once delayed ) May date and so for nearly all players, May will be a partial implementation. Some subsidiories, divisions, channels and customer groups will not be ready for May.
This means that there will be a need to be a phased implementation of the Faster Payments systems. (The mechanism is yet to be decided but could be by Sort Code, Bank, % of payments, customer etc). This means that implementation specific functionality such as a sort code switch cannot be specified, much less tested until the implementation approach has been designed.
Finally, the incorporation of other parties such as Agency Banks into the testing approach has also not been decided. Even if the scheme decides to simplify the initial launch and say that all non member parties that want to connect must wait until late 2008 or beyond there will still be complex testing with other companies. This is because many of the members will have collection account customers and indirect agency bank customers who will probably need to change their systems to receive Faster payments.
The result of this is that there are likely to be extra "releases" in 2008 and 2009 that will drag in extra functionality (both from members and the central infrastructure) but also batch up new participants such as Agency Banks and Corporates. Member and Agency banks should therefore plan on ongoing test cycles after the go-live in May 2008.
