Tuesday, 26 May 2020
A Testing Centre of Excellence is a central testing function in an organisation that operates to common standards, in line with the quality policy and overall test strategy. This allows a consistent approach to testing and the ability for an organisation to have a common toolset and specialist skills that can be used when needed on programmes and projects.
A key strength of a Testing Centre of Excellence (TCoE) is that knowledge (both testing and business), information and experience is built up centrally by the team, which can then be deployed onto programmes or projects of work using a central set of infrastructure.
As well as the establishment of a team, these individuals are supported by process, documentation and tools, with the aim of establishing best practice and measurable, predictable results across engagements in the organisation.
Key Elements of a TCoE
The three elements that must be in place are Team, Process and Tools.
Team
The most important element of a TCoE is having a team in place to service incoming project requests. It is important that the individuals in the team have the right skills and a consultancy-like mindset for the testing function to provide lasting value to its customers.
The following roles are typical in a TCoE, although not all of these roles will be appropriate for every implementation, as it depends on the organisation and particularly the projects:
- Head of Test / Test Service / Centre Manager
- Test Managers
- Test Architects / Principals
- Test Leads / Co-ordinators
- Test Automation Specialist(s)
- Performance Specialist(s)
- Information Security Expert(s)
Process / Documentation
Establishing a core set of standard documentation and a common approach to testing for different project types is a key part of the TCoE.
First and foremost is the Test Strategy. Depending on the size of the organisation and the amount of projects it has, will dictate whether multiple test strategies are necessary, or whether testing can be defined in a single document. Some organisations have a top level Test Policy document, which sets out the high level objectives of testing; this can then be supported by one of more Test Strategy documents. Ideally there needs to be a single test strategy for each group of similar projects in an organisation, for example, web developments, or SAP applications, etc.
Having a clear Test Strategy in place is core to achieving success in testing. The strategy should define the approach to testing on different types of project based on the changes made.
Examples of changes could be patches from third party suppliers, business as usual Agile or Waterfall development, change requests, bug fixes, and hardware changes. Each change should have a corresponding entry in the strategy to define what the organisation’s usual approach should be, which is essentially a decision based upon risk.
This doesn’t mean an inflexible strait jacket of test processes - there are always exceptions, but it is important to define the rule of where test phases are appropriate, what is the process, where can documentation be found and what tools to use.
A strategy should be accessible to all, so being based on a wiki or SharePoint site works well. This will not only help the testing team, it also helps project managers understand what testing will be needed on their projects, so that the right level of time and cost can be made available, up front.
Typical test phases that should be covered by the Strategy include:
- Unit Test: carried out mostly by developers, but it is useful to provide guidance/standards
- System Test: manual testing carried out in sprint
- Test Automation: regression packs, retrospective or developed in parallel/in sprint, tools, frameworks, data, coding conventions, continuous integration links
- Performance Testing: when relevant, core systems in scope, approach, tools and monitoring
- User Acceptance Testing: user involvement, co-ordination, guidance and management rather than active participation
The Test Strategy will also define quality gates and entry criteria at a high level, which can then typically be used in more detailed levels of documentation on projects (for example, test plans).
The methodology defined by the strategy will include a set of standard templates to be used, which are likely to include the following:
- Test Plans
- Test Charter (for exploratory testing)
- Test Scripts/Specifications
- Test Completion Reports
Note that there are likely to be different versions of the templates needed for different phases of testing (eg, system/performance). In addition, a feedback form should be completed after each project as part of the wash up/lessons learned process to share experience gained to the central team, so that the process can be improved.
Metrics are also very important to collect, such as estimate vs actual timescales, cycles of testing, defects opened/fixed/closed/per sprint and by cycle and release. Recording and analysis of metrics will help with measuring Return on Investment (ROI) and the success of the centre.
The usual testing activities are performed within each phase according to the workflow strategy – including test case design/execution and reporting.
It is vital for the TCoE to keep a track on resources, projects in flight and the project schedule. This ensures resources are allocated efficiently, all the projects in the pipeline are resourced and external assistance can be organised in plenty of time, if needed.
Elements sometimes overlooked when setting up a TCoE are establishing a process for customers in the organisation to request testing, but also processes to make sure everyone knows how to engage testing and what services can be provided from the TCoE.
This can be resolved by circulating the test strategy and high level approach to the project sponsors/programme management/project office; running workshops to increase visibility; and educating users and customers about what is available, how/when to request it and what to expect.
Tools
Choosing the right tools is key for any testing team, but arguably even more so when implementing a TCoE.
The types of tool likely to be needed include:
- Test Management Tool
- As a minimum, this should be a single tool capable of recording defects, test cases and results, and reporting on them across customisable views, but also potentially providing scheduling, release management and mapping of requirements to test cases and subsequently defects. It is very important to select a tool that meets the TCoE’s requirements and allows the recording of metrics required to provide ROI and cost savings over time, such as the Defect Detection Rate.
- Unit Testing
- If Test Automation is to be used at the unit test phase, the tools should be standardised across development teams, including integration into Continuous Integration (CI) tools.
- Functional Test Automation
- For the automation of regression tests on systems where regular releases are likely and risk of failure would be significant. This category would usually include device emulation, need to fit in with the development lifecycle in use, technical fabric of the application(s) under test and the skills of the team who will be using it.
- Performance Testing
- For testing non-functional attributes of applications once functionality has been proven. It is vital to ensure that systems perform as needed when accessed by the expected level of users, at a range of levels – usually normal, peak, spike and soak. The best tool for the job usually depends on the technical architecture of the applications under test. Open Source tools can be appropriate when considering single protocols, eg web, but there are some excellent enterprise grade tools available that can handle multiple applications/protocols as well as integrate with automation and test management tools, allowing full lifecycle management.
- Security Testing
- Security is often outsourced and executed remotely, however there are tools available that can integrate into the development process to improve the code and identify vulnerabilities as they appear, thus reducing the cost of remedial action when they are found later.
- Others
- Other tools to be considered include data management/masking, database query tools, and CI tools.
Is it for me?
One of the main aims of a TCoE is the need to Shift Left and eliminate defects early on, thus saving time and money. However, there do need to be enough projects and there does need to be an acceptance in the organisation that costs will not go down right away.
Only once established and bedded-in will overall costs be lower. Included in these calculations should be the number of defects reaching production, both before and after implementing the TCoE, as here is where the real savings lie. The increased confidence in the IT department should also be factored-in.
The benefits can be summarised as follows:
- Shared resources mean more specialist skills are affordable
- Centralised expertise and knowledge development – both of business/system knowledge but also of testing knowledge
- Consistency across projects
- Measurable, predictable results across projects
- Increased confidence in QA
Jonathan Binks
Head of Delivery - Testing (UK)
Need Help?
If you would be interested in hearing more about establishing a TCoE, or would like help or advice on your current implementation, we would be happy to help. Please contact us for a confidential, no-obligation chat.
