Performance Testing a Student Records System for Clearing Process

Challenges faced by UWE:

The University of the West of England, Bristol (UWE) needed to upgrade its on-premise Student Records System, Tribal SITS, in preparation for the crucial 2022 Confirmation and Clearing process. This period experiences a surge in user activity and transactions, as universities compete to admit students. Clearing, in particular, is one of the busiest times for university systems, exerting immense pressure on resources and system performance.

Our Approach:

UWE engaged Prolifics Testing to conduct performance testing services to verify the system's suitability and its ability to handle a substantial increase in usage, simulating the Clearing process. Leveraging their extensive experience with SITS in over 20 UK universities, Prolifics Testing devised a comprehensive test plan in collaboration with UWE. The plan encompassed the testing approach, simulated user journeys, expected user volumes, and the types of tests to be executed. This approach involved merging web application performance tests with a simulation of a high volume of students making simultaneous calls to the IVR contact centre, facilitated through a telephony partner.

The test plan was developed in consultation with UWE staff and presented to the management team for approval. It included a detailed schedule of activities and defined roles and responsibilities for all participants. The following user journeys were identified as within the scope of testing:

• Standard Clearing course application with an offer made, for both UCAS and non-UCAS applicants.
• Application resulting in an invitation to an interview (non-UCAS).
• Application resulting in a decline action due to applicant unsuitability (non-UCAS).
• Applicants accessing SITS e:Vision and checking the contents of the tabs presented.
• UWE backend activities such as applicant search, document upload, applicant change form, responding to offers, and confirming applicant details.

Each of these user journeys was scripted using JMeter and in-house accelerators. They were then combined into representative scenarios to generate the required load during each test. As part of this exercise, Prolifics Testing provided UWE with a profile of necessary test data, including the number and status of data items required for each test. This guidance enabled the UWE project team to prepare the required test data, which was backed up for use in multiple test iterations.

The selected profile of tests included:

Peak Load: Simulating expected peak volumes over 1 hour.

Combined SITS / Telephony Test: Simulating expected volumes, combined with simulated calls into the call centre IVR, over 1 hour.

Stress Tests: Increasing load until system instability or unacceptable response times were observed, lasting over 1 hour.

Soak Testing: Testing at peak volumes over an extended period (3 hours) to check for memory leaks or system degradation.

Spike Testing: Simulating major spikes in activity throughout the test to evaluate system recovery capabilities.

Failover Test: Forcing a failover to another server during a test to ensure the preservation of partially completed transactions.

Throughout these tests, the UWE technical team had access to an in-house developed Grafana dashboard, providing real-time results, including response times, concurrent user/transaction numbers, and overall pass/fail rates for each transaction type. This information, combined with server infrastructure statistics, allowed for quick identification and remediation of bottlenecks when necessary.

Seamless Performance Amidst Clearing Chaos:

The tests revealed several key findings:

1) Despite a 12% expected increase in potential students, response times for Clearing activities decreased, indicating the new SITS e:Vision version's efficiency improvement.

2) An issue with one of the four web servers not being utilised due to a configuration problem was identified and subsequently resolved before live Clearing.

3) Some high response times during the combined Telephony test were linked to a problem with the UCAS test system, related to the UWE test environment.

4) Soak testing uncovered a memory utilisation issue, reaching approximately 75% under prolonged heavy load, but this did not impact response times or user experience.

5) Failover tests demonstrated the system's ability to quickly recover from an outage.

Upon completion of the tests, Prolifics provided a comprehensive report, including comparisons against previous tests.

Additionally, in 2023, Prolifics conducted the same tests again to ensure UWE's readiness for the upcoming Clearing process.