Endpoint Resource Testing
The UC Test Lab provides endpoint resource testing as part of our overall meeting service evaluation. This article goes over the importance of understanding resource usage and how we perform our testing.
Resource testing involves reviewing how an endpoint reacts when different features are enabled for a meeting service. We closely monitor several key resources while enabling and disabling features and note both spikes and any rise or decline of resources. On a Windows PC or Mac endpoint, we monitor the following metrics: CPU, GPU, RAM and both send and receive bandwidth. On laptops and mobile devices we can monitor battery life. These metrics are important to pay attention to as increased resource usage affects the overall health of an endpoint and can significantly and negatively affect the user experience. For example, if a user has limited bandwidth available, they should not only know that using a gallery view lowers their incoming bandwidth, but if a remote participant is viewing them full screen it will affect their outgoing bandwidth.
Our test flow
First we start by monitoring all resources while the endpoint is idling with no meeting service client active. This gives us a baseline from which to gauge the meeting service against. Next the meeting service is launched. Since it is typical that meeting services include features like messaging, they are typically active on an endpoint constantly so users should be aware if simply having the meeting service open reduces their computer performance. Next, various features are tested while in meeting. This can include any features the client wants tested, like content sharing, active speaker versus gallery view, whiteboarding, etc while we monitor the endpoint’s resources. We run these tests multiple times in multiple meetings at random times during the day to attain an average of how the resources are affected. The network can also be conditioned to introduce packet loss or limit the bandwidth to assess the meeting service’s reaction. It’s important to note that we test all meeting services in the same manner. The same endpoints are used, the same camera feeds are utilized, and the same tools are used to condition the network to ensure accurate test results across all meeting services we test.
We can test resources across PC and Mac platforms. Battery run-downs can be performed on laptops and mobile devices.
What this provides for meeting service vendors is an objective third party validation and comparison against other competing platforms. This article only scratches the surface of the assessments we can perform. For more information, reach to us at bryan@uctestlab.com
