Recently, OnePlus has been making waves with their significantly cheaper but high-performing flagships like the OnePlus 3 and OnePlus 3T. But that might very well change soon, because they were caught cheating, red-handed.
That is, benchmark cheating. A few days ago, XDA Developers, a large community of smartphone tech devs and end-users, noticed something odd while tinkering with the Qualcomm Snapdragon 821. During test runs, the OnePlus 3T reported high clock performance without idling, even when they were supposed to. This led testers to suspect that something was up. And something was.
XDA sought the help of Primate Labs, developer of the standard benchmarking app, Geekbench, who came up with the idea of “disguising” a version of Geekbench 4 as an app called “Bob’s Mini Golf Putt”. The results were as they feared: OnePlus 3T returned significantly lower clock averages on apps it doesn’t “target”.
Apparently, the phone reports inaccurate clock values when tested with certain benchmarking apps. It sets off a trigger process when it detects a particular app running, and then returns inflated clock performance values to the benchmark app. The result? Performance is high for select apps, inclusive of benchmark apps like Geekbench. But for the majority of apps, the phone doesn’t function as advertised.
A string of benchmark tests on other phones ensued, which also found the Meizu Pro 6 returning inflated benchmark values. It appears that in order to market the phone as a multi-moded device with high-performance and efficiency modes, they shut off the big cores of the device when on “balanced mode” and turned them back on in “performance mode”. Other smartphones have since been reported to return similarly inflated values.
This was by no means the first time something like this happened. A few years ago, back in 2013, Anandtech and other hardware review sites found that, with the exception of Apple and Motorola, nearly all OEMs in the smartphone industry resorted to benchmark cheating. That includes even big-name manufacturers, including Samsung and HTC. These OEMs learned the hard way; most of them were banned from well-known benchmark sites until they decided to go clean.
XDA reached out to OnePlus for comment, and the company immediately admitted to wrongdoing. In addition, they promised that the feature will no longer be present in future builds:
"In order to give users a better user experience in resource intensive apps and games, especially graphically intensive ones, we implemented certain mechanisms in the community and Nougat builds to trigger the processor to run more aggressively. The trigger process for benchmarking apps will not be present in upcoming OxygenOS builds on the OnePlus 3 and OnePlus 3T."
This fiasco is testament to the present importance of benchmark tests to manufacturers and users alike. Users refer to benchmark tests in order to scout potentially good buys, and smartphone companies know this. The OnePlus 3T is by no means a bad phone. However, in the thick of the competition, anybody would want to look extra good on the metrics, however little of an advantage that might present.
However, although it’s easy to empathize with OnePlus, benchmark cheating is still cheating. It’s deceiving customers into thinking they got their money’s worth when they didn’t. Toward this, Steven Zimmerman, who broke the news on XDA’s forums, cautioned users that benchmark tests aren’t everything, consoling frustrated OnePlus users.
"Many of the biggest benefits of investing in improving a device’s software don’t always show up on benchmarks, with OnePlus offering excellent real world performance in their devices."
Read the full report here.
Moral of the story? Get a smartphone that doesn’t need to lie to you to let you know that it’s up to the task. Apple has been consistent in their quality toward that end, so why not buy an iPhone… or two? You’re certain to get exactly what you’re looking for—no lies.