Which Phone Is Faster: The iPhone 6 or the Galaxy S6?
Sam Rutherford and Alex H. Cranz recently published a report at Tom’s Guide crowning the Samsung Galaxy S6 “the world’s fastest smartphone.” “We pitted six of the latest smartphones against each other in nine rounds of competition,” they wrote, “and the Galaxy S6 blew away the field, finishing first in 6 out of 9 real-world tests and synthetic benchmarks.” But Daniel Eran Dilger reported for Apple Insider that the claims made by Tom’s Guide “relied on cherry picked figures and ignored the real world entirely.”
Samsung’s latest Galaxy S6 flagship phone was introduced in April, seven months after the iPhone 6 launched last September. Dilger reports that because each generation of smartphones delivers significant improvements over the previous year, Samsung’s latest flagship phone should be significantly faster than the iPhone 6. But when you take a less biased look at the figures than Tom’s Guide did, it isn’t.
When the first Galaxy S6 benchmarks appeared three months ago, Apple Insider reported that while the phone’s graphics performance scores in the GFXBench represented an improvement over those of the Galaxy S5, they were still well behind those of the iPhone 6. At its native resolution, the Galaxy S6 performed worse than 2013’s iPhone 5s because Samsung paired an ultra high-resolution screen with an underpowered CPU and GPU — which results in great specs on paper but poor performance in real life.
Rutherford and Cranz make no reference to the troubling GFXBench data, and instead cite four “real world” tests in which various tasks were performed. These tests tasked the phones with opening a PDF, launching the camera app, playing a video game, and transcoding a video, and their performance was filmed with a slow-motion camera to find millisecond differences.
The PDF “load time” test involved loading a 1.6GB file in an unspecified app. The difference between the Galaxy S6 and the iPhone 6 was 20 milliseconds — an interval so small you need the high speed camera to notice. But other Android phones performed slowly enough to produce a visible delay, and Google’s Nexus 6 took almost four times as long to open the file, pointing to the fact that stock Android isn’t necessarily all that fast.
A test of camera load time also pitted different apps against each other, but still required a slow motion camera to observe any difference between the iPhone 6 and two phones from LG and Samsung, which each opened the camera app within 10 milliseconds of each other. The Nexus 6 again brought up the rear, with a score more than twice as slow as the iPhone 6 and the higher-end phones from LG and Samsung.
A video gaming performance test profiled the phones’ performance with three minutes of Asphalt 8. The iPhone 6 achieved the same frame rate as the Galaxy S6. Despite being slower at its native resolution, the S6 plays most games at a lower resolution to keep up; and despite having more cores, more RAM, and a faster clocked application processor, the S6 didn’t claim any speed advantage over the iPhone 6.
A test that used an app to transcode a 1080p movie to 480p wasn’t even performed on the iPhone 6, since the benchmark app that Tom’s Guide used was for Android. As Dilger notes, the authors claimed that “this real-world scenario paints an accurate picture of CPU performance,” but the VidTrim app they used actually relies on specialized ARMv7 NEON hardware acceleration, and thus doesn’t reflect general CPU performance.
In all of their “real world” tests, Rutherford and Cranz failed to find any significant differences between the iPhone 6 and the Galaxy S6 — despite Samsung’s phone costing more, packing an “octa-core” processor versus the iPhone’s dual-core A8, and featuring three times as much system RAM using faster DDR4 chips. Android 5.0 Lollipop was expected to bring faster app performance by using ART “ahead of time” compilation to replace Android’s “just in time” virtual machine. But a year of hardware upgrades and the new version of Android haven’t helped Samsung to beat last year’s iPhone 6. And in its default native resolution, the Galaxy S6 delivers noticeably worse performance, with the GFXBench showing that Samsung’s Exynos 7-powered S6 drops to 15 fps, 78% of the frame rate of the iPhone 6 Plus.
But the Tom’s Guide authors ignored the GFXBench and instead selected the FutureMark 3D Mark Ice Storm Unlimited. In addition to GPU tests, it incorporates physics tests that use a software library that’s optimized for Android, but is neither used on nor optimized for iOS. So it returns slow iPhone scores that consistently deliver what Dilger terms a “false fail.”
The Tom’s Guide story says that the iPhone 6 tied or beat the fastest Android devices in the playback of a real video game, but shows that the iPhone 6 ranks at the bottom in the 3D Mark Ice Storm Unlimited, in which even the Nexus 6 scored higher despite its delivery of the worst real game performance. In Dilger’s assessment, “When you see FutureMark 3D Mark Ice Storm Unlimited test results appearing in a review or “shootout,” you know its a staged effort to make iPhones look bad and to make Android devices that flunk real tests (like the Nexus 6) look better than they actually are.”
The comparison of the iPhone 6 and the Galaxy S6 exposes an important insight into the choices Apple and Samsung made in engineering their chips. Apple’s A8 balances performance and efficiency with two large cores, while Samsung uses an eight-core design. Because most tasks use only one core at a time, the A8 can devote half the chip area to single core tasks. But Samsung’s chip leaves four cores idle and quarters the remaining chip surface area, delivering a smaller engine for real-world tasks.
In Dilger’s assessment, by looking at certain numbers in limited ways, Samsung effectively optimizes its product for scores rather than actual use. The Tom’s Guide story, unfortunately, perpetuates more of the same by ignoring the real-world conditions under which the phones would need to operate to achieve the benchmarks cited. It fails to prove that Samsung’s Galaxy S6 is actually the world’s fastest smartphone, and also seems a cogent argument against reducing phones’ performance to a comparison of milliseconds when there are much more significant differences to expose.