Tablets come in a wide assortment of operating systems with myriad processors and screen sizes, built with different users in mind. To determine how well the tablets will perform, we run a variety of synthetic and real-world tests that help simulate what a consumer would do after purchasing the device. The results from these benchmarks, which test nearly every component of a tablet, are then compared against competing devices, as well as the overall average, to help us determine a final rating for the product being reviewed. The following are descriptions of the tests we run.
Benchmarks
Laptop Mag Battery Test
This home-grown test, which is also used for laptops and smartphones, simulates how long a tablet will last while surfing the Web. We turn off all GPS and cellular radios, adaptive brightness settings and battery-saving applications. We then set the screen's brightness at 150 nits and connect to a Wi-Fi network. Then we run an app we developed that continuously cycles through a series of 50 popular websites, pausing at each site for 60 seconds, until battery is drained. Longer is better.
MORE: Our Favorite Tablets for Work and Play
PDF Open Test
We open a small, but extremely high-resolution 5.8MB PDF file of a detailed nautical chart in Adobe Reader (or the local PDF reader, if Adobe is unavailable). We record the time it takes to open the image from the instant we clicked to open. This tests both the device's processor, as well as the read/write speeds of the included storage. Faster is better.
Geekbench
A synthetic benchmark that tests processor, read/write and graphics performance. Higher scores are better.
3DMark Ice Storm Unlimited
A synthetic benchmark that tests the device's graphics performance by running a series of graphics tests "off-screen." This ensures that the resolution and quality of the display does not affect the test results. Higher is better.
Heat Test
We stream a 1080p video for 15 minutes in YouTube, and then measure the temperature at various points on the device. Temperatures higher than 95 degrees Fahrenheit are considered unacceptable.
Jetstream
A Web browser test designed to measure JavaScript performance. This helps us gauge how quickly large Web pages load. Higher is better.
Vidtrim
We time how long it takes to convert a 203MB video file from 1080p to 480p using the Vidtrim app. This test, exclusive to Android devices, tests the processing power of the device. Faster is better.
Display Brightness and Quality
It doesn't matter how fast a tablet is if the display looks terrible. To ensure you have the optimal content-consuming experience, we test the brightness and color performance of each tablet's display. A brighter display will be easier to view outdoors while better color performance (Delta E and color gamut volume) will make every YouTube cat video and Netflix original show you watch look spectacular.
Brightness
We turn off adaptive brightness, set the brightness of the device to its maximum level and open a file containing an all-white image. We then measure the light output using a Klein K-10a colorimeter and the Klein Chromasurf software.
Some devices, such as those from Samsung, will not adjust a display to its maximum brightness unless the device is in full sunlight. To mimic the sun, we use a very bright flashlight to trick the tablet's ambient light sensor.
Delta E
It's not a sorority. Delta E describes the accuracy of color representation on a display. Every color within (and beyond) the rainbow has a finite number associated with it, and deviation from that number is the Delta E. The closer to 0 a Delta E, the more accurate a display's production of that particular color. However, the human eye can't distinguish any difference between displays that rate between 0 and 3.
We measure 96 separate colors on each display using a Klein K-10a colorimeter and the dispcalGUI software. We then average those Delta E numbers. Lower is better.
Color Gamut
We measure the range of colors a tablet can display using dispcalGUI software and a Klein K-10a colorimeter. Most manufacturers optimize tablets for a specific gamut of colors — Rec. 709 or sRGB — so we measure specifically for that gamut. sRGB is widely used by Web designers, filmmakers and game developers. We specifically look for what percentage of that gamut the display is capable of producing.
Displays that produce less than 100 percent of the gamut generally appear washed out and cheap, while displays that show more than 100 percent appear very rich and vibrant — and in some cases too saturated.
Color gamut and Delta E combined give a better sense of a tablet's display quality. For example, a tablet that reproduces 150 percent of the color gamut, but has a Delta E of 10 will produce an image far less realistic than a display with only 90 percent of the gamut and a Delta E of 2.
