DxOMark Mobile, the ubiquitous, yet often controversial, suite of smartphone camera benchmarks recently received a much-needed boost thanks to the addition of two important new tests. However, the new scores can often prove confusing.
The New Tests
In a never-ending effort to keep up with smartphone technology and the new ways in which the devices are commonly used, the newly renamed DxOMark Camera benchmark will include brand new tests for wide-angle cameras and night photography modes. It arrives just in time for Apple’s latest iPhone 11 range which comes with both of these new features.
So what are the new tests?
Night Photography
A new ‘Night’ score replaces the DxOMark’s previous Flash score and adds to it a range of new test scenes designed to evaluate camera performance in low light situations with, and without, flash. Automatic flash functionality is also tested, when available, as are any specific night modes offered by the camera - Google ‘Night Sight’ fans rejoice! But not too much (see below).
Wide-angle Photography
DxOMark Camera also introduces an entirely new sub-score, called ‘Wide’. This test is designed to challenge and compare the ultra-wide-angle cameras found on many modern smartphones. It functions much like DxOMark’s existing Zoom test but works in the opposite direction. Think of the Zoom test as ‘Zoom In’ and the new Wide test as ‘Zoom Out.’
The new test looks for issues specific to wide-angle photography, such as facial distortions.
DxOMark Camera - Confusion Likely
Unfortunately for DxOMark, and despite comprehensive explanation on dxomark.com, the new results are sure to confuse readers - at least in the short term.
The new scoring system means smartphones that have already been tested are suddenly able to score extra points for features previously excluded from testing. You can see updated reviews of twelve newly re-tested smartphones now on dxomark.com and more re-tests will be added in due course.
While the new Wide test simply adds the possibility of extra points for smartphones equipped with ultra-wide lenses, the Night test isn’t such a simple upgrade to the overall benchmark.
The DxOMark Night test isn’t a pure ‘Night Mode’ benchmark as it also factors standard-mode and flash shots into the result. No specialized Night mode is required to achieve a Night score, but extra points can be gained if a feature, such as Google’s ‘Night Sight’ is available.
To do well in the Night test, a phone will, therefore, need strong flash performance as well as good low-light performance without flash when any specific Night modes are disabled.
For example, the updated Pixel 3 review which places Google’s flagship somewhere in the middle of the range of Night results, despite it having one of the best dedicated Night modes yet tested.
The confusion comes from the fact that readers can understandably, but mistakenly, equate the ‘Night’ result with the performance of a feature such as Night Sight. DxOMark’s own review calls the Pixel 3’s Night Sight “one of the best we have tested so far”, but the Pixel 3’s Night score is eventually dragged down by poorer low-light performance when Night Sight is turned off.
The biggest winners from the new tests are Samsung’s Galaxy Note 10+5G and Galaxy S10 5G along with the Huawei P30 Pro, all of which gain four extra points. Smartphones without a dedicated ultra-wide lens, such as Apple’s iPhone XS Max and the Google Pixel 3 gain a single point each.
In Conclusion
While DxOMark’s new tests are welcome and necessary additions to the current suite of benchmarks, the results still require expert interpretation if they are to be properly understood. Many important details are to be found only within the lengthy reviews accompanying the numerical scores which often fail to reveal the whole story.
If you’re looking to DxOMark to tell you which smartphone has the best Night mode, you won’t find the answer just by looking at the numbers. In fact, you may find yourself being nudged in the wrong direction unless you take the time to understand how the scores are calculated.
DxOMark isn’t stopping here. We can expect new macro photography tests next year along with several video-related updates. As the tests become more complex, it will become increasingly important to pay attention to the details.
