DxOMark Arrives

Comparing cameras to each other is difficult at best. There are so many unbound variables in most comparisons that the numbers all become confusing at best. This reduces many of us to looking at comparison photos and pouring over reviews at DPReview and other sites. But even then, that doesn’t tell all of the story as one has to ask which RAW converter was used, or were the photos shot in JPG. It’s enough to drive one mad. Today, however, there is a new tool that brings a little more sanity to the process of comparing some of the more numerically measurable properties of digital SLR cameras. It's called DxOMark.

Instead of just regurgitating manufacturer numbers and forcing one draw their own conclusions by peering at comparison photographs, DxOMark is the product of rigorous testing and analysis by DxO Labs. Using it, you can compare the attributes of sensitivity, noise, dynamic range, tonal range, and color sensitivity of a group of cameras. For example, here's the noise plots of two cameras I’ve been comparing myself lately, the D700 and 1D Mark III.

As you can see, the observations that everyone has made about the D700 (and D3) producing cleaner images than the 1D Mark III are illustrated in a very straightforward fashion. Showing a different parameter, here’s a dynamic range plot from these two cameras.

Pretty nifty, eh? All of the data presented on these two cameras by DxOMark fits my own expectations after having eyeballed several thousand images from each system. This includes the differences in measured versus indicated ISO sensitivity that each camera exhibits that, in turn, explains why I’ve been seeing slightly brighter images from the Nikon D700 than from the Canon 1Dmk3 when shooting with the same combination of ISO, aperture, and shutter. I hadn’t yet compared that difference yet in any meaningful form myself beyond noticing a basic trend in my photographs, but it’s clearly visible in the DxOMark data. You can check out the other plots of these two cameras on the DxOMark site.

Of course, as with all purely numeric comparisons of bench measurements, one should use this kind of data wisely. Simple numbers don’t tell the entire story of what a camera can do. The DxOMark numbers are only telling us what the imaging pipeline in each camera are capable of, and at that only a few characteristics. Furthermore, even the numbers themselves don’t tell the whole story with regards to a particular attribute. For example, noise can manifest itself in ways that are more or less objectionable.

Also, these numbers don’t give us an insight into how the rest of the camera outside the imaging pipeline operates, how easy it is to operate, and how well other systems like autofocus and metering work. The data presented also ignores the totally subjective metrics of how well any particular camera “meshes” with any particular human. Still, this is valuable stuff and I'm really happy to see this site appear as a resource that we can all use.

The graphs above are from the DxOMark website and are copyright DxO Labs. You can read other writeups of DxOMark at The Luminous Landscape and The Online Photographer.