Solving the Leica M8 DNG riddle

Written by:

The image on the left is the RAW data at its bayer format from the Nikon D200. The image on the right is the RAW data at its bayer format from the Leica M8. one is brighter then the other even though they both will look the same with any Raw software, such as Adobe Camera Raw. In this article we will explore the 8 bit riddle of the M8 DNG files.

Nikon D200
Leica M8

Yes, those are 8 bits

Right around the time first Leica M8 cameras reached the hands of some of the more advanced digital photographers, reports that Leica’s new M8 RAW files are 8 bit started to come in. This is actually a fairly easy thing to notice as all Leica M8 RAW files are 10.1MB, no matter what is actually in the image. So, if all RAW files are 10.1MB, this means there is probably no compression going on.

With computers, each byte is equal to 8 bits. So, if we have four pixels each holding 8 bits of data, we will have a file size of 4pixles X 8bit = 32bit = 4byte.

A simple calculation of the M8’s 10 megpaixels sensor reveals that with a resolution of 3936 x 2630 where each pixel is holding 8 bits, we will have a file size of:

3936×2630 = 10,351,680pixles X 8bit = 82,813,440bits = 10,351,680 bytes = 9.8MB (when kilobyte is 1024 bytes).

So, here is a proof that Leica’s DNG RAW files are really holding 8bit of pixel data. But, it is not as simple as that.

Understating the role of Gamma

Image sensors operate in a linear way. This means that for every time the amount of light is doubled the sensor also doubles it’s output. What you need to understand is that humans see the world in a logarithmic way, and so this is one of the reasons you see the image below as too dark. What’s more, our computers screens project light in a non-linear way.

Historically, to compensate for those problems camera makers have applied a gamma curve to the image, making it appear correctly on computers and TV screens. Today, camera makers not only use the gamma function for compensation, but also to design the image to look as the camera maker thinks it should look. In other words, every camera has its own gamma function rather then a standard value.

The image on the left is a linear image taken with the Canon EOS 30D. The image on the right has been applied with the Gamma function we have managed to extract from the Canon standard picture style profile.

16 or 14 bits?

Although Leica insist that the M8 is “16 bit-color resolution”, there isn’t any proof that this is actually the reality. What we are about to describe in this article is leading us to think the M8 is really a 14 bit camera, which is already not bad at all. Moreover, the chip that determines how many bits per pixel a camera has is called the AFE chip, or the A/D chip. As it turns out, dual cannel chips with 16 bit output are not very easy to come by. In fact, Analog Devices, one of the biggest AFE chip maker, hasn’t got any AFE chips with 16 bit output, but they do have some dual cannel 14 bit AFE chips.
We are going to assume in this article that the M8 is an 14 bit camera.

Leica M8 DNG files

Obviously, the reason the M8’s DNG files are 8 bit is because leica wanted to have small files that are easy to store and process. Leica is using a software processor on the M8, which is a good thing for a company with no experience with image processing. But this also means its power to energy ratio is quite bad compared to the hardware image processor in most DSLR cameras. So 8bit RAW data is going to help Leica get more battery life and processing time when the images are viewed.

The principle of Leica’s 8bit files is based on the fact that the data will need to be applied with Gamma. As discussed previously, humans see in a logarithmic scale. Human vision tend to have ability to distinguish shadows better then highlights, this could be exploited when 14 bit in linear scale are converted into non-linear 8 bits data.

A bit of reverse engineering

Leica’s DNG files are holding a DNG tag called “Linearization table”. This tag holds a lookup table with 256 entries where each entries is holding 14bit values. Lookup tables, or LUT in short are just big tables with input output values. Leica’s LUT is used to convert 256 input values into 14 bit output values.

There is a lot we can do with this LUT, and we’ll get to the actual use of it in a minute. Using a simple matching function we were able to reverse leica’s LUT to produce the actual LUT that maps 14 bits of linear data into 8 bits non-linear. The curve below exhibits Leica’s curve, reversed, in a 8 bit (with rounding)/8 bit scale.

Obviously when converting 14 bits (or 16 bits) into 8 bits, data gets compressed all over the scale. This LUT compresses data in such a way that data from the shadows gets compressed less than data from the highlights. The table below illustrates some key entries from the LUT (reversed) and the ratio of compressing.

In the 1st stop, every 111 bits are mapped to 1 bit in the 8 bit data
In the 2nd stop, every 77 bits are mapped to 1 bit in the 8 bit data
In the 3rd stop, every 55 bits are mapped to 1 bit in the 8 bit data
In the 4th stop, every 37 bits are mapped to 1 bit in the 8 bit data
In the 5th stop, every 28 bits are mapped to 1 bit in the 8 bit data
In the 6th stop, every 19 bits are mapped to 1 bit in the 8 bit data
In the 7th stop, every 16 bits are mapped to 1 bit in the 8 bit data

As you can see, the ratio in the shadows is lower then the ratio in the highlights. For example, in the shadows (6th stop) every 19 bits in the 14 bit data are mapped into 1 bit in the 8 bit data. Wheres at the highlights, every 111 bits are mapped to 1 bit in the 8 bit data. So steps in the shadows are more expressed.

Square root, a simple Gamma

Leica performs a Gamma function in order to transform the linear 14 bit data into non-linear 8 bit data. Normally, a camera manufacturer would use a Lookup table (LUT) to perform a similar function. A simpler way to perform the same task is by raising the pixel value with a power. Using exponentiation (power) is a limited function, capable only to shape the data in a single form. A LUT however, is completely flexible.

The term “Square root” has been used by leica to describe what is being done to transform 16 bit (according to leica) linear data into 8 bit non-linear. And indeed, the Square root of 65536 (16 bit) is 256 (8 bit). Also, because the way Square root is acting, by applying Square root on all the 16 bit data range, we would get a nice gamma curve similar to gamma 2.0.
This is because a gamma function is similar. Gamma is performed by raising a number with a power of gamma. For example, gamma of 2.0 would be: 65536^1/2.0 = 256.

Having explained that, I do not think Leica performs a Square root function on the data, because that would require 16 bit input data. It is more likely that Leica is applying a Lookup table (LUT) that performs a function that acts like a Square root, mapping 14 bit data in to 8 bit data. The chart bellow demonstrates Leica’s LUT along with the sRGB standard. Please note that part of the CIE lightness curve is being used by Nikon with their NEF compression method.

How it works

After Leica applies all necessary corrections on the RAW data, such as vignetting correction and cyan correction, Leica then applies the Gamma LUT on the data. The data is converted from what appears to be 14 bit of linear data into 8 bits of non-linear data. Leica then writes a reverse LUT, also called DeGamma, into the DNG header. This reverse LUT has 256 entries for input and 14 bits values for output.

The first step the Raw converter has to do, before it can continue with its normal image pipeline, is to reverse the data back into the linear domain. It does so by applying Leica’s stored LUT on the data, after which it can continue with its image processing pipeline. The diagram below should help you understand this process.

Capture 14 bit
LUT 14 bit => 8 bit
Result
Raw software
LUT 8 bit => 14 bit
Result
Proceeding
Color & Gamma
Final image
* use your mouse to roll over the above buttons

The negative

Other then small files, faster writing time and better battery consumption, there are no an other benefits. Although by using Gamma 2.0 Leica is taking advantage of the human vision and industry standards, the RAW data will always have 256 input bits. After applying DeGamma on the 8 bit non-linear data there are going to be gaps between the linear 14 bit data. Even if our eyes and computer screens can’t pickup those data gaps, there are still going to be problems with some issues:

Noise!

In the Poisson distribution light behaves, photon noise is larger as the data becomes brighter. By applying Gamma function on the data, noise is being averaged down in the highlights and starched out in the shadows, after which the DeGamma operation will reverse the function. Once the data has been manipulated that way, noise is being averaged down to tonal levels with no consideration of the structure of the image. Noise reduction algorithms are smart, and treat noise differently according to the surroundings of the pixel. For example, a simple algorithm will treat human hair differently than it would treat a flat surface.

The table below is the outcome of an experiment we tried. Using the Canon EOS 30D, we took a CR2 image of a B&W step chart. We then striped the CR2 file from any Canon headers and remained with pure 12bit Raw data in it’s bayer form. Using matlab, we then shifted the data to 16bit, and preformed a simple sqrt on all the data with rounding. The result was a 8 bit image of the chart, still in bayer form. The next step was to bring the data back to 16 bit, this was done by raising the values with a power of 2. The last step was to measure the standard deviation on each step in the image (still in bayer form).

Above: the step chart we used – evey step is 0.3EV.

Quantization problems

The M8’s files will not tolerant much of under exposure corrections. You are likely to see quantization quite easily when correcting your under exposed images. Also, post processing is likely to show some quantization problems. The histograms below are the actual photoshop histograms from the experiment we have done (described above). The histogram on the left is the full linear 16 bit Raw data and the image on the right is the decompressed image, also 16 bit linear. As you can see, there are gaps in Raw tonal data.

Highlights

Because of compression in the highlight, the M8’s DNG files would not have as much details as the next DSLR to recover by sliding the exposure tools of different RAW converters. Please note that ACR/Lightroom are using smart algorithms to recover details automatically once the slider is pooled into negative values – so ACR/Lightroom are not very suitable softwares to test this.

White Balance

In very extreme lighting temperatures, where the red channel or the blue channel are highly saturated, M8’s DNG files are most likely not holding much details at the highlights after the Green channel. Therefore, correcting white balance will likely result clipping and quantization problems of the dominant channel, red or blue.

Future improvements

We can’t be sure what people could think of next. Smart local toning? HDR printing? better smarter noise reduction? Maybe adobe will surprise us with something extra cool? Some of those could be relying on the RAW data being full 12 bits. The M8 DNG files might not have the ability to take the advantage of the next cool feature.


Digg this story   Add to del.icio.us





Related Posts:


6 Responses to “Solving the Leica M8 DNG riddle”

  1. Royi Says:

    Amazing analysis.
    How do you know Leica chose a “Software” Processor?

    Enjoyed reading.

  2. Guillermo Luijk Says:

    Interesting. I confirmed all these explanatios by studying 16-bit developed Leica M8 DNG linear histograms. The explanation of why holes appear in the high lights is clear looking at them. On the other side those 8 bit achieve almost the same detail in the shadow’s as Canon’s 12 bits.

    To see more histograms:

    Leica M8 vs Canon 350D histograms:
    http://www.ojodigital.com/foro/showthread.php?t=100967&page=42

    Leica M8 vs Canon 5D and DMR (16-bit RAW) histograms:
    http://www.ojodigital.com/foro/showthread.php?t=100967&page=43

    I developed the tool to plot 16-bit histogram. Can be downloaded here:
    http://perso.wanadoo.es/gluijk/soft/histo.htm

  3. Benoit Laplante Says:

    Dear Sir,

    If the M8 has a software processor, isn’t possible that they give us the option in the future by a new software release to choose the bit depth?

    Thanks for the very clear and well written article.

    Best regards,

    Benoit Laplante

  4. M8 now in KammaGamma color comparsion data base - Leica User Forum Says:

    […] Re: M8 now in KammaGamma color comparsion data base further reading at this site: 16 bit or 14 bit or 8 bit the Leica riddle KammaGamma Articles Solving the Leica M8 DNG riddle […]

  5. Best M8 DNG converter? - Leica User Forum Says:

    […] Zitat von KM-25 I have always exposed for the highlights with either the 5D or the M8 in medium to bright light. If I am on auto exposure with the M8, it is usually set at -1/3rd to -2/3rd as well…..but that is not really the issue here. Actually it is very much the issue here, one must understand how the DNG compresses the data then uses LUT’s to pad the data back out on decode. There is absolutely no headroom in highlights or the right 1/3rd of the histogram. Here a description to help you understand the limitations of the format (8bit squared) Leica has left us with. KammaGamma Articles Solving the Leica M8 DNG riddle […]

  6. oVan Says:

    That’s interesting… Sony stated that the DSC-R1 was a real 14-bit digital camera. Quote: “14-Bit DXP A/D Conversion: Sony’s 14-bit Digital EXtended Processor captures the range between highlight and shadow with up to 16,384 values, for extended dynamic contrast and detail.” The fact that it has 20Mb RAW files prove that it really contains that much information.

Leave a Reply