Blog / Foundations, Image Objects

Camera Raw Images

Peter Krogh
Wed Jun 10 2020

Today we turn our attention to camera raw images - those files produced by digital cameras and saved in a semi-processed state. They are quite different from conventional digital images which have an absolute value for each pixel’s brightness and color.

Camera raw images have revolutionized photography. They provide unparalleled flexibility for post processing by allowing photographers to fix defects. More importantly, raw files allow a broad range of subjective and stylistic interpretations.

Decoding raw images

A camera raw image file is encoded very differently than standard images described previously. This is due to the way camera sensors are designed as described below. The data must be decoded from its raw state into a more standardized digital image to be properly viewed or made use of. This decoding typically has three parts. These include demosaicing, linearization, and color transformation.

Mosaiced files

Unlike traditional color images which have a red, green and blue value for each pixel, nearly all digital cameras capture only one color value for each pixel sensor – typically red, green, or blue. Shown below is the Bayer mosaic sensor arrangement that many digital cameras use. (Cameras with the Foveon chip or those using large-format scanning backs create full RGB color for each pixel.)

In order to change the raw file into a standard rendered RGB format (even just for display on your monitor), the raw file conversion software needs to assign the missing two colors for each pixel. The software makes some educated guesses about how to interpret the color from neighboring pixels. This process is called demosaicing, and every program does it slightly differently. Demosaicing is the first part of raw image decoding because the interpolated RGB values are necessary for linearization and color transformation.

Most digital cameras only record one color value per pixel sensor. In order to change the raw file to a regular rendered file, the information for the missing two colors must be interpolated from surrounding pixels. This is a complex operation that throws a lot of math at your computer’s processor.

Linearization

The brightness information in a raw file is recorded very differently than the way it is stored in a standard digital image. If you were to display the raw file as a standard image, it would be very dark and low contrast. In order to decode the raw file, brightness information needs to be linearized which is done by applying a curve to the pixel’s brightness values. This transformation makes it look like a “normal” digital file, running from black to white with good contrast.

Color transformation

After demosaicing and linearization, the color for a raw file needs to be transformed. There are two basic parts to this. The first is a generic one that translates the color sensitivity of the camera sensor into “standard’ color. This is done through a color profile made for the specific camera model. The second part of the process is to set a white balance. This compensates for the color cast of the lighting in the original scene.

One of the primary advantages of raw file photography is the ability to choose a white balance in port production.

Here is what a raw file looks like as it gets decoded. The original image data is dark and compressed. It gets linearized and turned into a full color image. The image may then be optimized to the photographer’s satisfaction.

All software does it differently

The development of raw photography workflow has altered some of the basic understanding of digital images. Traditional image files and workflow are structured to produce identical renderings in different software. With camera raw images, there is no single “true” way to render a raw image. Every application renders raw files differently, and every program allows the user to make different adjustments. This is a fundamental break from the holy grail of predictable rendering that has been the goal of traditional image ecosystems.

Raw images are “capture” data, and inherently dependent on subjective interpretation. And as software and tastes in imaging evolve, we see that it’s quite common to reprocess images to get a better (or at least different) rendering. This malleability sets up an interesting dichotomy. We want the interpretations we’ve completed to be available, but we also want the ability to reinterpret in the future.

Raw files require read-only workflow

There are hundreds of variations of raw image files. And the nature of the files as described below make it essentially impossible to edit raw files by altering the raw image data. Instead, raw images require a non-destructive, read-only workflow. This reality was originally a major detriment to raw file workflow. But as tools have matured, the advantages of read-only workflow have turned the disadvantage into an advantage.

Even more components

In order to meet the challenges of predictability and flexibility that photographers want from camera raw workflow, an entirely new set of components is required. There is a lot of information that is needed to render a raw image, and there is an entirely different set of information that is needed to keep track and make sense of the renderings that have been created. Here’s a list of some of the information that is useful to bundle up with the raw image.

  • The raw image data, in original form - You’ll want to keep a copy of the mosaic image data if you want the most original version of the image.
  • Characteristics of the raw data - Because there are so many variations in how raw data is encoded, it’s important to define the exact characteristics of the original file creation, such as the exact Bayer pattern that is used and the layout of the sensor.
  • One or more profiles for converting the raw data - Changing the raw data into output-referred image data requires a linearization curve and a color profile. Given the large number of individual raw formats, it’s wise to bundle these with the image itself so it can be decoded in the future.
  • Settings used in raw file conversion - Saving a copy of the conversion settings made in the raw file software allows you to revisit the conversion and to either re-output an identical copy or to make incremental improvements without starting over in your adjustments. These settings should reference the application and version they were created with.
  • A rendered copy of the optimized image - Because the rendering you make from a raw file may not be easily repeatable, it’s often desirable to have a version of the file saved as a conventional image. Attaching an optimized version makes the final rendering available even if you no longer have access to the settings or software you used in the past.

The list is actually even longer than this. We’ll see a more complete listing when we look at the DNG format a little later in this chapter.

Is Mediagraph right for your organization?

Let’s find out together.

Book Your demo today