When the DNG format specification was announced at last year’s Photokina, a little touted free application was also announced called Adobe DNG Converter. The application is for the conversion of undocumented, proprietary Raw files into the publicly documented DNG format. At the time, many people played with the DNG Converter and decided that while interesting, it didn’t really add anything to their raw processing workflow except perhaps adding a step. As a result, most people don’t really use a DNG workflow.
(Updated 05/25/05 at 4:00PM Central)
Well, things have changed, and now both the DNG file format and the Adobe DNG Converter do have something substantial to offer.
To understand the benefits of a DNG workflow, you must first understand how to use it. In its simplest form, you take proprietary raw files and use the DNG Converter to convert them into DNG. The application itself is very simple. It can live anywhere, in your applications folder or even just sitting on your desktop-that’s where I tend to keep it. You can either double click DNG Converter to launch it or just drag and drop a folder of Raw files onto the app icon. Either way, you’ll be presented with the standard Adobe DNG Converter dialog.
At the top, at Step 1, you can browse to select the folder you wish to convert. If you are going directly from a compact flash card directory, you’ll want to select the option to Include images contained in subfolders. This option allows you to essentially blow past the 100 images per file folder limit on compact flash cards and process all Raw images on a card.
Step 2 is to select the destination folder. You can choose to convert in place, however, there are some significant reasons you really may not wish to do this. First off, keeping both Raws and DNG files in the same folder can cause problems when updating Camera Raw settings and adding metadata. Second, converting to DNG and keeping them in the same folder as the originals does nothing to duplicate the locations where the files are stored-and that is something you should be doing. Ideally, you should never have only a sinlge copy and location where your Raw files are stored. The rule is, duplicate locations and if possible, multiple media.
Step 3 is to choose whether or not you wish to alter the DNG file name. In the past, I’ve argued against altering the raw file names because of the risk of effecting the original proprietary Raw files. However, DNG Converter allows you to change the converted file while preserving the original Raw file format name and numbers inside of the resulting DNG file. So, if need be, you’ll always be able to determine the original file name that the DNG copy came from.
Step 4 is rather important and your options must be chosen wisely based upon your specific needs.
The DNG Converter preferences allow you to select various options in the conversion process. In general, you’ll always wish to select the Compressed (lossless) option. Depending on the proprietary Raw file format your camera produces, you can generally expect to save between 10-30% or more by compressing the DNG. You will lose nothing-the compression is completely lossless. For some Raw file formats, the compression can be considerable. In only a few cases, the compressed DNG may actually be slightly larger. The Nikon D70 for example uses a lossy compression scheme-DNG files made from D70 files will grow by the difference between the lossy scheme used by Nikon and the lossless scheme used by Adobe. But, in general you can expect to see file size savings-which if you shoot a lot can contribute to space savings on hard drives.
The Image Conversion Method allows you to either keep the DNG in the un-demosaiced form or to demosaic the file and convert to a linear file. Generally, you’ll want to keep the file in it’s un-demosaiced form as there is a space savings. Linear DNGs have already been converted to RGB files and therefore are larger.
Editor’s note: Bruce Fraser sent the following comment regarding Linear DNG: “The only reason to use Linear DNG is to feed the file to a DNG reader that can’t understand the particular flavor of DNG that DNG Converter creates for the specific camera, or to act as an interchange format-e.g., lens correx with DxO can write out a linear DNG that ACR can read. This is typically NOT an option people want, not because of size considerations, but because it’s no longer really raw-it’s half-baked. All the operations that take place during demosaicing are set in stone and can’t be redone.”
The big question is whether or not you wish or need to embed the original proprietary Raw file directly inside of the resulting DNG conversion.
If you do choose to embed the original proprietary Raw file inside the DNG file, you will be able to extract the original Raw file in the future. The DNG Converter can batch Extract Raw files just like it can do DNG conversions.
This is an important decision. If, for reasons of image verification or provenance, you may need to provide the original of a digital image, you should probably embed the original proprietary Raw file inside the DNG file. In scientific or forensics photography, the ability to extract the original raw may be required. The extracted Raw file from the DNG is a bit for bit copy of the original file so one could examine the original Raw file. The resulting DNG file will be the size of the converted DNG file plus the addition of the original proprietary Raw file combined. It will still be smaller that a converted 8 or 16 bit/channel tiff file. It will also probably be smaller than a linear converted DNG file. But, you will pay a file size penalty.
So, what about a DNG converted file will be different than the original proprietary Raw file? If you are shooting with a camera that outputs either NEF’s or CR2′s (Nikon and Canon), essentially, you’ll lose nothing. Starting with Adobe DNG Converter version 3.1 (the one released with Camera Raw 3.1 in May, 2005), the DNG Converter will take all the proprietary metadata from the original file format and safely move it to the private portions of the fully documented DNG file. This is a new, not so widely known addition to the DNG Converter v3.1. The next upgrade to the DNG Converter will add even more file formats.
Editor’s Note: Thomas Knoll has clarified what is and is not migrated from the proprietary raw file to DNG files upon conversion. DNG Converter 3.1 and above does indeed move all EXIF private maker note from all TIFF-EP based raw file formats, not just NEF and CR2 files. The file formats that do NOT use EXIF based private maker notes are not migrated. File formats from; Kodak, Foveon and Leaf do not use EXIF private maker notes, so those files are not supported for migration. Additionally, Canon CRW files and Fuji RAF do not completely adhear to the TIFF-EP and EXIF spec. So some metadata will be migrated and some will be stripped. Also, black masked pixels (the far outter edges usually cropped off by conversion software) for certain files, such as some CR2 files are also not preserved in DNG conversion. Future updates to the Adobe DNG Converter will continue to add to the scope of what is preserved and migrated to DNG.
So, if you shoot NEF or CR2 you will, in essence, loose nothing by converting to DNG. Even the Nikon D2X encrypted As Shot white balance is moved to the converted DNG file. It isn’t decrypted, but it is safely moved. As a result, if, at some time in the future Nikon grants Adobe permission to decrypt the WB data, it’ll still be there in the DNG file waiting to be used, as will all the other undocumented metadata.
If your Raw file format is not NEF or CR2, some or most of the proprietary, undocumented metadata found in the private maker notes of the EXIF specification will be stripped out. Is this important data? Hard to say since it’s undocumented. However, it’s metadata that Camera Raw can not currently use so if you are satisified with the way Camera Raw can convert your file, then the proprietary metadata may be meaningless to you.
The addition of the option to embed the entire original proprietary Raw file inside of the DNG file and the recent ability to move private maker notes into non-embedded DNG files have made a difference in the way you should look at a DNG Workflow. More about the benefits later.
Once you decide what flavor of DNG you wish to convert to, you simply press the Convert button.
How fast is the DNG Converter? The speed will depend upon the size of the original Raw file as well as both your processor speed and the speed of your hard drives. For example, the files shown above were Canon EOS 1D MII 8 MP CR2 Raw files. The processing times quoted below were done on a 1.5 GHZ G4 17″ laptop using the normal internal hard drives. Certainly not slow, but far from the top of the heap these days.
Straight DNG Convert: 3.5 seconds
DNG + Compressed: 4 seconds
DNG + Compressed + Embed: 6 seconds
DNG + Compress + Embed + Linear: 8 seconds
Compressing the file adds a bit to the processing time. So does embedding the original Raw file. Doing a compress, embed and linear conversion is the slowest. What were the file sizes after conversion?
The original Raw folder contained 12 CR2 files and was 83.2 MBs.
The DNG files without embedding but with compression was 66.8 MBs. A savings of about 20%.
Converted while compressing and embedding the original Raw file the size is 145 MBs.
Even with the embeded Raw file, that’s still far smaller than converting all the files to 16 bit/channel tiffs. DNGs with the embedded Raw files are even smaller than 8 bit/channel tiffs.
So, what are the tangible benefits to converting your undocumented, proprietary raw files to DNG? Aside from the very real and serious danger to long term preservation of undocumented and proprietary Raw files, (see the PSN story Digital Preservation) you will end up with Raw files that are smaller (in most cases), will allow for the safe adding of your own metadata such as IPTC and keywords directly in the DNG as well as storing the Camera Raw settings, again, right in the DNG file.
On the left are the original Raw files with their required XMP side car files. On the right, the DNG files have had the XMP metadata safely written directly into the converted DNG file format. If you must deliver your raw files to clients, this is an ideal raw delivery format since you can embed IPTC, keywords and your suggested Camera Raw settings directly in your files.
So, is the DNG Workflow for everybody? Perhaps not. As outlined above, it does add a step in your raw processing workflow. You get the benefit of knowing your proprietary Raw file formats have been converted to a fully documented Raw file format. You will probably see file size savings of 10-30% (or more, perhaps less). You have the ability to store not only the DNG converted Raw file but also embed, directly in the DNG a bit for bit copy of the original Raw file should you need it. You also get the benefit of having your Raw files stored with all your metadata embedded directly in your DNG file so you won’t need to worry about side car files at all. You get the potential benefit, depending upon the EXIF jpg thumbnail size in the original Raw file, of better performance inside of Adobe Bridge. If you shoot with multiple cameras possibly from multiple manufacturers, you also get the benefit of having all your Raw files in the same format.
If you need to deliver your Raw files to clients, something I try to avoid, it allows you to set all your metadata including Camera Raw settings directly in the DNG files. With your conversion settings from Camera Raw and your metadata locked into the files burned on CD or DVD, you can be assured that clients will have both your vision of what the file should look like as well as your name and contact info locked into the file.
Is all of the above worth an additional step in your raw processing workflow? It is for me. Since the release of Adobe DNG Converter 3.1, I’ve decided to convert all of my proprietary Raw files to DNG. I generally don’t bother with embedding the original inside the DNG file. There may be times, when creating a workshop demo for example, where I’ll also keep the original proprietary Raw files. But only for the purposes of demos will I be keeping the undocumented and proprietary raw files.
In the DNG Workflow Part II, I’ll outline a novel concept, using the Adobe DNG Converter at the point of original Raw file ingestion. Stay tuned.