D300 Raw compression

Discussion in 'Nikon DX DSLR' started by Ping279, Jul 26, 2008.

  1. So I have been shooting RAW 14 bit uncompressed and the files are coming out of the camera at about 25mb. The manual says that shooting with the lossless compressed does not lose any quality but the file size is 20-40% smaller. Is this true? Do the files actually have the same quality as the uncompressed?
     
  2. Panopeeper

    Panopeeper Guest

    Yes. There is no reason whatsoever to create uncompressed files.

    Note, that earlier models supported only lossy compression. Perhaps it is a psychological move of Nikon to offer the uncompressed version as well.
     
  3. tintingkc

    tintingkc Guest

    Been using lossless with great results. Don't see a difference in my eyes.
     
  4. Is there that big a difference between lossless compressed and plain ol' compressed (in 12 bit)? It was my impression that an NEF file is a lossless format to begin with.
    Without a doubt my eyes are far less "trained" than most, but is there that much of a difference that I should sacrifice twice the card space for it?
     
  5. mood

    mood

    Jun 27, 2007
    suburbia, ny
    I use 14 bit lossless

    I believe , this is what T. Hogan suggests also, FWIW
     
  6. Thanks Mood. Being primarily a wildlife shooter, I'd have a hard time sacrificing my fps on top of deciding between lossless comp. and plain ol' comp.
    Reading Hogan's review, as well as other threads, I can see where 14 bit will have a benefit in static situations (i.e., portraits and landscapes), but why is NEF standard compression now seemingly referred to as a "lossy" format in favor of lossless compression?
     
  7. mood

    mood

    Jun 27, 2007
    suburbia, ny
    good question
    I have no answer, but I'm sure someone will chime in here that does
     
  8. Panopeeper

    Panopeeper Guest

    There is no "standard compression" on the D3, D700 and D300. Whenever you want to create a compressed raw file, you have to decide if it should be loossily or losslessly compressed.

    Earlier models offered the choice from uncompressed or lossily compressed; the D70 (what else?) did not offer any selection, it always compressed the data lossily.

    There is no serious reason now to use the lossy compression, except for the speed (and that too can be solved). The dispute about the loss being relevant or not will go on for a while; however, one thing is sure: the lossless files are not worse than the lossy ones, while the other way is disputed.
     
  9. Well I think that I will continue using 14bit uncompressed since I shoot primarily products, still life, and other commercial "set up" shots for companies and magazines. Thanks for all the help guys.
     
  10. Panopeeper

    Panopeeper Guest

    I'm afraid you do not understand, that "lossless" means "lossless". There is absolutely no reason to create uncompressed files now, that the compression does not necessarily cause losses.
     

  11. I was merely using "standard" so there was a clear separation of the two choices I was referring to in the D300 menu. Far better than losslessfull....lesslossess...looselippishly.....awww heck you know what I mean! :tongue::biggrin:
    Certainly it will be a point of mild debate as to which is better, or whether it matters at all. It will come down to one's preferences and needs.
    I swear I'm not trying to beat a dead horse here, but I've never heard of a RAW file being a "lossy format" until I saw it in the D300 menu. I know the argument in the D200 era regarding compressed/uncompressed RAW was the ever so slightly more efficient recording of highlight information.

    I'm a RAW convert from 2 years back, and I'm grateful for it. Of the 3 choices now (aside from the bits), uncompressed and lossless compressed files take up pretty much the same amount of space on a card, give or take a handful of exposures. How much different are they from each other? There must be some difference I'm sure or why make it that way. Percentage wise, I wonder how detectable it is, and is it such a vast difference from the information I'll record if I choose "compressed" :wink: instead?

    With the advances in image processing applications and cameras shooting at 12+mp, are we simply delving into technological minutia? I'm thinking maybe. For the moment, I'd rather get 400 exposures on my 4GB card.
    But I'm not beyond being convinced otherwise. I'd love for someone to be able to show me or explain to me all that I may be losing by choosing compressed :wink: over lossless compressed.

    Now that I've gotten that off my chest.......who wants a beer? lol :biggrin:
     
  12. Well thats what the debate has been about. I have heard people, not on the forum, say that there are differences but on a minute scale. I do not know if it is worth it but I have nothing to lose since the majority of the shots will be going to clients for print use. I just want to cover my bases and make sure I have the utmost quality I can get from my camera.

    I dont mean to dig this hole any deeper but... why would the file size be so much larger with uncompressed? The data has to be coming from somewhere compared to the lossless compressed.

    Either way, thanks for trying to straighten this out for me :smile:
     
  13. Read up about the ZIP compressed format (for any type of file) for example. Or .SIT for the Mac (excuse my ignorance if there is a more recent format for that...), or gzip in Unix, etc...

    Compression is used to shrink the size some information takes without losing any information. That is good for text, as well as other things. The algorithms used behave better with certain types of data: for example text compresses a lot. Images don't compress as much, or as well with these algorithms, so other compression methods were developed to deal with images, some lossless, RLE encoding for example, and some lossy (because the human eye does not perceive all the information encoded in an image at once). JPEG is an example of a lossy format: that's because the (spatial) image data is replaced by the frequencies of information the image contains, effectively encoding the edges in the images (gross oversimplification, I know).

    The original NEF format is uncompressed and roughly is the size of the pixel data: so for a 10 Megapixel camera, at 12 bits per pixel (or 1.5 byte per pixel), you need 15 MB. Then, add a bit more for the EXIF, etc... The original (eg D70) NEF compressed format was called "visually losless" by Nikon, another way of saying lossy. Effectively, they reduced the number of levels encoded in the highlights and achieved a very good image quality vs. size compromise. The D70 NEF files are about 6MB, when they would have been 9MB uncompressed (6 megapixels x 1.5 byte).

    With the D300 and the D3, and now the D700, a new format, called "lossless compressed" is now available. It is equivalent to the ZIP, SIT, gz, etc... compression that you are familiar with for general computer use. I don't know which algorithm is used in this format, but lossless means that there is no difference at all in contained information (not one bit) between the compressed version and the uncompressed one.

    So, there is absolutely no reason whatsoever to ever want to use uncompressed NEF files on one of these recent cameras.

    So, why is the compressed file smaller? Because the data is reduced to its "information" content (not all data is information, think about that...), and that information is itself compressed. For an example of compressed encoding, read about Huffman codes
     
  14. Wow, I should get a diploma for just reading that! Pretty interesting stuff.
     
  15. Panopeeper

    Panopeeper Guest

    Look at following two histograms of the raw data. Each column represents one particular pixel value, the entire histogram reflects the pixel values from 0 to 511. Both are from the D200. Gaps in the bars mean, that no pixel with the respective value occured.

    The first one is from uncompressed data, i.e. lossless. Almost all pixel values are "used", except for some regular gap, let's ignore that here, for it has nothing to do with compression and lossiness.

    [​IMG]

    The next one is from the D200 compressed data, which is always lossy. Up to about 240 there is no loss, then the "thinning" starts; from about 340 the thinning gets stronger. All together, about 620 different pixel values (tonal level) are kept from the 4096 (minus the original "gaps").

    [​IMG]

    In effect several different pixel values are stored with the same value; this "thinning" gets stronger in the highlights, which is normal, as the non-linear mapping does the same anyway. Problem can occur in form of posterization, like in smooth skies or if the contrast gets increased very strongly.

    Finally, look at complete fine histogram of a 12-bit and of a 14-bit lossy files:

    D3 12 bit lossy compression

    D3 14bit lossy compression

    The difference in file size (compression gain) is achieved by storing the differences between adjacent pixels of the same color. When decompressing, the program calculates the original (absolute) pixel values. Thus there is no loss incurred; the result is bit for bit identical to the uncompressed version.
     
  16. That's some serious info Gabor and definitely worth chewing on. Might actually have to go with lossless compressed, and certainly keepp in mind the 14 bit when the circumstances allow, and even sometimes when it doesn't! :eek::wink:
    Thanks for sharing that info.

    Thanks to Nathan as well for starting this thread. Good stuff.
     
  17. ANDS

    ANDS

    166
    Aug 2, 2008
    Sacramento
    Reminds me of the "regular" versus "premium" gas debates that folks would have on motorcycle forums.
     
  18. NateD

    NateD Guest

    When I first got my D300, I immediately started shooting 14bit and it wasn't until I shot my first football game and noticed that I couldn't get the 6-7fps I got the camera for so I went back to 12bit
    I shoot 14 for my portraits and stills (landscape, real estate, etc) and whenever there's action, I compress....
     
  19. Panopeeper

    Panopeeper Guest

    My understanding is, that the frame rate depends on the bit depth, but not on compression.

    It is reasonable to record in 14bit such images, which may require extensive post-processing, and in 12bit those, which are recorded in raw only to change WB and make minor adjustments.

    However, I do not know of circumstances justifying recording uncompressed image data, no matter if 12bit or 14bit.