The Open RAW Initiative - response to Nikon and others

Bill;
Yes I have been reading what many people have been saying and then repeating themselves on many other forums.

I'm getting tired of reading the same individual complaining.

Now some people are solving their own problems by updating their
computers.

I also have written Nikon urging them to stand firm.

Now I am going to take pictures and enjoy life without reading
about any problems with anybody.

Birger

 

Bill;

If I were younger, I would fund a new company, find some smart programmers, call on Nikon [a visit not an Email] get the Nikon SKD and see what I could produce to meet the problems of the individuals who seem to take 2500 raw files per day or any day.



Birger

 

Bill,

I can understand the concern and desire for open format, but I do wonder though whether now is the right time for such a thing. Forcing an open format on the camera makers will mean restricting their ability to advance their technologies, and their technologies are not matured to the point where it makes good sense to apply such limitation, IMHO.

Forcing an open format on the camera makers would be almost like forcing every software maker to use the same SDK/API provided by the camera maker. So which one is really better?

And the reality is that it's much harder to get the camera makers to agree to an open format than for the software makers to simply keep up w/ the camera makers, IMHO. To wit, just look at what goes on in the audio and video industries. Look at what's been going on w/ SACD vs DVD-Audio, HD-DVD vs Blu-ray, and all the previous battles along those lines.

OTOH, I suppose it's debatable that it's best to go for an open format *before* the differing technologies mature and stances become intractable as in the cases w/ the audio and video worlds. But no matter what, getting these longtime rival competitors to sit down and cooperate is not much more than mere pipe dream. Even when faced w/ the serious threat of Microsoft and NT, the Unix community never got together well enough to get very far. Maybe if Microsoft bought out Adobe and one of the smaller camera makers (perhaps Olympus and push hard for 4/3 format or perhaps Fuji and develop around their SuperCCD) and started taking big market shares w/ DNG, then that might be enough to force the issue on Canon, Nikon, et al.

Birger,

I agree w/ you to some extent. I do find it a bit odd w/ those complainers who complain about the WB issue for dealing w/ tons and tons of RAW files. It's not like ACR ever got the colors right so that you can just use the in-camera WB setting and let things fly. I thought those folks just use a gray card for WB reference anyway.

Sure, it's a concern that Nikon *might* lock out other developers, but seems like most of them are just complaining for the sake of complaining.

_Man_

 

It seems to me that lots of people confuse "open format" with "open access". I really don't care if the format is open, so long as I can achieve what I want with the pictures I take.

I happen to prefer Capture's rendering over ACR's, probably because recovering highlights is less of an issue to me than overall image fidelity (I've heard ACR is better at recovering highlights, and I accept that at face value). For that reason, I'm not feeling the angst that a lot of other folks are right now, yet I still respect their feelings on the matter.

My point is simply that Nikon designs the system as a whole, and they have chosen to designate their API (as part of the SDK) as the "public" gateway to the data of a raw image; if that gateway doesn't provide the data that users need and want, then Nikon should be pushed to enhance the API instead of pushing them to publish the internals of a RAW file. I believe an open RAW format would ultimately create more problems than it's worth (my opinion of course, but based on a software development background).

I've never bought into the theory that Nikon is trying to sell more Capture boxes with all of this. But I do believe it is pointedly trying to establish the dividing line between where their system stops and the image data is made available to the user.

 

Thanks for the pointer Bill.

I am a fan of neither the FUD coming from Adobe nor the SDK lock-in advocated by Nikon. An open RAW format may indeed be the way to the future. I'd much rather that the owner of an existing format, such as NEF open it up, but I suspect all the growth of the format has been hardwired and kludged, hence the secrecy.

The open software movement, with few exceptions, has not produced the stellar successes it promised though. I would not mind an owned format that is open to inspection and extensible enough to be viable for future developments. I don't know the DNG is such a format, and it will be interesting to see what comes from Open Raw.

 

Whenever you work through an API you are limited to the parts the API designers think are important, thus limiting things that are currently outside the concept horizon of the designers.

An open format for something as complex as full color high resolution imagery will benefit from minds outside of the camera's manufacturer designing tools for manipulating the data. If outside designers are bottlenecked by too narrow of an api then the tools available to us users will be limited. I would like to be able to pick and choose which tools from different programs I would have in my toolbox. Unfortunately we are quite a distance from that type of modularity.

 

As a first step, I would suggest to compile a list of tags that allow for basic converter functionality, and ask camera-makers to "open" them.

This will also allow photographers to see how few of those tags are actually used by major third-party converters, and how irrelevant are "archival" fears.

Most important, that will allow everybody suggest the tags they want to have and think of how those tags will be used in practice.

 

Sounds like a good approach, Iliah, especially since the current atmosphere seems to be a bit too hostile towards the camera makers and not placing enough responsibility on the 3rd parties involved, especially Adobe, or even many of the complaining end-users, who seem not to want to understanding the situation -- there is definitely a consumerism mentality going on in the latter.

_Man_

 

Well, DPR just did an interview w/ David Coffin. It's an interesting read. Nice reference to the economic principle of "complements". Certainly, helps get a clearer big picture on what's going between Adobe and Nikon, et al. even though most of us probably had some ideas along similar longs already.

http://www.dpreview.com/news/0504/05042701davecoffininterview.asp

_Man_

 

The Problem is not thinking the Problem Through...

The problem currently is that camera manufacturers have not thoroughly vetted the issue of digital film.

Historically, for example, Nikon is a mechanical and optical engineering firm (That's what all their cameras were up until the early 80s). As time progressed, they had to start resorting to software to operate their cameras. (In the Aviation industry this has become known as fly-by-wire).

As they moved to digital, more and more software has been required to accomplish what a camera does. This software (aka firmware) is integral to how the image is accomplished in-camera and delivered to your Flash card.

So now, Nikon is a Mechanical/Optical Engineering (Hardware) AND Software Company! My guess, though, is that most of the Leaders of the company still are grounded in Mechanical Engineering and Optical backgrounds...and that the troops are the ones fighting the software issues. It takes a long time for a company grounded in hardware to understand the importance of software.

In the software business you have to define and account for the workflow that will be required to use the end product and you need to consider the full lifecycle of the resulting document. You also have to understand how to market it! It is my opinion that Nikon and probably all other current camera manufacturers have not done their homework here.

Best example: Flagrant disregard for the use of DEE as Nikon moved forward with its new NC feature of D-lighting. I have photos that are less than one year old in NEF formats that cannot be recovered with their DEE settings, NOW. I'm not even sure which NEF files those are!

Another example is simply the poor programming that has been accomplished with NC. It is in many ways outstanding and provides very good editing capability on your RAW files...but it still crashes way too often and it certainly is way too slow for many photographers who are trying to make a living. It is not a product that any software manufacturer would be proud of. It is a product that Nikon Management allows to be shipped because they do not understand the Software side of their business, yet. And these are the managers who are affecting your digital photo future.

By the way, the above problem would never have happened with a mechanical or optical issue.

Canon 300D users no longer have Manufacturer support for their RAW files. They are now totally dependent on 3rd party software vendors.

Unlike any problem we've ever faced in photography before, the integrity of your original is at stake! This is no small issue and should be the concern of every digital photographer who thinks "The Image Matters".

This should be your concern based on the fact that you take pictures and want them to last. This should be your concern no matter how young or how old you are. Now, if you are not bothered by this, at least make sure that you are archive your post-processed files in a TIFF format. The Tiff is one format that should protect one version of your image while the RAW issues get worked out.

I've tried to present my thoughts in a level headed manner. Am I passionate about this subject? Yes. Am I emotional - probably - but if you look at each of my statements I think you'll note they are mostly grounded in good logic. I'm typically a pretty happy-go-lucky kind of guy...but this is a serious subject that should concern every one of us here. (And I know that this forum is sort of a refuge away from the madness of DPReview...but some things you cannot totally get away from.)

Bottom line: Write Nikon a letter, please. You don't need to even advocate an open raw standard. Just ask them to think through the entire lifecycle of the photograph...and to consider the different use cases - From a fine arts photographer to an event photographer. If they did this we would not be worrying about the long term access to our precious soft copy negatives and we wouldn't be worrying about clumsy workflow with Nikon Capture, or inability to fully use other prominent 3rd party tools that we need to get our work done. If you are of a mind...tell them you'll spend $200 for Capture if that is what it takes to get it right!! I'm sure I lost $100 in productivity just writing this letter, alone.

With regards and respect -- Paul



Write to Nikon in Japan:
Makoto Kimura
President of Imaging Company
Nikon Corporation
Fuji Bldg., 2-3, Marunouchi 3-chome
Chiyoda-ku, Tokyo 100-8331
Japan

 

I think most of Paul's remarks are right on the money. The biggest benefit to opening standards (either by using a shared standard or just publishing the specs on internal standards) is that you know that in the future, it will be possible to read your original files.

I have a handful of old word processing files from a company that is long out of business. The only converters I could find (until recently) would, at best, extract the basic text from the files but lose all of the formatting. For some reason, after five or more years, I managed to find a free importer last week that successfully converted those old files to MS Word.

Now, I don't think Nikon is going out of business soon and I do understand their desire to protect their intellectual property, but in 10 years (which is a loooong time in technology years), will the latest software still read your original files?

A valid alternative is an intermediate format that preserves *all* of the information present in the original file, even if it requires a licensed (purchased) copy of software.

 

Paul (and others),

The thing is that this fear is in large part based on the assumption that 3rd party support would eventually cease just because of certain assumptions about what's going on now.

I would submit that's a very unlikely scenario unless the camera makers actually go to the extreme of properly using strong encryption to lock out 3rd parties *AND* effectively throw away the "key" themselves (by ceasing support themselves for whatever reason).

If there is actual $$$ to be made, somebody will more than likely pick up the slack. If there is no $$$ to be made, then no amount of letter writing will help. Now, I don't mean it has to be $$$ made directly from offering support, but could be indirect also. For instance, if Nikon went to the feared extreme and then ceases support for a lot of iterations of their NEF format while other makers do not do that, then Nikon will lose marketshare due to customers losing faith, which means $$$ lost.

We should recognize that much of what's happening right now is probably more about posturing between the Adobe and Nikon than Nikon necessarily heading toward that feared end. Even if Nikon decides to go the extreme route tomorrow for all future bodies, your current existing NEFs will still have 3rd party support and will still be accessible w/ today's Nikon software. Just don't buy Nikon anymore if it comes to that. You know the "funny" thing is that all this is coming back to bite all the Canon shooters who want a new model every 6 months because now Canon can't keep up w/ support -- today, it's D30, tomorrow 10D/20D/etc.

Of course, 3rd party support might not last forever if the costs involved are too great, but then again, if the costs are too great, then it probably means not enough people are willing to pay enough for the support. That would probably be the same underlying reason why a camera maker might cease support itself.

Sure, let all the camera makers, not just Nikon, know your fears, even if you only own Nikon gear -- afterall, you're a potential customer for non-Nikon gear. But I'd submit that most of the fears are a bit overblown and not accounting for market realities.

Also, if it really comes to that, it's a virtual impossibility for your RAW files to become so obsolete so fast that you have no opportunities to convert them to some other high fidelity format anymore, eg. 16-bit TIFF as you pointed out or maybe Adobe's DNG (ack! ). The only threat here regarding existing RAW files, and a somewhat remote one at that, involves future advancements of RAW conversion of your existing RAW files, not so much whether you can still convert them at all. Again, if and when support ceases permanently, just do a final conversion w/ the best latest available software. At least your RAW files and whatever other final format you choose are digital and won't wear away from errosion over time like film does.

RE: the example of word processing documents and such. I assume most of those cases either involved small companies w/ relative small customer base or are simply niche markets. Otherwise, somebody would be persuaded to make $$$ by offering support of some sort. Consider the computer programming field itself. Who's really writing new code in Cobol now? But you can bet there's still $$$ to be made just for the maintenance as long as the legacy stuff are still needed. If you know Cobol, go get that $$$ -- maybe I should go for that when I'm too old to keep up w/ the field. If you don't know Cobol, you can learn it.

Finally, I think we have to remember that these are all business entities w/ $$$ as their bottomlines (for the most part). If we believe there's a problem and want to be proactive to help solve it, we need to give proper consideration to the big picture for all involved, not just for our own fears of obsoleted photographs or whatever inconveniences or lack of choices. It does little good to merely yell at somebody to simply "do better" afterall.

_Man_

 

Bill,

I think I made that suggestion in OpenRAW list. Maybe it really deserves some discussion.

I know from experience that for now only few tags present in raw files are used in 3rd party raw decoders. Camera quality is often judged upon the results produced by those independent decoders; and it seems to be all the rage at the moment comparing cameras using same converter. But those decoders often level the cameras, using small subset of tags; and demosaicing/colour management/noise reduction (or lack of it, like the case of infamous banding in 20D) found in those converters do not make justice to the cameras. For camera makers this can become an inconvenience, as it is increasingly difficult to compete under given situation.

Paul,

Is it that previous versions of Nikon Capture supporting DEE are removed from Nikon site?