Lr and C1
Tagged: Capture One
AuthorTopic: Lr and C1 Read 26117 Times
ParticipantPosts: 169Lr and C1on: October 29, 2019 at 10:59 pm
You are welcome to rant John. …
Thank you. Kevin, I really appreciate your experience – both technical and artistic. That’s why I read your article and watch your videos. You are passionate about photography. I am too. I am also passionate about engineering. I beg you to consider that the (independent) engineer’s point of view is also important here.
I claim that there is nothing that proprietary RAW files can do that you can’t do with DNG. We can debate that. This is a testable hypotheses. It an issue that should be addressed in PhotoPXL forums, and elsewhere.
I am definitely interested in hearing Jeff Schewe‘s opinions. I read his book The Digital Negative some years ago – it was very influential. I’m not saying he was promoting DNG back then (although I thought he did, but regardless, he may have changed his opinion). He did say this in his book: “I need to point out that the DNG file format is advancing (currently at DNG spec version 1.4).” Actually, today the current version is still 1.4, which was published in 2012. In fact, DNG has only had to two widely adopted versions: 1.2 in 2008 and 1.4 in 2012. The fact that there have only really been two versions indicates that this format deals with features of data storage that are so fundamental they really don’t change with time – otherwise, Adobe would have addressed that. Moreover, the fact that there are such fundamental principles of image storage (and this is my turf) indicates that there is no need for proprietary RAW formats. I’d be happy to debate the technical aspects of those statements – but not here. However, note also that the current TIFF standard is 6.0 published in 1992, and I’ve noted that TIFF is still a popular image file format. The basic JPEG compression standard (T.81) was also published in 1992. There has been no reason to revise those standards because they got these fundamentals right. They have withstood the test of time, as has DNG.
My basic gripe, however, was not addressed in any of the responses. Proprietary RAW formats is one tool that this industry uses to stifle independent software development. The other issue I mentioned is that manufacturers don’t publish their WiFi, BT and even USB APIs (Application Programming Interface) is even worse. (As I pointed out, Sony used to do that, but they stopped – apparently to protect their in-house software.) I beg you to consider my comparison to the open developer networks supported by Apple and Android. Photographers could have substantially better software if this industry would adopt open standards.
If I’m right, then the photographic community is missing out on a plethora of innovative software that an independent developer community could provide. Moreover, that developer community already exists! We only need to turn them loose.
Gold MemberPosts: 132Re: Lr and C1Reply #1 on: October 31, 2019 at 1:58 am
With regards to DNG, I personally don’t use it in my normal workflow for several practical reasons. First, I use Lightroom and Camera Raw so if I convert to DNG and make a small parameter change in the file, the entire DNG file needs to be backed up on a files changed criteria. If I use a proprietary raw file and side car file then any changes only cause the tiny .xmp file to be updated. You might think this isn’t a big deal but I’ve gotten use to large digital captures and while DNG is an efficient format an 45MP capture is still about 50 MBs when converted to DNG. Ironically, the Nikon Z7 raw is 90 MBs so the DNG file is better compressed than the Nikon raw.
The other reason I don’t use DNG as a matter of workflow is personal in that I want to be able to have the original raw file for provenance so I can always say, “here’s the original untouched raw file”…
I do however strongly support Thomas’ efforts to provide a standardized raw file format and Adobe has offered DNG to the ISO for inclusion in the next rev of TIFF-EP (Tiff for electronic photography) which is the standard pretty much all camera makers use for their proprietary raw files.
As far as the impact DNG has had on the industry, it’s already done what Thomas really wanted to do which was to teach camera makers hw to create raw file formats–something Nikon and Canon had a hard time doing…remember the whole encrypted white balance the Nikon did in their raw files? Seems their software engineers didn’t know what they were doing and accidentally resorted to encryption to have their own software read their own raw files. Canon had their problems too…
As it is now, all of the camera makers’ raw files are well (or better) formed raw files that unfortunately are undocumented and proprietary :~(
The funny thing is most people don’t know that Camera Raw and Lightroom (in the Develop module) always convert the proprietary raw file to DNG on the fly. Thomas did that so ACR & LR could read the EXEF metadata and convert the file to DNG to apply the default settings too.
I will say that DNG does make for a nice and useful conservation and preservation raw file format container and is supported by the Library of Congress’ digital object conservation efforts. DNG also makes fo a good raw file format transport format in that the development settings are enclosed in the file and not in a separate sidecar file that could be lost.
BTW, there are two basic flavors of DNG files, there’s raw file DNGs such as what you would get out of a Leica or taking a raw file and converting to DNG and there’s also a Linear DNG that is no longer fully raw in that the color filter array (CFA) photo site data has ben interpolated by demosaicing. The file is still a linear gamma file without any color space conversions. So, one could say it’s a half-baked raw file :~)
The linear DNG is useful for certain files such as HDR and Panos that need demosaicing in order to do the merging of actual pixels.
Hopefully Capture One will expand their support for DNG…it would be useful for their user base.
ParticipantPosts: 169Re: Lr and C1Reply #2 on: October 31, 2019 at 11:04 pm
Jeff – thank you for your reply. I am particularly please with your support for open standards. I have further comments and questions.
“There are two basic flavors of DNG files, there’s raw file DNGs … and converting to DNG and there’s also a Linear DNG that is no longer fully raw.” True. Raw data is data as it comes off the sensor’s ADC. DNG supports raw data storage along with all the tags that store all the sensor specific data needed to linearize it. DNG can be a true RAW file. It is also true that DNG conversion produces “linearized-DNG” files. I would add that there is a third format, the floating point-DNG, which is the DNG file that Lightroom’s HDR merge produces. This came up this thread because Mike Nelson Pedde indicated that C1 doesn’t recognize Lr’s HDR.dng files. I pointed out that that is most likely because of the floating point data format used by those files.
By the way – all of these files are TIFF files. TIFF is Tagged Image File Format. The tags are like a table of contents for the file that point to various segments of the file contain data. There are tags for image data, tags for metadata, tags for data format, etc. TIFF-EP adds additional tags – mostly metadata. Proprietary RAW files are TIFF files with proprietary tags – they point to data, but you don’t know how to interpret that data unless you pay a royalty and sign a nondisclosure. DNG is a TIFF file with additional tags defined in the open standard DNG specification. JPEG files also use TIFF tags. An exceptions are the ICC profiles that use a different format. ICC profiles are embedded in TIFF and RGB files (to specify the color space) – but again as data blocks pointed to by a TIFF tag. I know this stuff because I’m write Mac software that accesses these files. I have to understands these various file format standards in great detail. Hopefully, you’ll see some of my work soon in an article on stops histograms that I submitted to this site.
“The funny thing is most people don’t know that Camera Raw and Lightroom (in the Develop module) always convert the proprietary raw file to DNG on the fly.” Of course they do! Moreover, it is not just Lr Develop and ACR, anybody’s raw conversion does exactly the same thing as the first step. C1 is no different. (I’m not saying they do a full DNG file-to-file conversion, but the definitely do linearization first.) The linearization steps are described in Chapter 5 of the DNG specification, which is just 1 page of the spec. These are linearization of the nonlinear photodiode response at the top end near full well capacity, and black level subtraction (which is a pattern bias largely due to the photodiode dark current). It is a single input – single output correction that depends only on the characteristics of the sensor’s photodiodes. It donesn’t change with time and there won’t be better new algorithms in the future.
So, you save your pristine raw data, but every time you want to access it you’re going to do the same linearization. I just don’t see the point. Just what are you preserving with the true raw data? Why are you willing to forgo the clear benefits of an open standard – a point of commonality justified by everything we know about image processing theory? Do you expect there will be an image that couldn’t get from your DNG conversion, but could from your proprietary RAW file? If so, please explain how that might occur.
Lastly, I don’t understand your statement “make a small parameter change in the file, the entire DNG file needs to be backed up.” I’ve heard this before, but I don’t understand where it comes from. My understanding is that when you make adjustments in Lr, the adjustments are stored in the Lr catalog, or sidecar file, – not in the DNG file. I tested this. I checked the modified date in the Mac finder of a DNG file – which was the same as the import date. I made adjustments, closed Lr, and back in the finder there was no charge to “modified date.” If the DNG file is changed, why doesn’t the Mac Finder recognize that? If I’m mistaken about this, I’d really like to understand what I’m missing here.
Please forgive me for my brashness. I know I am a nobody in the photography world, but I do understand data storage engineering. I’ve asked several photography experts these questions, and to date I haven’t gotten satisfactory answers.
- This reply was modified 4 years, 1 month ago by John Sadowsky.
Mike Nelson Pedde
ParticipantPosts: 641Re: Lr and C1Reply #3 on: November 6, 2019 at 8:28 pm
John: You and Jeff are way, way beyond me on this, but as I understand it, metadata is stored in one of two ways – internal to the image file, or as a sidecar file. Raw files each have an .xmp file that goes with the image data. The others: .dng, .tif, .jpg, etc. all store metadata as part of the image file, so what Jeff says is correct. If you modify the .dng file, you need to rewrite it. Now, with Lr, by default it reads metadata on file import and stores that information in its database (catalogue) – and then continues to store any metadata changes (keywording, image processing instructions, etc.) only in the catalogue. You can tell Lr to automatically write changes to the image or .xmp files or you can do it manually (Ctrl-S) but Lr won’t store the changes in your .dng file unless you tell it to do so. If you write out the metadata from Lr and then import the image into Capture One for example, it will read most of the information but the processing pipelines are so different that any edit changes will be ignored.
P.S. I know both Nikon (Capture NX) and Hasselblad (Phocus) have claimed that their proprietary raw formats allow them to do things in their (proprietary software) that would not otherwise be possible, but I’m not going to get on one side of that argument or the other.
Mike Nelson Pedde
Silver MemberPosts: 4Re: Lr and C1Reply #4 on: January 1, 2020 at 9:07 pm
When Apple dumped Aperture I evaluated both Lr and C1 extensively. I preferred C1 for editing but went with Lr for the DAM and integrated workflow enhancements like pano stitching, HDR, and geotagging. When Adobe went to a subscription model I avoided upgrading until this week. Apple’s latest operating system upgrade finally made Lr 6 unusable. I am not happy about the recurring expense for Lr, but the new version at least makes my images look better. I considered switching to C1, but C1 still has the same disadvantages.
I am a bit surprised that C1 has not made more advances in the area of DAM. The session workflow works well for some of their core customers, but is inadequate for others. Over the last three years I have used C1 with both Phase One and Nikon cameras as I developed imaging systems for my employer. Sessions work well for small projects. Both the session and catalogue systems are inadequate for enterprise level projects. In production the imaging systems I worked on generated terabytes of images that needed to be shared across the enterprise. Custom systems were in development to deal with the data. Sessions work well for many of C1 photographic customers, but industrial strength DAM should appeal to their cultural heritage and industrial customers.
DAM in Lr has shortcomings. The catalog is not conducive to a multi-user environment My wife has never liked that she cannot get to our family photos. Adobe’s cloud initiatives may offer good solutions to these problems, but the cost may be prohibitive.
If C1 invests in improving their catalog system they could entice many more Lr users to switch.
On APIs, Canon, Nikon, and Sony have well documented APIs to remotely control their cameras. Phase One has an API to interface with C1. The API can be a very important factory for camera selection for some applications. Existing APIs are good enough to enable exciting applications. Take a look at Sony’s RX0 Multi-Cam capabilities. Time and funding are bigger challenges to using the APIs than the functionality they contain. What would really help is an industry standard API. Then some of the smaller camera companies could have support for features like full tethering.
- This reply was modified 3 years, 11 months ago by Jeff Burns.
Mike Nelson Pedde
ParticipantPosts: 641Re: Lr and C1Reply #5 on: January 3, 2020 at 7:41 pm
I have both Lr 6.14 and C1 20. Like you, Jeff, I’ve been increasingly using Lr for DAM and C1 for processing my A7R III files. However, in the past little while Windows 10 has also made Lr increasingly unusable for me. Reboot the computer and it’s fine, but then it starts to d-r-a-g to the point where it takes several seconds to load an image, and I receive multiple not responding errors. I’ve tried any number of ways to circumvent this, but nothing works. This being a new month and a new year, I’m going to try making the jump more to C1 and leave Lr back in 2019. We’ll see what happens.
P.S. A good friend is still using Lr 6.14 under Windows 7 and he’s had no problems. Unfortunately, Microsoft is no longer supporting Windows 7 after January 14. Time marches on…
Mike Nelson Pedde
- You must be logged in to reply to this topic.