Lr and C1

Tagged: 

Viewing 15 posts - 1 through 15 (of 19 total)
  • Author
    Topic: Lr and C1 Read 1013 Times
  • Mike Nelson Pedde
    Mike Nelson Pedde
    Participant
    Posts: 456
    Capture One
    on: September 26, 2019 at 4:56 pm

    Hi There: I still have and use Lr 6.14 and am learning C1 12. Personally I find the DAM aspects of Lr better, at least for now, but with the A7R III files I prefer the processing in C1. My current workflow is to dump everything off the camera into Lr, sort out the dross and import the rest into C1. Yesterday I was out on a shoot and made a number of bracketed exposures. Brought everything into Lr, and used the built-in tools to created HDR images from the brackets. Also made a couple of quick panos. I don’t think Lr is necessarily the best tool for either of these, but in this case they looked fine to me. Lr renders both the HDR merges and the panos as .dng files. Today I was going through the images in C1 and kept thinking there were images missing… I finally figured out that C1 imported the pano renders, but none of the HDR merges. I even tried creating a collection in Lr of the 10 HDR merges, exporting that collection as a catalog and importing that into C1, but it just said the images were incompatible. Fortunately I still have the original raw files.

    Just an FYI.

    Mike.

    _____
    Mike Nelson Pedde
    Victoria, BC
    https://www.wolfnowl.com/

    Kevin Raber
    Kevin Raber
    Keymaster
    Posts: 554
    Re: Lr and C1
    Reply #1 on: September 26, 2019 at 9:23 pm

    Mike, C1 will work with all JPEG and Tiff files.  Once your panos are finished in LR make sure to export them as TIFF or JPEG and then import those into c1.

    Kevin Raber
    CEO & Publisher of PhotoPXL.com and Rockhopperworkshops.com

    Mike Nelson Pedde
    Mike Nelson Pedde
    Participant
    Posts: 456
    Re: Lr and C1
    Reply #2 on: September 27, 2019 at 1:26 pm

    Thanks, Kevin. As I mentioned, I have other tools for doing both HDR and panoramas; what surprised me was that C1 had no problems importing Panorama.dng but the HDR.dng files were incompatible. Go figure. Not a criticism of either program, just a head scratcher. In this case I did export/import .tif files. ?

     

    Mike.

    _____
    Mike Nelson Pedde
    Victoria, BC
    https://www.wolfnowl.com/

    Jeff Schewe
    Jeff Schewe
    Participant
    Posts: 42
    Re: Lr and C1
    Reply #3 on: September 29, 2019 at 1:18 am

    C1 (Phase One) doesn’t really fully support DNG…in fact the support is very limited to only those cameras C1 already supports (as far as I understand). To be fair, There are very few 3rd party apps that can read or write linear DNG files…DxO can write DNG files but I’m not sure DxO can read and process an ACR/LR created linear DNG because while the file has not he demosaiced, the white balance hasn’t been done.

     

     

    Mike Nelson Pedde
    Mike Nelson Pedde
    Participant
    Posts: 456
    Re: Lr and C1
    Reply #4 on: September 29, 2019 at 3:40 am

    Thanks for the explanation!

    Mike.

    _____
    Mike Nelson Pedde
    Victoria, BC
    https://www.wolfnowl.com/

    John Sadowsky
    John Sadowsky
    Participant
    Posts: 99
    Re: Lr and C1
    Reply #5 on: October 25, 2019 at 7:08 pm

    C1 (Phase One) doesn’t really fully support DNG…

    I’m very impressed with the videos on Capture One that Kevin has provided.  I’m starting to have concerns about Adobe – I think Lr CC is a disaster.  The videos are compelling enough to make me seriously consider dumping the Adobe CC Photographer $10/month plan.

    However, I’ve converted most of my RAW files to DNG, so not fully supporting DNG is a problem.  DNG converted files contain linearized raw data in a 16 bit fixed point format.  From the comments, it appears that C1 works fine with those file.  However, …

    The likely reason that C1 doesn’t recognize the HDR-DNG files is that those files store data in a 16 bit floating point format.  Those files can have a 30 stop dynamic range.  I personally believe that the HDR-DNG approach is one of the two “right” ways to do HDR merging of multiple exposures.  The other “right” way is just layers in photoshop.  HDR tone mapping is nothing but ordinary post processing – with lots of shadow pushing.  Give me Lr HDR merge, and let me do the tone mapping!  By the way, you can HDR merge images of the same exposure in order to reduce noise.

    By the way, it doesn’t have to be a DNG.  The TIFF standard supports floating point data formats.  Almost all image files (proprietary RAW, DNG, JPEG, …) are based on the TIFF standard.  So, floating point format is not an Adobe thing.

    So this is a problem for C1.  It should do an HDR merge, and it should be able to work with floating point data as in HDR-DNG or otherwise.

    JSS

    Kevin Raber
    Kevin Raber
    Keymaster
    Posts: 554
    Re: Lr and C1
    Reply #6 on: October 26, 2019 at 11:10 am

    Phase One has announced the Capture One 20 beta program.  There is a claim they handle DNG files better.  You may want to sign up for the beta program and see if the issues are solved.

    Frankly, I wouldn’t convert to DNG unless I’m years into my workflow.  Many RAW files contain references to sensor information like dead pixels or columns etc.  That’s not always converted over in DNG.

    Most RAW formats will be supported for a long time.  DNG was originally designed to be an archival format in case a camera manufacturer went out of business or a RAW processor couldn’t read certain older files.  Converting to DNG is an extra step in a workflow that may not offer any benefit.

     

    Kevin Raber
    CEO & Publisher of PhotoPXL.com and Rockhopperworkshops.com

    John Sadowsky
    John Sadowsky
    Participant
    Posts: 99
    Re: Lr and C1
    Reply #7 on: October 28, 2019 at 10:10 am

    Frankly, I wouldn’t convert to DNG unless I’m years into my workflow.  Many RAW files contain references to sensor information like dead pixels or columns etc.  That’s not always converted over in DNG. Most RAW formats will be supported for a long time.  DNG was originally designed to be an archival format in case a camera manufacturer went out of business or a RAW processor couldn’t read certain older files.  Converting to DNG is an extra step in a workflow that may not offer any benefit.

    Kevin, with substantial due respect, I think you missed a key point here.  I apologize in advance for my rant here.

    First, there is no real reason for camera manufacturers not to adopt DNG.  If there is nothing a proprietary RAW file that DNG can’t do.  DNG has two ways to support damaged pixels, for example.  The only processing of data that is done in conversion to DNG is linearization and black level subtraction.  (There is not demosaicing, etc.)  Those things are corrections for camera imperfection, as calibrated at manufacture time.  They are future proof, so there is no reason not to do that once for archival storage.  That’s the whole concept of the “digital negative.”  On the other hand, there is a huge advantage to having one common RAW format.

    In my opinion, and one can certainly disagree with my rant, the main reason of proprietary RAW files is that it is one of the tools the industry uses to shut down 3rd party software development.  The other is that they don’t publish APIs (Application Programming Interfaces).

    They want to keep you in their “ecosystem.”  Software manufacturers must pay a royalty and sign a non-disclosure agreement in order to get details of RAW format specification.  That makes it next to impossible for the rather large freelance and small company software developer community to develop new apps.   (Check out the audience size at Apple’s WWDC.)  Almost all camera companies have their own post processing software, but have you ever seen a positive review of, say, Sony’s Imaging Edge, or Nikon’s Capture software?   C1 is the exception.  It was originally developed for Phase One cameras – and then expanded to support other formats.  But seriously, how does Imaging Edge stack up to C1?  I don’t see too many sites like PhotoPxl doing videos on Imaging Edge.  If the industry would adopt DNG for raw capture, and again there is no reason not to, like some enlightened companies (Leica, Pentax) have, then anybody’s software would run on any camera’s RAW files.  No, we can’t have that!  Proprietary RAW files are nothing more than a log they throw in road of progress.

    Compare that to the Apple and Android model – which is to make all the APIs open and well supported, and turn the developer community loose to develop apps.  It is next to impossible to do this in digital photography, and photographers suffer because of it.

    The other part of this story is that camera manufacturers generally don’t support open APIs for WiFi, BT or even USB.  If we had access to those APIs, the software developer community could do some really cool stuff.  We just need to set them loose.  For example, why can’t we do focus stacks on the later Sony cameras (other than using tethered – yuk – Imaging Edge)?  It could be done easily with an iPhone app.  In fact, Sony used to publish their WiFi APIs, but they stopped doing that a few years ago.  Why? – to protect their in house software? – which … – well this is probably a good place to end my rant.

     

    JSS

    Kevin Raber
    Kevin Raber
    Keymaster
    Posts: 554
    Re: Lr and C1
    Reply #8 on: October 28, 2019 at 7:34 pm

    You are welcome to rant John.  What I am going to do is arrange for hopefully an interview with some folks that are really deeply involved in this whole RAW thing.  As an FYI in case you din;t know it I was VP at PHase One for 13 years.  I was involved in this discussion many times.  I also have some very deep connections into Adobe, and will be having coincidentally a conversation with one of their top management tomorrow.  I think it is important that everyone knows the ins and outs of this and maybe right from the source.  I am not claiming what you are saying is wrong in any way but there is a story here.

    I will also be traveling with Jeff Schewe for the next 12 days and I will bring him into the discussion.  Maybe I can do a video with him while traveling.  This is not to start any kind of argument but is to make sure we here the right info from the people that actually had things to do with creating the DNG format.

    Kevin Raber
    CEO & Publisher of PhotoPXL.com and Rockhopperworkshops.com

    Jeff Griffin
    Jeff Griffin
    Participant
    Posts: 18
    Re: Lr and C1
    Reply #9 on: October 29, 2019 at 3:58 am

    Maybe I can do a video with him while traveling. 

    It would be good if you could arrange that as Jeff Schewe has had, in the past, some educational  observations / tips  on image processing that I have found very useful.

    Javier Santa
    Javier Santa
    Participant
    Posts: 5
    Re: Lr and C1
    Reply #10 on: October 29, 2019 at 6:41 pm

    I have forgotten about DNG a long time ago for one main reason: You can convert a RAW to DNG but not the opposite. Maybe things have changed since then.

    If you have a look at how the RAW processors have evolved in a few years and have ever redevelop images  processed with, lets say, LR 3, you´ll get amazed at the difference. That´s why I think it´s better to keep the original RAW´s instead of the DNG just in case DNG stops working in a future. Those things happen.

    I keep a LR 6 version so that if I need it I can convert to TIFF or any other format (DNG included) if by any mean that camera model is not supported anymore.

    On the main topic, my opinion is that I would like to have a pro tool complete, so that I don´t have to be jumping back and forth from one software to others to finish my work. And, for me, it means having a good DAM included where I can have 400.000 + images without having to be switching from one catalog to other and not being able to do a thorough search into all of them at once.

    Of course (for me) it has to be a stand alone versión so that I always have the developed pictures with the correct software to read them as I processed them. Maybe for a non important photographer as me it´s not a big problem, but imagine we couldn´t read today Ansel Adam´s or any other master photographys.

    I can´t avoid thinking that, for instance, all the pictures that not so long ago appeared in a flea market shot by Robert Capa ( not really sure if it was him) would have finished in a trash if they where into a HD that nowadays computers couldn´t read and even if it could be possible none would bother to read it because normally there will be nothing inside.

    John Sadowsky
    John Sadowsky
    Participant
    Posts: 99
    Re: Lr and C1
    Reply #11 on: October 29, 2019 at 10:01 pm

    I have forgotten about DNG a long time ago for one main reason: You can convert a RAW to DNG but not the opposite. Maybe things have changed since then.

    In Lr Preferences under the File Handling tab, there is a check box “Embed Original RAW File”.  So, you actually can go back to your RAW format from DNG.  I don’t recommend that, because I don’t believe anything is lost by DNG conversion.  You would be right to point out that it is silly to double your storage in that way.  I’m not concerned about DNG going away because it is an open standard.  Even if Adobe goes under, someone will open that standard document and write software for it.  You can’t say that for proprietary file formats because they are not open standards.  My point is that we are always better off with an open standard.

     

     

    JSS

    John Sadowsky
    John Sadowsky
    Participant
    Posts: 99
    Re: Lr and C1
    Reply #12 on: October 29, 2019 at 10:59 pm

    You are welcome to rant John.  …

    Thank you.  Kevin, I really appreciate your experience – both technical and artistic.  That’s why I read your article and watch your videos.  You are passionate about photography.  I am too.  I am also passionate about engineering.  I beg you to consider that the (independent) engineer’s point of view is also important here.

    I claim that there is nothing that proprietary RAW files can do that you can’t do with DNG.  We can debate that.  This is a testable hypotheses.  It an issue that should be addressed in PhotoPXL forums, and elsewhere.

    I am definitely interested in hearing Jeff Schewe‘s opinions.  I read his book The Digital Negative some years ago – it was very influential.  I’m not saying he was promoting DNG back then (although I thought he did, but regardless, he may have changed his opinion).  He did say this in his book:  “I need to point out that the DNG file format is advancing (currently at DNG spec version 1.4).”  Actually, today the current version is still 1.4, which was published in 2012.  In fact, DNG has only had to two widely adopted versions: 1.2 in 2008 and 1.4 in 2012.  The fact that there have only really been two versions indicates that this format deals with features of data storage that are so fundamental they really don’t change with time – otherwise, Adobe would have addressed that.  Moreover, the fact that there are such fundamental principles of image storage (and this is my turf) indicates that there is no need for proprietary RAW formats.  I’d be happy to debate the technical aspects of those statements – but not here.  However, note also that the current TIFF standard is 6.0 published in 1992, and I’ve noted that TIFF is still a popular image file format.  The basic JPEG compression standard (T.81) was also published in 1992.  There has been no reason to revise those standards because they got these fundamentals right.  They have withstood the test of time, as has DNG.

    My basic gripe, however, was not addressed in any of the responses.  Proprietary RAW formats is one tool that this industry uses to stifle independent software development.  The other issue I mentioned is that manufacturers don’t publish their WiFi, BT and even USB APIs (Application Programming Interface) is even worse.  (As I pointed out, Sony used to do that, but they stopped – apparently to protect their in-house software.)  I beg you to consider my comparison to the open developer networks supported by Apple and Android.  Photographers could have substantially better software if this industry would adopt open standards.

    If I’m right, then the photographic community is missing out on a plethora of innovative software that an independent developer community could provide.  Moreover, that developer community already exists!  We only need to turn them loose.

     

    JSS

    Jeff Schewe
    Jeff Schewe
    Participant
    Posts: 42
    Re: Lr and C1
    Reply #13 on: October 31, 2019 at 1:58 am

    With regards to DNG, I personally don’t use it in my normal workflow for several practical reasons. First, I use Lightroom and Camera Raw so if I convert to DNG and make a small parameter change in the file, the entire DNG file needs to be backed up on a files changed criteria. If I use a proprietary raw file and side car file then any changes only cause the tiny .xmp file to be updated. You might think this isn’t a big deal but I’ve gotten use to large digital captures and while DNG is an efficient format an 45MP capture is still about 50 MBs when converted to DNG. Ironically, the Nikon Z7 raw is 90 MBs so the DNG file is better compressed than the Nikon raw.

    The other reason I don’t use DNG as a matter of workflow is personal in that I want to be able to have the original raw file for provenance so I can always say, “here’s the original untouched raw file”…

    I do however strongly support Thomas’ efforts to provide a standardized raw file format and Adobe has offered DNG to the ISO for inclusion in the next rev of TIFF-EP (Tiff for electronic photography) which is the standard pretty much all camera makers use for their proprietary raw files.

    As far as the impact DNG has had on the industry, it’s already done what Thomas really wanted to do which was to teach camera makers hw to create raw file formats–something Nikon and Canon had a hard time doing…remember the whole encrypted white balance the Nikon did in their raw files? Seems their software engineers didn’t know what they were doing and accidentally resorted to encryption to have their own software read their own raw files. Canon had their problems too…

    As it is now, all of the camera makers’ raw files are well (or better) formed raw files that unfortunately are undocumented and proprietary :~(

    The funny thing is most people don’t know that Camera Raw and Lightroom (in the Develop module) always convert the proprietary raw file to DNG on the fly. Thomas did that so ACR & LR could read the EXEF metadata and convert the file to DNG to apply the default settings too.

    I will say that DNG does make for a nice and useful conservation and preservation raw file format container and is supported by the Library of Congress’ digital object conservation efforts. DNG also makes fo a good raw file format transport format in that the development settings are enclosed in the file and not in a separate sidecar file that could be lost.

    BTW, there are two basic flavors of DNG files, there’s raw file DNGs such as what you would get out of a Leica or taking a raw file and converting to DNG and there’s also a Linear DNG that is no longer fully raw in that the color filter array (CFA) photo site data has ben interpolated by demosaicing. The file is still a linear gamma file without any color space conversions. So, one could say it’s a half-baked raw file :~)

    The linear DNG is useful for certain files such as HDR and Panos that need demosaicing in order to do the merging of actual pixels.

    Hopefully Capture One will expand their support for DNG…it would be useful for their user base.

    John Sadowsky
    John Sadowsky
    Participant
    Posts: 99
    Re: Lr and C1
    Reply #14 on: October 31, 2019 at 11:04 pm

    Jeff – thank you for your reply.  I am particularly please with your support for open standards.  I have further comments and questions.

    “There are two basic flavors of DNG files, there’s raw file DNGs … and converting to DNG and there’s also a Linear DNG that is no longer fully raw.”  True. Raw data is data as it comes off the sensor’s ADC.  DNG supports raw data storage along with all the tags that store all the sensor specific data needed to linearize it.  DNG can be a true RAW file.  It is also true that DNG conversion produces “linearized-DNG” files.  I would add that there is a third format, the floating point-DNG, which is the DNG file that Lightroom’s HDR merge produces.  This came up this thread because Mike Nelson Pedde indicated that  C1 doesn’t recognize Lr’s HDR.dng files.  I pointed out that that is most likely because of the floating point data format used by those files.

    By the way – all of these files are TIFF files.  TIFF is Tagged Image File Format.  The tags are like a table of contents for the file that point to various segments of the file contain data.  There are tags for image data, tags for metadata, tags for data format, etc.  TIFF-EP adds additional tags – mostly metadata.  Proprietary RAW files are TIFF files with proprietary tags – they point to data, but you don’t know how to interpret that data unless you pay a royalty and sign a nondisclosure.  DNG is a TIFF file with additional tags defined in the open standard DNG specification.  JPEG files also use TIFF tags.  An exceptions are the ICC profiles that use a different format.  ICC profiles are embedded in TIFF and RGB files (to specify the color space) – but again as data blocks pointed to by a TIFF tag.  I know this stuff because I’m write Mac software that accesses these files.  I have to understands these various file format standards in great detail.  Hopefully, you’ll see some of my work soon in an article on stops histograms that I submitted to this site.

    “The funny thing is most people don’t know that Camera Raw and Lightroom (in the Develop module) always convert the proprietary raw file to DNG on the fly.”  Of course they do!  Moreover, it is not just Lr Develop and ACR, anybody’s raw conversion does exactly the same thing as the first step.  C1 is no different.  (I’m not saying they do a full DNG file-to-file conversion, but the definitely do linearization first.)  The linearization steps are described in Chapter 5 of the DNG specification, which is just 1 page of the spec.  These are linearization of the nonlinear photodiode response at the top end near full well capacity, and black level subtraction (which is a pattern bias largely due to the photodiode dark current).  It is a single input – single output correction that depends only on the characteristics of the sensor’s photodiodes.  It donesn’t change with time and there won’t be better new algorithms in the future.

    So, you save your pristine raw data, but every time you want to access it you’re going to do the same linearization.  I just don’t see the point.  Just what are you preserving with the true raw data?  Why are you willing to forgo the clear benefits of an open standard – a point of commonality justified by everything we know about image processing theory?  Do you expect there will be an image that couldn’t get from your DNG conversion, but could from your proprietary RAW file?  If so, please explain how that might occur.

    Lastly, I don’t understand your statement “make a small parameter change in the file, the entire DNG file needs to be backed up.”  I’ve heard this before, but I don’t understand where it comes from.  My understanding is that when you make adjustments in Lr, the adjustments are stored in the Lr catalog, or sidecar file, – not in the DNG file.  I tested this.  I checked the modified date in the Mac finder of a DNG file – which was the same as the import date.  I made adjustments, closed Lr, and back in the finder there was no charge to “modified date.”  If the DNG file is changed, why doesn’t the Mac Finder recognize that?  If I’m mistaken about this, I’d really like to understand what I’m missing here.

    Please forgive me for my brashness.  I know I am a nobody in the photography world, but I do understand data storage engineering.  I’ve asked several photography experts these questions, and to date I haven’t gotten satisfactory answers.

    JSS

Viewing 15 posts - 1 through 15 (of 19 total)
  • You must be logged in to reply to this topic.