Does pixel-shift increase resolution?
-
-
Re: Does pixel-shift increase resolution?Reply #1 on: February 21, 2020 at 5:51 pm
I would expect the practical issues would overwhelm the 16-way pixel shift, starting with the fact that you have to have a rock-solid tripod and the high-resolution glass. And what about shutter shake? I’ve tried some 4-way pixel shift on my a7RM3 – I really couldn’t see much difference even under close pixel peeping. Compare pixel-shift to HDR merging, or even just image stack averaging of the same exposure for noise reduction. Layer-alignment in Photoshop (and other software) is quite accurate. But you can’t use that with pixel-shift. Pick your poison – sharpness or noise? Pixel-shift is limited to indoor studio photography, for studios located far from highways, rail tracks, or anything else that might produce a micro-shake.
I would gladly swap pixel-shift for focus stacks (as in the Nikons). Why doesn’t Sony give us something we can actually useful? (Not to mention that the Sony software used to merge the pixel-shift images is pathetic, IMHO.)
Having said that, as a signal processing engineer, I was not at all satisfied with that article.
The underlying assumption is that pixels have a 100% fill factor, meaning 100% of the pixel area is active in light detecting. That just can’t be true. The surface area of a pixel includes the light-sensing photodiode and several other components including at least 4 CMOS transistors. Modern sensors do have micro-lenses on top of the sensor pixels, and BSI moves metal layers to the dark side of the sensor. Those things do improve fill factor, but it is still a pretty big stretch of the imagination to think they achieved close to 100% fill.
If it were true (that these pixels have 100% area efficiency), then it is almost immediately obvious 16-way pixel-shift would have no resolution advantage over 4-way pixel-shift. The resolution would be determined by the “sampling aperture” of the square pixel area. Extra 1/2 pixel offsets of those squares don’t add anything to resolution over 4-way pixel-shift. 4-way still makes sense as it eliminates the need for interpolation in demosaicing interpolation.
I’d be happy to back up what I just said with more details from Fourier transform and Shannon-Nyquist sampling theory. Here it is in a nutshell. MTF = the Fourier transform of the PSF = point spread function. He uses the MTF because he can combine the lens MTF and the sampling aperture MTF by multiplication (as opposed to the convolution of PSFs in the spatial domain). But who thinks in terms of spatial frequency? He should have at least press the button on his Matlab software (or whatever he’s using) to transform back to the spatial domain to show us PSF. Then we could actually measure the parameter that photographers actually use to quantify resolution = the circle-of-confusion = the diameter of the PSF. MTF – WTF?
JSS
Re: Does pixel-shift increase resolution?Reply #6 on: February 26, 2020 at 12:28 amJim, I’m still not buying a 100% fill factor. I don’t think you can make such a claim without access to proprietary manufacturing process and design data. Nonetheless, it seems to defy physics. As with all features of semiconductor manufacturing, microlenses are masked and deposited on the sensor chip. It is highly probable that these lenses have circular symmetry, as sharp corners would be challenging to etch or deposit on that scale. (There are images online of microlenses – all the ones I found are circular.) A 100% efficient circular microlens inside of a square pixel is only 78% of the pixel area. We’re not talking about Zeiss optics here. There are going to be optical compromises.
OTF = FT of PSF, MTF = |OTF|^2 – my bad.
Yes, I do believe your use of the MFT in your blog confuses the issue. As I stated, photographers do have an operational definition of resolution, and it is not pixel count. It is the spatial domain circle-of-confusion (CoC), which can be measured as the RMS radius of the PSF.
I mostly agree with your conclusions. It is the combined PSF of the lens and the sampling-aperture (100% ideal square fill or otherwise) that limits the resolution, not the sampling density (megapixels). Still, I found your description of pixel-shift as a halving of pixel pitch with a 400% fill to be unnecessary and confusing. After all, the pixel doesn’t change size. Pixel-shift simply increases the sampling density, which expands the area of alias-free spatial frequencies. You only needed one MFT chart. Then simply mark on that chart where then alias cutoff point is for the various pixel-shift sampling densities. With that, I’ll concede that the frequency domain may the right place to explain the aliasing, but not the resolution. Either way, you’re still assuming that your readers understand something about Nyquist-Shannon sampling theory.
Notice that a smaller fill factor implies better resolution. (The reason we want a high fill factor is to collect more photons, which improves SNR.) That’s likely going to be counter-intuitive for most photographers. Since a more compact PSF implies a broader MFT, the value of pixel-shift is enhanced for lower (more realistic) fill factors. That’s the point I was trying to make when I challenged the 100% fill factor.
JSS
Re: Does pixel-shift increase resolution?Reply #7 on: February 26, 2020 at 5:06 pmYes, I do believe your use of the MFT in your blog confuses the issue. As I stated, photographers do have an operational definition of resolution, and it is not pixel count. It is the spatial domain circle-of-confusion (CoC), which can be measured as the RMS radius of the PSF.
Tell that to Zeiss:
Or Nikon:
Or LensRentals:
https://www.lensrentals.com/blog/2019/04/just-mtf-charts-sony-fe-mount-prime-lenses/
Or Rodenstock:
https://www.rodenstock-photo.com/Archiv/e_Rodenstock_Analog_Lenses_27-42__8226.pdf
or Schneider:
https://captureintegration.com/schneider-ls-lens-mtf-charts-for-phase-one-mamiya/
Or Canon:
Or Imatest:
https://www.imatest.com/products/imatest-master/
https://www.imatest.com/docs/sharpness/
CoC diameters are only useful if the PSF is roughly circular, and lens aberrations refuse to steadfastly toe that line. That’s why the sagittal and tangential MTFs differ. And there is much confusion on computing the diameter of PSFs that aren’t pillboxes; for example, the usual approximated diameter of the Airy disk is twice the radius of the first zero, which is unduly pessimistic.
Jim
Re: Does pixel-shift increase resolution?Reply #9 on: February 29, 2020 at 1:10 pmNotice that a smaller fill factor implies better resolution. (The reason we want a high fill factor is to collect more photons, which improves SNR.)
Of course, you’re right about the resolution. A canonically ideal sampler is a point sampler, which has (a Dirac delta-function-ish) 0% fill factor, and the flattest MTF curve. However, especially since having long experience with the GFX 50S and GFX 50R with their 50% fill factors, I disagree that the only reason for wanting a high fill factor is to improve SNR. A fill factor of nearly 100% is useful for reducing aliasing, as I’ve shown here:
https://blog.kasson.com/the-last-word/balancing-real-and-fake-detail-part-2/
- You must be logged in to reply to this topic.