The Megapixel Wars: MTF vs. SFR. vs. Resolution vs. Pixel Count
Articles stating that some camera phones have >30MP usually fail to investigate the actual output resolution and image quality of the device. Many of these newer sensors use fancy Bayer pattern outputs (e.g. "Tetra-Bayer") to reduce HDR motion blur effects but rarely are configured to output images using the sensor's full pixel count. This is not only confusing for consumers but also industry experts.
Let's begin with simple definitions of the four ways people frequently talk about Resolution. We defined these to resolve an international linguistic misunderstanding (between Optical Engineers, Image Quality Engineers, Sensor Engineering, and Computer Vision Engineers) during a conversation within the IEEE P2020 working group:
- Pixel Count Resolution: The total number of individual light sensing pixels on a physical image sensor, before debayering, binning, cropping (etc..) that can transmit a signal to the Analog-to-Digital Converter.
- Angular Resolution: The number of pixels per degree that a camera's field of view subtends. We prefer pixels per degree as the value can quickly be multiplied by the angular subtense of an object, allowing for ease of use in the computer vision engineering world. In the defense world, this can be inverted to IFOV (deg/px) if needed.
- Optical Resolution: The ability of an optical system to resolve two points. This is known as the Modulus of the Optical Transfer function (MTF) which characterizes the capabilities of the lens design and manufacturing effort.
- System Resolution: The digital Signal Frequency Response (SFR) which includes inputs from the Sensor, ADC, and Digital Image Processing Pipeline.
Selecting a camera sensor's Pixel Count Resolution is one of the first tasks an engineering team takes on when creating a new camera, but what needs to be focused on is the System Resolution. This is a result of many engineering teams succumbing to the marketing team, and thus focusing solely on compute/IO requirements. Let's suppose that you have a 12MP camera, like the Raspberry Pi High-Quality camera, and only use a lens with an MTF suitable for 3MP like the cheap off-the-shelf Pi HQ 6mm lens. The camera sharpness will underperform when any digital zoom is performed, and only meet 720P image quality at best!
To complicate matters further, smaller teams with production volumes under 1Mpcs/yr are typically deferred by sensor manufacturers such as Samsung, Omnivision, Onsemi, and Sony and are thus unable to properly co-optimize the Pixel Count Resolution with the System Resolution.
Two direct implications of building a camera with too many megapixels are:
- Decreased sensor sensitivity
- Increased image noise
Both of these directly lower the low-light performance of a camera system. This is one of the main reasons why security and surveillance cameras typically have lower resolution than cellphones, as ~8MP-10MP is the maximum overall system performance which can be achieved in 2020 with a camera Bill of Materials under $500 at 100kpcs/yr.
Here's an unrelated snippet that provides a non-technical perspective on the problem.
"Take the example of choosing a digital camera. Cameras advertise their megapixels, and the impression created is certainly that the more megapixels the better. This assumption is itself subject to question because photos taken with more megapixels take up more room on the camera's storage device and a computer's hard drive. But what is problematic for consumers is translating megapixels (not the most intuitive concept) into what they care about. Is it worth paying an additional hundred dollars to go from four to five megapixels? Supposed instead that manufacturers listed the largest print size recommended for a given camera. Instead of being given the options of three, five, or seven megapixels, consumers might be told that the camera can produce quality photos at 4 x 6 inches, 9 x 12, or "poster size."
Richard Thaler and Cass Sunstein "Nudge" page 94.