The Definition of Lens Chief Ray Angle and the dreaded CRA mismatch

Lens Chief Ray Angle Mismatch and Impact on Image Quality

What is the Chief Ray Angle of an Image Sensor?

Let's first start with the architecture of a modern Complementary-Metal-Oxide-Semiconductor (CMOS) pixel.

Here is a simplified pixel architecture from Sony's website that I've marked up.In this simplified marketing drawing, you can see the different components of a pixel.

An introductory textbook for diodes that we went through back in the day at UofR is Sze and Lee "Semiconductor Devices, 3rd ed."

What is the Chief Ray Angle of a Lens?

The chief ray of a lens is the ray that goes through the center of the aperture stop in an optical system.

If you look into a lens from object space, the chief ray is the ray that crosses the optical axis at the entrance pupil.

If you look from image space, this is the ray at the center of the exit pupil.

Hecht's Optics Fifth Edition has a great first-order optics diagram and description on page 185 for a general three element optical imaging system:

Chief rays exist for every illuminated point in object space. Let's see how this looks for a "Real World" lens.

When people discuss the Chief ray angle, they typically refer to the "Maximum CRA" which corresponds to the widest field of view of a lens combination.

To accurately compare the chief ray of a lens and the chief ray of a sensor, you must consider the CRA across the usable area of the image.

What does CRA mismatch look like Physically and why CRA Mismatch more important at high CRA Angles?

Low profile lenses (short TTL) typically have very high CRA, as optical design performance does not converge if a low CRA requirement is forced on the design.

To aide cell phone manufacturers with system-level image quality, sensor manufacturers adjust the spatial design of microlenses on the sensor to compensate for lens CRA. This microlens adjustment is generally only available to high volume (>10Mpcs/yr) companies, so the rest of us must just do our best to select the right sensor variant and matching lens.

Oblique Dependence of Microlenses at 25° CRA (CIL023 2.2mm F/2.2)

Oblique Dependence of Microlenses at 15° CRA (CIL039 3.9mm F/2.8)

Correcting for color shading due to CRA mismatch.

CRA mismatch CAN be corrected for in post process, but ONLY in applications with well controlled static illumination such as industrial machine vision for inspection.

When the light sources change, it becomes challenging to compensate. This is due the friendly topic of metamerism. We've seen a major CRA mismatch (20° non-linear mismatch) overcome before in a regular indoor environment, so it is doable to a "good enough" extent. This requires advanced ISP tuning with a calculated pixel-level spectral energy distribution 3DMLUT approach.This in turn will slow down other performance metrics in your camera and/or require more compute, so generally not the best practice to get into this situation.

Additionally, there are only a handful of leading image quality experts with the requisite knowhow and experience to get to a "good enough" quality with a >15° nonlinear mismatch with a sensor at 33°. I estimate <50 people in the world and it is near impossible to hire them as they are in high demand at big tech companies.

So unless you are fortunate enough to be on a team with one of these experts, we highly advise against venturing down the rabbit hole of thinking you can solve >15° nonlinear CRA mismatch in software: your project will likely have a 6-12 month delay and budget overrun.

Regardless of the approach and expertise there will be more color tuning corner cases that occur with huge CRA mismatch, than when you have a well-matched lens to sensor CRA.

Importance of CRA in Optical Design

The CRA is a critical parameter in optical design and digital imaging, as it significantly impacts the quality of captured images.

Accurate CRA is required to achieve high-quality images and mitigating artifacts such as vignetting and color shading.

The CRA is related to the angular relationship between the optical axis and the chief ray of the lens, making it a key factor in determining image quality. In optical design, ensuring that the CRA is properly aligned with the image sensor’s pixel architecture is crucial. This alignment helps in capturing images with precise color reproduction and sharpness, which is especially important in applications requiring high fidelity, such as medical imaging and high-resolution photography.

The Take-Away: Match the Lens Chief Ray Angle As Closely to the sensor as possible, but don't overindex on CRA mismatch.

We generally recommend matching CRA within +/-10° if the sensor's CRA is <10°, +/-7° if the sensor's CRA is >10° and <20°, and within +/-4° if the sensor's CRA is >20°.

However, it really depends on the pixel architecture and your application.

Jon Stern from GoPro's optics team provided his opinion publicly during a talk at the Embedded Vision Summit in 2020: View Slide 22 Here.

This mismatch tolerance must hold across the entire field of view, so make sure to compare a full plot if the sensor's specification sheet says "non-linear" on it.

Incorrect CRA matching can result in radial red to green color shading from the center of an image to the corner.

This shading is dependent upon illumination conditions, so it makes Image Quality Tuning extremely difficult.

This is a common issue when trying to build a camera using a "Mobile" Sensor with an "Industrial" Lens or vis-versa. We've seen multiple startup projects run into this issue, resulting in extensive cost (>$100k) and schedule (>1yr) overruns.