High Dynamic Range (HDR) in Computer Vision, Photography and Embedded Vision
What is High Dynamic Range?
Dynamic Range is the ratio of maximum to minimum detectable signal within an image.
Wide Dynamic Range (WDR) describes sensors that have above-average dynamic range with a single exposure.
High Dynamic Range (HDR) describes an image with an extended captured luminance range beyond a single standard exposure. Cameras commonly produce HDR using temporal, spatial, or split pixel multiplexing.
That WDR label is a vendor term, not a standard with a numeric threshold, so ask which mechanism a sensor actually uses. Three names cover most of them: a piecewise-linear response with a kneepoint above which the pixel compresses what it captures, on-chip companding that packs a high bit depth capture into a shorter output word, and dual conversion gain, where the pixel switches between a high-gain readout for the shadows and a low-gain readout for the highlights.
Each of those choices moves where noise and quantization land in the tone curve. Ask for the response curve, not just the decibel number.
What does Dynamic Range Look Like?
Dynamic range is noticeable when a camera's dynamic range is lower than a scene's dynamic range. Regions of the image will be too bright, or too dim, relative to the rest of the image which has correct exposure.
Camera dynamic range limitations occur in enclosed locations, like looking out of a tunnel.
In this Cathedral example, the low dynamic range image (left) has reduced feature visibility in the center. In the HDR image, the features and textures can be identified.
Hasinoff et al. “Burst photography for high dynamic range and low-light imaging on mobile cameras”
How is High Dynamic Range Achieved?
There are three different methods commonly used to compose High Dynamic Range (HDR) images:
- Temporal HDR uses multiple frames over time, each with different exposure length or gain. This is also known as exposure bracketing.
- Spatial HDR uses different exposure or gain values for different rows or columns of pixels, which is why it also goes by line-interleaved capture.
- Split pixel HDR uses sensors with multiple photodiodes or gain paths per pixel, each with a different combination of sensitivity, exposure, and gain. Dual conversion gain is the related in-pixel trick without the second photodiode: one photodiode, read out at two conversion gains and then merged. Some sensors pair split pixel with newer color filter patterns, such as Sony's Quad Bayer, but the color filter pattern and the split-pixel mechanism are not the same thing.
HDR Can Lead to Problematic Artifacts for Computer Vision and Machine Vision
The research team at Algolux shows a few common issues with HDR multiplexing. Even a careful implementation produces artifacts, and those artifacts become the edge cases that classification and detection methods fail on.
Three failure modes cover most of what goes wrong. Temporal multiplexing takes its exposures at different times, so anything moving between them ghosts along its own path. Line-interleaved capture reads neighboring rows at different exposures, which leaves zipper and row artifacts on horizontal edges and under flickering light. Merged exposures also carry SNR discontinuities at the kneepoints where the pipeline hands off from one exposure to the next, and a detector can see that noise floor step across a single object.
Nicolas Robidoux, Luis Eduardo García Capel, Dong-eun Seo, Avinash Sharma, Federico Ariza, Felix Heide. CVPR 2021, "End-to-end High Dynamic Range Camera Pipeline Optimization"
References and Related Links
Nicolas Robidoux, Luis Eduardo García Capel, Dong-eun Seo, Avinash Sharma, Federico Ariza, Felix Heide. CVPR 2021, "End-to-end High Dynamic Range Camera Pipeline Optimization"
Geese, Seger, and Paolillo "Detection Probabilities: Performance Prediction for Sensors of Autonomous Vehicles"
Trying to Determine Your Camera Requirements?
Use our free web-based AoV Calculator to determine your system's Field of View Requirements. Then, use the M12 Lens calculator to match your requirements with the available lenses. Our Depth of Field Calculator also provides the hyperfocal distance and depth of field for every sensor and lens combination.
We also have a couple of other calculators that many engineers find interesting.
