High Dynamic Range (HDR) in Computer Vision, Photography and Embedded Vision
What is High Dynamic Range?
Dynamic Range is the ratio of the largest signal a pixel can hold to the smallest one that still counts as detected. Both ends need a convention before the number means anything: the saturation level, the noise floor and the SNR that defines detection, the processing state the measurement is taken in, and the exposure mode. EMVA 1288 defines one such convention. A decibel figure quoted without those conditions cannot be compared against another vendor's.
Wide Dynamic Range (WDR) and High Dynamic Range (HDR) are used interchangeably across the industry. Neither is a standardized term, and a vendor calling a sensor WDR may mean a single-exposure mechanism, a multi-exposure one, or simply a large decibel number on a datasheet.
The useful question is which mechanism is running. Cameras reach an extended luminance range through temporal, spatial, or split pixel multiplexing, and dual conversion gain or split-pixel designs reach it inside a single exposure.
Three names cover most of the single-exposure mechanisms: a piecewise-linear response with a kneepoint above which the pixel compresses what it captures, on-chip companding that packs a high bit depth capture into a shorter output word, and dual conversion gain, where the pixel switches between a high-gain readout for the shadows and a low-gain readout for the highlights.
Each of those choices moves where noise and quantization land in the tone curve. Ask for the response curve, not just the decibel number.
What does Dynamic Range Look Like?
Dynamic range is noticeable when a camera's dynamic range is lower than a scene's dynamic range. Regions of the image will be too bright, or too dim, relative to the rest of the image which has correct exposure.
Camera dynamic range limitations occur in enclosed locations, like looking out of a tunnel.
In this Cathedral example, the low dynamic range image (left) has reduced feature visibility in the center. In the HDR image, the features and textures can be identified.
Hasinoff et al. “Burst photography for high dynamic range and low-light imaging on mobile cameras”
How is High Dynamic Range Achieved?
There are three different methods commonly used to compose High Dynamic Range (HDR) images:
- Temporal HDR uses multiple frames over time, each with different exposure length or gain. This is also known as exposure bracketing.
- Spatial HDR uses different exposure or gain values for different rows or columns of pixels, which is why it also goes by line-interleaved capture.
- Split pixel HDR uses sensors with multiple photodiodes or gain paths per pixel, each with a different combination of sensitivity, exposure, and gain. Dual conversion gain is the related in-pixel trick without the second photodiode: one photodiode, read out at two conversion gains and then merged. Some sensors pair split pixel with newer color filter patterns, such as Sony's Quad Bayer, but the color filter pattern and the split-pixel mechanism are not the same thing.
HDR Can Lead to Problematic Artifacts for Computer Vision and Machine Vision
The research team at Algolux shows a few common issues with HDR multiplexing. Even a careful implementation produces artifacts, and those artifacts become the edge cases that classification and detection methods fail on.
Three failure modes cover most of what goes wrong. Temporal multiplexing takes its exposures at different times, so anything moving between them ghosts along its own path. Line-interleaved capture reads neighboring rows at different exposures, which leaves zipper and row artifacts on horizontal edges and under flickering light. Merged exposures also carry SNR discontinuities at the kneepoints where the pipeline hands off from one exposure to the next, and a detector can see that noise floor step across a single object.
Nicolas Robidoux, Luis Eduardo García Capel, Dong-eun Seo, Avinash Sharma, Federico Ariza, Felix Heide. CVPR 2021, "End-to-end High Dynamic Range Camera Pipeline Optimization"
References and Related Links
Nicolas Robidoux, Luis Eduardo García Capel, Dong-eun Seo, Avinash Sharma, Federico Ariza, Felix Heide. CVPR 2021, "End-to-end High Dynamic Range Camera Pipeline Optimization"
Geese, Seger, and Paolillo "Detection Probabilities: Performance Prediction for Sensors of Autonomous Vehicles"
Trying to Determine Your Camera Requirements?
Use our free web-based AoV Calculator to determine your system's Field of View Requirements. Then, use the M12 Lens calculator to match your requirements with the available lenses. Our Depth of Field Calculator also provides the hyperfocal distance and depth of field for every sensor and lens combination.
We also have a couple of other calculators that many engineers find interesting.
