Image Noise and Noise Images in Computer Vision, Photography and Embedded Vision

What is Image Noise?

Image Noise is the random part of the variation in a captured signal, pixel to pixel and frame to frame. Photon shot noise, read noise, and dark current shot noise are the stochastic sources.

Not everything that moves a pixel off its true value is noise in that sense. Fixed pattern noise, PRNU and DSNU, is a repeatable offset and gain map rather than a random process. Bias, shading, distortion, dead pixels, and processing error are systematic too, and they are calibrated out rather than averaged down.

How Does A Noisy Image Impact Human Viewing and Computer Vision?

Noise is commonly pictured as salt-and-pepper speckle. That picture names the wrong mechanism for what most people are actually looking at. Salt-and-pepper is impulse noise: isolated pixels pinned at or near the ends of the range. Defective pixels are one source, and dead or stuck pixels do repeat in the same place.

Hot pixels are a weaker case, since their excess signal grows with temperature and integration time and can stay hidden at short exposures. Impulse values also arrive from bit errors on the interface, ADC or logic faults, radiation hits, and compression or interpolation artifacts, none of which is tied to a fixed location.

The grain in a dim still image is a different thing. That is photon shot noise plus read noise, spread across every pixel rather than a scattered few, and it moves from frame to frame.

In this sailboat example, researchers show that the confidence of class recognition drops rapidly when multi-pixel / spatial noise is present in an image. Noise is closely tied to camera exposure settings; see our related guide, Camera Exposure and Computer Vision, for how exposure time and gain affect image quality.

Sailboat image showing how multi-pixel noise reduces neural network classification confidence

Dodge and Karam, "Understanding How Image Quality Affects Deep Neural Networks"

What Types of Noise Exist in Cameras?

There are numerous types of noise in cameras. There are two main categories:

Single-pixel noise is a variation in signal (or pixel value) confined to one pixel. It is often temporal or random, changing frame to frame, though spatial extent and timing are independent properties.
Multi-pixel noise is a variation in signal across a group of neighboring pixels within one frame. It is often fixed pattern, repeating in the same spatial location frame after frame.

The sources named on a sensor datasheet usually include the following. This is not a complete list, and the categories overlap:

Photon shot noise, the Poisson variation in the arriving photons themselves. It is the floor no camera design gets under.
Readout noise, from the source follower, the amplifier chain, and the ADC. Reset noise (kTC) at the sense node belongs inside this category rather than beside it, and correlated double sampling removes most of it.
Dark-current shot noise is the random temporal fluctuation in thermally generated charge. Dark-signal nonuniformity (DSNU) is a separate, repeatable pixel-to-pixel variation in mean dark signal. A dark-frame correction can reduce DSNU but cannot remove each frame's random shot noise.
Fixed pattern noise is the part that lands on the same pixels from one frame to the next: DSNU in the dark, and PRNU (photoresponse non-uniformity) under illumination, a per-pixel gain spread that grows with signal.

Industrial cameras expose analog and digital gain rather than a photographic ISO setting, and the two behave differently. Analog gain is applied before the ADC, so it lifts the signal and the noise already on it above the quantization and ADC noise added after the gain stage; the pixel-side portion of read noise, from the reset and source-follower stages, is amplified along with the signal, not suppressed.

Where those post-gain terms matter, raising analog gain improves SNR. Dual conversion gain is a different mechanism: it switches the conversion gain at the pixel's sense node rather than amplifying after the pixel, so the high-gain state lowers effective read noise at the cost of full-well capacity. Once post-gain noise stops mattering, more analog gain costs highlight headroom without buying anything. Digital gain multiplies values that are already digitized. It does not improve input-referred signal-to-noise ratio, but it can affect clipping, rounding, and quantization in subsequent processing.

How much each of these sources matters depends on the sensor itself. See our image sensor selection guide for how resolution, shutter type, and pixel size factor into a sensor choice.

What About Denoising?

Denoising can be applied to mitigate noise. The approach is different for each of the two categories of noise.

Many multi-pixel (spatial) denoising algorithms use a range of methods, including frequency filtering, local and nonlocal averaging, and learned denoisers, to reduce the variation in pixel intensity. These methods trade bias against variance: they suppress noise and, at the same time, attenuate real detail wherever the two share a frequency band or a neighborhood.

A well-matched denoiser can raise task performance and give a better estimate of the scene than the raw frame, so the question is whether the bias it introduces costs less than the variance it removes. The failure modes are texture loss and, with learned denoisers, plausible detail that was never in the scene.

Spatial denoising with a frequency filter reducing pixel intensity variation across an image

Hasinoff et al. "Burst photography for high dynamic range and low-light imaging on mobile cameras"

Denoising (Spatial) Creates Texture Blur and Loss

Localized denoising costs texture whenever the filter cannot tell fine detail from noise, which is particularly visible with mobile phone cameras.

That texture loss is noticeable to the human visual system when zooming in on the image.

For computer vision, the impact can be even more noticeable depending on your filter size, and how much it costs depends on the filter, the scene, and the task.

Chen et al. show a brick wall building and the output from an edge detection network before and after filtering. The high frequency noise filter greatly reduces the texture in the image, particularly in regions where color is constant. This same tradeoff shows up when comparing sensor and lens choices; see our guide to spatial resolution in machine vision for how mm-per-pixel and the diffraction limit set the smallest detail a system can actually resolve.

Edge detection network output on a brick wall building before and after high-frequency noise filtering, showing texture loss

Chen, S.-C.; Chiu, C.-C. Texture Construction Edge Detection Algorithm. Appl. Sci. 2019, 9, 897. https://doi.org/10.3390/app9050897

References and Related Links

Dodge and Karam, "Understanding How Image Quality Affects Deep Neural Networks"
Stanford EE 392B, lecture notes (PDF)

Camera Exposure and Computer Vision, on how exposure time and gain interact with image noise.
Motion Blur Photography and Shutter Speed, another temporal artifact that, like noise, degrades computer vision confidence.
12+ Image Quality Metrics that Impact Computer Vision, for how noise fits alongside sharpness, contrast, and distortion.
Image Sensor Selection for Machine Vision, for resolution, shutter type, and pixel size tradeoffs that drive sensor noise performance.

Trying to Determine Your Camera Requirements?

Use our free web-based AoV Calculator to determine your system's Field of View Requirements. Then, use the M12 Lens calculator to match your requirements with the available lenses. Our Depth of Field Calculator also provides the hyperfocal distance and depth of field for every sensor and lens combination.

We also have a couple of other calculators that many engineers find interesting.