Image Noise and Noise Images in Computer Vision, Photography and Embedded Vision

What is Image Noise?

Image Noise is the variation in signal from the ground truth, from pixel to pixel or frame to frame.

How Does A Noisy Image Impact Human Viewing and Computer Vision?

Noise is commonly pictured as salt-and-pepper speckle. That picture names the wrong mechanism. Salt-and-pepper is impulse noise, and in a camera it points at defective pixels: dead, stuck, and hot pixels that report the same wrong value in the same place every frame.

The grain in a dim still image is a different thing. That is photon shot noise plus read noise, spread across every pixel rather than a scattered few, and it moves from frame to frame.

In this sailboat example, researchers show that the confidence of class recognition drops rapidly when multi-pixel / spatial noise is present in an image. Noise is closely tied to camera exposure settings; see our related guide, Camera Exposure and Computer Vision, for how exposure time and gain affect image quality.

Sailboat image showing how multi-pixel noise reduces neural network classification confidence

Dodge et al. "Understanding How Image Quality Affects Deep Neural Networks"

What Types of Noise Exist in Cameras?

There are numerous types of noise in cameras. There are two main categories:

Single-pixel noise is a variation in signal (or pixel value) confined to one pixel. It is often temporal or random, changing frame to frame, though spatial extent and timing are independent properties.
Multi-pixel noise is a variation in signal across a group of neighboring pixels within one frame. It is often fixed pattern, repeating in the same spatial location frame after frame.

The sources named on a sensor datasheet usually include the following. This is not a complete list, and the categories overlap:

Photon shot noise, the Poisson variation in the arriving photons themselves. It is the floor no camera design gets under.
Readout noise, from the source follower, the amplifier chain, and the ADC. Reset noise (kTC) at the sense node belongs inside this category rather than beside it, and correlated double sampling removes most of it.
Dark current shot noise, which accumulates with integration time and climbs steeply with sensor temperature. Its fixed-pattern component is DSNU, dark signal non-uniformity.
Fixed pattern noise is the part that lands on the same pixels from one frame to the next: DSNU in the dark, and PRNU (photoresponse non-uniformity) under illumination, a per-pixel gain spread that grows with signal.

Industrial cameras expose analog and digital gain rather than a photographic ISO setting. Gain amplifies the signal together with the noise already carried on it, so turning it up brightens an image without adding information to it.

How much each of these sources matters depends on the sensor itself. See our image sensor selection guide for how resolution, shutter type, and pixel size factor into a sensor choice.

What About Denoising?

Denoising can be applied to mitigate noise. The approach is different for each of the two categories of noise.

Many multi-pixel (Spatial) denoising algorithms use a range of methods, including frequency filtering, local and nonlocal averaging, and learned denoisers, to reduce the variation in pixel intensity. These methods also remove real information from the image.

Spatial denoising with a frequency filter reducing pixel intensity variation across an image

Hasinoff et al. "Burst photography for high dynamic range and low-light imaging on mobile cameras"

Denoising (Spatial) Creates Texture Blur and Loss

Localized denoising creates texture loss, which is particularly visible with mobile phone cameras.

This texture loss is noticeable to the human visual system when zooming in on the image.

For computer vision, the impact can be even more noticeable depending on your filter size.

Chen et al. show a brick wall building and the output from an edge detection network before and after filtering. The high frequency noise filter greatly reduces the texture in the image, particularly in regions where color is constant. This same tradeoff shows up when comparing sensor and lens choices; see our guide to spatial resolution in machine vision for how mm-per-pixel and the diffraction limit set the smallest detail a system can actually resolve.

Edge detection network output on a brick wall building before and after high-frequency noise filtering, showing texture loss

Chen, S.-C.; Chiu, C.-C. Texture Construction Edge Detection Algorithm. Appl. Sci. 2019, 9, 897. https://doi.org/10.3390/app9050897

References and Related Links

Dodge et al. "Understanding How Image Quality Affects Deep Neural Networks"

Stanford EE 392B lecture notes

Camera Exposure and Computer Vision, on how exposure time and gain interact with image noise.
Motion Blur Photography and Shutter Speed, another temporal artifact that, like noise, degrades computer vision confidence.
12+ Image Quality Metrics that Impact Computer Vision, for how noise fits alongside sharpness, contrast, and distortion.
Image Sensor Selection for Machine Vision, for resolution, shutter type, and pixel size tradeoffs that drive sensor noise performance.

Trying to Determine Your Camera Requirements?

Use our free web-based AoV Calculator to determine your system's Field of View Requirements. Then, use the M12 Lens calculator to match your requirements with the available lenses. Our Depth of Field Calculator also provides the hyperfocal distance and depth of field for every sensor and lens combination.

We also have a couple of other calculators that many engineers find interesting.