Underwater Image Enhancement

Underwater Image Enhancement. Submitted by Sanat Vibhor 18BEC10072.

Architecture of the proposed model. Input Image Histogram Equalization CLAHE White Balance G'" *otithm •dd.tional Gamma Correction Aggregated Weight Normalized Weight Output Image Sharpening Unsharp Masking Aggregated Weight Normalized Weight.

Noise Removal. Noise in an image can be introduced due to an inefficient camera system or lousy illumination of the scene. Different types of noises include Gaussian noise, Salt and Pepper noise etc. Various techniques have been developed to remove or reduce the effect of noise in an image..

Gaussian noise It is also called as electronic noise because it arises in amplifiers or detectors. Gaussian Noise is a statistical noise. It has a probability density function equal to normal distribution, also known as Gaussia.

Digital Image Processing Filters - Median Filter.

Adding Noise to the Image. np . random . normal ( 0 , 1 , img .size) it describes the commonly occurring distribution of samples influenced by a large number of tiny, random disturbances, each with its own unique distribution gauss .reshape( img .shape[ 0 ], img .shape[ 1 ], img .shape[ 2 ]).astype( 'uint8' ) Gives a new shape to an array without changing its data. cv2 . add ( img , gauss ) Gaussian-distributed additive noise..

100 100 100 200 200 100 200.

Code explanation. cv2.fastNlMeansDenoisingColored( P1, P2, float P3, float P4, int P5, int P6) Parameters: P1 – Source Image Array P2 – Destination Image Array P3 – Size in pixels of the template patch that is used to compute weights. P4 – Size in pixels of the window that is used to compute a weighted average for the given pixel. P5 – Parameter regulating filter strength for luminance component. P6 – Same as above but for color components // Not used in a grayscale image..

Histogram Equalization. The histogram of an image is a graph that gives the frequency of occurrence of each pixel intensity values. The size of each pixel gives the total number of intensity values. An 8-bit pixel has 256 intensity values ranging from 0 to 255. Grayscale images have a single histogram, while color images have multiple histograms. For example, an RGB image has three histograms, each for its color spaces Red (R), Green (G), and Blue (B). An ideal histogram uses the entire available range and has good contrast in the image. If the histogram of an image lies in a smaller interval, contrast is shallow and lacks color variations in the picture. Histogram Equalization spreads the histogram of the image over the entire available range. With this, pixels of lower contrast gain a higher contrast, and hence a better image is obtained..

Input Image Equalized Image. We should first separate the brightness of the image from the color and then run HE on the brightness. Now, there are already standardized color spaces that encode brightness and color separately, like- YCbCr , HSV , etc.; so, we can use them here for separating and then re-merging the brightness. The proper way: Convert the colorspace from RGB to YCbCr >> Run HE on the Y channel (this channel represents brightness) >> Convert back the colorspace to RGB.

Python: cv.COLOR_BGR2YUV convert between RGB/BGR and YUV Line 6 : img_yuv = cv2 . cvtColor ( img , cv2 . COLOR_BGR2YUV ) YUV is a color encoding system typically used as part of a color image pipeline . It encodes a color image or video taking human perception into account, allowing reduced bandwidth for chrominance components, compared to a "direct" RGB -representation. Historically, the terms YUV and Y′UV were used for a specific analog encoding of color information in television systems. Line 9 : cv2 . equalizeHist ( img_yuv [:,:, 0 ]) Here, src is the source image and dst is the destination image of the same size and type as src. Line 12 : img_output = cv2 . cvtColor ( img_yuv , cv2 . COLOR_YUV2BGR ) Convert back to BGR results in more saturated image..

Sa hist , bins = np . histogram ( img .flatten(), 256 ,[ 0 , 256 ]) NumPy also provides us a function for histogram, np.histogram(). s plit the whole histogram to 16 sub-parts and value of each sub-part is the sum of all pixel count in it. This each sub-part is called "BIN". In first case, number of bins were 256 (one for each pixel) while in second case, it is only 16. BINS is represented by the term histSize in OpenCV docs. hist is same as we calculated before. But bins will have 257 elements, because Numpy calculates bins as 0-0.99, 1-1.99, 2-2.99 etc. cdf = hist .cumsum() cumsum() function is used when we want to compute the cumulative sum of array elements over a given axis. cdf_normalized = cdf * float ( hist .max()) / cdf .max() hist returns a tuple that contains the histogram bin locations and y values. plt . plot ( cdf_normalized , color = 'b' ) Plots the graph in blue color. plt . hist ( img .flatten(), 256 ,[ 0 , 256 ], color = 'r' ) Flattening is a technique that is used to convert multi-dimensional arrays into a 1-D array,.

Gamma Correction. Images can look either too light or too dark. Gamma correction[2] is a method that allows to control the brightness of an image. The formula used to get gamma corrected image is given below: I – input pixel value [0, 255]. O – output pixel value [0, 255]. γ – gamma that controls image brightness. If gamma < 1 then image will be darker, if gamma > 1 then image will be lighter. A gamma = 1 has no effect. Gamma correction can be implemented using lookup table (LUT). It maps the input pixel values to the output values. For each pixel value in the range [0, 255] is calculated corresponding gamma corrected value. OpenCV provides LUT function which performs a lookup table transform..

Code. Original image Gamma corrected image. a5e1L'! paljanoj eLUILeCJ X a6eW! leu16uo.

Code Explanation. import cv2 import numpy as np def gc ( src , gamma ): invGamma = 1 / gamma table = [(( i / 255 ) ** invGamma ) * 255 for i in range ( 256 )] table = np . array ( table , np . uint8 ) return cv2 . LUT ( src , table ) img = cv2 . imread ( 'test1.jpg' ) gammaImg = gc ( img , 2.5 ) cv2 . imshow ( 'Original image' , img ) cv2 . imshow ( 'Gamma corrected image' , gammaImg ) cv2 . waitKey ( 0 ) cv2 . destroyAllWindows ().

Sharpening of Underwater Image. Unsharp masking [3] (USM) is an image sharpening technique, first implemented in darkroom photography , but now commonly used in digital image processing software. Its name derives from the fact that the technique uses a blurred , or "unsharp", negative image to create a mask of the original image. The unsharp mask is then combined with the original positive image, creating an image that is less blurry than the original. The resulting image, although clearer, may be a less accurate representation of the image's subject. The typical blending formula for unsharp masking is sharpened = original + (original − blurred) × amount. The algorithm works like this: Look at a pixel from the unsharp mask and find out its luminosity (brightness). If the luminosity is 100%, use the value from the high-contrast image for this pixel. If it is 0%, use the value from the original image for this pixel. If it's somewhere in-between, mix the two pixels' values using some weighting. Optionally, only change the value of the pixel if it changes by more than a certain amount.

Code [6]. Original. Sharpened.

Code Explanation. import numpy as np import matplotlib . pylab as plt from skimage . io import imread from skimage . filters import unsharp_mask im = imread ( 'test1.jpg' ) im1 = np . copy ( im ).astype( np .float) for i in range ( 3 ): im1 [..., i ] = unsharp_mask( im [..., i ], radius = 2 , amount = 2 ) plt . figure ( figsize =( 20 , 7 )) plt . subplot ( 131 ), plt . imshow ( im ), plt . axis ( 'off' ), plt . title ( 'Original' , size = 10 ) plt . subplot ( 132 ), plt . imshow ( im1 ), plt . axis ( 'off' ), plt . title ( 'Sharpened' , size = 10 ) plt . show ().

Fusion. Fusion is the process of combining information obtained from two or more methods for better results. Simple fusion considers each of the inputs equally, and the result is obtained. This can be improved by assigning weights to each of the inputs based on the information. In the proposed fusion method, three weight maps are calculated for each of the input images. They are 1. Laplacian Contrast Weight 2. Saliency weight 3. Saturation Weight After deriving the fusion inputs and their weights, naive fusion is followed to get the results. In naive fusion, element-by-element multiplication is performed between the input and its corresponding weight. The results of both the multiplications are added and normalized to get the output image..

Laplacian contrast Weight Laplacian weight map emphasizes the global contrast of the image. It is the absolute of the Laplacian filter applied over the Luminance channel. Saliency Weight The saliency weight map highlights the salient objects that lose their prominence in underwater scenes by eliminating the background. Saturation Weight Saturation weight gives the chromatic information of the image using the highly saturated regions. It is given by the deviation of the color channels (R, G, B) from the luminance (L) of the image. Luminance L of an image is calculated using the equation.

Normalized Aggregated Weights Aggregated weights for each of the inputs is calculated by summing the three weights obtained earlier. These weights are then normalized using the equations. A small constant δ (= 0.1) is considered so that the normalized weights have a definitive e value in the absence of aggregated weight of a pixel. After deriving the fusion inputs and their weights, naive fusion is followed to get the nal results. In naive fusion, element-by-element multiplication is performed between the input and its corresponding weight as shown in Figure..

Fusion Output Results.

Image Pyramid. Image pyramid refers to a way of representing the image at multiple resolutions. The feature that might go unnoticed at a particular location will get noticed at a different location. If in a dataset both the low and high resolution images are present analyzing them at different resolutions can be beneficial . The name pyramid because we stack the high resolution image at the bottom and then stack lower resolution over it then we get a pyramid. Two types of Pyramid are Gaussian Pyramid Laplacian Pyramid.

Gaussian Pyramid. Gaussian pyramid involves repeated applying gaussian blurring and downsampling an image until some stopping criteria can be met. This stopping criteria can be a minimum image size. Open cv has a dedicated function for applying Gaussian pyramid cv2.pyrDown(src[, dstsize[, borderType]]) Here, src is the image and dstsize refers to the size of the output image..

Laplacian pyramid As we know Laplacian is a high pass filter therefore each level of this pyramid we will get an edge image as an output. Laplacian can be approximated by the difference of Gaussian so taking this advantage of this condition we will obtain Laplacian pyramid. Thus, the Laplacian of a level is obtained by subtracting that level in Gaussian Pyramid and expanded version of its upper level in Gaussian Pyramid..

Performance Metrics. Underwater Color Image Quality Evaluation Metric (UCIQE) Underwater Color Image Quality Evaluation Metric (UCIQE) is denied for images in CIELab space. It considers the standard deviation of chroma σc, the contrast of luminance conl , and the average of saturation µs for assessing the images. Weights (c1, c2, c3) are assigned to these parameters and then added to get the nal value of the metric as given in Equation. A higher UCIQE value indicates a higher quality of the image For evaluating underwater images with color casts and distortions, the values of the coecients are c1 = 0.4680, c2 = 0.2745, and c3 = 0.2576..

Results. Histogram output. Color Histogram 1400 1200 1000 600 200 150 Color value 200 250.

Original test image denoise Histogram Equalization.

Conclusion and Future Work. In this project, the processing of the distorted underwater image is demonstrated. An alternative approach has been proposed that does not require any additional information other than a single input image. The whole process has been described in detail. It was observed that the deterioration in the images increases with the depth of image capture. Histogram equalization was helpful in improving the prominence of each color channel. And the controlled distribution of pixel intensities was obtained. The white balancing algorithm reduced the color casts. It had a significant role in restoring the original image. Later, the fusion process is described along with the derivation of both the fusion inputs. The fusion inputs preserve the colors and details of the image, respectively. The weights calculated were crucial in merging the fusion inputs to produce a better output..

Future Work. The fusion can be improved by decomposing the inputs into high and low-frequency components. These components can be fused using Multi-scale fusion techniques. Fusion process can also be implemented using Deep learning-based methods like Convolution neural networks(CNN) and Generative adversarial networks(GAN) for better extraction of the features. Non-Subsampled Shearlet Transform retains the image’s shape while decomposing into high and low-frequency components and hence better results..

THANK YOU.