SINGLE IMAGE SUPER RESOLUTION VIA COUPLED SPARSE AND LOW RANK DICTIONARY LEARNING

: Limitations in imaging systems and the effects of changes in sensing have caused limitation in acquiring high resolution images such as satellite images and magnetic resonance imaging (MRI). Sparsity can reduce the noises and improve the resolution. Super resolution in medical and satellite imagery is essential because low resolution image analysis is very difficult. Sparsity techniques have significant influence on computer vision specially when the main objective is extracting the meaningful information. The success of sparsity is related to the nature of signals such as image and sound which are naturally sparse because they were founded based on Wavelet and Fourier equations. In this research, we proposed a method for restoring a clear image from the related low-resolution parts of both MRI and satellite images. First, we proposed a widespread structure for learning the couple low rank and sparse main characteristic representation. Combined optimization of the nuclear and L 1 norms extracts the total low rank formation and the local patterns lodged in the image. In that case the reconstructed image will be more informative and matrix decomposition problem can recover a noisy observation matrix into an approximation of low rank matrix and a second matrix which contains some low dimensional structure. We assumed that by removing the blur and noise from these images, they would be reconstructed in the highest quality. The proposed method was compared with a variety dictionary learning approaches which addressed super resolution problem, such as tensor sparsity, Generative Bayesian and TV based methods. We demonstrated the results of applied method on MRI and satellite images, showing both visual and psnr improvements. Dealing with complex data in best manner shows the


INTRODUCTION
In the remote sensing, medical, military surveillance and reconnaissance and many other domains gaining high resolution (HR) images leads to huge consequence.However higher quality in imaging leads us to large and time-consuming problems.To address this problem, we proposed a super resolution approach using coupled dictionary learning incorporating dimension reduction strategy based on sparse and low rank structure.The key contribution of this work includes: 1.a new super resolution approach for low resolution (LR) satellite image and low resolution magnetic resonance imaging (MRI) are able to enhance various pairs of lowand high-resolution images 2. designing an effective coupled sparse dictionary, relying on alternative direction method due to the matrix-vector multipliers for preserving more spectral details The rest of the article follows: section 2 provides an overview of the related articles.In section 3 the prepossessing and the SR based image on DL Scheme for the proposed method presents.Section 4 reports the experimental results, and conclusion of this work is presented in section5.

RELATED WORK
In super resolution (SR), problem sparsity plays an important role.Each signal is estimated by a linear combination of examples described as dictionary components, resulting in simple and compressed representation (Donoho,2006).Image patches can represent sparsely over a particular dictionary then some extra information are gained to enhance the visual quality of the recreated image.Sparsity techniques and dictionary learning (DL) have significant influence on computer vision specially when the main objective is extracting meaningful information.
Dictionary learning has been successful in solving inverse problems such as medical imaging, single pixel detector and background subtraction (Cevher et al.,2008;Egiazarian et al.,2007, Gong et al.,2015) predicting missing values in image inpainting and demosaicking.Dictionary learning break up the data matrix  into a dictionary matrix  and a presentation of matrix , so that it's noted as matrix factorization.≈  In other words, signal  in ℝ  is estimated with a sparse linear combination of a few columns of a matrix  in ℝ × .Various dictionary learning methods exist.Structured dictionary learning considers a particular structure among dictionary elements to overcome l1-norm deficiency in modeling interactions among dictionary elements.Hierarchical dictionary learning considers a predefined hierarchical structure exist among dictionary elements.Zhou et al (2009) considered  as Group-Lasso penalty.A tree is described by the user and one dictionary component is dedicated to every node of the tree.Each group structure  of subsets of {1, … , }including one node and all its descendants in the tree.The penalty function  is designated as .
In this tree structure dictionary components with low frequency gather near the root and high frequencies near the leaves.Inspired by topographic independent component analysis, Hyvarinen et al (2001) introduced a twodimensional grid structure for dictionary learning.The main assumption is that dictionary elements can represent on a grid and neighbor relations between them are definable (Marial et al.,2011).Inspired by this method Szlam et al (2011) modeled inhibition impacts between dictionary components.Sahebi et al (2018) proposed a sliding window-based joint sparse representation model for hyperspectral anomaly detection.Huang et al (2018) proposed an image super resolution algorithm based on an advanced sparse autoencoder.Ayas et al (2020) proposed a single image super resolution dictionary learning and sparse coding with multi directional Gabor feature representation.They used Gabor filter to extract image features at various scales and orientations.Therefore, the difficulty of capturing the complex local structures in all scales and variations is resolved.The effective mapping between low resolution and HR images achieves a high-quality reconstruction result.The learningbased dictionary has a comparatively powerful adaptive skill.
Therefore, Barman et al (2021) proposed a GPU accelerated adaptive dictionary learning and sparse representation for multispectral (MS) image super resolution.For edge preserving, they extracted high frequency characteristics presented in the input low resolution of MS image using Butter which worth low pass.Then various parallel algorithms are created for adaptive Dictionary learning.In all of these applications choosing the proper dictionary is essential

Data
Magnetic resonance imaging (MRI) of brain downloaded from the health section of https://www.data.gov/ in double complex format.IKONOS satellite image downloaded from ZENDOO dataset.This data contains 20% sampling of RGB, NIR and panchromatic bands.

Preprocessing of magnetic resonance imaging (pMRI)
The data acquiring level in conventional MRI is a relatively slow sampling process.To improve the scanning speed of MRI an efficient and feasible way is needed to acquire the data in parallel with multi-channel coils (Chen et al.,2013).The scanning time relies upon the number of measurements in the furrier domain and it will be considerably reduced when each coil only acquires a small fraction of the total measurements.In literature this issue called pMRI.
For furrier pulse sequences parallel imaging approaches invariably reduce the number of phase encoding steps needed to sample k-space and thereby decrease the imaging time.(Sahebkheir et al.,2019)In order to reduce the imaging time, MRI was reconstructed with pMRI based on the following formula:   = 1, … ,     =   +  where  is the unknown image,   is the vector of computed partial Fourier coefficients at the jth receiver,  is a diagonal sub-sampling operator,  is the Fourier transform,  is the Gaussian noise, and  is the total number of coils.The operator   is a diagonal matrix sensitivity mapping for the jth receiver, as it is used to refund for the crumble of signal intensity with distance from each pixel (Chen et al.,2013).

Image SR algorithm using dictionary learning
The super resolution problem can be defined as  =  +  . ∈   is the HR image,  ∈   is the LR image,  is the down sampling operator,  is as the additive noise.Expecting that there is an overcomplete HR dictionary  ℎ ∈  × and a LR dictionary   ∈  × sharing the same sparse coding.Then, the HR image  can be stated as a sparse linear combination of ∈   ,  ℎ =  ℎ .The model of recovery solution for sparse coding is as follows: where  notes as approximation error and ‖‖ 0 denotes as l0-psedu norm which computes the non-zero elements in a vector.l0-psedu norm replaced with the l1-norm because sparsify the solution and efficient optimization.Therefore, the optimization problem using Lagrange multiplier formulated as: where the parameter  controls the sparsity of the solution.
By solving the equation ( 2) the sparse coefficient  can be obtained and the HR image is reconstructed by mapping onto the high-resolution dictionary  ℎ =  ℎ  * .

Training set of preprocessing method
The training set containing 91 HR image derived from literature of (Huang et al.,2017). ℎ ∈  × represents the HR images and the corresponding   ∈  ×  constructed using scale down operator.Bicubic interpolation used to create related middle images   of the equal size as the HR images.

Create the HR training set
The HR images are deducted from middle images to eliminate their low frequencies.The difference images  ℎ can be obtained via  ℎ =  ℎ −   (Zeyde et al.,2011).Then the HR training set can be gained by accomplishing feature extraction on different images  ℎ .

Create the LR training set
In order to pull out local features related to the high frequency contents  high pass filters conducted on   {  *   }  ,  = 1,2, … ,  ( * denotes as convolution operator).Two kinds of high pass filters are preferred: gradient filters or Laplacian filters.
After this preprocessing step feature extraction is carried out on these filtered images.Then the LR training set   ′ can be achieved.Dimension of   ′ increased after operating an interpolation and linear filter.In that case, the last step before dictionary learning is dimensionality reduction.Sparse principal component analysis (SPCA) was conducted to reduce the dimension of   ′ and computation complexity.SPCA is based on PCA but it can provide more interpretable results.SPCA uses lasso constraint on the regression coefficient.This leads to representative and more accurate sparse principal components Merola (2014).Finally, the joint training set  = [ ℎ ,   ] obtained by joining together the HR training set  ℎ with the LR training set   .

Coupled sparse dictionary learning
The image background generally has a low rank structure so the dictionary should have a low rank formation.Different objects can be distinguished with their specific characteristics.So, they should have sparse structure i.e., (  ) ≤  , ‖ ℎ ‖ 0 ≤  1 . and  are two integers representing the prior information on the upper of the sparsity and the rank, respectively.The identity information of the same objects under different cameras is the same and should be as comparable as possible.
Therefore, the same objects should have the same coefficients even under different cameras because of that  ℎ =   .These methods rely on generating coupled sparse dictionary while jointly encoding two coupled feature spaces considering low rank information.Our aim is to discover a coupled dictionary pair   and  ℎ for the HR  ℎ ∈  × and LR   ∈  × images.The proper coupled dictionaries   and  ℎ can be approximated by solving the following sparse decompositions: Where   is the sparse coefficient matrix related to LR image. ℎ represents the sparse coefficients of HR image  ℎ and   controls the sparsity penalty.Alternative direction method of multipliers (ADMM) (Jiao et al.,2016) can convert the constrained dictionary learning problem represented in (3) into an unconstraint form.This can be formulated as bellow: For easier calculation the equations in ( 4) formulated as l1minimization problem: ADMM is a dominant algorithm for solving structural convex optimization problems (Boyd et al.,2011).ADMM uses augmented Lagrange so that the objective function of the original problem splits into several sub problems.The DL step for sparse representation of signals considering low rank information formulates as:  , {‖ − ‖  2 } , . .‖  ‖ ≤  ,  = 1,2, … ,  where  is the training matrix,  is the dictionary and  denotes the projection of the signals onto the dictionary ,  is the upper bound of the sparsity coefficients.
For solving this optimization equation ADMM and a given initial dictionary  and training matrix  are used.Then OMP algorithm performed for the sparse coding stage.Therefore, coefficients  can be solved.By considering  is fixed, we updated dictionary .Iteration continues till the convergence satisfies error of the signal representation.Indeed, DL based on ADMM algorithm changes the problem to this format using the Lagrange function: where Λ  is the th column of Lagrange multiplier matrix.

EXPERIMENTAL RESULT
In this section, we represent the results of the proposed method and its comparison obtained from ASDS Dong et al., (2011), Generative Bayesian SR of Zhang et al., (2012), Low Rank Tensor Completion by Liu et al., (2013), FISTA by Beck et al., (2009) and Sparse representation based iterative incremental image deblurring from Zhang et al., (2009) implemented on both satellite and MRI images.We studied several deblurring and SR methods based on sparse representation modelling and learning to find the best dictionary.Then the proposed method for image SR implemented in the same environment for both pMRI result and satellite image.The proposed methos uses coupled DL incorporating low rank information.The ADMM algorithm used for DL which was faster than the state-of-the-art DL methods used in validation algorithms such as K-SVD or BCD.The average running time of proposed method and its validations for MRI imaging and satellite image listed in table 1.
As it's shown, the proposed method is the fastest.The sparse coding stage in the proposed method performed by OMP.For fair comparison, the proposed method and the validation algorithms used the same training set extended by 90° rotation considering the visual importance of edges.
Degradation factor set  = 3 and white Gaussian noise with standard deviation of 2 were adopted for all algorithms.Feature extraction employed by Laplacian filters and gradient.Nearly 130,000 training patch pairs were gathered.In the proposed method, SPCA applied for dimensional reduction.For the proposed method, the HR dictionary ℎ and the LR dictionary   utilizing coupled sparse dictionary learning.Sparse representation coefficients calculated by OMP algorithm and the HR image calculated via  =  ℎ .The psnr (peak of snr) calculated by Eq (8): Figure 1 illustrates the result of pMRI reconstruction and employment of proposed and validation methods on it.

First
, we used pMRI technique for reconstructing magnetic resonance imagery in complex double type which was downloaded from https://www.data.gov/.(Sahebkheiret  al.,2019)The reconstruction procedure took about 3 minutes implemented in Matlab 2016b on a desktop Intel corei7 8550U CPU.The result of pMRI is shown in Figure1(a).After the reconstruction, some blurs appeared; this time we assumed that by performing super resolution algorithms and removing the blurs, the image should be reconstructed in the best of its quality.The result of pMRI is the test image for the deblurring and image super resolution step.
Figure1illustrates the result of pMRI reconstruction and employment of proposed and validation methods on it.Figurebillustrates the result of SR algorithms on satellite image.Figure1(b) shows that ASDS suppresses the noise but some details are lost.Figure1(c) shows fine reconstruction result but it took 338.37minutes.In BCD method incorporating tensor sparsity leads to cause ghost artifacts enclosing the edges in figure1 (d) and figure 2 (e).TV based methods like FISTA are efficient in overpowering the noise, however, they produce oversmoothed results and exterminate much details represented in figure1 (e) and figure2.(f).The l0-norm sparsity-based methods are effective in rebuilding smooth parts but it fails to rebuild sharp edges in figure1.(f).The proposed approach has the best visual quality and the distances between white matter and gray matter which are constructed accurately in figure1(h).Figure2.(c) shows that ASDS suppress the noise but produces piecewise constant block artifact.Figure2.(d) (f) (g) color distortion is obvious.In Figure2 (e), the reconstructed image is jaggy and contains ringing effects.Figure2.(l) shows that the proposed approach has the finest observable quality and edges are preserved.Table2 represent the PSNR and SSIM related to MRI and satellite image reconstruction under different methods.The experimental results on both MRI and satellite image clarified that the proposed approach outperforms various state-of-the-art methods in both PSNR, SSIM and visual quality.