1 Introduction

Infrared target detection has a wide range of applications in optical remote sensing，air defense early warning，and night surveillance. Especially in military application scenarios such as enemy aircraft tracking and minefield detection，where have various factors such as long imaging distance，low target pixel share，inherent sensor noise，and a complex natural environment background，make the detection of weak infrared targets very difficult^［1-2］.

In the past decades，a variety of methods have been proposed in the field of infrared small target detection^［3-7］. Traditional infrared small target detection algorithms rely on background subtraction. Firstly，the background of the infrared target is predicted，and then the difference between the original infrared image and the background is calculated^［8-10］.Deshpande et al^［8］ proposed maximum median filtering and maximum mean filtering to estimate the background. Although median filtering and mean filtering can suppress some of the burst noise，the detection effect is not satisfactory when the signal-to-noise ratio of the image is low. Zeng et al^［9］first introduced top-hat morphological filtering into the field of infrared target detection for estimating the background to obtain a residual map，but it could not suppress background clutter and was very sensitive to noise. Bai et al^［10］ defined a novel top-hat transform and applied it to infrared small target detection，but the applicable scenario of the algorithm was very limited and could not be adapted to infrared targets of different sizes. Although the method based on background subtraction is simple to implement and efficient to detect，it lacks the use of features of the target which leads to the detection effect being hardly satisfactory.

Compared to visible images，infrared images lack features such as color and texture details，but the contrast is very obvious. Therefore，researchers began to work on using contrast features for infrared small target detection tasks^［11-14］. Chen et al^［15］first proposed the local contrast method（LCM）based on the human visual system. Han et al^［16］ improved the detection efficiency by modifying the definition of local contrast and defining pixel sub-blocks instead of pixel-by-pixel detection. Pan et al^［17］， inspired by LCM，designed a two-layer contrast model to expand the detection area of the nested structure to obtain more complete contrast information. The contrast-based approach is capable of enhancing the target，but the algorithm is very sensitive to pepper noise in the background because the maximum value of contrast is taken into account in the calculation.

Gradient features are also one of the research hotspots in the field of infrared small target detectio^n［18-20］. Zhang et al^［21］ divided the detection structure into four quadrants of a two-dimensional coordinate system to locate the position of a small infrared target by calculating the pointing of the gradient of the pixel points. However，there is a lot of structural noise in the infrared image with similar structures to small targets，which has the same gradient pointing as the target in some directions，resulting in a high false alarm rate of the algorithm in complex backgrounds. The averaged absolute gradient^［22］ uses the property of local averaging to suppress background clutter and achieves a robust detection effect，but it lacks the use of directional information，and the algorithm cannot correctly detect the location of small targets when one side of the small target is near the highlighted edge. Saeid Aghaziyarati et al^［23］weighted the directional derivative with the average absolute gray difference to address the effect of highlighting edges on the detection effect，but the detection performance of the algorithm decreases significantly when the pixel value of the target is low，or when there are multiple targets with large target brightness disparity in one image.

Based on the above problems，an algorithm based on associated directional gradient and mean contrast（ADGMC）is proposed in this paper：

（1）Four detection structures with different directions are designed to obtain the gray gradient between the target area and the surrounding background area，and the concept of "associated directional gradient" is proposed to suppress background noise and eliminate the effect of highlighting background edges on gradient changes.

（2）Based on the characteristic of "similar contrast in all directions" for small targets，the idea of mean filtering is used to construct a mean contrast detection structure to eliminate isolated noise and enhance the target at the same time.

（3）For the case that there exist multiple targets with large grayscale differences in one image，after calculating the associated directional gradient，the associated directional gradient feature map is nonlinearly mapped to ensure that the algorithm can detect all the small targets in the image completely.

1　Proposed Method

The method proposed in this paper consists of two main modules：the associated directional gradient module and the mean contrast module. First，in the associated directional gradient module，the gradient is redefined as the sum of the gradients in a single direction and adjacent directions，called the joint gradient. Based on the gradient distribution characteristics of the small target，four different detection structures are designed to obtain the associated directional gradient characteristics of the small target in different directions（up，down，left，and right）. Second，considering the case that there are multiple small targets with large luminance differences in one image，a nonlinear mapping function is used to reduce the disparity of feature values between different targets. Then，in the mean contrast module，a contrast detection structure is designed based on the directionality of the small target grayscale distribution，and the minimum value of contrast in the four directions of small target is taken as the final contrast value. The idea of mean-filter is introduced into the calculation of contrast，which makes full use of the directionality of small target grayscale distribution and eliminates the influence of isolated noise on the detection results. Finally，a simple threshold segmentation method is used to segment the target. The flowchart of the ADGMC algorithm is shown in Fig. 1.

图 1. ADGMC流程图

Fig. 1. flowchart of ADGMC

下载图片 查看所有图片

1.1　Calculation of Associated Directional Gradient

The traditional definition of gradient only focuses on the grayscale change in a single direction of the target，as shown in Fig. 2，when the target is close to the high-brightness edge，although the small target has high gradients in the upper，left，and right directions，the lower side of the target is very close to the background high-brightness edge，and the gradient in that direction may plummet to zero.

图 2. 目标靠近高亮边缘

Fig. 2. target near the highlighted edge

下载图片 查看所有图片

To solve the problem that the gradient is easily affected by the highlighted edges，this paper proposes the concept of associated directional gradient，redefining the gradient calculation from a single direction as the sum of the gradients of that direction and another adjacent direction. Four gradient detection structures are designed as shown in Fig. 3；T is the region where the target center is located，B1，and B2 are the background regions for calculating the associated gradients.

图 3. 每个联合方向上的梯度检测结构 (a) 左上联合方向,(b)上右联合方向,(c)右下联合方向,(d)下左联合方向

Fig. 3. Gradient detection structure of each associated direction (a) is the upper associate left, (b) is the upper associate right, (c) is the right associate lower, (d) is the lower associate left

下载图片 查看所有图片

For the detection structures shown in Fig. 3，the associated directional gradient is defined as

The sky background shown in Fig. 4（a）is one of the main application scenarios for IR small target detection. Assuming that the small target is surrounded by complex clouds and the brightness of the clouds exceeds that of the small target，the pixel distribution of the target and the background is shown in Fig. 4（b）. The blue area is the area where the small target is located，the red pixel point denotes the brightest point of the small target，and the rest is the background area. Taking Fig. 3（a）as an example，it is assumed that the brightest point of the small target is exactly in the T area of the detection structure. Since the small target is surrounded by brighter clouds，if the size of the detection structure shown in Fig. 3（a）is 3*3，the return value of the upper-left associated gradient at this point is 25；if the detection structure size is 5*5，the return value will reduce to -5. But for the associated gradient at the bottom right of the target，because the background at the bottom right region is lower than the center of the small target，the larger the detection structure，the larger the gradient value returned.

图 4. 红外小目标靠近高亮边缘 (a)局部放大图，(b)像素示意图

Fig. 4. Infrared small target near the highlighted edge (a) Local enlargement，(b) Pixel schematic

下载图片 查看所有图片

Considering various complex backgrounds，the multi-scale sizes of 3*3，5*5 and 7*7 are set for the detection structures shown in Fig. 3，and the maximum value of the multi-scale is taken as the associated directional gradient value in that direction. Therefore，the multi-scale associated directional gradient is defined as

Where I0 is the grayscale value of the pixel in the center of the detection structure，Iiis the grayscale value of the pixel numbered i in the detection structure. Taking a 7*7 detection structure as an example，each pixel is numbered in the order as shown in Fig. 5.

图 5. 7*7大小的上左联合方向检测结构

Fig. 5. structure of left associate upper direction，with size of 7*7

下载图片 查看所有图片

According to Eq.（2），in each joint direction，the associated gradient is extracted using three different scales of 3*3，5*5 and 7*7 detection structures，and the maximum value of the multi-scale calculation result is selected as the gradient response of the current pixel in the joint direction：

where Gn*ni denotes the associated gradient of the current pixel in direction-i shown in Fig. 3 for a detection window of size n*n.

Define the product of the gradients in the four associated directions as the associated directional gradient（ADG）of the current pixel point：

Since the product is used in the calculation of ADG，when there are multiple targets in one image and the target grayscale difference is large，the product operation will further enlarge the grayscale difference between different targets，making the threshold segmentation phase very difficult. In order to effectively reduce the gradient gap between different targets，a nonlinear function is used to map the ADG to reduce the feature-response gap between different targets. The structural clutter and noise in the background has a very low response in the ADG. To avoid low values of clutter being mapped to high values，a low value truncation of the ADG is performed using Eq.（8） before the nonlinear mapping.

Where MaxADG is the maximum value among all elements of ADG matrix，and α is a coefficient. By the simulation calculation of extreme cases，it can be obtained that α taking 0.1 can better balance the effect of noise elimination and complete retention of multiple targets. Then normalize ADGT with Eq.（9） to obtain ADGN and perform ADGN a nonlinear mapping as Eq.（10）.

Fig. 6 shows the grayscale differences between different targets before and after the nonlinear mapping. Comparing Fig. 6（c）and Fig. 6（e），it can be seen that after the nonlinear mapping，the grayscale difference between targets becomes significantly smaller.

图 6. 非线性映射前后对比 (a)原始红外图像，(b) ADG，(c) ADG的三维灰度图，(d) ADGmappedx,y，(e) ADGmappedx,y的三维灰度图

Fig. 6. Comparison before and after nonlinear mapping (a) original infrared image，(b) ADG，(c) 3D grayscale view of ADG，(d)ADGmappedx,y，(e) 3D grayscale view of ADGmappedx,y

下载图片 查看所有图片

1.2　Mean-Contrast based on eliminating isolated noise

Due to the characteristics of the imaging mechanism，there is often a large amount of isolated noise in infrared images. Various smoothing filters are the most common means to eliminate isolated noise in image processing. In this paper，the idea of mean-contrast filtering is combined with the concept of local contrast，and a mean contrast filter structure is designed as shown in Fig. 7，where T is the target region and Bi is the background region.

图 7. 均值滤波结构

Fig. 7. structure of mean contrast filter

下载图片 查看所有图片

When the size of the target region T is n*n，the contrast between the target region T and the background region Bi is defined as Eq.（11）.

Infrared small targets are centered on the brightest point，and the grayscale value decreases in a radial fashion in all directions. Therefore，the maximum contrast value can be obtained when the small targets are exactly or mostly in the T-zone of the detection structure shown in Fig. 7. To accommodate targets of different sizes，multiscale is set in the target region T. For each Bi，the maximum value of the multiscale response is used as the contrast response between Bi and T：

Since some structural noise only has high contrast in some directions，while the infrared small target has similar contrast in all directions，so select the minimum value among contrastB1、contrastB2、contrastB3、contrastB4 as the contrast between the target region and the surrounding neighborhood，which can effectively filter the structural noise and further reduce the potential false alarm rate of the algorithm. The mean-contrast（MC）of the final background region T is defined as Eq.（16）.

1.3　Calculation of the algorithm

After obtaining the ADGmapped and the MC，the final result obtained by the algorithm based on the association directional gradient and the mean contrast（ADGMC）is defined as

1.4　Threshold segmentation

A new image matrix can be obtained after calculating the ADGMC for the original infrared image. As the calculation of ADGmapped involves a threshold truncation operation to eliminate a large amount of noise in the background，and the multiplication operation of two feature maps further reduces the possibility of false alarms of the algorithm，it is only necessary to determine the threshold value based on the maximum value in ADGMC to obtain a better detection effect. The threshold T is defined as in Eq.（18）：

Where MaxADGMC is the maximum value among all elements of the ADGMC matrix，β is the coefficient between 0 and 1. Experiments have shown that β taking values between 0.4 and 0.6 can better complete the target detection tasks under various scenarios such as single target and multi-target.

The final segmentation result Binary(x,y) is obtained by Eq.（19）.

2　Experiments and results

In this section，firstly，the advantage of associated directional gradient over single direction gradient is verified. Secondly，the suppression effect of mean contrast proposed on independent noise and structural background is verified. Finally，the algorithm proposed in this paper is applied to four real single-target infrared image sequences and a single-frame infrared image group to analyze the performance of the algorithm and compare it with other algorithms. The image data are described in detail in Table 1. The experiments were done on a computer configured with an Intel（R）Core（TM）i5-4460 CPU @ 3.20 GHz 3.20 GHz.

2.1　Experimental procedure and results

2.1.1　Validation of the associated directional gradient module

Fig. 8 shows an example of the "target close to the highlighted edge" case described above. It contains three small targets，with the lower side of the target indicated by the arrow being close to the highlighted edge. The associated directional gradient features and the four individual directional gradient features are computed separately，and the results are shown in Fig. 9：

图 8. 测试图像

Fig. 8. test image

下载图片 查看所有图片

图 9. 联合方向梯度与单方向梯度对比 (a) 左方梯度，(b) 上方梯度，(c) 下方梯度，(d) 右方梯度，(e) 单向梯度乘积图，(f) (e)的mesh图，(g) 左上方联合梯度，(h) 上右方联合梯度，(i) 下左方联合梯度，(j) 右下方联合梯度，(k) 联合放i像梯度乘积图，(l) (k)的mesh图

Fig. 9. Comparison of assoicated directional gradient and single directional gradient (a) left gradient，(b) upper gradient，(c) down gradient，(d) right direction，(e) single-direction gradient product，(f) mesh of (e)，(g) left-upper gradient，(h) upper-right gradient，(i) down-left gradient，(j) right-down gradient，(k) associated directional gradient product，(l) mesh of (k)

下载图片 查看所有图片

From Fig. 9（c），it can be seen that the return value of the single-direction gradient below the target is almost zero because the target is close to the highlighted edge；whereas the return value of the associated directional gradient is related to the surrounding adjacent gradient，so in Fig. 9（i），the return value of the associated directional gradient of the small target near the highlighted edge is still more obvious. Comparing the product of the four directional gradients in the two ways，we can see that combining the two adjacent directional gradients into the joint directional gradient can eliminate the effect of the highlighted edge on the detection of small infrared targets. Comparing the results of the product of the gradients in the four directions under the two gradient calculation methods in Fig. 9（f）and Fig. 9（m），it can be seen that binding the gradients in two adjacent directions as associated directional gradients can eliminate the effect of the highlighted edges on the detection of small IR targets.

2.1.2　Validation of the mean contrast module

As shown in Fig. 10，the red boxes are small infrared targets，and the red circles are marked with artificially added isolated noise.

图 10. 添加了孤立噪声的红外图像

Fig. 10. IR image with isolated noise added

下载图片 查看所有图片

Calculate the MC for Fig. 10 to obtain Fig. 11. Fig. 11（a）to Fig. 11（d）correspond to the mean contrast in the upper，down，left，and right directions，respectively. Fig. 11（e）is the result of taking the minimum value of the mean contrast in four directions.

图 11. 均值对比度计算过程图 (a)左方均值对比度，(b)上方均值对比度，(c)下方均值对比度，(d)右方均值对比度，(e)最终均值对比度输出

Fig. 11. MC calculation process diagram (a) left mean contrast; (b) upper mean contrast; (c) down mean contrast; (d) right mean contrast; (e) final MC

下载图片 查看所有图片

From Fig. 11（a）to Fig. 11（d），it can be seen that the isolated noise is eliminated due to the mean filter structure used in the calculation of MC，and the structural background，such as the cloud boundary，has different contrast in each direction. Fig. 11（e）shows the result of taking the minimum value of the mean contrast in four directions. Since the small target has a high contrast in all directions，while the structural background has a large difference in contrast in different directions，the operation of taking the minimum value can eliminate most of the structural background while having no effect on the small target.

Fig. 12 shows the results of calculating the ADGMC for Fig. 10. From Fig. 12（b）and Fig. 12（c），it can be seen that during the calculation of ADG，the isolated noise is also significantly enhanced along with the target，which undoubtedly affects the detection results. Using Fig. 11（e）to multiply with Fig. 12（d）to obtain Fig. 12（e），ADGMC. As can be seen from the Fig. 12（e），the isolated noise is completely suppressed.

图 12. ADGMC计算过程图 (a) MC的灰度图，(b) MC的mesh图，(c) ADG的灰度图，(d) ADG的mesh图，(e) ADGMC的灰度图，(f) ADGMC的mesh图

Fig. 12. Diagram of ADGMC calculation process (a) grayscale diagram of MC; (b) mesh of MC; (c) grayscale diagram of ADG; (d) mesh of ADG; (e) grayscale diagram of ADGMC; (f) mesh of ADGMC

下载图片 查看所有图片

2.1.3　Experiment on infrared data

In this section，the ADGMC algorithm is applied to the real IR images listed in Table I. The processing process of each single-target image sequence（Data1-Data4）by the ADGMC algorithm is shown in Fig. 13. Fig. 13（a）shows the original infrared image，which contains scenes of sky，ocean，bright and dark backgrounds，near the highlighted edge，and multiple targets. Fig. 13（b）and Fig. 13（c）correspond to the ADG and ADGmapped，respectively. By comparing the two figures，it can be seen that after nonlinear mapping，the low-value pixel points in the image are mapped to higher values，and the gray-scale difference between multiple real targets becomes smaller，which is more conducive to the complete detection of small targets. Fig. 13（d）shows the MC calculation results，and it can be seen that there is isolated noise with higher response in ADGmapped，which is suppressed to some extent in the MC calculation. Fig. 13（e）is the final ADGMC image，which is obtained by multiplying ADGmapped and MC. Fig. 13（f）shows the detection results after threshold segmentation. Data5 is used as a supplementary experiment to verify the robustness of the algorithm in a variety of scenes，and Fig. 14 shows the detection results with the forest as the background in data5. The comprehensive experimental results show that the algorithm proposed in this paper can adapt to a variety of scenes such as sky，ocean，and forest to accomplish the task of infrared weak target detection.

图 13. 从上到下对应Data1-Data4 (a)原始图像，(b) ADG，(c) ADGmapped，(d) MC，(e) ADGMC，(f)阈值分割结果（右上角是目标附近区域放大图）

Fig. 13. top to bottom: Data1-Data4 (a) original image，(b)ADG，(c)ADGmapped，(d) MC，(e) ADGMC，(f) Result(The top right corner is a zoomed-in view near the small target)

下载图片 查看所有图片

图 14. Data5部分数据的处理过程图 (a)原始图像，(b) ADG，(c) ADGmapped，(d) MC，(e) ADGMC，(f)阈值分割结果（右上角是目标附近区域放大图）

Fig. 14. Part of Data5 processing process (a) original image，(b)ADG，(c)ADGmapped，(d) MC，(e) ADGMC，(f) Result(The top right corner is a zoomed-in view near the small target)

下载图片 查看所有图片

2.2　Performance comparison of algorithms

In this section，four general evaluation metrics，signal-to-clutter ratio gain（SCRG），background suppression factor（BSF），receiver operating characteristic curve（ROC）and algorithm runtime efficiency，are chosen to compare the algorithm proposed in this paper with some classical algorithms and current advanced algorithms，including AAGD，AMWLCM，LEF，Max-mean，Max-med，MPCM，Top-hat，RLCM，TLLCM.

2.2.1　Signal-to-clutter ratio gain

The signal-to-clutter ratio（SCR）describes how difficult it is to detect small infrared targets. The higher SCR the easier the target is to detect. SCR is defined as.

whereμt denotes the average pixel value of the target，theσb andμb denote the standard deviation and mean pixel values of the adjacent regions.

To assess the ability of the algorithm to enhance the target，the SCRG is defined as

Where theSCRout and SCRin denote the SCR of the processed image and the original infrared image. A larger SCRG value means that more information about the target is extracted from the original image，indicating that the algorithm has better enhancement capability for the target. In the comparison experiments，the mean SCRG values of single-frame images corresponding to each algorithm are shown in Table 2，with the bolded representing the optimal performance. It can be seen that the algorithms proposed in this paper exhibit optimal target enhancement capability under each set of image data.

2.2.2　Background suppression factor

The BSF is used to assess the ability of various detection methods to suppress background noise. Generally，the higher the BSF，the better the algorithm's ability to suppress background noise. This is defined as follows.

where σin and σout represent the Standard Deviation of the background area（the whole image except the target area）in the original image and the image after processing by each algorithm，respectively.

In the comparison experiments，the mean BSF values of the single image corresponding to each algorithm are shown in Table 3. The bolded represents the optimal performance. It can be seen that the algorithms proposed in this paper exhibit optimal background noise suppression under all sets of image data.

2.2.3　Receiver operating characteristic curve

The ROC can quantitatively describe the dynamic relationship between False Positive Rate（FPR）and True Positive Rate（TPR）. In the performance evaluation of infrared small target detection，TPR and FPR are defined as

The larger the area under the ROC curve（AUC），the better the detection performance of the corresponding algorithm，which is represented in the ROC curve graph by the concentration of the coordinate points in the upper-left corner area of the image. The ROC curves for the different detection methods for each group of images in the experiment are shown in Fig. 15. It can be seen that under various thresholds，the false alarm rate of the method proposed in this paper remains at a low level for both single-target sequences and multi-target complex image groups，while the detection rate is higher，highlighting the superiority of the algorithm in this paper.

图 15. Data1至Data5对应的ROC曲线 (a)Data 1，(b)Data 2，(c)Data 3，(d)Data 4

Fig. 15. Data1 to Data5 corresponding ROC curves (a) Data 1，(b) Data 2，(c) Data 3，(d) Data 4

下载图片 查看所有图片

2.2.4　Time efficiency

The performance of an algorithm in terms of time is one of the criteria used to evaluate it. The average time efficiency for single-frame image detection in the comparison experiments is shown in Table 4. The algorithm proposed is not as fast as the classical background subtraction method due to the multiple scales used to accommodate targets of different sizes. The TLLCM algorithm has excellent time efficiency，but it does not perform as well as the proposed method in other evaluation metrics such as SCRG，BSF，etc. The background noise and the enhanced target can hardly be suppressed by using a single filter to estimate the background. Moreover，the time efficiency of the proposed method is controlled to the order of hundredths of a second，which is sufficient for various scenarios.

3　Conclusion

In this letter，a new method for infrared small target detection called ADGMC is proposed. A new feature detection structure with associated directional gradients has been designed in the associated directional gradient module，and a contrast detection structure incorporating directional information and the idea of mean filtering has been designed in the mean contrast module. Both modules can effectively avoid the influence of high-intensity edges on the gradient variation pattern of small target models. In addition，a non-linear mapping function has introduced into infrared small target detection to ensure the method can complete the detection of all targets in multi-target scenarios. The experimental results show that ADGMC has strong target-enhancement and background-suppression capabilities，especially for scenes with high intensity edges and multiple targets.