1. INTRODUCTION

A conventional RGB camera can only obtain chrominance information under specific conditions. The color data are affected by the characteristics of the device and the illumination environment. It cannot reveal the objective color information. RGB data change when the illuminating environment changes. Spectral reflectance data can objectively characterize the color data, as much as possible to avoid the inauthenticity of color replication caused by metamerism. Therefore, the study of spectral reflectance data acquisition becomes the focus of current research. The spectral reflectance reconstruction method that is used to obtain multispectral images uses the pseudo-inverse reconstruction ^[1], Wiener reconstruction ^[2,3], and the method of finite-dimensional reconstruction ^[4,5] to obtain information of spectral reflectance that has nothing to do with the equipment and the scene. At present, multispectral imagers have been constructed using a variety of technologies, including mechanically switched bandpass filters, electronically tunable bandpass filters ^[6], snapshot multispectral imaging ^{[79" target="_self" style="display: inline;">–9]}, and fixed gratings to make light splitting. The complexity of the optical system of the first two methods listed above is bigger and the reliability is poor; the cost of the last two methods is more expensive, and the craft is more complex ^{[69" target="_self" style="display: inline;">–9]}. Therefore the paper ^[10] used liquid crystal tunable retarders to modulate the collected spectral information randomly and used a single-pixel detector to collect the modulated spectral energy, and multispectral reflectance information is reconstructed by the algorithm of compressive sensing on the condition that the sampling rate is lower than that of the Nyquist sampling rate needed. A single-pixel detector and a compressive sensing algorithm that is used to reconstruct the spectral reflectance greatly reduce the complexity of the optical system and the reconstruction time; the reconstruction precision is improved, but the modulation range of liquid crystal tunable retarders is limited, because it is difficult to achieve high precision reconstruction.

The problem above can be solved by compressing the totality of spectral channels into a smaller set that can be more easily managed. This set is made by taking a few linear combinations of the integrand products of the illuminant spectrum, narrow-band filter shapes, and the spectral sensitivity of the camera so as to approximate the first few principal components of a particular basic training sample spectral reflectance database. Multiplying these principal components by the testing spectral reflectance can be well fitted by a linear combination of the same principal components; the principal component vectors can be eliminated because of the orthogonality of the vectors, and then the fitted coefficients of the testing spectral reflectance based on the principal component vectors can be obtained and the testing spectral reflectance can be reconstructed.

This paper uses an illuminating source with a specific relative spectral power distribution to illuminate the objects and to modulate the spectral reflectance information of the object. The spectral energy is collected by a single-pixel detector, and high-resolution spectral reflectance information is reconstructed by using the method of sparse decomposition based on the principal component orthogonal base. In this paper, the research can make full use of the sparse prior knowledge (which means that the spectral reflectance are mostly zeros, with only occasionally nonzero values) of spectral reflectance and the relative spectral power distribution of the illuminating source based on the principal component orthogonal basis, which reduces the optical complexity of the multispectral data acquisition system and reduces the sampling number; the efficiency of spectral reflectance reconstruction is improved, and the reconstruction accuracy is improved. It has a certain reference significance for improving the multispectral image acquisition technology.

2. PRESENT SPECTRAL REFLECTANCE RECONSTRUCTION ALGORITHM

After the multichannel spectral imaging system shoots the scene, it outputs a multichannel image that is related to the color space of the camera. The image is affected by the illumination and spectral reflectance of the scene and the spectral characteristics of the camera: where k is the number of the channel, uk is the output image of the channel k of the system, s(λ) is the spectral sensitivity of the camera, l(λ) is the spectral power distribution of light, τk(λ) is the spectral transmittance of the filter of each channel, and r(λ) is the spectral reflectance of the scene.

This is shown as the discrete matrix

Q=M(MTM)−1 can be obtained by the least-square method min‖Qu−r‖22; then the spectral reflectance of the testing sample sets is reconstructed:

3. METHOD OF SPECTRAL REFLECTANCE RECONSTRUCTION BASED ON SPARSE PRIOR KNOWLEDGE BY PRINCIPAL COMPONENT ORTHOGONAL BASIS

According to the smooth characteristic of the spectral reflectance, r can be well fitted by a linear combination of J(<N) feature vectors, and the finite-dimensional model can be obtained: where bj represents the fixed basic vectors of the linear model, and aj represents the corresponding coefficients of the spectral reflectance based on the basic vectors. If the spectral reflectance r is sampled uniformly by N times in the visible scope, the linear model can be expressed as where r is an N×1-dimension spectral reflectance vector, B is an N×J-dimension basic function vector matrix, and a is a J×1-dimension coefficient of the basis function vector. According to the above analysis, Eq. (2) can be expressed as ^{[1113" target="_self" style="display: inline;">–13]}

The particular measurement matrix M can be a sparse decomposition based on the principal component orthogonal basis:

Then

The bj of Eq. (8) is obtained by the method of principal component analysis. The contribution of the first three principal components [b1,b2,b3] is above 95%, when the color card of X-Rite 24,140, the Pantone cotton color card, and the 1296 Munsell color card are used to be the training sample set. The principal component is selected as the sparse basis. By the orthogonal characteristics of the principal component base vector, Eq. (8) can be expressed as

In Eq. (9), (a1,a2,⋯aJ) is obtained by constructing a specific (ȃ1,ȃ2,⋯ȃJ); then the spectral reflectance is reconstructed.

The method principle of spectral reflectance reconstruction based on sparse prior knowledge by a single-pixel detector of the principal component orthogonal basis is shown in Fig. 1. Using a constructing illuminating source with a specific relative spectral power distribution to illuminate the color lump, the spectral reflectance energy value that reflected from the color lump is received by a single-pixel detector. A series of lighting sources, for which the relative spectral power distribution is bJȃJ, is constructed, and the measurement matrix M=[b1,b2,⋯bJ][ȃ1,ȃ2,⋯ȃJ]T is composed of a series of relative spectral power distribution matrices. (The principal component orthogonal basis is obtained by the basic training sample set.) The spectral reflectance of the testing sample is r, and the matrix, which consists of a series of spectral energy values that are received by a single-pixel detector, is defined as u. The coefficient (a1,a2,⋯aJ) is obtained by calculating Eq. (9); then the spectral reflectance is obtained by Eq. (5).

Fig. 1. Spectral reflectance reconstruction principle based on the sparse prior by a single-pixel detector.

下载图片 查看所有图片

For the acquisition of spectral reflectance of the large-sized object, the spectral reflectance energy value of modulated objects is collected by using a CCD. (Each pixel of the CCD is made as a single-pixel detector to measure the reflecting light energy that is reflected from the corresponding position of the object.) Then the spectral reflectance of the object is obtained.

4. RESEARCH ON SIMULATION

The simulation of the spectral reflectance reconstruction based on sparse prior knowledge by a single-pixel detector of the principal component orthogonal basis is carried out in the MATLAB simulation platform. The experiment uses a piece of the training sample set of 1296 Munsell color as multispectral reflectance data of the testing sample; the detecting range of reflectance is from 380 to 780 nm, and the sampling interval is 5 nm. The relative spectral power distribution of the modulation illuminating source illuminates the color lump. A single-pixel detector is used to collect the energy value when each modulated illuminating source illuminates the multispectral color lump.

The process of the spectral reflectance reconstruction based on sparse prior knowledge by a single-pixel detector of the principal component orthogonal basis is as follows: (1) the 1296 Munsell color card is selected as the basic training sample, and the first three principal components of spectral reflectance data are obtained by the method of principal component analysis, as shown in Fig. 2, and made as B=[b1,b2,b3] of Eq. (7). (2) According to obtained principal components, the relative spectral power distribution bJȃJ of the illuminating source is constructed; it is shown in Fig. 3. A piece of the training sample of Munsell color is illuminated (for which spectral reflectance is shown in Fig. 4) and selected as the testing sample, the matrix that consisted of the relative spectral power distribution of the modulated illuminating light is made as M=[b1,b2,⋯bJ][a1,a2,⋯aJ]T, and the spectral energy that was collected by a single-pixel detector is made as u in Eq. (9). (3) The spectral reflectance is reconstructed by Eq. (9); it is shown in Fig. 5. Then the spectral reflectance reconstruction error is compared with the one by the pseudo-inverse method of Eq. (3); the reconstruction error is shown in Table 1. Therefore spectral reflectance can be reconstructed accurately by the spectral reflectance reconstruction algorithm based on sparse prior knowledge by a single-pixel detector; its reconstruction accuracy is better than that of the pseudo-inverse method.

Fig. 2. Principal component of training sample set of 1296 Munsell colors.

下载图片 查看所有图片

Fig. 3. Relative spectral power distribution of modulated illuminating light based on the principal component of training sample set of 1296 Munsell color. (a) Relative spectral power distribution of modulating light based on the first principal component. (b) Relative spectral power distribution of modulating light based on the second principal component. (c) Relative spectral power distribution of modulating light based on the third principal component.

下载图片 查看所有图片

Fig. 4. Spectral reflectance of a piece of training sample set of 1296 Munsell color.

下载图片 查看所有图片

Fig. 5. Results of spectral reflectance reconstruction based on the sparse prior by a single-pixel detector.

下载图片 查看所有图片

The multispectral reflectance and the relative spectral power distribution of the illuminating source can be sparse decomposition based on sparse prior knowledge of the principal component orthogonal basis, and the orthogonal property of the principal component orthogonal basis is used to eliminate basis. Then the sparse coefficient is obtained by calculation and the spectral reflectance data are obtained. The high accuracy spectral reflectance data are obtained with only a small amount of sampling. The detector is a single-pixel detector without spatial resolution ability, which reduces the cost of the multispectral image.

4.9 A. Effect of the Number of Base Vectors of Principal Component Orthogonal Basis

The training sample set of 1296 Munsell color is selected as the basic training sample, and the first four principal components of spectral reflectance data are obtained by principal component analysis. The relative spectral power distribution of illuminating light is modulated and constructed by using one, two, three, and four principal components, respectively, and a piece of the training sample of 1296 Munsell color is illuminated and selected as the testing sample. The spectral energy value obtained by a single-pixel detector is made as u in Eq. (9). The spectral reflectance is reconstructed by Eq. (9); it is shown in Fig. 6.

Fig. 6. Effect of the number of base vectors of principal component orthogonal basis.

下载图片 查看所有图片

From Fig. 6, we can conclude that (1) with the increase of the number of constructed modulation illuminating light sources based on the principal component orthogonal basis, the reconstruction error based on sparse prior by a single-pixel detector of the spectral reflectance reconstruction algorithm decreases, and (2) when the number of structured modulation illuminating light sources based on principal component is above 3, due to the contribution rate of the first three principal components being more than 95%, with the increase of the number of structured modulation illuminating light sources based on the principal component orthogonal basis, the reconstruction error changes less.

4.10 B. Effect of Sample Selection

The color card of X-Rite 24,140, the Pantone cotton color card, and the 1296 Munsell color card are used to be the training sample set, and the chromaticity spatial distribution maps of the selected training sample set are drawn up, and shown in Figs. 7(a)–7(d). The first three principal components that made up the B=[b1,b2,b3] of Eq. (7) of the training sample set of X-Rite 24,140, the Pantone cotton color card, and the 1296 Munsell color card are selected to structure the relative spectral power distribution of the illuminating source. A piece of the training sample of the 1296 Munsell color card is illuminated and selected as the testing sample, and the matrix constituted by the relative spectral power distribution matrix of modulated illuminating light is made as M=[b1,b2,⋯bJ][ȃ1,ȃ2,⋯ȃJ]T. The spectral energy collected by a single-pixel detector is made as u in Eq. (9). The spectral reflectance is reconstructed by Eq. (9). The reconstruction error is calculated by the root mean square of Eq. (10), where ȓ is the reconstructed spectral reflectance: RMSE=1n∑i=1J(ri−r^i).(10)

Fig. 7. Chromaticity spatial distribution of training sample sets (a) X-Rite 24, (b) X-Rite 140, (c) Pantone, and (d) Munsell.

下载图片 查看所有图片

It can be seen from Figs. 7 and 8 that (1) the chromaticity spatial distribution of the color card of X-Rite 24 and 140 is distributed uniformly in the color space, and the similarity of the chromaticity space distribution of the training sample set of the 1296 Munsell color card with the X-Rite 24 and 140 is small. When the illuminating light source is structured based on the first three principal components of the selected training sample set of X-Rite 24,140, the reconstruction error is larger. The similarity of the chromaticity space distribution of the Pantone cotton color with the 1296 Munsell color card is larger, and the reconstruction error is smaller. (2) When the illuminating light source is structured based on the first three principal components of the selected training sample set of the 1296 Munsell color card, the reconstruction error is the lowest. (3) When the number of the selected training sample set increases, and the chromaticity space distribution is more uniform, the spectral reflectance reconstruction error decreases.

Fig. 8. Reconstruction error of selecting different training samples.

下载图片 查看所有图片

5. EXPERIMENT

A piece of a training sample set of 1296 Munsell color is made as multispectral reflectance data. The acquisition range is from 610 to 670 nm, the sampling interval is 10 nm, and the relative spectral power distribution of illumination light is modulated; the relative spectral power distribution of modulation illuminating light is structured by selecting the principal component of the training sample set of the 1296 Munsell color card. A CH253 voltage type photon counter whose spectrum sensitivity changes gently from 610 to 670 nm (the influence of detection spectrum sensitivity is eliminated) is selected to collect the energy value by each modulation illuminating source illuminating the multispectral color card.

An illuminating source for which the center wavelength of the LED is 610, 620, 630, 640, 650, 660, and 670 nm is selected to compose the illuminating source (because the LED has a certain bandwidth, the sampling interval is 10 nm), and the relative spectral power distribution of the illuminating source is modulated by controlling the control current and add attenuator before the single LED, respectively. It is shown in Fig. 9. The matrix structured by the relative spectral power distribution of modulation illuminating light is made as M=[b1,b2,⋯bJ][ȃ1,ȃ2,⋯ȃJ]T of Eq. (9). The modulated light illuminates a piece of the training sample set of 1296 Munsell color. The energy value that is reflected from the color lump is collected to constitute the u. The spectral reflectance is reconstructed by Eq. (9). It is shown in Fig. 10. Therefore, the spectral reflectance can be reconstructed accurately based on sparse prior by a single-pixel detector.

Fig. 9. Relative spectral power distribution of the LED.

下载图片 查看所有图片

Fig. 10. Experimental results of spectral reflectance reconstruction based on sparse prior by a single detector.

下载图片 查看所有图片

5.5 A. Effect of the Number of Principal Components Base Vector

The 1296 Munsell color is selected as the basic training sample, and the first four principal components of multispectral reflectance data are obtained by the principal component analysis method. The relative spectral power distribution of the illuminating source is modulated and structured by using one, two, three, and four principal components. A piece of the training sample set of 1296 Munsell color is illuminated, and the spectral energy value collected by a single-pixel detector is made as the u in Eq. (9). The spectral reflectance is reconstructed by Eq. (9), and the reconstruction error is calculated by the root mean square of Eq. (10). It is shown in Fig. 11.

Fig. 11. Effect of the number of the principal component base vector.

下载图片 查看所有图片

It can be seen from Fig. 11 that (1) with the increase of the number of structured modulation illuminating light sources based on the principal component orthogonal basis, the reconstruction error decreases, and (2) when the number of structured modulation illuminating light sources based on the principal component is above 3, the reconstruction error based on sparse prior by a single-pixel detector changes less.

5.6 B. Effect of Sample Selection

The relative spectral power distribution of the illuminating source is structured by selecting the first three principal components of the training sample set of X-Rite 24,140, the Pantone cotton color card, and 1296 Munsell color, and a piece of the training sample of 1296 Munsell color is used to be the testing sample. The spectral energy value detected by a single-pixel detector is used to reconstruct the spectral reflectance by Eq. (9). The reconstruction error is calculated by the root mean square of Eq. (10); it is shown in Fig. 12.

Fig. 12. Effect of training sampling selection.

下载图片 查看所有图片

It can be seen from Fig. 12 that (1) when the illuminating source is structured based on the first three principal components of the selected training sample set of X-Rite 24 and 140, the reconstruction error is larger; (2) when the illuminating source is structured based on the first three principal components of the selected training sample set of the 1296 Munsell color card, the reconstruction error is the lowest; and (3) with the increase of the number of the selected training sample set, the spectral reflectance reconstruction error is smaller.

6. CONCLUSION

At present, multispectral images generally use a grating spectrometer, a mechanical rotating filter, a liquid crystal tunable optical filter, and snapshot multispectral imaging to split, and the optical system complexity is higher, the price is expensive, and the craftwork is complex. This paper makes full use of the sparse features of spectral reflectance and the relative spectral power distribution of the illuminating source based on sparse prior knowledge by the principal component orthogonal basis, and the object is illuminated by using a lighting source with a specific relative spectral power distribution. The spectral reflectance information of the object is modulated, and high-resolution spectral reflectance information is reconstructed by using a single-pixel detector to collect the spectral energy. The method can reduce the optical complexity of the multispectral data acquisition system and reduce the sampling number, and the efficiency of spectral reflectance reconstruction is improved.

By studying the traditional spectral reflectance reconstruction method, spectral reflectance and the relative spectral power distribution of the lighting source based on a priori knowledge of the principal component orthogonal basis are sparse decomposition, and the orthogonal property of the principal component orthogonal basis is used to eliminate basis. Then the spectral reflectance data are obtained. On the basis of theoretical analysis, the spectral reflectance reconstruction based on a priori knowledge by a single-pixel detector of the principal component orthogonal basis is carried out by software simulation and experiments. It can be concluded that with the increase of the number of structured modulation illuminating light sources based on the principal component orthogonal basis, the reconstruction error based on sparse prior by a single-pixel detector decreases; with the increase of the number of training sampling sets, the reconstruction error decreases.

The spectral reflectance reconstruction method based on sparse prior knowledge by a single-pixel detector can use a smaller sampling amount to obtain high precision spectral reflectance data. The detector is a single-pixel detector without spatial resolution ability, which reduces the complexity and cost of the system, and it has certain reference significance for the improvement of multispectral image acquisition technology.

7 ACKNOWLEDGMENT

Acknowledgment. This study is supported by the National Natural Science Foundation of China (Grant No. 61405115), the Natural Science Foundation of Shanghai (Grant No. 14ZR1428400), and the Innovation Project of Shanghai Municipal Education Commission (Grant No. 14YZ099), National Basic Research Program of China (973 Program) (Grant No. 2015CB352004).