Recently, with increasing requirements and applications on ubiquitous location-based services (LBS), indoor positioning has become one of the most promising technologies and has many potential industrial and commercial benefits^[1]. Unfortunately, suffering from high attenuation and strong multipath effects, the global positioning system (GPS) cannot work effectively in indoor scenarios. On the other hand, researches on visible light positioning provide a new approach on high-precision indoor positioning with several advantages such as no electromagnetic interference (EMI) on existing devices and reuses the existing lighting infrastructures as reference points in positioning, which performs low-cost, energy-saving, and widely applicable localization services^[2].

According to the kind of detector on the receive side, the research about visible light positioning can be categorized into two types: photodiode (PD) based and image sensor based. In previous research, many PD-based visible light positioning algorithms and systems are proposed^{[35" target="_self" style="display: inline;">–5]}. Jung proposed a visible light-positioning algorithm based on the time difference of arrival (TDOA), and proved by simulation that the indoor location accuracy of the algorithm can be less than 1 cm^[3]. Zhang proposed an asynchronous receive signal strength (RSS)-based indoor positioning system with the positioning error of several centimeters^[4]. For these visible light positioning systems, the receiver should include a high-speed PD, which is not included in typical commercial mobile devices. Thus, extra accessories should be installed. It will increase the cost of the positioning system and bring inconvenience to users. On the other hand, research about indoor visible light positioning based on image sensors are also emerging^{[6–12" target="_self" style="display: inline;">–12]}. Kuo demonstrated an imaging positioning system taking advantage of the rolling-shutter effect of image sensors, and achieved decimeter-magnitude indoor positioning^[6]. Hossen estimated the performance of imaging positioning systems with image sensors of different resolution^[7]. A fish-eye lens-equipped imaging positioning system is demonstrated by Nakazawa with a maximum horizontal error of about 10 cm^[8]. For these imaging positioning systems, multiple reference light sources should be captured by image sensors for 3-dimensional (3D) positioning, which raises a requirement on the density of light sources in a positioning cellular and has limitation in scenarios where the arrangement of light sources is sparse.

In this Letter, an indoor imaging visible light positioning scheme based on a sampled sparse light source and a mobile device that includes an image sensor and a gyro is proposed. The scheme is demonstrated in a positioning cellular with a single light source and an off-the-shelf mobile phone. The experimental results show that the average 2-dimensional (2D) and 3D positioning error are only several centimeters even with a rotated, rolled, and pitched mobile phone. Moreover, the distributions of the 2D and 3D positioning errors are analyzed. To consider the identification of different cellular, space coordinates, and parameters of the reference light source in a specific cellular can be transported via the rolling-shutter effect of an image sensor^[13] or take advantage of visible light communication^[14]. This work will focus on the high-precision positioning scheme in a typical cellular.

A typical scenario of indoor imaging positioning is shown in Fig. 1. The reference light sources with the diameter of d are mounted evenly with interval D, and the image sensor is placed upward vertically. In previous research, to make sure that at least three light sources in a positioning cellular can be captured by the image sensor, the density of light sources should follow D+d/2≤h×tan(φ/2),(1)where h is the vertical distance between the mobile device and the ceiling and φ is the field of view (FOV) of the imaging system in the mobile device. This means that the performance of this kind of positioning system is restricted by the arrangement of the reference light sources and the parameters of the imaging system on the mobile device.

Fig. 1. Typical scenario of indoor imaging positioning.

下载图片 查看所有图片

In this Letter, a imaging positioning scheme that only needs a single reference light source in a cellular to achieve 3D localization is proposed. The imaging positioning scheme in a positioning cellular, as Fig. 2(a) shows, is discussed. At the transmitter, a single reference light source is used, and it will be sampled into 5 reference points. At the receive side an image sensor usually equipped by a mobile device is considered. There will be a projective light spot of the reference light source on the image sensor, which includes 5 corresponding projective points.

Fig. 2. (a) Positioning cellular and principle, (b) azimuth, roll, and pitch angle of mobile device.

下载图片 查看所有图片

In the imaging positioning scheme, a 3D Cartesian coordinate system is defined in the room. The aim of positioning is to obtain the space coordinate (x,y,z) of the centroid O of the imaging lens according to the image captured by the image sensor. It is clear that rotation, pitch, and roll of the mobile device will influence the captured image. For an arbitrary orientation of a mobile device it can be expressed as three angles, as Fig. 2(b) shows, namely the azimuth angle θ, roll angle γ, and pitch angle ω. A 2D Cartesian coordinate system is defined on the image sensor whose origin is the centroid of the image sensor. When the axis x′ and y′ of the image sensor are parallel to the axis x and y of the space coordinate system in the room, respectively, θ, γ, and ω are defined as zero.

The flow chart of the proposed imaging positioning algorithm is shown in Fig. 3. First, the image is captured by the image sensor on the mobile device. At the same time, the azimuth angle, roll angle, and pitch angle of the mobile device are measured by the gyro integrated in it. If the mobile device is rotated, pitched, or rolled, the position and shape of the light source’s projective image on the image sensor will be changed. In order to remove these influences, the captured image is restructured according to [xre′yre′]=[cosθ−sinθsinθcosθ]−1[cosω00cosγ]−1[xca′yca′],(2)where (xca′,yca′) is the coordinate on the image sensor of each pixel in the original captured image and (xre′,yre′) is the corresponding coordinate in the restructured image of the original pixel. Moreover, to reduce the complexity of the algorithm, only pixels in the reference light source’s projective image on the image sensor are computed.

Fig. 3. Flow chart of the imaging positioning algorithm.

下载图片 查看所有图片

With the restructured image, the imaging scheme is introduced as Fig. 2(a) shows. First, 5 reference points described as A(x1,y1,z1), B(x2,y2,z1), C(x3,y3,z1), D(x4,y4,z1), and E(x5,y5,z1) are sampled from the reference light source on the ceiling. The z coordinates of these points are the same because they are in the same plane. A is the centroid of the reference light source, B, C, D, and E are 4 vertices of the circumscribed rectangle of the light source, and BC is parallel to axis x of the space coordinate system. The corresponding projective points in the restructured image of these reference points are described as A′(x1′,y1′), B′(x2′,y2′), C′(x3′,y3′), D′(x4′,y4′), and E′(x5′,y5′). According to the homothetic triangle theory and geometrical optics, the vertical distance between the ceiling and the imaging lens can be calculated as h=f×d(d1+d2)/2,(3)where f is the focal length of the imaging lens, d1(d2) is the distance between B′ and C′(E′). For reference point A and the corresponding projective point A′ on the image sensor, Eq. (4) should be followed: h2f2=(x−x1)2+(y−y1)2x1′2+y1′2.(4)

Since the focal length f of the imaging lens can be read out and the coordinate of A′ on the image sensor plane, namely (x1′,y1′), can be obtained, Eq. (4) can be simplified as C1(x−x1)2+C1(y−y1)2−h2=0,(5)C1=f2/(x1′2+y1′2).(6)

Similarly, to consider the other 4 pairs of reference points and the corresponding projective points, {C1(x−x1)2+C1(y−y1)2−h2=0C2(x−x2)2+C2(y−y2)2−h2=0C3(x−x3)2+C3(y−y3)2−h2=0C4(x−x4)2+C4(y−y4)2−h2=0C5(x−x5)2+C5(y−y5)2−h2=0,(7)should be followed, where Ci=f2/(xi′2+yi′2)i∈[2,3,4,5].(8)

Finally, by taking advantage of the Levenberg–Marquardt method, the least square solution of (x,y) can be obtained by solving Eq. (7). Consequently, the location of the mobile device (x,y,z1−h) is obtained. In general, compared with^[7], the proposed positioning algorithm needs to restructure the captured image and the complexity is slightly increased. On the other hand, it can achieve indoor localization with a rotated, pitched, and rolled mobile device, and only needs a single light source in a cellular, which is effective and very suitable for scenarios with a sparse lighting infrastructure.

To evaluate the performance and robustness of the proposed positioning scheme an imaging positioning cellular is built with off-the-shelf devices. Proof-of-concept experiments are made accordingly, and the block diagram is shown in Fig. 4. At the transmitter a commercial light-emitting diode (LED) lamp is driven to emit visible light for both illumination and positioning. To consider that the space coordinate and diameter of the LED lamp can be transported by visible light communication they are assumed as known constants here. At the receive side, a mobile phone is used to capture image and gyro information to position itself. To reduce the influence of the halo around the LED lamp the exposure time of the image sensor is controlled by an integrated imaging software. Considering that a mobile phone is usually handheld in practical application the front camera of the mobile phone is used to capture the images and the distance between the ceiling and the mobile phone is about 1.8 m. For simplicity, the captured information is transmitted to a computer and processed offline, and it can be achieved by a cloud service in practical usage. The key parameters of the system are listed in Table 1, and the scenario of the experimental cellular is shown in Fig. 5.

Fig. 4. Block diagram of the imaging positioning system.

下载图片 查看所有图片

Fig. 5. Scenario of the imaging positioning cellular.

下载图片 查看所有图片

In experimental verifications the performance of the proposed imaging positioning scheme in a practical system is evaluated with different positions and orientations of the mobile phone. A typical captured image and the corresponding restructured image for a specific location (0.8, 0.2, 0.2) of the mobile phone are shown in Fig. 6. In Fig. 6(a), the image is captured with azimuth angle 270°, roll angle 5°, and pitch angle 5°. Figure 6(b) is the corresponding restructured image and 5 reference points are sampled in it. It indicates that for a mobile device with azimuth, roll, and pitch angles, the influences of these angles on the captured image can be removed by the proposed algorithm and reference points can be sampled successfully. Moreover, due to the short exposure time of the image sensor, the projective light spot of the light source is very clear in the captured image, which made the proposed scheme not sensitive to the background on the ceiling and reflected lights from the scenario.

Fig. 6. (a) Captured image at (0.8, 0.2, 0.2), (b) corresponding restructured image at (0.8, 0.2, 0.2).

下载图片 查看所有图片

For the positioning experiment, first the mobile phone is placed upward vertically with azimuth, roll, and pitch angles equal to 0°. The distributions of the 2D/3D positioning errors and the corresponding cumulative distribution functions (CDFs) are shown in Figs. 7 and 8(a), respectively. The results show that in the positioning cellular the average 2D/3D positioning error is 0.89/1.68 cm. Moreover, for 95% of the points on the receive plane, the 2D and 3D positioning errors are less than 1.67 cm and 3.44 cm, respectively.

Fig. 7. (a) 2D positioning errors with azimuth, roll, and pitch angles equal to 0°, (b) 3D positioning errors with azimuth, roll, and pitch angles equal to 0°.

下载图片 查看所有图片

Fig. 8. CDFs of positioning errors with (a) azimuth, roll and pitch angles equal 0°, (b) azimuth angle equals 270°, roll and pitch angles equal 0°.

下载图片 查看所有图片

The positioning performance of the proposed scheme is shown in Figs. 8(b) and 9 with an azimuth angle. For the same positioning cellular the scheme is evaluated with an azimuth angle equal to 270° and roll/pitch angles equal to 0°. The average 2D/3D positioning error is 1.16/2.05 cm, both slightly larger than the errors with an azimuth angle equal to 0°. Moreover, for points on the receive plane, 95% of them are more accurate than 2.38/5.09 cm, respectively, for 2D/3D positioning.

Fig. 9. (a) 2D positioning errors with azimuth angle equal to 270°, roll, and pitch angles equal to 0°, (b) 3D positioning errors with azimuth angle equal to 270°, and roll and pitch angles equal to 0°.

下载图片 查看所有图片

Fig. 10. (a) 2D positioning errors with azimuth angle equal to 270°, and roll and pitch angle equal to 5°, (b) 3D positioning errors with azimuth angle equal to 270°, and roll and pitch angles equal to 5°.

下载图片 查看所有图片

For a practical indoor positioning scenario the handheld mobile phone usually works with an azimuth angle and is slightly pitched and rolled. To consider that the receiver can capture the image of a light source at each point on the receive plane, the performance of the proposed imaging positioning scheme is evaluated with an azimuth angle equal to 270° and the roll/pitch angles equal to 5°. The results are shown in Figs. 10 and 11. The average 2D/3D positioning error is 1.50/2.67 cm. 95% of the measured points on the receive plane are localized with positioning errors no more than 3.46/4.96 cm for 2D/3D localization. It indicates that the proposed scheme is effective even with tilted mobile devices. The resolution of the front camera of the used mobile phone is only 1.2 million pixels. With a higher resolution camera the positioning error can be decreased further. Moreover, the distribution of the positioning error is nearly even for a tilted mobile phone on the receive plane. By combining more cellular with a single light source in a practical scenario together high-precision indoor localization can be achieved in a large area. It is worth noting that, in these experiments, the exposure time of the image sensor is less than 2 ms. Thus, for a moving receiver with a typical walking speed, the positioning system is still robust and the performance will not be degraded greatly.

Fig. 11. CDFs of positioning errors with azimuth angle equal to 270°, roll, and pitch angles equal to 5°.

下载图片 查看所有图片

In general, for both 2D and 3D positioning in a cellular, the proposed imaging positioning scheme can achieve precise localization with a centimeter-magnitude positioning error at every measured point (including the points far from the light source) on the receive plane, which is accurate enough for high-precision indoor navigation and LBS. Moreover, with the association of a gyro in the mobile device and the proposed restructure algorithm, the imaging positioning scheme can still work with a rotated, rolled, and pitched mobile device. It means that the proposed scheme is robust and suitable for practical usage. According to the good positioning performance of the proposed scheme in a cellular, high-precision and robust positioning services are guaranteed by placing many positioning cellulars evenly in various indoor scenarios. Moreover, the identification and information transportation of LEDs in different cellulars can be achieved via the rolling-shutter effect of the image sensor as^[13] shows.

In this Letter, an indoor imaging visible light positioning scheme with a sampled sparse reference light source and a mobile device is proposed. Proof-of-concept experimental verifications are made with a single light source and an off-the-shelf mobile phone in a positioning cellular. The results indicate that the localization performance of the proposed scheme is high precision even with a rotated, rolled, and pitched receiver and the 2D/3D positioning errors are centimeters in magnitude. Moreover, the entire positioning system is made of slightly modified existing lighting infrastructures and a commercial mobile phone, which means that the proposed imaging positioning scheme can provide accurate, robust, convenient, and cost-effective localization services and is suitable for indoor positioning in various scenarios such as a warehouse, supermarket, office, parking lot, and airport.