Localization and Route Planning for the Pedestrian Using 360-degree Images

Chief investigator:

Jui-Chiu Chiang, Senior Member, IEEE and Ke-Sin Yan

Department of Electrical Engineering,

National Chung Cheng University, Taiwan

Abstract:

Over the last decades, with the development of mobile devices, smart phones and tablets have become an indispensable part of our lives. The navigation system is one of the often-used functions in the mobile devices. For example, Google Maps not only provides street view services, but also provides route information to guide people to destinations. Therefore, the accurate localization and navigation technology are essential. The global positioning system (GPS) technology is heavily used for outdoor localization. However, its performance is not always guaranteed, in particular in the crowed urban and in the bad weather. Therefore, an accurate outdoor localization technology is still a challenge.

Image assisted GPS could be an effective solution. However, it needs to collect large amounts of data for image matching, and the database usually contains tens of thousands of images. This will significantly increase the computation cost. To tackle this problem, in this work, 360-degree images are used to match the query image taken by the user.

Usually, the navigation system is designed for vehicle. In addition to the localization technique, we also develop a technique for pedestrian navigation. Considering different properties of pedestrians and the external environmental, the target is to provide the most effective and suitable pedestrians route planning. In addition, the conventional navigation systems do not provide the orientation information in a friendly way. Therefore, in the proposed navigation design, we provide the direction for pedestrians in a comprehensible manner and help they arrive the destination safely and comfortably.

Equirectangular representation(ERP)

  • The most classical mapping of a sphere onto a 2D plane is the so-called equirectangular.

        a)  the horizontal angle  𝜃∈[−𝜋,𝜋]

        b)  the vertical angle  𝜙∈[−𝜋/2,𝜋/2]

1

Motivation

  • Even though the current smartphones have integrated with various sensors, the performance of positioning in city urban is still not promising.

        a)  Error is 50-100 meters

        b)  Large amount of data

        c)  Long retrieval time

  •   Pedestrians from heterogeneous group with varieties of needs and personal preferences, route selection for pedestrian navigation system according to user’s subjective preference is more difficult.

2

Usage scenarios

  • In the paper, we provide a service that allows the pedestrian to take the surrounding environment pictures (e.g., query image) using the 3C equipment.

  • Through the query image, the system can offer the current location of the pedestrian. Final, it will provide the optimal the route for the pedestrian.

3

Architecture of the proposed system

4

Implementation environment

  • The experiments are conducted the campus of National Chung Cheng University. The campus area is 134.24 hectares. We collect the 360-degree images and the GPS information of 12 landmarks on the campus.

  • The places are displayed in Table, respectively school gate, administration building, activity center, Zhiyuan building, dorm room, swimming pool, lakeside, library, fountain, playground, hall, and gym.

  • In the map, we construct the node is 48, and edge is 66.

5

Experimental results

  • For the query image taken by elderly people, we test four places to present the experimental results.

  • The place of the image which taken at the daytime are respectively library, hall and fountain.

  • The query image of the administration building at the different time which take at 4 a.m., 6 p.m., and 8 p.m.

6

  • Select the image within the range of less than 100 meters for image matching, thus increasing the efficiency of image matching.

  • It records the distance between the current location of the elderly people and all of the place in the dataset, and then selects the distances less than 100 meters.

  • Match the dataset images, we can get the number of the matching points between the two images. We calculate the average matching points and select the highest matching point number of the category. It presents the current location of the pedestrians is the library.

7

  •   The system will suggest the route for the user.

         * The green lines represent recommendation route for the pedestrian of older.

         * The blue lines represent recommendation route for the pedestrian of children.

         * The yellow lines represent recommendation route for the pedestrian of sun condition.

         * The red lines represent recommendation route for the pedestrian of fast condition.

8

Conclusion

  1. In order to solve the problem of signal instability of traditional position technology and the large amount of image construction dataset, we use the GPS information combine with the 360-degree image to locat the        current  position of the pedestrian.
  2. Using saliency map to extract the viewpoint images and construct a dataset to reduce the search complexity and achieve accurate positioning results.
  3. The experimental results show that the outdoor location of the GPS information combined with the 360-degree image matching method can accurately locate the current location of pedestrian.

Reference

[1] N. M. Drawil, H. M. Amar, and O. A. Basir, "GPS localization accuracy classification: A context-based approach," Intelligent Transportation Systems IEEE Transactions, 2013.


[2] "Algorithm descriptions of projection format conversion and video quality metrics in 360Lib Version 5" Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting: Macao, CN,  18-24 Oct. 2017, JVET-H1004.


[3] D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, pp. 91-110, 2004.


[4] G. Yu and J.-M. Morel, "Asift: An algorithm for fully affine invariant comparison," Image Processing On Line, vol. 1, pp. 11-38, 2011.


[5] B. Dedhia, J. C. Chiang, and Y. F. Char, "Saliency prediction for omnidirectional images considering optimization on sphere domain," in Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019.


[6] B. Chakraborty and T. ashimoto, "Computational Framework for Subjective Preference based Route Selection in Pedestrian Navigation System," in Proc. of IEEE TENCON, pp. 1067-1071, 2011.


[7] Dijkstra, E. W, "A note on two problems in connexion with graphs," Numer. Math., 1959.


[8] O. Czogalla, "A Model for Pedestrian Routing in Walkable Networks," IFAC Proceedings Volumes Volume 45, Issue 24, September 2012.


[9] H. Furukawa, "Empirical evaluation of the pedestrian navigation method for easy wayfinding," in Proc. of the International Conference and Workshop on Computing and Communication, 2015.


[10] S. Bao, T. Nitta, K. Ishikawa, M. Yanagisawa, and N. Togawa, "A safe and comprehensive route finding method for pedestrian based on lighting and landmark," in Proc. of Global Conference on Consumer Electronics, pp. 589-593, 2016.