2D and 3D Enrichment of Panoramas - 27/02/2012
Visualisation of Topographical Features Improves Functionality
The use of panoramic images has become widespread in many governmental agencies and companies, especially in the last couple of years. It is possible to integrate 2D spatial features (e.g. building outlines, parcels, utilities etc.) with panoramic images to enable the proper visualisation of topographical features in a 3D environment. Verbree et al. (2004) have shown the advantages of augmenting underground conduit pipes and cables into panoramic images, thus allowing better visualisation of otherwise invisible features. The geometric augmentation of 2D features into panoramic images enables users to identify, query or analyse any geographically related information in a 3D environment. This 3D environment provides a more natural way of interfacing with the spatial features.
In addition, this augmentation gives the user a clearer visual impression of the region to be analysed, which will in turn enable better visual checks of 2D features when updating topographical datasets or in change-detection applications, for instance. However, the integration procedure may lead to misalignments on a large scale, particularly at locations with a high topographical gradient. For professional applications, it is crucial to properly determine the relative positions of topographical objects in 3D environments. This paper investigates a solution for more accurate and better geometric augmentation to integrate 2D features with panoramic images.
Collaboration and scope
This joint project has been carried out by CycloMedia and the Dutch Kadaster. The Dutch Kadaster holds the position of national mapping agency which provides and maintains small and medium-scale topographical data. CycloMedia is one of the leading companies and solution providers in GIS data in The Netherlands. Its web client platform called GlobeSpotter enables the projection of raster and vector data into panoramic images and onto aerial imagery. The goal of the project was to expand the functionality of panoramic images by integrating topographical data. The GlobeSpotter platform was used for the integration of 2D spatial features into panoramic images, so that the panoramic image could be geometrically augmented with 2D spatial features.
To achieve the integration of topographical data with panoramic images, the 2D features were directly projected onto the panoramic images. The result can be seen in Figure 1, which shows the test area located in Rotterdam, The Netherlands. It is noticeable that the projected 2D features are misaligned with respect to the objects to be fitted, with the misalignments mainly being in the vertical direction in non-flat regions.
To enable more accurate visualisation of the topographical features to the standard required for professional usage, it is crucial to overlay geometric features properly and hence prevent misalignments.
The project was divided into two parts: (1) enriching the 2D features with corresponding ground elevation values from airborne laser altimetry data, in order to minimise the misalignments, and (2) visualising this new dataset through a Web Feature Service (WFS).
There is no doubt that GIS is used by many organisations and individuals to manipulate, query, analyse and visualise various sets of geoinformation. However, integrating different sources of spatial datasets has always been challenging and problematic. One of the particularly challenging aspects of this project is that three different datasets are used, with each dataset having completely different positional and vertical accuracies.
The first dataset is the Large-scale Base Map of The Netherlands (GBKN) containing 2D building outlines. The most detailed topographic dataset in The Netherlands, GBKN is collected by photogrammetric and terrestrial measurements. The positional accuracy in urban areas is about 28cm. For the ground elevation value assignment, a second dataset is used: the high-resolution airborne laser altimeter dataset called Actual Height Model of The Netherlands (AHN2). AHN2 is the digital height model of The Netherlands captured using Lidar (Light Detection and Ranging) technology. The minimum point density of AHN2 is 8-10 points/m2 and can go up to 30 points/m2 with an accuracy of 5cm. The third dataset is made up of 360-degree panoramic images, the so-called Cycloramas. These have a spatial resolution of 0.075 degree/pixel and a standard recording interval of 5 metres at a maximum speed of 80km/h. The positional accuracy is about 10cm. The Cycloramas are seamless, parallax free and geometrically correct (Heuvel, 2008).
From 2D to 3D
The procedure for enriching 2D features into 3D consists of: (1) pre-processing the raw laser dataset for detecting and removing outliers and masking the water areas, (2) generating a Digital Terrain Model (DTM) in a regular grid with a resolution of 50x50cm, (3) extracting the edge vertices of the 2D building outlines, and (4) assigning the height values of the ground surface to the corresponding edge vertices of the 2D building outlines.
As a result of the applied procedure, each feature vertex of the building outline is updated with the corresponding height value from the terrain. In doing so, it is crucial that the height value assignment is completed without manipulating the original horizontal position of the building outlines. The result is a dataset with 3D spatial features (although the enriched data is actually 2.5D, it will be called 3D for the sake of simplicity). This is then imported into GlobeSpotter as GML served by a Web Feature Service-enabled (WFS) web server and projected onto the panoramic images.
Projecting 3D Features
To be able to visualise this 3D dataset through the WFS protocol, a WFS is needed that is capable of outputting the 3D data in a compound spatial reference system. The spatial reference system currently in use in The Netherlands is EPSG:7415, RD 2004 new + RD NAP. The Deegree software package, which is one of the available WFS components, is used to serve the 3D dataset, since it is one of the few software packages that correctly supports the EPSG:7415 (RD 2004 new + RD NAP) compound spatial reference system. The Deegree WFS has a connection to a postgreSQL database containing the 3D dataset.
Each time a panorama is opened, GlobeSpotter requests all 3D features in the local area from the WFS server (Figure below). Because the position and orientation of the panorama are known, the 3D features are projected with relative orientation onto the panorama as an overlay. Any additional information associated with the features, such as the type of object they represent, can be accessed by the user by clicking on the projected representation of each feature in the panoramic image.
Concluding Remarks and Future Developments
In conclusion, enriching 2D feature geometries into 3D and projecting them onto panoramic images clearly improved the geometric augmentation of panoramic images. Particularly at locations with high topographical gradient, providing accurate height information minimises misalignments on a large scale.
Two main challenges were faced and overcome in this project:
1) Ensuring consistency of the horizontal and vertical coordinates among the different datasets
2) Generating 3D Deegree WFS
The project continues to further improve the geometric augmentation of the features by excluding occluded data and performing quality analysis of the projection result. Occlusion means that some features are visible in the projection that should be invisible from the point of view of the panoramic image. Parts of features behind a wall, for example, should either not be shown or should be rendered with thinner lines and a different colour.
Since the quality of the projection is also determined by the quality of positional and vertical consistencies, it is very likely that some shifts in vertical and horizontal directions are observed according to the accuracy of each dataset used. Therefore, the quality analysis is still continuing in order to be able to evaluate which locations are most suitable for the projection.
As soon as the AHN2 dataset is available for the whole country, the applied methodology for this research can be implemented with the use of 2D topographical objects together with the utility datasets used to augment panoramic images.
The authors would like to thank Haico van der Vegt of Dutch Kadaster, and Bart van Velden and Arjen Swart of CycloMedia for their help and valuable comments.Last updated: 27/02/2018