Dynamic 3D world rendering using offline OpenStreetMap data

I would like to share some progress on my open source project: UtyMap. This is a library which provides highly customizable API for procedural world generation based on real map data, e.g. OpenStreetMap, NaturalEarth. Core logic is written on C++11 and can be used on many platforms as it has no dependency to specific game engine or application framework. Demo application is built using Unity3D and it provides examples with interactive world creation at different zoom levels, including globe and ground:

Full source code is available here: GitHub - reinterpretcat/utymap: Highly customizable library for procedural world generation based on real map data
Latest release binaries: https://github.com/reinterpretcat/utymap/releases/tag/ver1.1
Released on Google Play: https://play.google.com/store/apps/details?id=com.utymap.demo

Update from July 2016:
Initial post referred to ActionStreetMap project which is now deprecated and not supported, but full source code is available.

3 Likes

It looks really cool, nice work!
Maybe i can help you a bit with the shaders.

Sure, pull requests are welcomed.

Some of upcoming changes inside ASM framework:

  • Reimplemented logic of processing map data: use custom spatial index inside and no need to use splitter tool.
  • Support import of o5m, pbf and xml formats.
  • Reimplemented logic of interpolating height map for elevation mode: it’s more accurate now (bilinear interpolation is used).
  • Added support flat map mode: rendering is faster in ~30% and no need in elevation data.
  • Fixed issues with missing elements crossing tile borders: limitation of old map data processing logic.
  • Added simple search for elements with given tag’s key/value pair: you can find the nearest bars, restaurants, other places, features.
  • Smart road processing: build road graph, detect road junctions in order to simplify further AI logic.
  • Async loading by Rx: dynamically load map tile without blocking UI thread.
  • Auto downloading and saving of current elevation file from NASA server (or custom one)
  • Auto downloading of map data for current tile from openstreetmap server (or custom one)
  • Added reverse geocoding support by querying of Nominatium server (or custom one).
1 Like

Decided to use flat shading as default rendering style:

2 Likes

Added pseudo 2D overview mode (see first post updated)

1 Like

Two questions:
-Why flat shading?
-Are all the object meshes merged into one mesh?

First attempts were with unity terrain and buildings with predefined textures. But result wasn’t nice:

  • Unity terrain introduced some issues with performance. Mostly because heightmap resolution should be high to display terrain regions more accurate. Dynamic creation also introduces some memory traffic and didn’t find a way to fix it.
  • In general, OSM data doesn’t contain information about facade styles. Only few buildings have information about material/color. It’s really difficult to prepare textures for different cities (compare architecture styles of Germany, France, Asian countries…) and apply them dynamically for different areas. I think it may sense only for design time (some products are already doing that). Otherwise user may fell you’re cheating: I had some old demo where I tried to apply random textures to buildings in place where I leave and result wasn’t nice.

Terrain consists of chunks which are single mesh (except water in case 3D tile). Buildings, trees are separate meshes.

Is it possible for me to use OSM and terrain data for my city? If so, how can I use the data?

Yes, it’s possible in general. Furthermore, choosing any location is main feature of project. Please note, it’s not possible so far in current web demo builds due to web sandbox security :frowning:
But you can download demo project, build it manually and check in editor. I created some simple GUI for choosing location by performing reverse geocoding. It will try to resolve your city name to geocoordinate. After you press Start, app will try to download SRTM and OSM data dynamically (and cache it for future usage) for given location.

Another approach, change geocoordinate in code once (see ApplicationManager.cs) to skip these steps for regular runs.

P.S. I prefer to build offline index once for whole region (e.g. Berlin) as it works better so far.

Hello magnificent job I have some suggestions please:

-Make an internal search in the program such as Google Maps or even more specific. As the ArcGIS; Find related object with same attribute (the object name, location, etc.).

-a simple editor of 2D polygons to create new buildings by extruding the polygon in the ground surface.

Please take this into account

Thanks for your attention

Hi, thanks for your feedback! Actually, it has internal search which is working for internal map index quite good and it’s offline which means it doesn’t require internet connection. This feature is available by debug console (type help and you will see list of all commands). Search command has the following syntax:

Usage: search [/h|/H]
search /q:tag_key=tag_value [/f:element_type] [/r:radius_in_meters]
search /q:any_text [/f:element_type] /r:radius_in_meter → Just realized this syntax is broken so far

For example, to get the list of bars in 500 meters you can do the following:

search /q:amenity=bar /r:500

So far, you have to know OSM tags meaning in order to use it (not a problem to extend it with some custom aliases to avoid this). This command is used internally for current address resolution
Another potentially useful command is geocode - it performs reverse geocoding by requesting osm nominatim server.

The problem that I have no time to build nice GUI to show all potential features.

Yes, I was thinking to implement something like that. So far, buildings and terrain are different meshes and you can modify their shapes by clicking. Internally, ASM build index for these objects to locate affected vertices. Then you can control what to do with them on application level.

Thanks for your feedback!

1 Like

Next demo version will contain simple editor tool which allows you to add/remove buildings, barriers, trees, etc and adjust terrain height.

1 Like

I’m able to change geocoordinate for my city in ApplicationManager.cs file and “walked” around my city. I tried to add a car to the scene so that I can “drive” around my city but only the first part of the city was generated. The other parts of the city was not generated. How do I fix this problem? I think that it is possible to replace the faceted skybox with regular one but I’m not sure if this will work.

Is there any log message in editor?. Possible reasons of issue:

  • Issue in ASM related to rendering (most likely)
  • No data for given region
    Could you provide geocoordinate of that place? Then I can investigate what is wrong here.

Actually, the project is still under development and I’m testing it using only few places. So, arbitrary place may contain some data which is not handled properly and fwk may fail there. Unfortunately, OSM has some ambiguity in data format and that introduces additional level of complexity in implementation and testing.

My focus now is not rendering and fixing processing of data ambiguity, but new features which may affect internal architecture. That’s why building is still not vertically placed in correct way and mapcss has no styles for some objects - it’s quite easy to fix in comparison to implementing of editor and multiplayer features (of course, performance should be always measured and taken into consideration).

I used geocoordinate of 40.603621,-74.072954 for my city. When I “walked” around the city using the construction worker, all areas of the city are rendered. I tried to add the car from Unity 5 to the scene and added a camera to the car but only the first area (tile?) is rendered. I think I have to add scripts to the car but which scripts? You used a white material for the ground and buildings. I changed the materials to green for ground and red for buildings but the roads are dark green and I would like them to be dark gray.

If I understand correctly, it works for default character (construction worker), but don’t work for car. In this case, the reason that you need attach Assets/Scripts/Character/ActionStreetMapBehaviour.cs script to your newly created character (car) and remove old one (worker). ASM listens for position changes of character and loads/reloads tiles based on current position.

Actually, you needn’t change any material to do that. I’m using vertex colors and few shaders for all objects (possible to have only one material). What and how to render are defined in declarative way in single mapcss file:

/Assets/Resources/Config/default.mapcss

For example, if you want to change default properties of default building then you should take a look at this entry from default.mapcss:
area|z19[building] { builder: building; height:12; min_height: 0; levels: 5;
fill-color:gradient(#c0c0c0, #a9a9a9 50%, #808080); facade-builder:empty; material:Materials/Buildings/Building;
roof-color:gradient(#808080, #606060 50%, #505050); roof-builder:flat; roof-material:Materials/Buildings/Building; }

It is similar to OSM’s mapcss with my custom declarations. Also it allows to change visual representation in runtime without compilation by having different mapcss files (e.g. summer and winter “themes”).

Is it possible for you to create the “game” using Unreal Engine 4?

This question is too generic (I think I can build simple game using UE4 just by following tutorials for beginners). Did you mean “can ASM to be ported on Unreal Engine4”? I just was asked about this on github. Short answer: no, it’s not possible at the moment. You can follow discussion here

For that location I noticed one of the known issues: ASM misses some objects in 3d tiles, but has them in 2d preview tiles. This is a bug of online mode in which ASM is fetching data from openstreetmap.org directly. I have to fix it in future versions.

Actually, the best way to avoid this is to use offline mode in which map of your region is downloaded and converted from osm format to internal spatial index (but ASM will still be able to fetch data from osm server if you will move outside that region). Offline index has much better performance.

At them moment, ASM provides MapIndexUtility class (don’t like *Utility suffix, so name of the class may change in future) to build spatial index. However, nothing is documented yet…