In response to repetitive customer requests seeking spatial datasets based on OpenStreetMap service, CYBERTEC decided to start an initiative to address this demand.

In short, CYBERTEC implemented a download OpenStreetMap service, which periodically generates extracts of OpenStreetMap data in various flavors, outputted as sql dump to streamline and simplify its usage. Extracts cover a particular region of interest and typically comply with boundaries of well-known countries or continents. Flavors pose datasets usage and implicitly state how one dataset has been initially imported into our database.

The main entry point of our service, which is free of charge, is located at https://gis.cybertec-postgresql.com/.

Dataset flavors

Importing OpenStreetMap data into PostGIS can be a hard and time-consuming task. To simplify this process and speed up the import, we decided to kick off a service offering two dataset types served as sql dumps periodically.

The basis for all flavors are pure OSM extracts, provided as pbfs, which are generated first.

The first type refers to the main purpose and usage of OpenStreetMap, which is indeed solving spatial questions by analyzing and visualizing spatial data. There are endless ways to import and map OSM data to PostGIS to serve this purpose. To support most typical use cases and retain flexibility, we decided to implement a generic import following on OpenStreetMap Carto (https://github.com/gravitystorm/openstreetmap-carto). Incidentally, this enables us to use this dump seamlessly as a source for tiling services, such as https://switch2osm.org/serving-tiles/ or https://github.com/Overv/openstreetmap-tile-server.
A brief description of the import chain can be found at https://gis.cybertec-postgresql.com/osmtile/readme.txt.

The second type addresses the demand to solve various kinds of routing-related questions. This time, native datasets are imported utilizing osm2po (https://osm2po.de/), a great java application turning pure OpenStreetMap data into a routable graph. Ideally, the resulting table will be used together with pgRouting (https://pgrouting.org/), another great extension, bringing geospatial routing functionality to PostgreSQL.
Checkout the import chain at https://gis.cybertec-postgresql.com/osmrouting/readme.txt.

Dataset coverage

CYBERTEC offers extracts for the whole world. We’re not inventing the wheel again, thus datasets spatial extents belong to boundaries of countries or continents. To simplify access and download, datasets are structured hierarchically, as figure 1 states.

Dataset hierarchyDataset hierarchy

Dataset frequency

How often will our datasets be generated? Native extracts (pbfs) are generated daily, dumps are produced 1 to 2 times per month. Currently, we’re trying to optimize the whole process to provide current dumps more frequently for our customers.

Dataset usage and import

So how can customers consume our service? Let’s go through a typical customer scenario.

As a customer, we plan to setup up a tiling server offering raster data covering Austria. Let’s assume tiling infrastructure is already up and running, the missing link is a database containing OpenStreetMap data. Instead of utilizing osm2pgsql to import our dataset, we take a nice shortcut and directly download the appropriate sql dump from our service 😊.

The general instructions are as follows:

Firstly, start by heading to https://gis.cybertec-postgresql.com/ and choose your region of interest. Figure 2 shows available datasets for Austria at present time. As we want to setup a tiling service, our dataset flavor of choice is “Analysis, Tiling”. Let’s download the latest dump for this region from https://gis.cybertec-postgresql.com/osmtile/europe/austria/osmtile_europe_austria_latest_compressed.dump

database download
Dataset download, restore instructions

 

Secondly, we must import our dump. How can this be accomplished?
Instructions are published beneath each published dump as readme.txt. For Austria, instructions can be accessed at https://gis.cybertec-postgresql.com/osmtile/europe/austria/readme.txt.

Let’s quickly look at the instructions to better understand how we must proceed. As requirement, a database must be prepared, with extensions PostGIS and hstore enabled.

postgres=# create database tilingdb;
CREATE DATABASE
postgres=# \c tilingdb
psql (13.3 (Ubuntu 13.3-1.pgdg20.04+1), server 13.4 (Ubuntu 13.4-1.pgdg20.04+1))
You are now connected to database "tilingdb" as user "postgres".
tilingdb=# create extension postgis;
CREATE EXTENSION
tilingdb=# create extension hstore;
CREATE EXTENSION

 

Finally, the dump can be restored by executing

pg_restore -j 4 --no-owner -d tilingdb
osmtile_europe_austria_latest_compressed.dump

 

This results in a new database schema osmtile_europe_austria.

tilingdb=# \dn
          List of schemas
                   Name | Owner
------------------------+----------
 osmtile_europe_austria | postgres
 public                 | postgres
(2 rows)

The upcoming listing shows generated tables within our schema.

tilingdb=# \dt+ osmtile_europe_austria.
                                           List of relations
         Schema         |        Name        | Type  |  Owner   | Persistence |  Size   | Description
------------------------+--------------------+-------+----------+-------------+---------+-------------
 osmtile_europe_austria | planet_osm_line    | table | postgres | permanent   | 1195 MB |
 osmtile_europe_austria | planet_osm_nodes   | table | postgres | permanent   | 3088 MB |
 osmtile_europe_austria | planet_osm_point   | table | postgres | permanent   | 323 MB  |
 osmtile_europe_austria | planet_osm_polygon | table | postgres | permanent   | 1865 MB |
 osmtile_europe_austria | planet_osm_rels    | table | postgres | permanent   | 102 MB  |
 osmtile_europe_austria | planet_osm_roads   | table | postgres | permanent   | 130 MB  |
 osmtile_europe_austria | planet_osm_ways    | table | postgres | permanent   | 1996 MB |
(7 rows)

Outlook

This article briefly introduces CYBERTEC’s brand new OpenStreetMap download service. The service has been recently released and we are very curious what customers think about it. Please leave a message at gis@cybertec.at to give us feedback, discuss further dataset flavors, or just in case you need assistance. Check this post if you want to know how to start with PostgreSQL and PostGIS.