---
url: /2022/03/eoxmaps-the-update-part1.md
---

The update of our [EOxMaps](https://maps.eox.at/) infrastructure has been long overdue. Finally, in
2021 we have completely reworked our former EOxMaps infrastructure, which allows
us to generate, alter and update EOxMaps more frequently than before.

Firstly, we focused on updating the somewhat outdated map label layers.

## Data Foundation - OpenStreetMap & Natural Earth

Like with most OpenData maps today we have opted to use a mix of [OpenStreetMap](https://planet.openstreetmap.org/planet/)
and [Natural Earth](https://www.naturalearthdata.com/) as sources for our
map labels update.

## PostgreSQL + PostGIS Database

The OSM dataset is imported into a PostGIS database via an import tool:
[imposm3](https://github.com/omniscale/imposm3). Here we specify the nessesary
parameters to extract the attributes needed by the map layers from the
OpenStreetMap `.pbf` file.

Given the size of the imported data and the complexity of some spatial features,
we have generalized some layers to make the vector tiles generation more performant.

### Centerlines

For large geographical features like oceans, lakes, mountain ranges, etc.
we use the custom centerlines of these objects for labeling. The EOX has
a custom solution to generating these centerlines as described in the ["Curved labels"](https://eox.at/2015/12/curved-labels/)
article. The output of these centerlines is then imported into the PostGIS
database.

## Custom Tiles-API as requests handler

Since numerous tools are used in this pipeline, we were forced to consolidate
these tools under a single endpoint that handles input and output (I/O) from
the database over the vector tiles to rendering of raster tiles in the final
map layers.

For this data throughput management we have utilized [FastAPI](https://fastapi.tiangolo.com/)
to serve vector-tiles and rendered raster tiles as well as some metadata around the
whole EOxMaps infrastructure.

### FastAPI Introduction

For those unfamiliar with [FastAPI](https://fastapi.tiangolo.com/) here is
a python3 ['Hello World'](https://fastapi.tiangolo.com/tutorial/first-steps/)
on how to set up a simple API instantly.

1. Save a `main.py` python file with:

```python
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello World"}
```

2. Run the app from `bash` terminal via:

` $ uvicorn main:app --reload`

3. Open a browser at localhost: <http://127.0.0.1:8000>.
   Response should be following `json` output: `{"message": "Hello World"}`.

### Vector Tiles (Mapbox-Vector-Tiles)

At EOX we are already familiar with vector tiles ([MVT](https://docs.mapbox.com/vector-tiles/specification/))
as for instance this article from last year shows: [Vector tiles with dynamically updated style in Leaflet LPvis](https://eox.at/2020/01/vector-tiles-with-dynamically-updated-style-in-leaflet-lpvis/).
Having already had some idea on how we should proceed in handling the data from the
PostGIS database, we decided that apart from our agriculturally driven projects
we will use a similar data pipeline here as well.

Vector tiles are extremely handy for anything with high I/O vector data throughput.
As the files are independent from each other, this enables convenient concurrently
driven utilization. There are some PostGIS native functions that create MVTs on
the fly: [ST_AsMVT](https://postgis.net/docs/ST_AsMVT.html) and [ST_AsMVTGeom](https://postgis.net/docs/ST_AsMVTGeom.html).
If you are comfortable with the SQL language, you can customize and add some handy
SQL functions to clip and reproject the geometries and edit attributes of the vector layers.
These operations can increase or decrease the performance of the MVT generation,
so be careful.

Finally, we serve the optimized vector tiles to the renderer via our `Tiles-API`.

## TileServer GL as renderer

To make a smooth transition between our former and current maps infrastructure
we still do render raster (`.png`,`.jpg`) tiles to serve them via our WMS/WMTS services.
For this we have used [TileServer GL](https://github.com/maptiler/tileserver-gl)
written by [Maptiler](https://www.maptiler.com/). This tool is based on the
open version of [Mapbox-GL-Native](https://github.com/mapbox/mapbox-gl-native).

TileServer GL allows for an easy rendering of vector tiles into raster tiles.
It also comes with some handy [configuration options](https://tileserver.readthedocs.io/en/latest/config.html#)
that give somewhat more control over the whole rendering procedure. It also
provides vector tiles (source data), styles and rendered tiles (WMS/WMTS)
[endpoints](https://tileserver.readthedocs.io/en/latest/endpoints.html).
This is particularly handy if you don't want to handle the requests yourself like
we do via the `Tiles-API`.

### Mapbox-GL Styles

The final layer look is defined in a [Mapbox Style File](https://docs.mapbox.com/mapbox-gl-js/style-spec/),
which is a json file specifying the outcome of your map.

This is the most time-consuming part where fiddling with the map details comes
into play. There are numerous ways on how to configure and review your styling;
our primary style creation tool was [maputnik](https://maputnik.github.io/).
Maputnik allows editing [Mapbox Style Files](https://docs.mapbox.com/mapbox-gl-js/style-spec/)
in a nice map editor and exporting the final style so that you can use it in the
TileServer GL endpoints.

Example of a **mapbox-gl-js json style** config:

```json
{
  "version": 8,
  "name": "osm_borders", // name of the style
  "metadata": { // some metadata information, this can be whatever you need
    "name": "osm_database",
    "version": 1.0,
    "timestamp": "2022-03-08",
    "info": "OSM Borders Tutorial",
    "sources": ["openstreetmap"],
    "maputnik:renderer": "mbgljs"
  },
  "sources": {       // sources where to look for vector tiles,
    "osm_borders": { // in this case we use the regular EPSG:3857 MVT grid
      "minZoom": 0,
      "maxZoom": 8,
      "tiles": ["http://{VECTOR_TILES_HOST_ADRESS}/vt/xyz/overlay_mercator/{z}/{x}/{y}.pbf"],
      "type": "vector"
    }
  },
  "glyphs": "http://{FONTS_HOST_ADRESS}/fonts/{fontstack}/{range}.pbf",
  // one layer for this example should be enough
  "layers": [
    {
	  "id": "osm_border_linestrings_countries",
	  "type": "line",
	  // where to look in the sources list above
	  "source": "osm_borders",
	  // name of the layer in the MVT files
	  "source-layer": "osm_border_linestrings_countries",
	  // filter what you need based on attributes in the MVT
	  "filter": ["all"],
	  "layout": {
	      "line-cap": "round",
	      "line-join": "round",
	      "visibility": "visible"
	    },
	  "paint": {
	    "line-color": "rgba(255, 130, 40, 1)",
	    "line-width": {"stops": [[0, 0.5], [8, 1], [11, 2]]}
	  }
    }
  ],
  "id": "12345"
}
```

## Preview and Map Layer Cache

Having everything set up and running, it is nice to see your map layers in action.
For the preview we have taken our own [Openlayers](https://openlayers.org/) extension - [EOxElements](https://eox-a.github.io/elements/examples/map-basic.html)
package and provided it with access to the vector tiles, WMS and WMTS endpoints—all of these served by the custom `Tiles-API`.

### MapServer MapCache

Serving already cached map layers increases the performance significantly. Therefore we are
using [MapCache](https://mapserver.org/mapcache/), which provides a tile based
caching solution and also offers various serving [endpoints](https://mapserver.org/mapcache/services.html).

The caching itself happens in a command called ["mapcache_seed"](https://mapserver.org/mapcache/seed.html),
where a lot of options are offered to tailor the (Map)cache into the final
endpoints and services, which will utilize the cache once it has been generated.

## Overview

As this is the first article about our maps technical details, we hope
that it was an appropriate introduction into the EOxMaps topic on how we utilize
available tools to provide up-to-date solutions for one of our many services.

It might seem that a lot of tools in this scheme are somewhat doubled in the
infrastructure, which is partly true. For the purpose of a flawless transition
from our old EOxMaps solution we have adapted available geospatial tools and
combined them into one big pipeline which is capable of serving map tiles on the fly.

Here is a graph visualising the content of this article.

## What comes next?

Like in my [last article](https://eox.at/2021/03/sentinel-2-cloudless-2020/), there
is more stuff happening at EOX right now. After the EOxMaps labels, we have started
working on even more advanced automatization of our [EOxCloudless](https://cloudless.eox.at/)
data processing.

As for the maps, in a bit more distant future, apart from labels we aim to update
our other [EOxMaps](https://maps.eox.at/) layers (i.e. the "Terrain" and "Terrain-Light" layers) so that they get a fresh
coat of paint as well.

***

Petr Ševčík, EOX IT Services GmbH Email: <petr.sevcik@eox.at>

Data ©[OpenStreetMap](https://www.openstreetmap.org/copyright) contributors and others.

Contains Modified Copernicus Sentinel-2 Data 2020.

Our thanks to [maptiler](https://www.maptiler.com/) for [TileServer GL](https://github.com/maptiler/tileserver-gl/blob/master/LICENSE.md)
