Open Source GIS Data Processing and Cartography Tools

header_auckland_lidar


This list is designed as a basic overview of tools I use for open source geospatial data processing and visualization. This list is not definitive. Instead, it is a collection of tools I use at various times to complete static or scalable cartographic projects as of mid-2018. This particular post is not a ‘how to’. It is just the tools with a brief description. Other posts on this site describe in detail how they are implemented.

Eight percent of my work is processing data leading to a visual product. Although unseen, it is this data work that remains the bulk of my cartographic projects and the portion I can truly say remains open source.

Some of my visual cartographic work does not involve open source tools. One disclaimer I feel is necessary to confess to the open source world is this: I use open source tools for geospatial processing; however, I rely heavily on Adobe CS for post processing static visual work. There are open source tools replacing InDesign, Illustrator, and Photoshop; however, in my professional experience, very few programs compare to the image processing capabilities of Adobe CS. Also, I must confess occasionally look to ESRI for interpolation and visualizing LiDAR point clouds.

Open Source Tools in Geospatial Data Processing

Ubuntu
I work with Linux (Ubuntu) and Windows. I use Ubuntu for all my GIS data processing and base cartographic work. I run Adobe CS off a Windows machine for post processing static published work.  Linux has a number of flavors and all provide the ease and control of terminal processing. Ubuntu, one flavor of Linux, has several GIS packages, like GDAL as part of the standard install.

QGIS
QGIS is a GIS application/editor. It is my go to for testing data and static visualization work.  QGIS is always evolving. Every year it gets better.  I rely on QGIS for the construction of base images. Basically, the base map visuals that go into more complex cartographic layouts and posters. Layouts, legends, supporting visuals, and labeling are most often handled using Adobe.

GDAL
GDAL is a geospatial command line raster processing tool. It is fast, reliable, and covers almost all the basic needs of geospatial raster processing. Reprojection, transformation, interpolation, masking, calculations, and a number of things I am completely unaware of. GDAL commands are easy to add to scripts using BASH or Python and are a breeze for batch processing. Many QGIS raster tools are built using GDAL in the backend.

Along with GDAL, users get OGR2OGR for their vector processing and conversions.

PostgreSQL/PostGIS
PostgreSQL is an open source database manager. PostGIS is the geospatial extension added to this database. Processing geospatial data inside a database is clean and efficient. PostgreSQL offers all the power of scripting and allows for complex operations and analysis on data. PostgreSQL integrates well with QGIS and tables are easily visualized using the QGIS DBManager. I use PostgreSQL/PostGIS for as much vector processing as I can and integrate it into my BASH scripts.

See also psql2shp and shp2psql for moving shapefiles between data bases. (http://www.bostongis.com/pgsql2shp_shp2pgsql_quickguide.bqg)

BASH
Ubuntus’ command processor tool, BASH offers a selection of programming tools like: for loops, while loops, variables, arrays , and lists to wrap around command line operations. BASH does not get too complex, but will go a long way. Anything that can be executed in the terminal can be built into a BASH script. Need to locate five hundred rasters of the same resolution, move and rename them? BASH can do that. Need to set up a series of GDAL processes in succession that might take three days to complete? Run them through BASH. Need to create a geospatial plugin for QGIS? BASH will NOT do that. See Python for all your plug in construction needs.

MapProxy
MapProxy is an open source tool for creating raster tile caches and serving scalable maps. For me, its most useful advantage is in its ability to create tile caches in any projection. MapProxy also acts as a map server. It will render tiles on the fly or serve tile caches as needed. This unassuming tool is easy to use and allows for a lot of customization in outputs. It also creates WMTS GetCapabilites documents and serves WMTS for testing.

CartoCSS
CartoCSS is a syntax language used to style geospatial data into dynamic, scalable maps. CartoCSS allows for ease of styling across multiple zoom levels. CartoCSS is a good language to incorporate raster blending and gradients.

Yes, I still prefer raster tiling and use Tilemill to design scalable maps. I like raster tiling because it still allows me to use raster data. Half the data used in GIS. Plus very little compares to the visual quality. Please do not leave me comments extolling the benefits of vector tiling. I do know these benefits, especially in labeling, and will discuss them later.

As part of CartoCSS, I should also note the use of Mapnik XMLs to format CartoCSS for MapProxy. Although a small step, knowing how to manipulate these documents goes a long way in developing custom projections.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s