INTRODUCTION TO POSTGIS

Prepared By: Paul Ramsey

Refractions Research Inc.
400 - 1207 Douglas Street
Victoria, BC, V8W-2E7
pramsey@refractions.net
Phone: (250) 383-3022
Fax: (250) 383-2140
TABLE OF CONTENTS

1 SUMMARY ...............................................................................................................................3
1.1 REQUIREMENTS...................................................................................................................3
1.2 CONVENTIONS.....................................................................................................................3
1.3 DOWNLOADS .......................................................................................................................3
2 INSTALLATION......................................................................................................................4

3 SETUP .....................................................................................................................................10

4 USING POSTGIS ...................................................................................................................13

4.1 SIMPLE SPATIAL SQL .......................................................................................................13
4.2 LOADING SHAPE FILES......................................................................................................14
4.3 CREATING SPATIAL INDEXES ............................................................................................16
4.4 USING SPATIAL INDEXES ..................................................................................................17
4.5 INDEXES AND QUERY PLANS ............................................................................................18
4.6 POSTGRESQL OPTIMIZATION ...........................................................................................20
4.7 SPATIAL ANALYSIS IN SQL ..............................................................................................21
4.8 DATA INTEGRITY ..............................................................................................................22
4.9 DISTANCE QUERIES...........................................................................................................22
4.10 SPATIAL JOINS ..................................................................................................................23
4.11 OVERLAYS ........................................................................................................................24
4.12 COORDINATE PROJECTION ................................................................................................25
5 EXERCISES............................................................................................................................26
5.1 BASIC EXERCISES..............................................................................................................26
5.2 ADVANCED EXERCISES .....................................................................................................27
6 MAPSERVER & POSTGIS ..................................................................................................29
6.1 BASIC MAPSERVER CONFIGURATION ...............................................................................29
6.2 MAPSERVER FILTERS AND EXPRESSIONS .........................................................................30
6.3 MAPSERVER WITH SQL ....................................................................................................32

-2-
1 SUMMARY
PostGIS is a spatial database add-on for the PostgreSQL relational database
server. It includes support for all of the functions and objects defined in the
OpenGIS “Simple Features for SQL” specification. Using the many spatial
functions in PostGIS, it is possible to do advanced spatial processing and
querying entirely at the SQL command-line.
This workshop will cover
• installation and setup of PostgreSQL,
• installation of the PostGIS extension,
• loading of sample data,
• indexing,
• performance tuning,
• spatial SQL basics,
• mapserver configuration,
• advanced mapserver configuration, and
• spatial SQL best practices.

1.1 Requirements
This workshop will use the new Windows native PostgreSQL, and a data package
of shape files, which are both included in the workshop CDROM.

1.2 Conventions
Interactive sessions will be presented in this document in grey boxes, with the
text to be entered in boldface, and the results in normal text. In general,
directions to be performed will be presented in boldface.

1.3 Downloads
The PostgreSQL source code is available from http://www.postgresql.org/.
The PostGIS source code is available from http://postgis.refractions.net/.
The GEOS source code is available from http://geos.refractions.net/.
The Proj4 source code is available from http://proj.maptools.org/.
The PgAdmin administration tool is available from http://www.pgadmin.org.

-3-
2 INSTALLATION
The PostgreSQL / PostGIS installation package is in the CDROM at
Software\Windows\PostgreSQL\postgresql-8.0.1.zip.
• Double-click the postgresql-8.0.msi file.
• The installer will start up. Choose your language and follow the
installation instructions.

-4-
• When the “Installation options” selector comes up, do not enable the
PostGIS extension.

• You will have to select an account the run PostgreSQL under. The default
“postgres” user name is standard.

-5-
• If you plan on accessing PostgreSQL from any machine other than the
machine you are installing it on, you have to check “Accept connections
on all addresses” box. The default locale and encoding are good for
English.

• Even if you accept connections in principle, you will have to specifically

indicate which hosts or networks you will accept connections from by
editing a configuration file. This is covered later in the workshop.

-6-
• The PL/PgSQL language is required for PostGIS, so ensure it is selected.
PL/Perl is handy for Perl programmers who want to write triggers and
functions in Perl, but not required by PostGIS.

• The B-Tree GIST, Fuzzy String Match, and Tsearch2 modules might come
in handy if you work with PostgreSQL enough to become a power user.

-7-
• That is it for decisions! Complete your installation of PostgreSQL
• Now it is time to install PostGIS.
• Double-click the postgresql-8.0.msi file.

• De-select the “Create Database” option.

-8-
• Accept the default install location.

• Change the database destination to “bc” and set the postgres user
password to “osg05”.

• Complete the installation, all done!

-9-
3 SETUP
The installer will set up a PostgreSQL menu in your start menu.
• Go to the PostgreSQL menu and run PgAdmin III.
• Double click on the “PostgreSQL Database Server” tree entry. You will be
prompted for the super user password to connect to the “template1”
database.

• Navigate to the “Databases” section of the database tree and open “Edit

New Object
New Database”. Add a new database named “bc”, with
“postgres” as the owner “template1” as the template and “pg_default” as
the tablespace.

- 10 -
• Open up the new “bc” database tree and navigate to the languages. Check
that PL/PgSQL is already installed. This language is needed for PostGIS.

• Now we are ready to install PostGIS. Open up the SQL window by clicking
the SQL button (the one with the pencil).

• Choose “File
Open…” and navigate to

C:\Program Files\PostgreSQL\8.0\contrib\share\lwpostgis.sql
• Press the “Run” button. (The green triangle.) The lwpostgis.sql file will
execute, loading the PostGIS functions and objects into the “bc” database.
• Choose “File
Open…” and navigate to
C:\Program Files\PostgreSQL\8.0\contrib\share\spatial_ref_sys.sql

- 11 -
 • Press the “Run” button again. The spatial_ref_sys.sql file will
execute, loading the EPSG coordinate reference systems into the PostGIS
spatial reference table.
The “bc” database is now set up and ready for data to be loaded. Things we
should remember for other applications:
Database Name: bc
User Name: postgres

Administration Note: We have created our database as the “postgres” super-

user. In a real multi-user system, you will probably have different users with
different access privileges to various tables and functions. Setting up users and
access permissions can be a complicated DBA exercise, so for workshop
purposes we are using the all-powerful super-user.

Unix Note: If you install PostgreSQL and PostGIS on a Unix server, you can still
use PgAdmin to administer your database. However, when loading the
postgis.sql file, you must use the postgis.sql file from your Unix distribution, the
one created during the build and install of PostGIS. This is because the
postgis.sql file includes references to machine library files. The Windows version
of the file will not make sense to a Unix machine.

- 12 -
4 USING POSTGIS
Connect to the database using the psql SQL command line.
• From the Start Menu, go to the PostgreSQL menu, and select “psql to
template1”.
• When prompted for a password, enter your postgres database user password.
• When you have connected to template1, use the \c bc command to connect
to the “bc” database.

4.1 Simple Spatial SQL

Now we will test creating a table with a geometry column, adding some spatial
objects to the table, and running a spatial function against the table contents.

postgis=# create table points ( pt geometry, name varchar );

CREATE TABLE
postgis=# insert into points values ( 'POINT(0 0)', 'Origin' );
INSERT 19269 1
postgis=# insert into points values ( 'POINT(5 0)', 'X Axis' );
INSERT 19270 1
postgis=# insert into points values ( 'POINT(0 5)', 'Y Axis' );
INSERT 19271 1
postgis=# select name, AsText(pt), Distance(pt, 'POINT(5 5)')
from points;

name | astext | distance

--------+------------+------------------
Origin | POINT(0 0) | 7.07106781186548
X Axis | POINT(5 0) | 5
Y Axis | POINT(0 5) | 5
(3 rows)

postgis=# drop table points;

DROP TABLE

Note that there are two spatial database functions being used in the example
above: Distance() and AsText(). Both functions expect geometry objects as
arguments. The Distance() function calculates the minimum Cartesian distance
between two spatial objects. The AsText() function turns geometry into a simple
textual representation, called “Well-Known Text”.

- 13 -
4.1.1 Examples of Well-Known Text
POINT(1 1)
MULTIPOINT(1 1, 3 4, -1 3)
LINESTRING(1 1, 2 2, 3 4)
POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))
MULTIPOLYGON((0 0, 0 1, 1 1, 1 0, 0 0), (5 5, 5 6, 6 6, 6 5, 5 5))
MULTILINESTRING((1 1, 2 2, 3 4),(2 2, 3 3, 4 5))

4.1.2 Canonical Form

What happens when we select geometry columns without using the AsText()
function?
postgis=# select name, pt from points;

name | pt
--------+--------------------------------------------
Origin | 010100000000000000000000000000000000000000
X Axis | 010100000000000000000014400000000000000000
Y Axis | 010100000000000000000000000000000000001440

The “funny looking” information in the “pt” column is the “canonical form” of the
geometry data. It is an enhanced hex encoded version of the “well known binary”
format. PostGIS uses hex encoded binary so that database dumps and restores
(which convert the data to ASCII and back again) do not cause coordinate drift in
the geometry.

4.2 Loading Shape Files

Now we will load our example data (C:\ms4w\apps\postgis\data) into the
database.
The sample data is in Shape files, so we will need to convert it into a loadable
format using the shp2pgsql tool and then load it into the database.
• Run the pg_setenv.bat file first, to add the PostgreSQL install data to your
path, so you can easily use the shp2pgsql tool.
Our data is in projected coordinates, the projection is “BC Albers” and is stored
in the SPATIAL_REF_SYS table as SRID 3005. When we create the loadable
format, we will specify the SRID on the command line, so that the data is
correctly referenced to a coordinate system. This will be important later when we
try the coordinate re-projection functionality.

- 14 -
We may either create the load format as a file, then load the file with psql, or
pipe the results of shp2pgsql directly into the psql terminal monitor. We will
use the first option for the bc_pubs data, and then bulk load the remaining data.
C:\> cd \ms4w\apps\postgis\data

C:\workshop\data> dir *.shp

Directory of C\ms4w\apps\postgis\data>

16/02/1998 03:27 PM 1,196,124 bc_2m_border.shp

16/02/1998 03:27 PM 1,523,396 bc_2m_lakes.shp
16/02/1998 03:27 PM 1,317,004 bc_2m_rivers.shp
16/02/1998 03:27 PM 2,137,928 bc_2m_rivwide.shp
01/02/2005 04:06 PM 5,768,300 bc_elections_1996.shp
01/02/2005 04:06 PM 5,918,336 bc_elections_2000.shp
04/06/2004 01:50 PM 1,332 bc_hospitals.shp
04/06/2004 01:50 PM 1,332 bc_voting_areas.shp
02/06/2004 12:05 PM 278,184 bc_municipality.shp
06/02/2005 12:18 PM 18,540,700 bc_parks_2001.shp
02/06/2004 01:22 PM 10,576 bc_pubs.shp
20/07/2001 11:03 AM 19,687,612 bc_roads.shp
11 File(s) 56,379,492 bytes

C:\…\data> pg_setenv.bat
C:\…\data> shp2pgsql -s 3005 bc_pubs.shp bc_pubs > bc_pubs.sql
C:\…\data> write bc_pubs.sql
C:\…\data> pg_shpsql.bat
C:\…\data> psql –U postgres –f bc_data.sql
Password:

BEGIN
INSERT 215525 1
. . . . . .
COMMIT

- 15 -
4.3 Creating Spatial Indexes
Indexes are extremely important for large spatial tables, because they allow
queries to quickly retrieve the records they need. Since PostGIS is frequently
used for large data sets, learning how to build and (more importantly) how to use
indexes is key.
PostGIS indexes are R-Tree indexes, implemented on top of the general GiST
(Generalized Search Tree) indexing schema. R-Trees organize spatial data into
nesting rectangles for fast searching. (For the seminal paper, see
http://postgis.refractions.net/rtree.pdf)
C:\…\PostgreSQL\8.0\bin> psql postgis

postgis=# create index bc_roads_gidx on bc_roads using gist

( the_geom gist_geometry_ops );
CREATE INDEX

postgis=# create index bc_pubs_gidx on bc_pubs using gist

( the_geom gist_geometry_ops );
CREATE INDEX

postgis=# create index bc_voting_areas_gidx on bc_voting_areas using

gist ( the_geom gist_geometry_ops );
CREATE INDEX

postgis=# create index bc_municipality_gidx on bc_municipality using

gist ( the_geom gist_geometry_ops );
CREATE INDEX

postgis=# create index bc_hospitals_gidx on bc_hospitals using gist

( the_geom gist_geometry_ops );
CREATE INDEX

postgis=# \d bc_roads

Run a table describe on bc_roads (\d bc_roads) and note the index summary at
the bottom of the description. There should be two indexes, the primary key
index and the new “gist” index you just created.
Now, clean up your database and update the index selectivity statistics (we will
explain these in more detail in a couple sections).
postgis=# vacuum analyze;

- 16 -
4.4 Using Spatial Indexes
It is important to remember that spatial indexes are not used automatically for
every spatial comparison or operator. In fact, because of the “rectangular”
nature of the R-Tree index, spatial indexes are only good for bounding box
comparisons.
This is why all spatial databases implement a “two phase” form of spatial
processing.
• The first phase is the indexed bounding box search, which runs on the
whole table.
• The second phase is the accurate spatial processing test, which runs on
just the subset returned by the first phase.
In PostGIS, the first phase indexed search is activated by using the “&&”
operator. “&&” is a symbol with a particular meaning. Just as the symbol “=”
means “equals”, the symbol “&&” means “bounding boxes overlap”. After a little
while, using the “&&” operator will become second nature.
Let’s compare the performance of a query that uses a two-phase index strategy
and a query that does not.
First, time the non-indexed query (this looks for the roads that cross a supplied
linestring, the example linestring is constructed so that only one road is
returned):
postgis=# select gid, name from bc_roads where crosses(the_geom,
GeomFromText('LINESTRING(1220446 477473,1220417 477559)', 3005));
gid | name
-------+--------------
64555 | Kitchener St
(1 row)

Now, time the two-phase strategy, that includes an index search as well as the
crossing test:
postgis=# select gid, name from bc_roads where the_geom &&
GeomFromText('LINESTRING(1220446 477473,1220417 477559)', 3005) and
crosses(the_geom, GeomFromText('LINESTRING(1220446 477473,1220417
477559)', 3005));
gid | name
-------+--------------
64555 | Kitchener St
(1 row)

You will have to be pretty fast with your stopwatch to time the second query.

- 17 -
4.5 Indexes and Query Plans
Databases are fancy engines for speeding up random access to large chunks of
data. Large chunks of data have to be stored on disk, and disk access is
(relatively speaking) very, very slow. At the core of databases are algorithms
tuned to search as much data as possible with as few disk accesses as possible.
Query plans are the rules used by databases to convert a piece of SQL into a
strategy for reading the data. In PostgreSQL, you can see the estimated query
plan for any SQL query by pre-pending “EXPLAIN” before the query. You can see
the actual observed performance by pre-pending “EXPLAIN ANALYZE” before the
query.

Hit the Up Arrow to recall a previous query.

Hit Ctrl-A to move the cursor to the front of the query.
Add the EXPLAIN clause and hit enter.

postgis=# explain select gid, name from bc_roads where

crosses(the_geom, GeomFromText('LINESTRING(1220446 477473,1220417
477559)', 3005));
QUERY PLAN
---------------------------------------------------------------------
Seq Scan on bc_roads (cost=0.00..7741.71 rows=54393 width=17)
Filter: crosses(the_geom, 'SRID=3005;LINESTRING(1220446
77473,1220417 477559)'::geometry)
(2 rows)

No “&&” operator, so no index scan. The query has to test every row in the table.

postgis=# explain select gid, name from bc_roads where the_geom &&
GeomFromText('LINESTRING(1220446 477473,1220417 477559)', 3005) and
crosses(the_geom, GeomFromText('LINESTRING(1220446 477473,1220417
477559)', 3005));
QUERY PLAN
--------------------------------------------------------------------
Index Scan using bc_roads_gidx on bc_roads (cost=0.00..6.02 rows=1
width=17)
Index Cond: (the_geom && 'SRID=3005;LINESTRING(1220446
477473,1220417 477559)'::geometry)
Filter: crosses(the_geom, 'SRID=3005;LINESTRING(1220446
477473,1220417 477559)'::geometry)
(3 rows)

With the index scan, the query only has to text a few rows using the crosses
filter.

- 18 -
4.5.1 When Query Plans Go Bad
A database can only use one index at a time to retrieve data.
So, if you are making a query using a filter that specifies two indexed columns,
the database will attempt to pick the index with the “greatest selectivity”. That
is, the index that returns the fewest rows. It does this by using statistics
gathered from the table and indexes about the makeup of the data.
As of PostgreSQL version 8.0, the spatial selectivity analysis for PostGIS indexes
is integrated into the main selectivity system of PostgreSQL. So, to ensure that
your statistics are kept up to date as you change your data, you just have to run
the “ANALYZE” command occasionally. You must compile PostGIS with
USE_STATS enabled to have this functionality (this is the default).
With selectivity turned off, PostGIS is tuned to force the query planner to always
use the spatial index. This can be a terrible strategy, if the index filter is very
large – if, for example, your query is “give me all the streets named ‘Bob’ in the
south half of the province”. Using the index on street name will be much more
selective (assuming there are not many streets named Bob in the province) than
the spatial index (there are hundreds of thousands of streets in the south half of
the province).
If the spatial index is (incorrectly) chosen, the database will have to sequence
scan every road from the south of the province to see if it is named ‘Bob’, instead
of sequence scanning every road named ‘Bob’ to see if it is in the south half of
the province.
So, it is important to use selectivity, and ensure that selectivity statistics are
generated using the “ANALYZE” command regularly on your database.

- 19 -
4.6 PostgreSQL Optimization
PostgreSQL is shipped by the development team with a conservative default
configuration. The intent is to ensure that PostgreSQL runs without modification
on as many common machines as possible. That means if your machine is
uncommonly good (with a large amount of memory, for example) the default
configuration is probably too conservative.
True database optimization, particularly of databases under transactional load,
is a complex business, and there is no one size fits all solution. However, simply
boosting a few PostgreSQL memory use parameters can increase performance.
The database configuration file is in the database’s \data area, and is named
postgresql.conf.
Open postgresql.conf in any text editor. The following lines offer quick boosts
if you have the hardware:
shared_buffers = 1000 # min 16, 8KB each

Increase the shared buffers as much as possible, but not so much that you
exceed the amount of physical memory available for you on the computer. After
about a few hundred MB, there are diminishing returns. Since each buffer is
8Kb, you will have to do a little math to calculate your ideal buffer size.
For example, if you want 120MB of shared buffers, you will want to set a value of
120 * 1024 / 8 = 15360.
Shared buffers are (surprise) shared between all PostgreSQL back-ends, so you
do not have to worry about how many back-ends you spawn.
work_mem = 1024 # min 64, size in KB
maintenance_work_mem = 16384 # min 1024, size in KB

Working memory is used for sorting and grouping. It is used per-backend, so

you have to guesstimate how many back-ends you will have running at a time.
The default value is only 1MB.
Maintenance memory is used for vacuuming the database. It is also used per-
backend, but you are unlikely to have more than one backend vacuuming at a
time (probably). The default value is 16MB, which might be enough, depending
on how often you vacuum.
NOTE: Some optimizations, like shared_buffers have to be synchronized with
operating system kernel tuning. For example, the default number of
shared buffers allowed by Linux is fairly low, and that number needs to
be increased before a tuned-up PostgreSQL will be able to run. See
http://www.postgresql.org/docs/8.0/static/kernel-resources.html.

- 20 -
4.7 Spatial Analysis in SQL
A surprising number of traditional GIS analysis questions can be answered using
a spatial database and SQL. GIS analysis is generally about filtering spatial
objects with conditions, and summarizing the results – and that is exactly what
databases are very good at. GIS analysis also tests interactions between spatially
similar features, and uses the interactions to answer questions – with spatial
indexes and spatial functions, databases can do that too!

4.7.1 Exercises
These exercises will be easier if you first peruse the data dictionary and functions
list for information about the data columns and available functions. You can
enter these exercises in order, or for a challenge, cover up your page with
another piece of paper and try to figure out the answer yourself before looking at
the SQL.
“What is the total length of all roads in the province, in kilometers?”
postgis=# select sum(length(the_geom))/1000 as km_roads from bc_roads;
km_roads
------------------
70842.1243039643
(1 row)

”How large is the city of Prince George, in hectares?”

postgis=# select area(the_geom)/10000 as hectares from bc_municipality
where name = 'PRINCE GEORGE';
hectares
------------------
32657.9103824927
(1 row)

”What is the largest municipality in the province, by area?”

postgis=# select name, area(the_geom)/10000 as hectares from
bc_municipality order by hectares desc limit 1;
name | hectares
---------------+-----------------
TUMBLER RIDGE | 155020.02556131
(1 row)

The last one is particularly tricky. There are several ways to do it, including a
two step process that finds the maximum area, then finds the municipality that
has that area. The suggested way uses the PostgreSQL “LIMIT” statement and a
reverse ordering to pull just the top area.

- 21 -
4.8 Data Integrity
PostGIS spatial functions require that input geometries obey the “Simple
Features for SQL” specification for geometry construction. That means
LINESRINGS cannot self-intersect, POLYGONS cannot have their holes outside
their boundaries, and so on. Some of the specifications are quite strict, and
some input geometries may not conform to them.
The isvalid() function is used to test that geometries conform to the specification.
This query counts the number of invalid geometries in the voting areas table:
postgis=# select count(*) from bc_voting_areas where not
isvalid(the_geom);
count
------
0
(1 row)

4.9 Distance Queries

It is easy to write inefficient queries for distances, because it is not always
obvious how to use the index in a distance query. Here is a short example of a
distance query using an index.
“How many BC Unity Party supporters live within 2 kilometers of the Tabor Arms
pub in Prince George?”
First, we find out where the Tabor Arms is:
postgis=# select astext(the_geom) from bc_pubs where name ilike 'Tabor
Arms%';
astext
-----------------------------------------
POINT(1209385.41168654 996204.96991804)
(1 row)

Now, we use that location to pull all the voting areas within 2 kilometers and
sum up the Unity Party votes:
postgis=# select sum(upbc) as unity_voters from bc_voting_areas where
the_geom && setsrid(expand('POINT(1209385 996204)'::geometry, 2000),
3005) and distance(the_geom, geomfromtext('POINT(1209385 996204)',
3005)) < 2000;
unity_voters
--------------
421
(1 row)

- 22 -
4.10 Spatial Joins
A standard table join puts two tables together into one output result based on a
common key. A spatial join puts two tables together into out output result based
on a spatial relationship.
We will find the safest pubs in British Columbia. Presumably, if we are close to a
hospital, things can only get so bad.
“Find all pubs located within 250 meters of a hospital.”
For clarity, we will do this query without an index clause, since the tables are so
small.
postgis=# select h.name, p.name from bc_hospitals h,
bc_pubs p where distance(h.the_geom, p.the_geom) < 250;

Just as with a standard table join, values from each input table are associated
and returned side-by-side.
Using indexes, spatial joins can be used to do very large scale data merging
tasks.
For example, the results of the 2000 election are usually summarized by riding –
that is how members are elected to the legislature, after all. But what if we want
the results summarized by municipality, instead?
“Summarize the 2000 provincial election results by municipality.”
postgis=# select m.name, sum(v.ndp) as ndp, sum(v.lib) as liberal,
sum(v.gp) as green, sum(v.upbc) as unity, sum(v.vtotal) as total from
bc_voting_areas v, bc_municipality m where v.the_geom && m.the_geom
and intersects(v.the_geom, m.the_geom) group by m.name order by
m.name;

name | ndp | liberal | green | unity | total

---------------------------+-------+---------+-------+-------+-------
100 MILE HOUSE | 398 | 959 | 0 | 70 | 1527
ABBOTSFORD | 1507 | 9547 | 27 | 575 | 12726
ALERT BAY | 366 | 77 | 26 | 0 | 500
ANMORE | 247 | 1299 | 0 | 0 | 1644
ARMSTRONG | 433 | 1231 | 199 | 406 | 2389
ASHCROFT | 217 | 570 | 0 | 39 | 925
BELCARRA | 93 | 426 | 37 | 0 | 588
BURNABY | 15500 | 31615 | 8276 | 1218 | 58316
BURNS LAKE | 370 | 919 | 104 | 92 | 1589
CACHE CREEK | 76 | 323 | 0 | 27 | 489
CAMPBELL RIVER | 2814 | 6904 | 1064 | 0 | 11191
. . . . . . .

- 23 -
4.11 Overlays
Overlays are a standard GIS technique for analyzing the relationship between
two layers. Particularly where attributes are to be scaled by area of interaction
between the layers, overlays are an important tool.
In SQL terms, an overlay is just a spatial join with an intersection operation. For
each polygon in table A, find all polygons in table B that interact with it.
Intersect A with all potential B’s and copy the resultants into a new table, with
the attributes of both A and B, and the original areas of both A and B. From
there, attributes can be scaled appropriately, summarized, etc. Using sub-
selects, temporary tables are not even needed – the entire overlay-and-
summarize operation can be embedded in one SQL statement.
Here is a small example overlay that creates a new table of voting areas clipped
by the Prince George municipal boundary:
postgis=# create table pg_voting_areas as select
intersection(v.the_geom, m.the_geom) as intersection_geom,
area(v.the_geom) as va_area, v.*, m.name from bc_voting_areas v,
bc_municipality m where v.the_geom && m.the_geom and
intersects(v.the_geom, m.the_geom) and m.name = 'PRINCE GEORGE';
SELECT

postgis=# select sum(area(intersection_geom)) from pg_voting_areas;

sum
------------------
326579103.824927
(1 row)

postgis=# select area(the_geom) from bc_municipality where name =

'PRINCE GEORGE';
area
------------------
326579103.824927
(1 row)

Note that the area of the sum of the resultants equals the area of the clipping
feature, always a good sign.

- 24 -
4.12 Coordinate Projection
PostGIS supports coordinate reprojection inside the database.
Every geometry in PostGIS has a “spatial referencing identifier” or “SRID”
attached to it. The SRID indicates the spatial reference system the geometry
coordinates are in. So, for example, all the geometries in our examples so far
have been in the British Columbia Albers reference system.
You can view the SRID of geometries using the srid() function:
postgis=# select srid(the_geom) from bc_roads limit 1;

The SRID of 3005 corresponds to a particular entry in the SPATIAL_REF_SYS

table. Because PostGIS uses the PROJ4 library or reprojection support, the
SPATIAL_REF_SYS table includes a PROJ4TEXT column that gives the
coordinate system definition in PROJ4 parameters:
postgis=# select proj4text from spatial_ref_sys where srid=3005;
proj4text
--------------------------------------------------------------------
+proj=aea +ellps=GRS80 +datum=NAD83 +lat_0=45.0 +lon_0=-126.0
+lat_1=58.5 +lat_2=50.0 +x_0=1000000 +y_0=0
(1 row)

Coordinate reprojection is done using the transform() function, referencing an

SRID that exists in the SPATIAL_REF_SYS table. For example, in the panel
below we will view a geometry in the stored coordinates, then reproject it to
geographic coordinates using the transform() function:
postgis=# select astext(the_geom) from bc_roads limit 1;
astext
----------------------------------------------------------
MULTILINESTRING((1004687.04355194 594291.053764096,1004729.74799931
594258.821943696,1004808.0184134 594223.285878035,1004864.93630072
594204.422638658,1004900.50302171 594200.005856311))
(1 row)

postgis=# select astext(transform(the_geom,4326)) from bc_roads limit

1;
astext
--------------------------------------------------------------------
MULTILINESTRING((-125.9341 50.3640700000001,-125.9335 50.36378,
-125.9324 50.36346,-125.9316 50.36329,-125.9311 50.36325))
(1 row)

Note the longitude/latitude coordinates in the transformed geometry.

- 25 -
5 EXERCISES

5.1 Basic Exercises

“What is the perimeter of the municipality of Vancouver?”
postgis=# select perimeter(the_geom) from bc_municipality where name =
'VANCOUVER';
perimeter
------------------
57321.7782018048
(1 row))

”What is the total area of all voting areas in hectares?”

postgis=# select sum(area(the_geom))/10000 as hectares from
bc_voting_areas;
hectares
------------------
107000936.980171
(1 row)

“What is the total area of all voting areas with more than 100 voters in them?”
postgis=# select sum(area(the_geom))/10000 as hectares from
bc_voting_areas where vtotal > 100;
hectares
------------------
41426649.2880661
(1 row)

”What is the length in kilometers of all roads named ‘Douglas St’?”

postgis=# select sum(length(the_geom))/1000 as kilometers from
bc_roads where name = 'Douglas St';
kilometers
------------------
19.8560819878386
(1 row)

- 26 -
5.2 Advanced Exercises
”What is the length in kilometers of ‘Douglas St’ in Victoria?”
postgis=# select sum(length(r.the_geom))/1000 as kilometers from
bc_roads r, bc_municipality m where r.the_geom && m.the_geom and
r.name = 'Douglas St' and m.name = 'VICTORIA';
kilometers
------------------
4.89151904172838
(1 row)

”What two pubs have the most Green Party supporters within 500 meters of
them?”
postgis=# select p.name, p.city, sum(v.gp) as greens from bc_pubs p,
bc_voting_areas v where v.the_geom && setsrid(expand(p.the_geom, 500),
3005) and distance(v.the_geom, p.the_geom) < 500 group by p.name,
p.city order by greens desc limit 2;
name | city | greens
----------------+-----------+--------
Bimini's | Vancouver | 1407
Darby D. Dawes | Vancouver | 1104
(2 rows)

”What is the latitude of the most southerly hospital in the province?”

postgis=# select y(transform(the_geom,4326)) as latitude from
bc_hospitals order by latitude asc limit 1;
latitude
------------------
48.4657953714625
(1 row)

”What were the percentage NDP and Liberal vote within the city limits of Prince
George in the 2000 provincial election?”
postgis=# select 100*sum(v.ndp)/sum(v.vtotal) as ndp,
100*sum(v.lib)/sum(v.vtotal) as liberal from bc_voting_areas v,
bc_municipality m where v.the_geom && m.the_geom and
intersects(v.the_geom, m.the_geom) and m.name = 'PRINCE GEORGE';
ndp | liberal
-----+---------
16 | 59
(1 row)

- 27 -
”What is the largest voting area polygon that has a hole?”
postgis=# select gid, id, area(the_geom) as area from bc_voting_areas
where nrings(the_geom) > 1 order by area desc limit 1;
gid | id | area
------+--------+------------------
3531 | NOC 60 | 32779957497.4976
(1 row)

“How many NDP voters live within 50 meters of ‘Simcoe St’ in Victoria?”
postgis=# select sum(v.ndp) as ndp from bc_voting_areas v,
bc_municipality m, bc_roads r where m.the_geom && r.the_geom and
r.name = 'Simcoe St' and m.name = 'VICTORIA' and distance(r.the_geom,
v.the_geom) < 50 and m.the_geom && v.the_geom;

ndp
------
2558
(1 row)

- 28 -
6 MAPSERVER & POSTGIS
The University of Minnesota Mapserver (aka “Mapserver”) can read spatial data
directly out of PostGIS databases. This provides a quick and easy way to publish
database data directly to the internet without a “staging” process of dumping
data out of the database for publication.

6.1 Basic Mapserver Configuration

The most basic configuration of Mapserver using PostGIS simply references a
spatial table and indicates to Mapserver which column contains the spatial
information to be mapped. All other configuration is standard Mapserver map
file syntax:
LAYER
NAME "elections"
CONNECTIONTYPE postgis
CONNECTION "host=localhost port=5432 dbname=bc password=postgres
user=postgres"
DATA "the_geom from bc_voting_areas"
TYPE POLYGON
STATUS ON
PROJECTION
"proj=aea"
"ellps=GRS80"
"lon_0=-126"
"lat_0=45"
"lat_1=50"
"lat_2=58.5"
"x_0=1000000"
END
CLASS
NAME "Voting Areas"
OUTLINECOLOR 0 0 0
COLOR 255 255 200
END
END

The example above takes polygons from the bc_voting_areas table in our PostGIS
database and maps them all as yellow polygons.
Note that the PROJECTION is defined in the map file, not read from the
database. The definition here is for the BC Albers projection.

- 29 -
6.2 Mapserver Filters and Expressions
The only “standard” Mapserver map file contructs that work differently when
using PostGIS as a data source are the FILTER parameter and the EXPRESSION
parameter.
When using shape files as a data source, FILTER takes in Mapserver “expression
format” logical strings. When using PostGIS, the FILTER takes in SQL “where
clause” expressions. That is, SQL fragments that would be legal in a SQL
“where” expression. Note that the example below does not have square brackets
[] around the attribute name, as it would if it where using Mapserver syntax.
LAYER
NAME "elections"
CONNECTIONTYPE postgis
CONNECTION "host=localhost port=5432 dbname=bc password=postgres
user=postgres"
DATA "the_geom from bc_voting_areas"
FILTER "lib > 100"
TYPE POLYGON
STATUS ON
PROJECTION
"proj=aea"
"ellps=GRS80"
"lon_0=-126"
"lat_0=45"
"lat_1=50"
"lat_2=58.5"
"x_0=1000000"
END
CLASS
NAME "Voting Areas"
OUTLINECOLOR 0 0 0
COLOR 255 255 200
END
END

- 30 -
Mapserver expressions are only slightly different when using PostGIS. When
using shape files, the attributes in Mapserver expressions are expressed all in
UPPER CASE. When using PostGIS, the attributes are expressed using all lower
case.
LAYER
NAME "elections"
CONNECTIONTYPE postgis
CONNECTION "host=localhost port=5432 dbname=bc password=postgres
user=postgres"
DATA "the_geom from bc_voting_areas"
FILTER "lib > 100"
TYPE POLYGON
STATUS ON
PROJECTION
"proj=aea"
"ellps=GRS80"
"lon_0=-126"
"lat_0=45"
"lat_1=50"
"lat_2=58.5"
"x_0=1000000"
END
CLASS
NAME "Voting Areas Strong Liberal"
OUTLINECOLOR 0 0 0
COLOR 255 255 200
EXPRESSION ([lib] > 100)
END
CLASS
NAME "Voting Areas Weak Liberal"
OUTLINECOLOR 0 0 0
COLOR 255 255 200
EXPRESSION ([lib] <= 100)
END
END

- 31 -
6.3 Mapserver with SQL
Sometimes the information you want to map is not directly available in your
table – it is the result of a calculation or a comparison with some other data.
In these cases, you can construct an arbitrary piece of SQL, and have Mapserver
render the result into a map. Once you get used to the power of spatial SQL, you
will see how useful this Mapserver/PostGIS capability is.

6.3.1 Thematic Mapping base on Calculations

As a simple example, consider the bc_voting_areas table. In includes a vtotal
column which records the total number of votes, and a vregist column that
records the total number of registered voters. Mapping the total number of votes
would be interesting, but a more useful map would be of the percentage voter
turnout. We can calculate percentage turnout on the fly in SQL and have
Mapserver render that. The mapping file fragment below shows the data
statement:
DATA "the_geom from (SELECT gid, the_geom, (case when vregist = 0 then
0.0 else vtotal::REAL / vregist::REAL end) AS percent FROM
bc_voting_areas) as foo using srid=3005 using unique gid"

The statement includes the following components:

• the_geom indicates the spatial column to use.
• (select … ) as query is the arbitrary SQL query.
• using srid=3005 tells Mapserver what SRID to use when building a
bouding box to subset the data. This should be the SRID of the data table in
the database.
• using unique gid tells Mapserver what column in the SQL query to use as
a unique key. This is needed to support map querying against your SQL.
Everything outside the () is mapserver specific, everything inside is PostGIS
specific. You will always have to ensure that you return (a) a geometry and (b) a
unique identifier so that your arbitrary SQL can be mapped and queried,
respectively.
Note the use of the “case” operator within the SQL portion of the statement. By
using a database construct we can avoid the divide-by-zero case. This would not
be possible using Mapserver expressions along.

- 32 -
6.3.2 Ordering based on Calculations
This example shows a Mapserver LAYER that reads from the full bc_roads table,
but only labels the 15 most “important” roads on the map. The layer determines
which roads are “important” by summarizing their lengths by unique road name.
Roads with a large amount of length for a name are deemed more “important”
than shorter roads, so they get mapped first. Once the first 15 records have been
returned, the query ends.
DATA "the_geom from (
SELECT
name,
Sum(Length(the_geom)) AS length,
Collect(GeometryN(the_geom,1)) AS the_geom
FROM bc_roads
WHERE the_geom && setsrid(!BOX!,3005)
GROUP BY name
ORDER BY length DESC
LIMIT 15)
as foo using SRID=3005, using unique name"

The aggregate length is calculated with the Sum() function.

Because we are grouping on name we can use it as a unique key (note that it is
declared as the unique key in the Mapserver portion of the data statement at the
end).
The output geometry is calculated by using Collect() to join up all the similarly
named roads into one aggregate feature. The GeometryN() function removes one
level of nesting to the output collection, so that Mapserver can handle it for
mapping. (This works around a mapserver limitation which can hopefully be
removed in the future.)
The WHERE statement using a special substitution variable !BOX! that is
rewritten to the map extents for the purposes of forming an indexed bounding
box query. By using !BOX! we can control exactly where in the SQL statement
the spatial index is used, which can sometimes (not usually) improve
performance.
Finally, we group by the name and order by the length in a descending manner,
so the largest lengths come first. The LIMIT statement stops the query after 15
return records. It is faster to use a LIMIT in SQL than a MAXFEATURE in the
map file, because with a LIMIT the database can plan for a short query, while
with a MAXFEATURES it must plan to return the whole result set, even if
mapserver stops the query partway through.

- 33 -
6.4 Mapserver and Dynamic SQL
Doing fancy things with data and styling in Mapserver has traditionally been the
exclusive domain of Mapscript. However, by using variables and dynamically
written SQL, it is possible to make a very flexible system using only the
Mapserver CGI program and PostGIS.
LAYER
<.. snipped ..>
DATA "the_geom from (SELECT the_geom,gid, distance(the_geom,
geometryfromtext('POINT(' || (%mx% + %img.x% * %mw% / %iw%) || ' ' ||
(%my% - %img.y% * %mh% / %ih%) || ')',3005)) AS dist FROM bc_roads) as
foo using srid=3005 using unique gid"
<.. snipped ..>
CLASSITEM dist
CLASS
NAME "Roads < 1km from Click"
EXPRESSION ([dist] < 1000)
COLOR 0 255 0
END
CLASS
NAME "Roads > 1km from Click"
EXPRESSION ([dist] >= 1000 AND [dist] < 2000)
COLOR 255 255 0
END
CLASS
NAME "Roads > 2km from Click"
EXPRESSION ([dist] >= 2000 AND [dist] < 4000)
COLOR 192 192 0
END
CLASS
NAME "Roads > 4km from Click"
EXPRESSION ([dist] >= 4000)
COLOR 255 0 0
END
END

The trick here is using the Mapserver CGI “variable substitution” system in
conjunction with the PostGIS arbitrary SQL system. The variables mx, mw,
img.x, iw, etc, are all being created as part of the CGI template form, and are
passed back to the CGI with each mouse click.
• mx, my: map minx and maxy in spatial coordinates
• mw, mh: map width and height in spatial coordinates
• iw, iw: image width and height in pixels
• img.x, img.y: mouse click location in pixels from top left

- 34 -
Mapserver replaces the %variables% with their passed in values prior to
executing the data query. Once executed, PostGIS does the math to convert the
image sizes, bounds and click point into a spatial location, which it in turn uses
to calculate distances for each feature. Mapserver does the final step of
converting the distance numbers into colors on the final map via the usual
CLASS mechanism.

- 35 -
6.5 Mapserver and Very Dynamic SQL
The ultimate in dynamic SQL in passing arbitrary SQL statements into
Mapserver on the fly. Using such a system, it becomes possible to map and
visualize any spatial table in the database, in any combination of filters,
operations, and joins available at the backend.
LAYER
NAME qlyr
CONNECTIONTYPE postgis
CONNECTION "host=localhost port=5432 dbname=test password=postgres
user=postgres"
DATA "the_geom from (%sql%) as foo using SRID=3005 using unique
gid"
TYPE LINE
STATUS ON
PROJECTION
"proj=aea"
"ellps=GRS80"
"lon_0=-126"
"lat_0=45"
"lat_1=50"
"lat_2=58.5"
"x_0=1000000"
END
CLASS
NAME "Query Results"
SYMBOL solid
SIZE 2
COLOR 250 0 0
END
END

Note how simple the DATA statement is. Simply accept the contents of the
%sql% variable, and map the geometry from it. The only requirements of the
%sql% variable are that:
• It return a single geometry column named the_geom.; and,
• It return a unique row identifier column named gid.
Otherwise, the world is your oyster. Try some example queries:
• All voting areas where the Liberals got more than 150 votes:
SELECT gid,the_geom FROM bc_voting_areas WHERE lib > 150
• All roads that start with “S”:
SELECT gid.the_geom FROM bc_roads WHERE name LIKE ‘S%’

- 36 -