Wazimap

Wazimap is a Django application for exploring census and other similar data. It makes it easy to understand a place through the eyes of the data, and to explore data across a range of places. It is most suited for census data but can easily be used with other data that is similarly focused on places in a country.

Check out Wazimap South Africa and Wazimap Kenya to get an idea of what Wazimap is about.

Wazimap is a fork of the excellent Censusreporter project which was funded by a Knight News Challenge grant. You can also find Censusreporter on GitHub.

Wazimap builds on Censusreporter and makes it easier to re-use. Wazimap was originally built by OpenUp with the support of Media Monitoring Africa. It is maintained by OpenUp.

Contents

Getting Started

Wazimap is a Django application that you’ll need to configure correctly before you can use it. You’re going to need to import place metadata, census data and write some code to tell Wazimap what census data you want to show.

Wazimap is a bit different to most Django applications. It comes with all the URLs and most of the settings that you’re going to need, so you need to tell Django to load the details from the Wazimap application, rather than your Django application like normal.

Note

You’ll need to choose a short name for your application. In these examples, we’ll be using wazimap_ex for Wazimap Example. We suggest using the two-letter country code of your country in place of the ex.

  1. Setup a python virtual environment, we recommend using virtualenv.

  2. Install Django (version 1.8.0 or newer) and start a new Django project.

    pip install 'django<1.10'
    django-admin startproject wazimap_ex
    cd wazimap_ex
    
  3. This will have created some files that are unnecessary because Wazimap already provides them. You can safely delete them.

    rm wazimap_ex/urls.py wazimap_ex/wsgi.py
    
  4. Wazimap needs GDAL installed to allow users to download data in KML, GeoJSON, Excel, etc. GDAL can be complicated to install. For development, we recommend you follow the Django instructions.

  5. Install Wazimap and its dependencies

    pip install wazimap
    
  6. At this point you have the absolute basics in place. We’re now going to configure your Django application to load its settings from Wazimap.

    Change your settings.py file to import most settings from Wazimap, and then override the settings that you want to change. Replace the contents of your file with the following:

    # pull in the default wazimap settings
    from wazimap.settings import *  # noqa
    
    # install this app before Wazimap
    INSTALLED_APPS = ['wazimap_ex'] + INSTALLED_APPS
    
    # Localise this instance of Wazimap
    WAZIMAP['name'] = 'Wazimap Example'
    # NB: this must be https if your site supports HTTPS.
    WAZIMAP['url'] = 'http://wazimap.example.com'
    WAZIMAP['country_code'] = 'EX'
    

    See also

    See the Configuration section for more details on configuration options.

  7. You’ll need a running PostgreSQL server. The default database settings are:

    • Username: wazimap
    • Password: wazimap
    • Database: wazimap
    • Hostname: localhost

    So create the user and database locally:

    createuser -P wazimap
    createdb -O wazimap wazimap
    

    Or alternatively from inside psql:

    CREATE USER wazimap WITH PASSWORD 'wazimap';
    CREATE DATABASE wazimap OWNER wazimap;
    

    Note

    If you want to use a different database, configure the default database as you normally would for a Django application.

  8. Run migrations to setup the database:

    python manage.py migrate
    
  9. Create a superuser to manage your site:

    python manage.py createsuperuser
    
  10. You now have a Wazimap application and matching database ready to load geographies and data into. Those steps are covered in the rest of this guide.

Configuration

There are a number of options to configure how Wazimap behaves. Options are set in settings.py in the WAZIMAP dict.

name
The full name of the website. Default: Wazimap Example
url
The site’s base url, including scheme, WITHOUT and ending slash. If your site supports HTTPS, this must be an https url, otherwise embeds that are over HTTPS might not load correctly. Default: http://wazimap.example
twitter
Twitter handle. Default: @Wazimap
ga_tracking_id
Google Analytics tracking id.
country_code
Two-letter country code for the country this site relates to.
google_geocode_api_key
Google Geocoding API key, if you want to use more than the free limit.
cache_secs
How many seconds should cacheable Wazimap pages be cached for? Default: 60 * 60
embed_cache_secs:
How many seconds should Wazimap embed pages be cached for? Default: 24 * 60 * 60
geodata
The dotted-path of the class to use for geo data helper routines. See Geographies for more info. Default: wazimap.geo.GeoData
levels

Geography levels. This must be a dict similar to the following:

{
  'country': {
    'name': 'country',      # optional, defaults to the level code
    'plural': 'countries',  # plural version of the name
    'children': ['province'],
  },
  ...
}

Default:

'country': {
    'plural': 'countries',
    'children': [],
}
comparative_levels
Ordered list of levels to compare a place with; should generally be only 2-3 levels. Default: []
geometry_data

Map from geometry names to local static files with geometry data. This is used to both draw maps in webpages and for geolocation. Wazimap uses Django’s static files finder to find these files, so the files must be stored in your application’s static directory.

For maps, topojson is preferred because it’s smaller. But for geolocation, because it’s done server side, it requires geojson (shapely doesn’t understand topojson). Ideally you should have both .geojson and .topojson files and specify the .topojson version here. Wazimap will look for geojson equivalents of topojson files.

The top level of this dict MUST be the geo version. If you’re not using versioned geographies, use the empty string.

If a level isn’t specified here, Wazimap will try to find a file under the key ''.

Default:

{
  '': {
    '': 'geo/all.geojson',
  }
}
map_centre, map_zoom
Centre coordinates and zoom level defaults for maps. Centre must be a [lat, long] pair and zoom is a zoom level (1-12). If not set, both are determined from the geometry.
embed_url
The full url, including scheme (but no ending slash) where embeds are hosted. This is generally not used. If your site supports HTTPS, this must be an https url, otherwise embeds that are over HTTPS might not load correctly. The default is the same as url.
na_label
The label to display for stats which have no data available. The default is N/A.
default_geo_version
Default geo version to use when loading geographies. If None, the most recent version in the Geography’s table (geo_data.latest_version) is used.
legacy_embed_geo_version
The geo version to use for legacy embeds that don’t specify a geo version. If None, uses the latest version (geo_data.latest_version). If you’re introducing versioned geographies and your users have already embedded charts, you probably want to set this to your earliest version so that embeds continue showing the original data.
primary_release_year
The primary release year to use for each geo level. The default is to use the latest release. Set this if you have newer releases at some geo levels, such as a 2010 national census down to the city level, and a 2015 partial census to the provincial level.

Localisation

You can customise how Wazimap formats numbers shown on a profile page by using Django’s FORMAT_MODULE_PATH setting. These settings in particular apply:

THOUSAND_SEPARATOR
The thousand separator used when formatter numbers.
DECIMAL_SEPARATOR
The decimal separator used when formatter numbers.
CURRENCY_SYMBOL
The currency symbol to be used before currency amounts. Not in Django. Default: $

Additional Options

These options are specific to Wazimap and are set at the top-level of your settings.py file, not in the WAZIMAP dict.

STRIP_WWW
Set this to True to redirect users visiting your site at www.example.com to example.com. This is the reverse of Django’s PREPEND_WWW setting. Default is False.

Geographies

A geography (or place) is identified uniquely by a level and a code which, when combined, form its geoid. Each level describes a different way of breaking up a country and can be units such as country, province, municipality, state or county. The code is some ID that is unique for that level, either a number or a string.

Some example geoids are:

  • province-WC: the Western Cape province in South Africa
  • country-KE: Kenya
  • ward-19100057: ward 19100057 in Cape Town, South Africa
  • county-1: the Mombasa county in Kenya

Geographies are stored in the wazimap_geographies table using the Geography Django model.

Adding Geographies

The geography database table looks something like the following. It’s generally easiest to import this data using PostgreSQL’s CSV import support.

geo_level geo_code version name year parent_level parent_code
country KE   Kenya 2009    
county 1   Mombasa 2009 country KE
county 2   Kwale 2009 country KE
county 3   Kilifi 2009 country KE

Level Hierarchy

The geography levels form a hierarchy with a single root geography (generally a country). The levels configuration option describes each level in the hierarchy.

Geography Versions

Wazimap supports geographies that may change over time. For instance, a municipality’s boundaries may change while it’s code and name stay the same.

Every Geography has a version linked to it, even if it’s just an empty string (the default). The version can be anything that is sortable. By default, Wazimap will show data linked to the most recent version of a geography, if there are multiple versions.

We recommend using the year as the geography version.

Note

If you don’t need versioned geographies, you can simply use an empty string as the version, wherever it is needed.

Maps and boundary data

Wazimap needs to know boundary information for each geography so that it can draw shapes on maps and include the boundaries in download formats such as KML and GeoJSON.

The easiest way to provide this information is as a GeoJSON FeatureCollection. You can either have one big file, or one file for each geo level.

  1. Create a .geojson file with single FeatureCollection object. Each Feature in the collection must be a geography in Wazimap. It must have the following attributes:

    • name: the name of the place (same as the geography name)
    • level: the geography level of the place, same as the geo_level in the database
    • code: the code for the geography, same as the geo_code in the database
    • geoid: the unique id for this geography, which must be the level-code.
  2. Store this file in your Django app’s static folder, such as wazimap_ke/static/geo/.

  3. Set the geometry_data configuration option in your settings.py to indicate where each version and level’s geojson file is stored, relative to you static folder. Note that the first key in the dict is a geo version – the default geo version is the empty string. For example:

    geometry_data = {'': {
      'country': 'geo/country.geojson',
      'province': 'geo/province.geojson',
    }}
    
  4. GeoJSON files can be quite big for geographies that share lots of boundaries. If you want an improvement in speed and size, also convert each file to a TopoJSON file using the topojson NPM tool. Then place these files alongside your geojson files and change your geometry_data settings to point to the .topojson files:

    geometry_data = {
      'country': 'geo/country.topojson',
      'province': 'geo/province.topojson',
    }
    

    Note

    It’s important to have both the TopoJSON and GeoJSON formats. Wazimap will look for the GeoJSON format when loading the boundary information for use on the server, such as for geolocation. This is necessary because Python doesn’t have a good TopoJSON library.

Geo Data API

Wazimap makes geography data available through the wazimap.geo.geo_data object, which is detailed below.

The class used for this object can be configured using the geodata setting. This allows you to change this behaviour if you need to.

Statistical Data

Wazimap presents statistical data about geographies, such as the number of English speakers in a city. You need to tell Wazimap what data is available and for which geographies.

Wazimap stores statistics in what it calls data tables. There are two different types of data tables: Field Tables and Simple Tables.

Field Tables
Field Tables are more flexible than Simple Tables. They allow Wazimap to store combinations, such as the number of people in an area by both age and gender. Most census data is best stored using Field Tables. Field Tables almost always have only one column that contains a number, the total column.
Simple Tables
Simple Tables look a lot like spreadsheet tables. Each column is a different statistic about a place. They are easy to think about and work with but are limited in their flexibility. Simple Tables often have many columns with numbers, in addition to a total column.

Both types of tables have some metadata linked to them, such as an id, year and a description of the population that it covers.

Datasets and Releases

In Wazimap, a dataset is a collection of related data tables, such as a national census. A dataset can be updated with new releases every few years. Not all data data tables will always be updated in every release, so Wazimap lets you link data tables to releases individually.

Sometimes a release has a different name to the original dataset. For example, South Africa conducts a full census every decade, but releases a community survey in between each full census. A community survey is a statistical sampling and is not a full census, so it would be incorrect to call them both “census”. The results of the community survey are very similar to the census and are directly comparable. We consider census and community surveys to be different releases of the same dataset.

Important

You must add at least one dataset and one release before you can add any data tables. See below for details on how to do this.

Create a Dataset and Release

  1. Go to the Django admin section at http://localhost:8000/admin and log in.
  2. Under Wazimap, click the Add button alongside Datasets.
  3. Give your dataset a name.
  4. Under Releases, fill in the name and the year of your first release. For example, you could use Census and 2017.
  5. Click Save.

Configuring Tables

Datasets, releases and data tables are configured through the Django admin interface, at http://localhost:8000/admin.

Once you have told Wazimap about your tables, it’ll ensure that they exist in the database. You can then import the raw data from CSV.

Field Tables

A Field Table is a logical collection of fields and values that describe numeric facts about a geography, along with some extra metadata about the table such as a name.

A field is generally an attribute of a place or a person in that place, such as language or gender. A field has corresponding keys such as English or Female. Fields and their keys describe a collection of people that match those attributes, such as all the English-speaking females in a province. The value associated with a collection of fields and keys is the number of people with that attribute.

For example, here is a Field Table with two fields, language and gender:

language gender total
English Male 298
English Female 312
French Male 128
French Female 779

Most census Field Tables describe a partitioning of the population: the population is broken into groups (such as by language or gender) and every person is counted exactly once. If we added up all the values for all key combinations, we’d get the total population. That’s useful because it means we can express the value for each combination of keys as a percentage of the total.

Each Field Table is stored in a physical PostgreSQL database table. Each entry in a Field Table is linked to a geography, since a row is a statistic about a place, and so each row has the geography level, code and version associated with it.

geo_level geo_code geo_version language gender total
province WC   English Male 283
province WC   English Female 199
province WC   French Male 324
province WC   French Female 287
province GT   English Male 298
province GT   English Female 312
province GT   French Male 128
province GT   French Female 779

Adding a Field Table

First, ensure that you have created at least one dataset and release.

  1. Go to the Django admin section at http://localhost:8000/admin and log in.
  2. Under Wazimap, click the Add button alongside Field tables.
  3. Choose the dataset the table belongs to.
  4. Name the Universe this table describes, such as Population, Households or Youth aged 14 to 25.
  5. Provide a comma-separated list of the Fields in your table.
  6. Leave the Description blank and it will be generated for you.
  7. Click Save.

Now import the data into the table. The easiest way of doing this is to look at the database to understand the columns in your new table, shape your data accordingly, and import it using psql’s CSV import support.

Simple Tables

A Simple Table looks a lot like a spreadsheet. It contains statistics for many places, one geography per row. Each column has a name and the cell values are the numerical statistics for that row’s geography. Each Simple Table is stored in a physical PostgreSQL database table.

For example, here is a Simple Table with two columns, votes_cast and registered_voters.

geo_level geo_code geo_version votes_cast registered_voters
province WC   829 1024
province GT   773 990

You can see that in contrast with a Field Table, a Simple Table can have multiple statistics per geography.

A Simple Tables usually has a column which represents a total value, usually (but not always) called total. It is used to calculate percentages for other columns in the table. In the example above, the registered_voters column is the total column, because we can express votes_cast as a percentage of the registered voters in each province.

Wazimap uses this to allow the user to switch between absolute values and percentages when viewing data for the table. You can also tell Wazimap that a table doesn’t have a total column, in which case it always shows absolute values.

Note

If your table has a total column, it’s important that all the statistics in it are related. If it doesn’t make sense to express a column as a percentage, put it in another table that doesn’t have a total column.

Adding a Simple Table

First, ensure that you have created at least one dataset and release.

  1. Go to the Django admin section at http://localhost:8000/admin and log in.
  2. Under Wazimap, click the Add button alongside Simple tables.
  3. Give your table a descriptive name.
  4. Choose the dataset the table belongs to.
  5. Name the Universe this table describes, such as Population, Households or Youth aged 14 to 25.
  6. Add a Description of your table.
  7. Click Save.

Now import the data into the table. The easiest way of doing this is to look at the database to understand the columns in your new table, shape your data accordingly, and import it using psql’s CSV import support.

The Profile Page

You need to tell Wazimap what data to show on a place’s profile page, and how to show it.

The Profile Builder Function

When Wazimap builds the profile page for a place, it calls the function defined the profile_builder configuration setting. This function must return a specially-formatted dict-like structure that contains all the information that will be shown on the profile page.

Create a profiles.py file in your project folder and add a function get_profile like in the example below. Remember to set the profile_builder setting in your settings.py to the dotted path to this function, for example:

WAZIMAP['profile_builder'] = 'wazimap_ex.profiles.get_profile'

Here’s a simple example of a profile builder:

from wazimap.data.utils import get_session, get_stat_data

def get_profile(geo, profile_name, request):
    # get a SQLAlchemy database session
    session = get_session()
    data = {}

    try:
        data['demographics'] = get_demographics_profile(geo, session)
        # ... load other sections here

        return data
    finally:
        # tidy up the session
        session.close()

def get_demographics_profile(geo, session)
    # get the number of people for each sex
    sex_dist_data, total_pop = get_stat_data('sex', geo, session)
    return {
        'sex_distribution': sex_dist_data,
    }

In get_profile we simply get a database session and then call get_demographics_profile to do the heavy lifting.

In get_demographics_profile we use the get_stat_data function to get data on the place’s population for the field sex from a relevant Field Table. It does all the work of formatting the results correctly, and we return it under the key sex_distribution.

All that data is then passed into the Profile Page template where you choose how to show the data.

Get Stat Data for Field Tables

The get_stat_data function is powerful and flexible. It finds the appropriate Field Table for the fields you want, fetches data, remaps fields if you need it, calculates percentages if necessary, and adds the metadata required by the profile page.

Get Stat Data for Simple Tables

If you’re working with Simple Tables, you’ll want to use get_datatable and SimpleTable.get_stat_data.

The Profile Page Template

You need to tell Wazimap how to display your stats on a place’s profile page.

The file you want to override is templates/profile/profile_detail.html, you can see what it looks like in the repo. You generally only need to change the profile_detail block.

Create a new file in your project called templates/profile/profile_detail.html that extends the existing template and provides your new content for the profile_detail block:

{% extends 'profile/profile_detail.html' %}

{% block profile_detail %}
... your stats go here ...
{% endblock %}

If you reload your site you’ll see the homepage has your new content. Django uses this template instead of Wazimap’s version, relying on Wazimap for the blocks you don’t override. There are lots of other blocks you can change, take a look at the original file for more ideas.

See also

There’s more information on changing Wazimap templates in Customising Wazimap.

You must still provide the content that goes into each stats block. The easiest right now is to see how other countries do it, such as South Africa’s default census profile.

Profile Page Charts

The Django template for the profile page creates empty slots for each chart, which are filled by Javascript when the page loads. These placeholders look something like this:

<div class="column-half" id="chart-histogram-demographics-age-distribution" data-stat-type="scaled-percentage" data-chart-title="Population by age range"></div>

The column-* class isn’t really important here; that’s just a structural setting that gives the block an appropriate amount of width that can be governed with media queries. What we really care about are the id and data-* attribute values. The id value tells the constructor what type of chart to draw and which data to use.The data attributes allow you to optionally make changes to how the chart is drawn.

Chart ID

The id tells Wazimap everything it needs to know to create this chart from the profile data. The id is broken into a few parts:

chart-<chartType>-<chartData>

The chartType, in our example case histogram, tells Wazimap the type of chart to draw. Wazimap supports:

  • pie
  • column
  • grouped_column
  • histogram
  • bar
  • grouped_bar

The chartData provides the path to the data that should fill this chart. Wazimap starts at the top, in this case demographics, and then drills down based on the rest of the keys: demographics > age > distribution. That’s where Wazimap expects to find the data to draw the chart.

Data Attributes

You can use optional data attributes to change how the chart is shown.

Use data-chart-title to specify a title to place above the chart.

Use data-initial-sort to change how pie charts are sorted. Determines which category to highlight when the chart is drawn. Using data-initial-sort="-value" will display the highest data value in the chart first. Otherwise the first value in the chart data will be used.

Use data-qualifier to add a trailing line below the chart, prepended with an “*” character. This is useful when charts require a little extra context.

Use data-stat-type to provide formatting hints for the chart’s language and display. Standard chart behavior may be overriden with these values:

  • percentage: Adds a “%” character after figures in the chart. Sets chart domain to 0-100. Uses “rate” in comparison sentences.
  • scaled-percentage: Does the same things as “percentage,” but also scales the chart so that the highest category value takes up the full vertical space available.
  • dollar: Adds a “$” character before figures in the chart. Uses “amount” in comparison sentences.

Customising Wazimap

Overriding Templates

The easiest way to change page layouts and content is to add your own templates, as you would for any Django application. Most pages have blocks which make it simple to override just specific parts of a page.

  1. Find the existing template that you want to change by looking in the wazimap/templates directory of the wazimap package. You can also browse the repo.
  2. Create an empty file with the same name inside your project’s templates directory.
  3. Use {% extends %} to tell Django that your template extends the original Wazimap template
  4. Override the page’s blocks to make your changes.

In the next section we provide an example.

Changing the homepage

Let’s say you’d like to change the homepage. You’re happy with the header and the footer, but would like to change some of the text in between.

The file you want to override is templates/homepage.html, you can see what it looks like in the repo. You want to change the text in the homepage_detail block.

Create a new file in your project called templates/homepage.html that extends the existing template and provides your new content for the homepage_detail block:

{% extends 'homepage.html' %}

{% block homepage_detail %}
your new content!
{% endblock %}

If you reload your site you’ll see the homepage has your new content. Django uses this template instead of Wazimap’s version, relying on Wazimap for the blocks you don’t override.

See also

There’s more information on changing profile page template in The Profile Page.

Changing CSS theme

In addition to providing your own CSS files by overriding the templates, you can also override the SCSS variables that control the Wazimap colour scheme (theme). This is the fastest and easiest way to match Wazimap’s colours to your brand.

Note

If you customise the CSS using this method, you MUST call python manage.py compilescss before running collectstatic when you deploy your application.

Create a new file in your project called static/css/_custom.scss. In it, override the default Wazimap colour variables <https://github.com/OpenUpSA/wazimap/blob/master/wazimap/static/css/_variables.scss>.

These are the most important ones:

  • $brand-primary-color: a bright primary colour
  • $brand-secondary-color: a darker, secondary colour. Variations on this colour are used throughout the site.
  • $brand-secondary-{lightest, lighter, darkest, darker}-color: lighter and darker shades of the secondary colour. We recommend simply lightening or darkening your $brand-secondary-color by 10% and 20%.
  • $general-bg-color and $general-bg-{lightest, lighter, darkest, darker}-color: general background colours and lighter and darker shades. We recommend simply lightening or darkening your $general-bg-color by 10% and 20%.

Deploying

Wazimap can be deployed like any Django application. It’s particularly easy to deploy on Heroku or a Heroku-like environment such as Dokku. You can use the Wazimap WSGI application variable from wazimap.wsgi.

Important

Always ensure that you set the DJANGO_SETTINGS_MODULE environment variable to the module path of your settings.py file.

Deploying on Heroku or Dokku

See also

You can find example of the files necessary to deploy on Dokku or Heroku in the Wazimap deploy directory.

To deploy on Heroku on Dokku, create your application in the normal way. Then set these config variables:

  • DATABASE_URL=<your database URL>
  • DJANGO_SECRET_KEY=<a random secret key>
  • DJANGO_DEBUG=False
  • DJANGO_SETTINGS_MODULE=<your settings module>
  • DISABLE_COLLECTSTATIC=1=

You’ll need a Procfile, too:

web: gunicorn --worker-class gevent wazimap.wsgi:application -t 120 --log-file -

GDAL

Wazimap requires GDAL to support data downloads in formats like KML. The easiet way to get these installed on Heroku or Dokku is to use multiple buildpacks. Create a file .buildpacks in your project’s root directory:

https://github.com/cyberdelia/heroku-geo-buildpack.git
https://github.com/heroku/heroku-buildpack-python.git

That tells Heroku and Dokku to install GDAL and then continue with the usual Python install. Alternatively, install GDAL for your platform manually.

By default, Wazimap doesn’t install the libraries to use GDAL because it can be difficult to install. Tell Wazimap to install everything it needs for GDAL by installing it with wazimap[gdal] or specify GDAL>=1.11.0,<2.0 in your requirements.txt.

Be sure that the platform GDAL and Python GDAL versions match.

Dependencies

Wazimap requires Django 1.9.

Add Wazimap and Django as dependencies for your project in your requirements.txt file:

wazimap[gdal]
Django==1.9.2

Copy Wazimap’s ``app.json``<https://github.com/OpenUpSA/wazimap/blob/master/wazimap/static/css/_variables.scss> into your project:

{
  "scripts": {
    "dokku": {
      "predeploy": "python manage.py compilescss && python manage.py collectstatic --noinput && rm -rf /var/tmp/wazimap_cache"
    }
  }
}

Finally, just use git push heroku or git push dokku master to deploy as you usually would.

Should you need to do a database migration, you can run dokku run <app-name> python manage.py migrate on your server.

HTTPS and SSL

Note

We recommend running your website over HTTPS.

We recommend that you run your site over HTTPS (SSL). If you don’t use HTTPS, then any website that does use HTTPS will not be able to embed a chart from your Wazimap. This is because websites using HTTPS cannot load content from non-HTTPS sites.

Upgrading

Upgrading from 1.x to 2.x

As of version 2.0.0, Wazimap supports multiple releases of a dataset. This means that you can, for example, have two releases of a national census in Wazimap at the same time. You will need to make some changes to how you declare your tables and build profile pages.

  • Data table management is done through the Django admin interface.
  • Datasets are a concrete concept, they bring together a collection of related data tables. A dataset can have multiple releases (ie. updates) over the years. A good example of a dataset is a national census, which could be released every few years.
  • Data tables must have at least one release, which is a set of data for that table, released at a particular point in time.

Run migrations

Before anything else, run migrations:

python manage.py migrate

Upgrade existing tables

The old way of declaring tables in tables.py is no longer used. Before you remove those declarations, use them to upgrade your table descriptions into the new format. Run:

python manage.py upgradetables

This will upgrade the tables into the database.

Note

Do not delete the data tables from the database, they are still needed and will be used by the upgraded tables.

Visit the admin area

Create a superuser so you can login to the admin area:

python manage.py createsuperuser

Visit http://localhost:8000/admin to log in and check that your tables are correct.

Remove the old table descriptions

If you’re happy that your tables have been imported correctly, you can remove them from your tables.py file.

Wazimap Version History

2.1.0 (09 October 2019)

  • Upgrade to python 3
  • Upgrade to django 2
  • Return fieldtable id from as_dict
  • better object representation of fieldtable object.

2.0.0 (?)

This release of Wazimap introduces some backwards incompatible changes. You WILL need to follow the upgrade steps. Be sure to upgrade to 1.x before upgrading to 2.x.

  • Wazimap now supports multiple releases of a dataset. This means that you can, for example, have two releases of a national census in Wazimap at the same time. You will need to make some changes to how you build profile pages.
  • BREAKING: SimpleTable and FieldTable are now Django models.
  • BREAKING: Table management is now done through the Django admin interface. You must upgrade your existing tables.
  • BREAKING: get_objects_by_geo has been replaced by FieldTable.get_rows_for_geo.

1.1.1 (21 November 2017)

  • FIX another bug for versioned geos for static maps

1.1.0 (23 October 2017)

  • FIX support for versioned geos for static maps

1.0.1 (21 September 2017)

This release of Wazimap introduces some backwards incompatible changes.

  • Support versioned geographies.
  • BREAKING: The Geography model has changed. You must run python manage.py migrate when updating.
  • BREAKING: The method signature of the profile_builder function has changed. It is now called as get_profile(geo, profile_name, request). You will need to update your profile methods to work with a geo object rather than a level and a code. Use geo.geo_level and geo.geo_code instead.
  • BREAKING: Some methods on the GeoData object now take a Geography object rather than a geo_level and a geo_code.
  • BREAKING: The geometry_data setting must include a geography version at the top of the dict. Use '' by default. eg: {'': {'country': 'geo/country.geojson'}}
  • Some methods on the GeoData object now take an optional version parameter.
  • A new method GeoData.get_comparative_geos makes it easier to build the profile page for comparative geographies.
  • FIX: load streetmap tiles over https if necessary
  • FEATURE: site menu
  • FEATURE: use SCSS stylesheets, support customising using SCSS variables.
  • Attribution updates to reflect OpenUp, the new name for Code for South Africa.

0.8.2 (20 April 2017)

  • Make the ‘show data / embed’ links for charts more visible

0.8.1 (18 April 2017)

  • Fix metadata for field tables that re-use db tables

0.8.0 (12 April 2017)

  • Support FieldTables that share a common database table
  • Remove support for FieldTable.table_per_level parameter.

0.7.6 (3 April 2017)

  • Tell robots not to crawl /api/

0.7.5 (15 March 2017)

  • Remove unnecessary Auth, Admin, Messages and Sessions middleware. If you need these middleware, add them to your INSTALLED_APPS and MIDDLEWARE_CLASSES in your settings file.
  • New setting STRIP_WWW to redirect www.example.com to example.com, turned off by default.

0.7.4 (13 March 2017)

  • Allow overriding of header block in map view

0.7.3 (9 March 2017)

  • Add a stub help page at /help

0.7.2 (3 March 2017)

  • Don’t exclude partial matches from google geocoding results

0.7.1 (25 January 2017)

Important This version makes GDAL an optional install. You must ensure you add GDAL to your requirements file, or install wazimap using wazimap[gdal]]. See http://wazimap.readthedocs.io/en/latest/deploying.html

  • Make GDAL and Shapely optional. This makes it much easier to get up and running with Wazimap without getting GDAL installed, which can be tricky.
  • Support GDAL 1.x or 2.x (2.x not fully tested).

0.6.1 (11 January 2017)

  • FIX bug in testing database names

0.6.0 (11 January 2017)

  • Support null values in a table. This allows you to indicate to Wazimap that you don’t have data for a particular field/indicator in a particular place. It is still necessary to ensure that all places have an entry in the database for all indicators.
  • Automated build-time testing on Travis.

0.5.8 (21 November 2016)

  • Support percentage values in the map view

0.5.7 (17 November 2016)

  • Support tables with raw values that are percentages
  • Version 0.5.6 was not released

0.5.5 (7 November 2016)

  • Fix embedded charts (contributed by mevey)

0.5.4 (28 October 2016)

  • Don’t show error for empty FieldTable with has_total=True

0.5.3 (21 October 2016)

  • Fix loadGeometryForGeoIds bug (contributed by cliftonmcintosh)

0.5.2 (28 September 2016)

  • Fix profile page tables for non-percentages

0.5.1 (27 September 2016)

  • Correctly send comparative_levels setting to profile page

0.5.0 (22 September 2016)

  • Support floating point values in tables

0.4.6 (20 September 2016)

  • Be more tolerant of GDAL versions >= 1.11.0 and < 2.0
  • Lazy load OGR/GDAL
  • Don’t allow Django 1.10

0.4.4 (8 September 2016)

  • Allow overriding of favicon.

0.4.2 (23 August 2016)

  • Support decimal places in chart tables

0.4.1 (22 August 2016)

  • FEATURE control the number of decimals for stats shown with the _stat_list include by using the decimals parameter.
  • FEATURE adjust the currency symbol used for currency stats shown by _stat_list by setting CURRENCY_SYMBOL in the file specified by Django’s FORMAT_MODULE_PATH.
  • FEATURE get_stat_data supports percent_grouping and slices

0.3.3 (17 August 2016)

  • FEATURE get_stat_data now applies the only and exclude parameters in the database. This makes it possible to specify filters on fields that don’t need to be fetched.
  • FIX get_stat_data now uses the denominator_key to determine the table total for FieldTables that have denominator_key specified.
  • FIX correctly serialise Decimal() in additional places
  • Don’t use whitespace in JSON responses.

0.3.1 (15 August 2016)

  • FEATURE a FieldTable can now optionally not support percentages (has_total parameter)
  • FIX correctly serialize Decimal() values in JSON

0.2.25 (12 August 2016)

  • FIX reordering pure-integer keys in javascript

0.2.24 (4 August 2016)

  • FIX splitting geographies into child levels

0.2.23 (26 July 2016)

  • FIX geography search in table view

0.2.22 (22 July 2016)

  • Easier to override header blocks

0.2.21 (20 July 2016)

  • Footer Wazimap links use WAZIMAP.name where appropriate
  • Change GitHub link in Footer
  • Easier to override logo
  • Twitter link is now optional

0.2.20 (11 July 2016)

  • FIX correct comparative levels

0.2.19 (20 June 2016)

  • Add table detail view, linked from homepage, to help users find data rather than always starting with a place.

0.2.18 (17 June 2016)

  • Make it easier to add provinces up the hierarchy in the data table view

0.2.17 (17 June 2016)

  • FIX how geo level ancestors are determined to fix issues with geography levels deeper than two.

0.2.16 (26 May 2016)

  • Add leaflet images included by css

0.2.15 (25 May 2016)

  • FIX ordering of keys for nested values

0.2.14 (5 May 2016)

  • Load leaflet locally so that it works over https
  • Load fonts over https if necessary

0.2.13 (29 April 2016)

  • Serve most (all?) assets over https if site is loaded over https
  • NOTE: this release still doesn’t work correctly over https

0.2.12 (28 April 2016)

  • FIX: place search is wildcarded on both sides
  • Make it easier to subclass from the default Wazimap Geography model by using GeographyBase

0.2.11 (20 April 2016)

  • Make homepage easier to override
  • DOCS: how to override templates
  • DOCS: profile page chart options
  • Improve limiting searches to specific geo levels

0.2.10 (10 April 2016)

  • FIX: show slippy map on desktop, fixing bug introduced in 0.2.9.

0.2.9 (7 April 2016)

  • FIX: correctly hide slippy map on mobile, making elements clickable again

0.2.8 (5 April 2016)

  • FIX: embed over HTTPS

0.2.7 (5 April 2016)

  • Remove unnecessary rewrites for /static/iframe.html, this is only needed by wazimap-za.

0.2.6 (4 April 2016)

  • FIX: cleanly hide homepage map on small (<768px) screens

0.2.5 (2 April 2016)

  • Change the way geometries are loaded in comparison views
  • Simplify template overrides in wazimap templates
  • Fix link to GitHub repo in homepage template
  • Add map back to the homepage [#4]
  • DOCS: update deployment documentation and example files
  • DOCS: make note that we recommend running securely over HTTPS

0.2.4 (8 March 2016)

Django models have changed in this release. You will need to run migrations with python manage.py migrate.

  • FIX: content_type for robots.txt
  • Remove numpy as a dependency
  • Remove unused osm_area_id from geo models
  • Add root_level to geo_data
  • Don’t include root level geography when computing geography full names
  • Add optional long_name attribute to geo model
  • Change profile maps JS to make it easy to override

0.2.3 (15 February 2016)

  • FEATURE: use Google place search to find places
  • FEATURE: new country_code config setting
  • FIX: redirect to slugged URLs when possible
  • BREAKING: Data tables now default to one table for all geo levels
  • Include deployment file examples in the deploy directory
  • Remove uservoice.js

0.2.2 (15 February 2016)

  • Initial release

Indices and tables