An interactive map of the Sri Lanka census data.
Made by Alex Storer on behalf of the World Bank.
https://github.com/alexstorer/
- app
- deploy
- ingest
This contains the final code, data and libraries, ready to be uploaded to your desired server.
To run locally, make sure you run the server.py
file with Python version 2.7. This will create a local webserver that you can use to preview the app by going to the following directory:
file:///Users/astorer/Dropbox/Work/sl/app/choropleth-example.html
You will need to change the /Users/astorer/Dropbox...
portion to the path on your local machine.
deploy/deploy.sh
My simple script for deploying the app. Presently, the app is being deployed to my personal S3 space on Amazon, which is publicly accessible here:
http://s3-us-west-2.amazonaws.com/worldbank-srilanka/choropleth-example.html
deploy/deploy_zip.sh
This script will zip the app up and copy it to s3. It is located here:
http://s3-us-west-2.amazonaws.com/worldbank-srilanka/app.zip
To ingest all of the source files and build the necessary resources for the app, run this script
ingest/ingest.sh
This directory is where we will put the shapefiles, data and scripts to process the data in the right way for the app.
We need the following data in order to build the app:
- CSV files (in
CSV
directory) template_cutpoints.csv
- Shapefiles
For the CSV files, we expect the following format:
,,,,Census of Population and Housing - 2012,,,
,,,,Table 104: Population by Province/ District/ DS Division/ GN Division and Sex,,,
,,,,Province/ District/ DS Division/ GN Division,,Sex,
a1p,a1,a3,a4,,Total Population,Male,Female
0,0,0,0,Sri Lanka,20359439,9856633,10502806
1,0,0,0,1,5851130,2848649,3002481
2,0,0,0,2,2571557,1229795,1341762
3,0,0,0,3,2477285,1194540,1282745
Any variation from this format will lead to unexpected errors.
We also require a list of the cutpoints for each variable in each table, as well as whether to omit it. This file should take the following form:
Filename,Title,Variable,Label,Omit,Province,District,DSD,GND
CPH2012_GNT_100.csv,Total Population,Total Population,,,"{""min"":1000000,""max"":6000000,""categories"":5}","{""min"":0,""max"":2500000,""categories"":10}","{""min"":0,""max"":350000,""categories"":7}","{""min"":0,""max"":30000,""categories"":6}"
CPH2012_GNT_101.csv,Ethnicity of Population,Total Population,,True,,,,
CPH2012_GNT_101.csv,Ethnicity of Population,Sinhalese,,,"{""min"":0,""max"":100,""categories"":10}","{""min"":0,""max"":100,""categories"":10}","{""min"":0,""max"":100,""categories"":10}","{""min"":0,""max"":100,""categories"":10}"
For shapefiles, we need the following directories in the data/shape
directory:
DSDivisions/
Districts/
GNDivisions/
Provinces/
In each of these, we expect files labeled like:
Provinces.dbf Provinces.prj Provinces.shp
Provinces.qpj Provinces.shx
If they are named differently, as in, Provinces_2016_new.shp
, the script will break.
The following scripts are present in the scripts directory, and are called by the ingest.sh
script, in this order.
split.R
-- split the .shp files using Rconvert_topojson.py
-- convert the .shp files to topojson and move themconvert_csv_to_json.py
-- make sure you change the source of the directory containing the CSV files- Outputs data.json and fields.json
var_json.py
-- splits the data.json into variablesconvert_table_list.py
-- converts tables.csv (table ordering) to tables.jsonconvert_fields.py
-- adds cutpoints, changes field labels. make sure you change the output directory. it puts tables, fields and cutpoints in the output directory.split_json.py
-- splits the data.json into smaller components for faster loadingmake_summary_gnd.py
-- summarize the GNDs so it's easy to display names
This is the contents of the ingest.sh
script.
RScript split.R; python convert_topojson.py; python convert_csv_to_json.py; python var_json.py; python convert_table_list.py; python convert_fields.py; python split_json.py; python make_summary_gnd.py; mv *.json ../app/data