Home

Welcome to the Q-and-A wiki!

This is a wiki set up to ask questions and post answers regarding the technical challenges associated with using open-source geographic information systems.

How can I add a question?

Go into editing mode using the edit button at the top-right. Then scroll the end of the document and add a new line, starting with h3, or three hash-tags. Type a short title to your question. Then start a new line and add more information.

Hint for model builder

You can right click an algorithm and deactivate temporarily, causing that part of the model to be saved, but not run. This can be helpful for testing and model development.

I can't calculate new geometries based on my custom projection

First, make sure your new projection is really in the spatial_ref_sys table, and that all the parameters are correct. Second, make sure your geometry column is registered in the geometry_columns table, and that it's registered to the new SRID you're trying to project to. Third, make sure the geometry you're translating from really exists, and has data. Check this with the astext() function and check it's registration in the geometry_columns table. Finally, make sure the geometry you're translating from has all the same SRID's as the column it is registered to. Check this with the srid() function (similar to 'astext()', but it tells you the SRID for each record in the geometry column).

I can't map a field from a database with Graduated Symbols in QGIS

This is probably caused by the field being recognized as a data type other than a number. Double check the fields in the layer's properties to make sure it's the right data type. Then double check the column's data type in the database. Remember to specify the data type when you create a new column in the database by adding REAL or INTEGER to the ALTER TABLE ... ADD COLUMN ... statement.

If you have calculated a numeric field on the fly in a SELECT query and added the results to the QGIS map canvas, the data type might be registered as TEXT, making it impossible to use for mapping numerical data. In this case, you're better off adding a new column in DB Manager with the correct data type and using UPDATE to make a permanent record of the calculation. See Pre Lab 3, where we calculated DistJoin based on information selected from another table.

Why won't my new column in the database show up in QGIS?

If you first add a table to QGIS from DB Manager and then you change the schema of that table in DB Manager by adding a table, the new column does not show up in the QGIS attribute table. It doesn't seem to matter if you re-connect the database or add new versions of the layer, or even disconnect the database, close QGIS, start a new QGIS project, and re-connect the database. This problem even persisted through making a new copy of the database! This observation led me to believe there must be some metadata about the table stored in the database itself... and there is, in the geometry_columns_field_infos and geometry_columns_statistics metadata tables.
You can update these with a SpatiaLite function: SELECT UpdateLayerStatistics('table-name'). For example, if I need to update columns/fields for a table named Districts, you can use SELECT UpdateLayerStatistics('Districts') to update all the metadata for that table. The next time you add Districts to the map, it will have all of its columns. If you're wondering why SpatiaLite wasn't programmed to update these statistics automatically, read the programmers' discussion about this issue here. Believe it or not, similar issues crop up in ArcGIS as well-- sometimes you have to use a calculate statistics tool to update the statistics for a raster dataset before the dataset can be displayed or used in analysis.

Why won't my shapefile layer import from QGIS to SpatiaLite using DB Manager?

SpatiaLite creates a primary key column for each table it imports, and tries to name it pk by default. However, if your shapefile was previously exported from SpatiaLite, then it already has a pk column, and the conflict causes DB Manager to crash. There are two solutions to this:

1. delete the pk column from the shapefile before you import it. -or-
2. while using the import layer tool in DB Manager, specify the primary key to use cat as a primary key, since it is of type integer and all the values are unique.

Why doesn't my custom projection work?

This question could come from observing that the custom projection will not display in QGIS or from observing that a new column with the custom projection is not properly storing transformed coordinates. The most common solutions I've seen are:

1. Make sure you haven't accidentally switched latitude and longitude.
2. Make sure you have included one and only one datum, either in the form of +datum, or a combination of +ellps and +towgs84. Also note that these seem to be case-sensitive, i.e. wgs84 is not the same as WGS84.

Questions about Lab Eight

1. If you're really stuck on this lab, please take some time away from the computer to do the readings.
2. The general idea is to find & choose the best way to deal with the SLC-off gaps before running any kind of classification tool. Between patching any combination of granules, closing gaps, or closing gaps with spline, choose the best option.
3. It looks like "clustering for grids" is now named "k-means clustering for grids".
4. Saving grids in SAGA: the easiest way is to set any of the output grid settings when you run a module. Outputs usually have "<<" before the output grid option. (and likewise, ">>" for any input grids). You can also save the data for grid files by right-clicking the relevant grid in the Data panel of SAGA. You will also be prompted to save any unsaved grids when you close the SAGA gui, where you can simply check any of the unsaved grids to get them saved upon closing (this is a riskier option, in case SAGA were to crash during the middle of your work).
5. Trouble with optical calibration (see below, and also, you may try running Image Concatenation and Optical Calibration in the Orfeo program rather than through QGIS. You'll have slightly more control over the modules by going through Orfeo directly).

Optical calibration in Orfeo returning a raster of 0's

If you run optical calibration and get a result of all 0's, you most likely saved the output as an integer data type. Since optical calibration converts raw integer radiance values at the satellite sensor to percentage reflectance values at the top of the atmosphere, the output raster must be of type double or float to accommodate decimal numbers.

After exporting a SAGA grid or stack of grids to the GeoTIFF format, the result is all white

This may not really be an error at all. Look at the actual values stored in your new GeoTIFF by using the identify tool in QGIS or moving the cursor around the scene in Orfeo. It's likely that the background values are -999999 (the default nodata value in SAGA) and your legitimate data values are still in the range you'd expect (from 0 to 1). Since nodata information is lost in the file conversion, hopefully all you need to do is set the new nodata value to -999999 (or whatever the background values may be) using the Orfeo No Data Management module.

Adjacent Landsat scenes look very different (one darker or lighter than the other) after Optical Calibration

There are three possible explanations:

1. The brightness of the scenes actually did change due to real changes in the landscape or atmospheric conditions.
2. Optical calibration did not work correctly: check to make sure all of your values are in a range from 0 to 1, and check to see whether objects that should look very similar over time (e.g. a dark water body or patch of forest) have similar values in both scenes.
3. The landscape didn't change much and optical calibration did work correctly, but the composition of each scene differs enough that Orfeo's automatic histogram stretching makes them appear differently. As in (2), the actual data values for similar features should be similar, in which case you either just need to go ahead to Mosaic the images so that they both render with the same histogram for each band, or you can play around with the percentage cut values in the Color Dynamics tab.

Unexpected results in NoData areas of Landsat scenes (particularly the SLC-off bands of Landsat 7) while using Mosiac in Orfeo

This problem arises because Mosaic may be trying to interpolate values for the unknown locations. Try to use the "simplest composition mode" for the Feathering Method in Orfeo, and make sure that the output image has the same number of bands as the input.

What is going on with Superimpose Sensor?

The video tutorial apparently makes this sound more complicated than it is. You have satellite scenes for two time periods. Let's call them Time A and Time B. They mostly overlap, but not quite, because the satellite orbits drift a little over time. You want to create one unified boolean mask with a 1 wherever there's legitimate data for all the bands for both periods of time. Before you can do that, the two scenes must share the exact same grid system. In SAGA, you'd accomplish this with the Resample module, by resampling one grid (let's say Time A) into the other grid's system (Time B's). You can then use band math, crosstabulation, and other tools on resampled grid A and the original grid B. In Orfeo, the tool is Superimpose Sensor, and it does the same work as RESAMPLE. In the video tutorial, I only make an additional copy of Grid B so that I could start the next section of the workflow in a new folder with consistently named scenes. To be absolutely clear, you resample or superimpose one grid (let's say Time A) into the other grid's system (Time B's), and then move along with the analysis using the resampled/superimposed Time A and the original Time B. The next step is creating a boolean mask so that you can cut out all the extraneous data before you classify.

Google Earth is struggling to render my polygons

In case your cloud mask polygon is extremely complex, Google Earth might have a hard time drawing it. In this case, try looking at video L10a_Resampling and apply this resampling method to your boolean mask grid. This resampling will simplify the grid, after which you may or may not want to use majority filter again to further simplify it. This simplified grid should be easier to convert to polygons and render in Google Earth. So, if anyone has trouble loading their image to google earth, I would recommend going through lab video 10a and following the steps to simplify your polygon so that google earth will actually be able to handle the size of the kml file - Hannah

Clarification of what we need for Monday

Hey everyone, I just spoke with Joe about what he wants us to have for Monday, and I figured I would post it here so that everyone can see it. He went over this in class, but I thought it couldn't hurt to have it written in multiple places.

Our main objective is to create a poster that we can use to tell a story of what happened in our region of interest. Five components we can use to tell this story are:

1. Population Change Maps
2. Population Change Tables
3. Land Cover Classification Maps
4. Land Cover Classification Tables
5. Scatterplot connecting the two (once the first four parts are complete)

In order to draw polygons to create our classes, it is helpful to have different visualizations of the satellite imagery (e.g. RGB, False Color, NDVI). An unsupervised classification can be helpful to make us think & interpret the landscape like the computer will, and guide us to create polygons to differentiate between classes the algorithm might confuse. It is not necessary, however, to have a nice looking map and legend for the unsupervised classification. RGB, NDVI, and other maps can be helpful to tell a story and also they can look cool.

Hope this is helpful! - Jonah

SAGA's supervised classification module returns error: Could not initialize classifier from training areas

Has anyone else encountered the error "could not initialize classifier from training areas" when trying to run supervised classification in SAGA? If so, did you find your error / how to get past it? The error window asks if I want to continue anyway, but then kills the program when I click okay. Thanks! - Jonah

I haven't seen this before, but my best guesses are to try: 1) check that the polygons overlap your satellite image grids on the map and have the same projection system as the grids and 2) check that polygon attribute field you're using for classification has values for each of the polygons.

Supervised classification results in all one class.

This is probably due to having a set of classes that are too similar to one another for the maximum likelihood supervised classifier to figure out. Check a couple of things: 1) if you open the classification statistics file (a kml file, you can open with Notepad ++ ) look at the mean values for each of the classes (I show this in the classification video) and see if multiple classes are confusing because they have very similar means for their bands. 2) if you look at your polygons overlaid with RGB composites of your satellite images, are the polygons including a wide variety of colors/reflectance values? Everything included in a polygon is ultimately averaged to create a spectral signature, so you should try to keep the contents of each polygon fairly homogeneous. 3) At some point you may be trying to create too many training areas and too many classes: with too many classes it's unlikely that all the classes are really generating distinct spectral signatures. Try simplifying your training areas for classification.

Workflow for final project

Hey everyone, I've been compiling a workflow while doing remote sensing analysis for the final project on the page "Land Cover Change Analysis Workflow". It has each step linked to relevant parts from the lab/videos so you wouldn't have to spend too much time looking for specific steps. Please feel free to contribute to it so we can have a comprehensive workflow! - Trisha

Cannot move forward with classification

I had to return to classification because my old image was causing significant trouble and I cannot move forward with the mask for the supervised classification because my k-means clustering map was so bright that clouds are reflecting the same as certain bodies of water and plots of land. Therefore I have classes that include clouds (which I want to exclude) as well as pieces of land and lakes, which I would rather keep (urban areas and shoreline is also mixed with clouds in certain classes). Did anyone else encounter this problem at any point? -Casey

Your final analysis does not depend directly on the results of K-means clustering. It helps to know where the clouds are so that while you're digitizing in Google Earth, you can avoid digitizing training areas over clouds. You can still use the K-means clouds class(es) as rough guide to the locations of clouds with a class that is partially clouds and partially other features: you'll just have to cross-reference that with RGB composites of your study area as you digitize training areas.

Very low kappa statistic from cross-tabulation

If your kappa statistic is very low, you should check if you deselected the "include unclassified cells" option. I just realized I was overlooking this step in my analysis!