eCognition Tutorial: Classifying a QuickBird Image
Jared Newell and Pontus Olofsson Dept. of Geography and Environment, Boston University
1. Setting up an eCognition project A Quickbird Image of the BU area can be found on /data/bs16/Bostonpan/. Download the contents of this folder and put it in a local folder on your computer. (There should be two tiff images; a four‐band Quickbird image and an NDVI image). Open eCognition Developer from your start menu. Choose the ‘Rule Set Mode’ from the prompt. Click File > New Project. A screen will appear asking you to import image layers. Navigate to the BostonQB.tif and NDVI.tif images and open them. Here we can specify a project name, thematic data, and which bands to use. Most importantly we can select a subset from the image (‘Subset Selection’ button); subsets are particularly useful if you want to try out various rulesets on a large image. For this tutorial we’ll use the whole 2,300 x 2,300 pixel image. Click the edit button for each layer and rename the ‘layers’ so you can differentiate between them. Quickbird layers 1,2,3,4 are B,G,R,NIR; rename the NDVI layer accordingly. Click OK to load the data. The main Developer screen will open up and you should see your image. To change the layer composition of the bands, click on the top toolbar. What’s neat about eCognition is that you can mix more than three bands. For example you can display an image in true color RGB, and then for added vegetation enhancement add in the NIR or NDVI in the green band. You can also do layer weighting, and give weights for each band (use the right click to increase, and the left to decrease). Play around with the layer mixing and weighting. I like this mix:
1
2. Segmentation eCognition is a unique image processing software, as it uses an ‘object based’ processing and classification scheme rather than traditional pixel‐based methods. What this means is the software runs segmentation algorithms to group similar and adjacent pixels together into polygons or ‘image objects’. In eCognition, the segmentation algorithm uses a bottom‐up, low‐to‐high value region growing method coined the ‘Fractal Net Evolution Approach’ (Baatz and Schape 1999). First, a seed pixel is selected from within the image, and then neighboring pixels with similar values are consolidated into objects based on the smallest growth of heterogeneity, and defined by spectral variance and geometry of the object (Benz et al. 2004). This region growing process stops when the object growth exceeds a user defined scale parameter. The segments (image objects) are then classified through fuzzy logic based object metrics and statistics such as pixel values, texture, shape, and size. Let’s learn a bit about this whole segmentation thing… First make sure you have the proper windows open. We’ll be playing around with some processes and algorithms, so let’s load up the processes tree window. Go to View>Develop Ruleset View. Some windows should show up on the right hand side of your screen. eCognition uses a hierarchical process tree when classifying or manipulating any image data. The processes are defined by a ‘Parent‐Child’ concept; parents being the root and children being the 2
branches. First let’s start by adding a new Parent. Right‐click in the process tree and ‘Append New’. Give it the name ‘coarse segmentation’. Right click on the ‘coarse segmentation’ in the process tree and ‘Insert Child’. Here we are going to start with the usual first step in object‐based image classification ‐ Image segmentation. In the ‘Edit Process’ window that pops up, select ‘Multiresolution Segmentation’ from the Algorithm dropdown. Multiresolution segmentation is probably the most widely used algorithm in eCognition. It has three main settings or criterion that define how the image will be segmented: ‘scale parameter’, ‘Shape’, and ‘Compactness’. Scale parameter will determine the size of the segments; it’s an arbitrary number not related to scale or pixel size. The Shape parameter ranges from 0‐1.0. A value of 0.1 will create segments heavily weighted on color (e.g. differentiating between tree species, rock type, urban substrate), while a value of 0.9 will segment based on shape (e.g. buildings, roads, stands of trees). The compactness parameter defines how condensed and compact image objects will be; a value of 0.1 will give you squiggly and irregular segments (maybe useful if you’re looking at extracting hydrologic data or roadways), while a value of 0.9 will yield blocky and compact segments (good for extracting buildings and patchwork type land cover). Additionally you can give weights to your image layers (ranging from 0‐ 1.0). Think about what you want to classify (veg, buildings, land cover), and adjust accordingly. Give the segmentation level a name ‘coarse seg’. You can try a few different segmentations; the settings below will yield very small segments heavily weighted on color.
Play around with the different parameters and see what you get. After each segmentation you’ll have to delete the layer
.
3
3. Classification Let’s try and make a land cover classification from the image. In many image scenes there are different scales used for analysis (think forest > stand > individual tree). In some cases there are areas in images that you might want to process with greater detail. Conversely there are areas in a scene that may not need much analysis at all (fields, water).
3.1 Coarse Scale Segmentation Take a look at the Boston image; what broad land cover categories do you see? There are trees, buildings, roads, water, etc, overall a pretty complex and detailed landscape. At a very general level there’s water and land. What I’ve found helpful is to pre‐classify large homogenous land covers before concentrating on the fine‐scale stuff. In this case it’s best to classify the water first and remove it from further analysis. We all know that shadow and water can easily be confused at a fine scale, so this will let us deal with those pesky shadows at a later time. Let’s run a coarse resolution segmentation on the image, and try to extract the Charles River the best we can. The BU bridge and RR tracks on the left side of the image divide the river, so we can’t get too coarse with our segmentation… I found that these parameters work the best for delineating water and land. Delete your previous segmentation and use these settings:
4
Hit execute. It will take a few seconds or up to a minute. You should get an image and segmentation that looks something like this:
You can change various visualizations by selecting the buttons to the right of the ‘eye’ on the toolbar. Move your mouse over the various buttons to learn what they do. You can turn image objects on and off, view classifications, see classification samples, etc with these buttons. The two buttons to the right of the eye are the ones I use the most. They toggle between image view and classification view. Now that the image has been segmented let’s start with a very basic classification, separating the land and water. First let’s create two classes land and water. To so do simply ‘right click > insert class’ in the class hierarchy window (top right) and add the two classes. Give them appropriate colors and labels. We are now going to use the standard ‘Nearest Neighbor’ classifier to create this first classification. From the top toolbar click classification > nearest neighbor > edit standard nearest neighbor feature space.
5
The window that pops up has two sides to it; on the right are all the various parameters you can use for classifying your image. Toggle through some of them… there are a lot! eCognition is unique in that it classifies the image based on image objects; there are very few classification inputs that are pixel based (e.g. no maximum likelihood, isodata, etc). Since we’re doing a very basic classification, there’s really no need to use many object features in the NN classifier. Take a look at the image objects over the Charles River vs. those over land. Obviously the land has higher reflectance and a generally positive NDVI, while the water is dark and uniform in color. Also the image objects over the water are much larger. Let’s add a few object features to the classifier and see what we get for a classification…
Double clicking various feature space parameters on the left hand side of the nearest neighbor window, adds them to your NN classifier. Here I’ve added mean values for brightness and NDVI, standard deviation for B and NDVI, and image object area. This should do a good job of classifying water, given 6
that water objects are low in brightness and NDVI, have little variation in reflectance or color, and are generally larger in size compared to land objects. Add these inputs and click ok. Let’s now add these inputs to our two classes
Click Classification > Nearest Neighbor > Apply Standard NN to Classes. Double click or use the all button to bring the land and water classes to the selected classes box. Now we’re going to add a new process (the classification). In the process tree click on your parent (coarse segmentation), and insert a new child.
7
Then in the Algorithm dropdown box, under “Basic Classification’, select classification.
This process window will then pop up. Make your active classes land and water; keep everything else the same. Click ok. 8
We will now select samples for land and water. If you don’t already have the samples toolbar up, add it to your toolbar (View > Toolbars > Samples).
You can use both the tool with the arrow (select objects one at a time), or the cross sight (brush tool) to select objects for classification. Click either button and then the land class under the Class Hierarchy window (upper right window) to begin to assign samples.
Here I used the brush tool to get samples of land. I tried to get a wide variety of samples (veg, impervious, shadow, etc). Do the same for water.
9
Let’s now execute the classifier. Right click the classification in the process tree and hit execute.
10
Looks pretty good! If you get some misclassified objects, you can go back and add more samples for the appropriate class. Just use the sample tool, select the misclassified samples and then hit execute again. We now have a very basic land/water map. At this point you might want to save your project. Go to File > save as, and find an appropriate spot to save. Your project will be saved as a .dpr file; a similar sort of project file as an ArcMap .mxd file. The next step in classification is to break the land class into more detailed land cover classes. This will require another segmentation…
3.2 Fine Scale Segmentation We’re going to want to re‐segment the land area into smaller segments. We’ll shoot for a minimum mapping unit of around 2‐5m (trying to capture, buildings, roads, trees). Add a new parent to the Process tree; right click on the ‘coarse segmentation’ > Append new. Call this process ‘fine segmentation’ – click ok. Then add a child to the process. In this case we are going to want to add another multi‐resolution segmentation, and re‐segment only the land objects. In the edit process window select multiresolution segmentation from the algorithm dropdown, apply it to the level ‘coarse seg’, and select a class filter for just ‘land’ – keep all the other parameters on the left the same. On the right, we’re going to specify the level usage as ‘use current’ and input layer weights and parameters as so:
11
You can try different scale and segmentation parameters, hitting execute will delete the previous layer. I found the parameters above to work pretty well in segmenting different land covers and image objects. Hit execute and see what you get.
Next we’ll start classifying the fine‐scale segments. To classify these much smaller image objects, we need to first ask ourselves: What do we want to classify? We’ll stick to a fairly basic classification system (Tree, Grass, Bare Ground, Building/Impervious, Road, and Shadow). Although we only have six classes, there are very different types of each class (e.g. bright and shadowed sides of a tree, white and black buildings, green and dry grass). We’ll want to try and capture this variability in our classification. I’ve found that the best way to classify a heterogeneous and spectrally diverse hi‐res scene is to start with many classes, and then merge them together when you’re ready to deliver a final product. Give a very thorough look over your image, and think about what classes you might want to include. Also think about how the classifier might ‘see’ the image objects – you can’t expect the classifier to differentiate between road and building, without giving it some measure of shape and compactness. Start building different classes by right clicking in the Class Hierarchy window. We’ll start with various tree classes… I added five‐classes based on the general ‘brightness’ and mixing seen with different image objects. I also tried to keep the class colors representative of true color.
12
I put together a classification that looks something like this:
Similar to what we did before, we are going to want to edit the NN feature space; go to classification > nearest neighbor > edit standard nearest neighbor feature space. Given that we’re classifying both vegetation, as well as urban substrate, I added a few geometric object features, such as length/width, rectangular fit, and shape index. Hopefully these will help in classifying some of the road and building objects.
13
Set your feature space up similarly (or different!) and click ok. Then apply the feature space to your classes: Classification > Nearest Neighbor > Apply Standard NN to Classes (select all the classes except ‘land’ and ‘water’). Next we will start collecting samples for each class. Use the sample toolbar and find good representative samples for each class. It can be helpful to use the ‘image object information’ window to see the layer values within each object; once you start classifying, you can also look at membership probabilities of each object). My samples looked something like this:
14
Make sure you make use of the image object information window, and observe class probabilities of various objects (the classification tab), as well mean object spectral values (the features tab). This can help you make correct samples, and also see if your classes are too similar. By clicking an unclassified image object, you can see class probabilities which are based on your classifier and samples you’ve chosen. Add a new child under your fine segmentation parent. For the Algorithm select ‘Classification’. In this step we are only going to classify the fine segmentation and the ‘land’ class, so our active classes will be everything except ‘land’ and ‘water’. Input these parameters:
Hit execute. It may take a couple minutes to classify depending on how many object features you put in the NN classifier. My classification looks something like this:
15
Take a look at your classification. There will some error in it, most likely between the ‘road’ and ‘built’ classes. It can be helpful to use a variety of visualization tools such as transparencies (the slider in lower left of the screen), splitting the window (under the window pulldown), as well as the standard layer, object, and classification buttons within the top toolbar.
16
Here, I split the screen vertically and used a ‘side by side’ view (both under the window pulldown), and then made a transparent classification overlay on the left and a classified polygon view on the right. This makes it easier to see where there’s error in the classification.
4. Manual Editing One of the last steps in creating a classification is manual editing and clean‐up. This is usually the most tedious and time consuming step. Thankfully it’s much easier to do clean‐up in eCognition compared to the roundabout ENVI/ArcMap process that’s typically used. What we are going to do here is add a new parent in the process tree called cleanup – right click in the process tree > append new > name the parent ‘cleanup’ > and drag it to the bottom of the ruleset.
Right click on ‘cleanup’ > insert a new child > select manual classification towards the bottom of the pulldown.
17
Then select one of your classes on the right window. Here I’m using a ‘bare’ class. If you want you can use a brush tool for editing. I like to use a small weight of 2, so I can select both individual objects and have the functionality of using the brush.
Add a ‘Manual classification’ for each class.
18
After you have added all the clean‐up processes, start with a section of your map. I’ll start at the lower right by Fenway Park. One of the major mis‐classifications can be seen between the road and built classes. Right click on the ‘road’ manual classification in the process tree and hit execute. Your cursor should turn to a brush tool; you can then manually paint and reassign image objects. We’ll clean‐up some of the errors in this portion of the image…
19
Here I did some cleanup on the roadways, although it’s not perfect, it’s better than what we had before. At this point you could even add additional classes (parking lots, sidewalks, highway) and manually edit them in. We’ll keep it simple for now though.
I spent a couple minutes cleaning up this section of the image. Overall, it looks pretty good! It’s usually worth giving the image a solid look over, just remember you’ll never get a 100% perfect product. 20
5. Export Result Once you’re happy with your classification you can export it to a variety of formats (tif, img, shp). This can be helpful if you’re going to use the classification in Arc or ENVI.
Under the Export pulldown, select ‘export results’. For export type choose ‘raster file’, content type ‘classification’, and format ‘erdas img’. Img files allow you to keep class names and color schemes ‐ this is typically what I use.
Hit export and save it in a proper directory. 21
The image can now be viewed and analyzed in Arc. To get rid of the different brightness classes, I often group classes and use the reclassify tool in Arc.
There’s a lot to learn in eCognition, this tutorial just scratches the surface. You can use this software for any sort of remotely sensed data. I’ve processed LiDAR data, for vegetation analysis. You might even be able to add in LiDAR data to differentiate between building heights and different types of urban substrate – use your imagination! Unfortunately there aren’t many good Definiens tutorials out there so the best way to learn is to simply play around with the software. Good luck and have fun!
Learn more M. Baatz and A. Schäpe, Object‐oriented and multi‐scale image analysis in semantic networks, Proceedings of the 2nd international symposium on operationalization of remote sensing, ITC, Enschede (1999), pp. 16–20.http://www.definiens.com/ U.C. Benz, P. Hofmann, G. Willhauck, I. Lingenfelder and M. Heynen, Multi‐resolution, object‐oriented fuzzy analysis of remote sensing data for GIS‐ready information, ISPRS Journal of Photogrammetry and Remote Sensing 58(3‐4) pp. 239‐258 http://www.ecognition.com/ and http://www.definiens.com/ 22
