Geoprocessing with Python

This week was a little less stressful than last week.  Having used geoprocessing tools in ArcMap, I think I had a more clear understanding of what I needed to accomplish my task.  This week we needed to create XY coordinates from an existing point feature class.  Next we needed to create a 1000 meter buffer around the points, and then dissolve that buffer.  Our script needed to print all messages from each of the geoprocessing steps.  These are the result messages as they appear when using Python in ArcMap .

Script results

I must admit, I always get a little nervous when I see someone else has taken a different route to get the desired results.  I have flashbacks to six grade algebra and the nuns at Catholic school yelling "You have to show your work!"

Python Fundamentals Part 2

What the holy hell happened!  Man o man, did I hit a wall this week.  The first few week of this course and I'm zipping through it thinking this is going to be a great summer and then week 4 happened.  I realize, that I am a visual learner.  I need to see the action in order for it to click in my head, I need to be able to dissect the working code in order to understand the ins and outs.

This week, we were given the start of a script.  The import random module was already loaded at the start of the script.  We then needed to find some errors in a block of code that would allow the scipt to run properly. Luckily, the lecture helped to clear this up.  And then step three happened.  In step three we were to create a list from scratch using 20 random numbers between 0 and 10.  I knew some of the modules that I needed to use, and individually I could run them with proper results, however, the while loop, threw me.  I had a very hard time comprehending how to include the append function and the random integer. Once I got past step 3, things seemed to move along a little better.  Once I had my list created, I needed to create a random unlucky number that would be counted and then removed from the list.  I'm not sure why this while loop was easier to understand than the last one, perhaps because I was able to dissect the previous one.  Our final results were to include the original block of code that we fixed, our created list, a statement declaring how many of the unlucky number would be removed from our list; if any, and our final list with the unlucky number removed. Below are the result of the my script.

Script results

Here's hoping my clicker works better next week.

Mapping the Herrin Massacre Potter's Field

The Herrin Massacre took place in Williamson County Illinois in 1922.  A chance meeting between a historian looking for the gravesite of a WWI vet killed in the massacre and a geospatial scientist initiated the mapping of the Herrin Massacre Potter’s Field and the entire Herrin City cemetery.   The project took approximately four years to complete, mostly because of researching the data that was not previously digitally recorded or was missing completely.  Data was input into a sql database and a custom enterprise geodatabase was created and shared through use of various services via ArcGIS for Server.  

A GIS model was created from an old hand drawn map using known dimensions.  This model was used as a template since it was just the conceptual drawing.  Using data created and recorded over the history of the cemetery, the team was able to map the routine of each of the sextons over the years to approximate the location of the potter’s field. 

Detailed typography, headstone outlines, and satellite imagery were processed in ArcGIS 3D Analyst.  This gave the team the ability to locate unmarked sites by analyzing the subtle changes in the ground surface. 

Though these combined processes, the team was able to sumise the lot in which the potter’s field was potentially located.  Their continued research and proven geographical, historical, and geospatial data was enough evidence to convince the city to allow excavation.  On Nov 12, 2013 the first vault and coffin were discovered and confirmed based on historical accounts describing the coffins and photos taken prior to burial.

What I found most interesting about this article was the use of new age technology to bring back the past.  Even though the records were missing, the team was able to analyze the available data to correctly locate burial sites that lay unnoticed for nearly 90 years.  Being able to analyze subtle difference in ground depression to find the unmarked graves could not have been accomplished using just satellite imagery or aerial photography.

http://www.esri.com/esri-news/arcuser/spring-2014/finding-the-victims-of-the-herrin-massacre

Herrin Massacre Information

Python Fundamentals Part I

This week we started compiling basic Python code. The key to understanding code is to understand the psuedocode.  For example, where do you start, what are the steps in the middle, and what should the end result be.  For our lab, the psuedocode would look like this:

1. Create a string for your name
2. Split that string into a list
3. Print last name
4. Find the length of the last name
5. Multiple the length of the last name by 3
6. Print length of last name times 3

This took some trial and error. As I was going along in my script, I would use the print command after each line of code to ensure that I was getting the correct response.  It took a while to wrap my head around the concept of creating the variable lastNameLen.  I had to realize that I had to tell the code what the variable lastNameLen would be equal to the last list item of my last name.  Once it clicked, the rest of the script seemed to fall into place.

The image below is the results of my script. You can see that my last name Williams is printed (step 3) and so is the number 24, which is the multiple of 8 (the number of letters in Williams) x 3.

Script Result

Geoprocessing In ArcGIS

For this weeks lab we created a geoprocessing model that was used to create a script that was used to create a tool.  The results of the model can be seen below.  Creating the model was a first a bit confusing. The instructions were very basic, however, it made me think a bit more about what I was doing to get to the desired results. Once I got in the right frame of mind, setting up the model was rather simple.  When written out it seems rather long and complicated, but it is mostly repetitive. 

The model I created was used to: 

1. Clip the Soils layer to the Basin layer

2. Select the Soils that were classified as "Not Prime Farmland"

3. Erase the previously selected soils from the basin layer

Basin soil with non prime farmland removed

The steps I tool to create the model included:

1.       Open ArcMAP

2.       Add basin.shp and soils.shp to the map via the Add Data tool or ArcCatalog

3.       Right click Mod2_jwilliams.tbx (previously created)

4.       Click on Model

5.       Drag the Basin and Soil layers into the Model window

6.       Open ArcToolbox

7.       Expand the Analysis Toolbox and the Extract tool set

8.       Click and drag the Clip tool to the Model window

9.       Click the Connect tool in the Model window

10.   Click on the soils layer and then click the clip tool

11.   Choose Input Feature

12.   Click on the basin layer and then click the clip tool again

13.   Choose Clip Feature

14.   The Clip tool and output feature class shapes will change to solid colors and the output feature class will be renamed to soils_clip.shp

15.   Right click the clip tool and run it – this will populate the output which is necessary for the next step

16.    Go back to the ArcToolbox

17.   Clip and drag the Select to the Model window

18.   Click the Connect tool

19.   Click on the soils_clip.shp  and then click the Select tool

20.   Double click the Select tool and configure the expression as "FARMLNDCL" = 'Not prime farmland'

21.   The Select  tool and output feature class shapes will change to solid colors and the output feature class will be renamed to soils_clip_select.shp

22.   Right click the select tool and run it – this will populate the output which is necessary for the next step

23.   Go back to the ArcToolbox

24.   Expand the Overlay toolset

25.   Click and drag the Erase tool to the Model window

26.   Click the Connect tool

27.   Click the Basin layer and then click the Erase tool

28.   Select Input Feature

29.   Click the  soils_clip_select.shp output and then click the Erase tool

30.   Select Erase Feature

31.   The Select  tool and output feature class shapes will change to solid colors and the output feature class will be renamed to soils_clip_select_erase.shp

32.  Right click the select tool and run it

33.  You can add the layers to the map from the Add Data tool or the ArcCatalog

Intro to Python

Welcome back!  For me, this is the final leg of my GIS Certification journey.  Programming is not something I have ever wanted to do, but over the years I have seen where knowing programming can be beneficial.  I know a little bit of HTML and some SQL and Access proramming, but at this point, that is the extent of my programming knowledge.  Python is an open source programming language that is supplied with ArcGIS.  The current version available with ArcGIS 10/10.1 is Python 2.7.  The textbook is telling me that it is easy to learn, we shall see.  Already I can see that Python is a very helpful language.  Say for example you begin to type a command that begins with the letter P.  Python will prompt you with a list of commands that begin with the P.

One thing I have learned in my brief programming experience has been the importance of syntax.  One missing or misplaced double quote can ruin your whole day.  Something I hope to gain from this course is a greater understanding of the roll scripting can play in GIS and how it can help me in my day to day use of ArcGIS.

Our first assignment was to open, examine and run a script that creates a module folder for each week of our GIS Programming course.  The results of the script can be seen in the image below.

GIS 4930 Final Project

This should be my final week of GIS certificate courses, but due to a scheduling error last summer, I will be taking one more class this coming summer. This final week of GIS 4930, Special Topics, was once again focused on the forestry category.  I have to say, this has not been my favorite topic, but I think a lot of that comes from all of the confusion that I have been faced with over the last six weeks.  It certainly didn't make it any easier, but it was not nearly as bad as statistics.

I chose to find the maple sap production potential of a 1,400 hectare study area in New Brunswick Canada. Finding the production potential included using an array of ArcGIS tools including ArcToolbox tools Merge, Multiple Ring Buffer and Overlay, joining feature classes and tables, Select by Attribute, and Field Calculator.  Also used to assist in analysis was the use of Pivot Tables in Microsoft Excel.

Use the links below to see a brief abstract and the results of my analysis.

Abstract (Word format)
Abstract (PDF format)
Deciding Maple Syrup Production Potential (pptx with audio)
Deciding Maple Syrup Production Potential (PDF - no audio)