Module 8 - Cursors....

My Ever-Growing GIS/Programming Library

This module was an excellent view of cursors.   Cursors are essential to the toolbox of any programmer that works with data of any sort.   I was surprised to see the similarities in concept to the T-SQL environment of Microsoft SQL Server.

Notes to Future Self:

• When debugging, remember not only to look at the line(s) of code where you THINK the problem is, but also at other lines that may execute.  I ran into a problem where I was convinced that the UpdateCursor just wasn’t working and then I went back a few lines and saw that my “query” was limiting the results that the operation was running.

• When debugging, it is helpful to comment out lines and re-add them in smaller quantities to focus in like a laser on the area you are concerned with.
• Also when debugging, remember to try various ways of running the program… PythonWin, IDLE, and by double clicking the script.   Insight might be gained from each one of these attemps.
• Don’t reinvent the wheel…   The ArcGIS Online Help is GREAT on each command, including code snippets that might help as well.   Also, be sure to be competent at using your search engine of choice to look up both arcpy as well as general python fixes to get around any issues you are experiencing.

Module 7 - Listing Tools

This week’s lesson taught us some skills that will prove to be essential to our success in the real world. We learned how to formulate our approach to creating a solution by asking some questions about the problem. 
We learned skills such as path manipulation, DESCRIBE methods, Listing tools, Usage methods, and even learned how to check and toggle license usage.
Notes to Future Self:

• When struggling with one of the tools try it in different editors to get further hints.   I had problems with the ExtractByMask function in Python Win, and couldn’t figure out why it would work in the Interactive and even in IDLE.    Turns out I needed to have my Scratch workspace defined.   Once I match my code to have the same scratch as my ArcGIS environment variable had, it was much more reliable.
• Use the Online Help.   When working on a tool it is great to use the online help to grab code from.   No need to reinvent the wheel if they already have a working version of it.
• When you get frustrated, walk away for a bit…   You’ll be surprised when the idea comes to you (hint: it usually isn’t when you are conveniently near a computer….)
• Break down tasks to their simplest elements.   When working with the ExtractByMask, I ended up having to create a separate Python script to try to use just the bare minimums to get the function to run to a hard coded file.    I took the lessons learned here back to my assignment.
• I found this assignment to be great system for using the data that we learned in the exercises.   I struggled at points, but kept at it and was able to finally make the program work.   That is tough to find assignments that are tough, but not impossible and I think this week was a good example of the challenge level that the assignments should present.
• One problem I had with the lab was on page 3 where it asked us to save a raster out to the GDB with a period in the file name.   I was unable to do this, so just left the .TIF of the file name.

GIS Programming - Module 6 Assignment

I am still polishing some of the code required for this week's assignment, but wanted to go ahead an make my blog posting about some of the "Lessons Learned"

PART III: Module Summary
Write a summary of what you learned this Module. Try to come up with specifics from each exercise and assignment. List any problems (and fixes) you encountered, and any other notes you feel would be helpful to your ‘future-self’. Post this summary to your blog.

Lessons Learned:

Exercise 6a:

• It is important to understand the pros and cons of relative and absolute paths (especially in terms of sharing your data with others)
• Relative paths cannot span multiple drives
• Absolute paths work best on static data
• Relative pathnames work best when sharing data (remember you have to provide the data as well!)

Exercise 6b:

• You can effectively share data with others by proper directory planning and relative paths.

Exercise 6c:

• It is common to provide users with the tools and scripts in addition to the data
• A recommended folder structure should have a data, scripts, scratch, and docs subdirectory
• Getting messages out to the results window can be done with the arcpy.AddMessage() function
• The ArcGIS tools have a section of their help documentation dedicated to sample code snippets
• Password protection of embedded code is a great feature to help programmers maintain intellectual property (or keep the so called “power users” from messing up your code then entering a help desk ticket, but I digress….)

Exercise 6d:

• Packaging data to deliver to clients is pretty straightforward in ArcGIS 10.x
• You can use the Consolidate Map tool (which stores the references to data)
• You can use the Package Map tool (which stores the data itself).   Think of the benefits of this when you are in the field with a laptop 20 miles from the nearest cell phone tower….

Assignment

The assignment felt like the real world kinds of things I need to do every day which is:

• Understand the assignment
• Research the tools
• Struggle through a couple of false starts
• Have that “a ha” moment where things start clicking

Module 05 - Debugging and Troubleshooting Python Scripts 

Summary:

I found this week’s assignment to have a lot of relevance in the “real world”.    The majority of the code you write will not work properly the first time you run it.   This is where your ability to debug and correct the course of the program is your most valuable asset.  

This week we learned several valuable skills towards that goal:

·         Benefits of using IDLE or PythonWin in different situations to improve debugging.

·         Identifying some of the most common errors and how to troubleshoot them.

·         The importance of indentation (and the difficulty in tracking down bugs related to indentation)

·         How to use the Alt-G command to jump to a specific line number to debug (very useful!!!)

·         How to use PDB to see behind the scenes.

·         How to use breakpoints and watches to better understand what is going on in your program.

·         How to understand the error messages that ArcGIS will display.

·         The use of some arcpy funtions like GetMessageCount() and GetMessages()

·         How to use Python to Programmatically buffer a point using the Geoprocessing tools.

I think you could attend an entire semester on this phase of the programming process and still not know all there is to know.   I have found that the ability to debug code is one of the most important skills you have as a programming professional.   It gets even more difficult when you have to debug code that someone else wrote.  

I picked up a couple of good books that might help the people in this class learn a bit more about ModelBuilder and Python.

They are :

Getting to Know ArcGIS ModelBuilder – by David W. Allen  ISBN-10: 1589482557

And

A Python Primer for ArcGIS by Nathan Jennings -  ISBN-10: 146627459X

In particular, I really like the David W. Allen Book.

Module 4 - Review of Python Syntax

Greetings fellow Argonauts,

This week’s assignment was a very good overview of some of the critical Python skills we will need to improve to (eventually) become GIS Development Masterminds.  I am lucky enough to have a background in programming and can imagine that if you had never programmed before, that these exercises might be difficult and frustrating.   All I can do is to tell you to hang in there !   Completing a project (in the real world as in a class) is all about iterative problem solving.   Rarely will something work the first time and it usually will take some research time to solve the problem (Thanks Google!).  Trust me, your programming skills WILL improve with each problem that is placed before you.   Your mind may not be used to thinking like the computer so it is almost like a workout at the gym for your brain.   At first you won’t be able to do much, but with persistence, you will amaze yourself (and your coworkers).   So hang in there, use the message board, use the Internet, talk to others you know that might be struggling with similar problems.

Now on to what I learned this week:

1.      I REALLY prefer Python WIN over IDLE.    I leaned on the copy/paste and find/replace functionality heavily, especially during the 4th assignment

2.      You should familiarize yourself with some of the basic Python functions.  One of the things I’ll do when I learn a new language is look for a Python Quick Reference or Cheat Sheet that gives me a list of the basics.

3.      Lists are a very useful data type that I wasn’t really familiar with from my other programming languages.  I can foresee them being very useful in ArcGIS when we start thing about layers, features, etc...

4.      Some of the skills you are learning in this class are going to help you learn other languages (if you choose to go that path).   The “import module” for example and the “.” Hierarchy of functions is in several languages

5.      You WILL see CSV files in the real world quite a bit, so that lesson is very useful.

6.      Reading and Writing to text files is also a very valuable skill that will come up quite a bit in your development duties.

7.      Loops are another great place to use a cheat sheet to remember the format (although the indentation format Is kind of growing on me)

 

Intro to Python - Module 3

Summary of what I learned:

Exercise 3a:

·        While the Shell is good for a lot of things, working on multiple-line code segments is not one of them.

·        One of the big benefits of working in a Text Editor is the global Find/Replace functionality that we have become accustomed to.

·        Notepad ++ can be temperamental.   Whenever I tried to work with it, it would muck up my paths to my S: drive not only from within Notepad ++ but even from Windows Explorer.

Exercise 3b:

·        It is important to recognize places where you will need to use both single and double quotation marks, such as in contractions used in text strings.

·        Embedded variables are an extremely yet simple way to personalize your text strings.

·        Variable naming can lead to a lot of confusion.   You want to make sure to check for proper case sensitivity to ensure that you reference the correct variable.

Exercise 3c:

·        Be sure to take full advantage of the auto-complete functionality that ArcGIS gives you with Python.

·        In addition to auto-complete, Python also offers extensive drag and drop functionality to create code for you.

·        Don’t forget to import your arcpy library !

·        Be mindful of your environment settings.   These are key to finding all the awesome data results you have created !

Assignment 3a:

·        Python has great tools for automating repetitive tasks such as directory creation.

·        Lists are a very useful concept that can help reduce “code-bloat”

Assignment 3b:

·        String literals are very helpful in this assignment.  These really help minimize the time it takes to format your strings

·        You can use a “while” loop to make this code more streamlines

·        Using the data in a LIST format really helps with readability

Challenge Exercise:

·        For this exercise, you can set the parameters using the ArcGIS interface rather than trying to do it from within the Python script.

·         You can then read these parameters as follows:    inYear = gp.GetParameterAsText(0)

This was a fun assignment and challenge exercise.    I am looking forward to what comes next !

SWOPPAT Announcements - New Tool

It has come to our attention that there has been a great demand for a tool that will allow SWOPPAT staff to add two additional columns to their shapefiles.   Acres and HectaAcres.    We at SWOPPAT GIS Programming have made this possible with the new SWOPPATAAHACT "The Save the World One Python Program at a Time Acre and HectaAcre Calculation Tool" (™ and © pending).

This tool can be accessed from your toolbar as follows:

This tool can save you countless hours.   Lets say for example that you create 3 shapefiles per week.  And lets say that calculating the Acres and HectaAcres tool you 3 minutes each time.    That would be 9 minutes a week.   When you look at this over the course of the year, you see that this tool can save you 468 minutes a year.    That is over 7 and a half hours a year, almost an entire work day !



GIS Programming Orientation

The programming language I am really looking forward to learning this semester is Python.   Python is an open-source project and it is freely usable and distributable, even for commercial use.   It strives to achieve “readability” which means it is relatively easy to read someone else’s code to determine the functionality.  It is "portable" which means that the code can be used on several platforms with very little modification.    It is "versatile" because it can be used as a scripting language but also supports fully object oriented programming as well.   ESRI has leveraged the strength of Python into its product line by implementing the ArcPy package which can “perform geographic data analysis, data conversion, data management, and map automation” ¹    Python is the successor to VBA (Visual Basic for Applications) as the primary development language within the ESRI product line.   ModelBuilder can be used to generate Python code from a GUI (Graphic User Interface) view.  

An example of a program that could be generated using Python would be a drive time estimator.   You could click at any point on a map and Python could then compute the drive time to a particular location.  Python would leverage the geographic information that ArcGIS can provide to retrieve these results.

It is a great time to learn Python with the release of 10.1 coming out next month. 

Some pages I found to help fellow Argonauts include:

• http://wiki.python.org/moin/BeginnersGuide   
• http://video.esri.com/watch/1120/the-power-of-python
• http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//000v00000001000000

 Sources:

¹ –“A quick tour of ArcPy” by ESRI Help http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//000v00000001000000