Saturday, January 17, 2009

Geo-coding Your Constituents

Everyone likes maps & map mash-ups. There seems to be something very appealing about being able to see things visually compared to textually. Since we launched our Interactive Class Map about 8 months ago it's become one of the longest viewed pages on our website. The average constituent spends almost 4 minutes and 30 seconds doing nothing but panning and zooming around the map looking at their classmates. We have had a simple NetCommunity Directory Part that displayed the same information just in list form for years. It's still widely used but users don't spend anywhere near the amount of time viewing it as they do the Interactive Class Map. 

In this post I'm going to cover what is really the most important part of working with maps on your NetCommunity website, geo-coding. So, what is geo-coding? It's the process of taking an address and finding its latitude and longitude. Once you have done that it becomes much easier to display that location on a map. Just a warning, this is going to be a bit technical. 

I hacked out a quick and dirty webservice that I'll make available at the end of this post for download. It uses the REAPI to get a constituent’s address from RE and then uses Google to find the lat/lng. I'll take you through the code step by step. 

CAUTION: This method is NOT the best approach if you are planning on processing a lot of records. Google imposes a 15K limit on the number of requests per day from a single IP address. They also frown upon making requests too quickly and may block you for that. If you are planning on using this method to process your entire database you really should batch it up in small chunks over the course of a few days. Also be sure your timer makes each request to Google a few seconds apart. The code I'm going to show you does not do that, but it wouldn't be too hard to add. Since you have the source code you should be able to add that functionality.

So let’s get started.

The first step is to create a new Web Service project in Visual Studio 2008. The idea behind this project is that other NetCommunity custom parts could use it. If you are planning to bulk geo-code your constituents you could also use it in something like a Windows Service that runs over a few days to process your whole database.

Now that the webserivce project has been created we want to go ahead and start setting up the two functions we're going to use.

We're going to need to import these references;
Imports System.Web.Services
Imports System.Web.Services.Protocols
Imports System.ComponentModel
Imports Blackbaud.PIA.RE7.BBREAPI
Imports System.Xml
Imports System.Net
Imports System.IO

You'll probably need to add Blackbaud.PIA.RE7.BBREAPI to your project. If you don't have the REAPI you'll need it. Otherwise you'll have to modify this code to use the NetCommunity API which is free. I know the REAPI better for this stuff which is why I used it. 

    ''' this method will take a REID and get the address
    ''' then it will use google maps to lookup that address address
    ''' note: the value of the returned array with the lat/lng is nothing if
    ''' either the address coudln't be geocoded or it wasnt' specific enough to get back a single result
    Public Function getConstitLatLng(ByVal sREID As String) As ArrayList
        Dim oConstit As New CRecord 'this is the constit object
        Dim aLatLng As New ArrayList(2) 'we'll be returning this.
        Dim RE7 As REAPI = initAPI() 'this is located in the source. be sure to change up your serial, username, pass and db number. 
        Dim oAddress As CConstitAddress 'address object
            With oConstit
                .Init(RE7.SessionContext) 'init
                .Load(sREID) 'load the passed in REID
                For Each oAddress In .Addresses 'loop over all the addresses
                    If oAddress.Fields(ECONSTIT_ADDRESSFields.CONSTIT_ADDRESS_fld_PREFERRED) = -1 Then 'grab the prefered one, yes I know it's -1 trust me on this -1 = true for some strange reason in the BBREAPI...I don't ask questions if it works. 
                        'pass the retrieved criteria to the seekCoords method
                        aLatLng = getCoords(oAddress.Fields(ECONSTIT_ADDRESSFields.CONSTIT_ADDRESS_fld_ADDRESS_BLOCK) & " " & oAddress.Fields(ECONSTIT_ADDRESSFields.CONSTIT_ADDRESS_fld_CITY) & ", " & oAddress.Fields(ECONSTIT_ADDRESSFields.CONSTIT_ADDRESS_fld_STATE) & " " & oAddress.Fields(ECONSTIT_ADDRESSFields.CONSTIT_ADDRESS_fld_POST_CODE))
                        Exit For
                    End If
            End With
        Finally 'close down your stuff...always
        End Try
        Return aLatLng 'return the value
    End Function

Now lets look at the getCoords function. This one just takes the address we retrieved from the API and uses a Google web service to return the lat/lng. One important thing to note is that it will return Nothing if the address can't be geo-coded OR if the address isn't specific enough and Google returns more than one possible result. I did this for simplicity but you can modify the project so things are handled how you want them to be. 

Private Function getCoords(ByVal sAddress As String) As ArrayList
        Dim stream As IO.Stream = Nothing
        Dim doc As XmlDocument
        Dim req As HttpWebRequest
        Dim aLatLng As New ArrayList
        If sAddress <> "" Then 'we check to make sure that we have an address
            req = DirectCast(WebRequest.Create("" + sAddress), HttpWebRequest) 'pass the address to googole and assign the results to req
            'now we parse up the results 
            'generally speaking kml = xml so we can use the std handlers in .net for this
            'why google doesn't just call it xml is beyond me
            Dim response As HttpWebResponse = DirectCast(req.GetResponse(), HttpWebResponse)
            stream = response.GetResponseStream()
            Dim sr As New StreamReader(stream)
            Dim text As String = ""
            text = sr.ReadToEnd()
            doc = New XmlDocument()
            longitudes = doc.GetElementsByTagName("longitude")
            latitudes = doc.GetElementsByTagName("latitude")
            'for simplicty we only want to worry about results that have 1 possible result
            'none, means the address wasn't geocodable
            'multiple means the address wasn't specific enough
            If longitudes.Count = 1 Then
            End If
        End If
        Return aLatLng
    End Function

So that's pretty much it. You can download the full project below, enjoy. 

Don't forget to Join the LinkedIn and/or Facebook group and be sure to follow me on Twitter.

Thursday, January 15, 2009

What's cooking

I'd like to first apologize for the lack of substantive posts recently. The combination of the Holidays, work projects and an increasingly energetic toddler has left me with less time to blog. I think the energetic toddler is mostly to blame.

I wanted to take a moment to update everyone about what we have been working on over at lately with NetCommunity. As Blackbaud continues to build on the API the possibilities of what can be done within NetCommunity expands. Here are a few of our recent projects.

Pledge (reoccurring gifts) Payment - This application is relatively simple. It reads the logged in constituents active pledges from RE and then displays to them their outstanding & next payment(s). It also displays any EFT pledges they have on the books and the date of the next charge. They can choose which pledge(s) to pay and how much. The transaction is processed through IATS, written to their RE gift record and a receipt is sent to them. There is also another custom part that sits on our main Online Giving page that reminds a constituent they have an outstanding pledge before they make a new gift. 

Top Content (aka Top Five) - This was a fun project. Like most of you, we have a landing page where we display to our constituents personalized and targeted content after login. We were looking to expand our offerings on that page and this application came about. After login it will search the stats table in NetCommunity and display three items in an AJAX tab panel control;
  • The top five new articles on our site in the last 7 days.
  • The top five non-news article pages in the last 30 days.
  • The top five pages the logged in user has visited in the last six months.
This last piece is very useful for users who are trying to get back to a page they were on recently. 

Event Feeder - We use events pretty heavily especially around reunion season. This is a vey nice little tool that shows the logged in constituent the future events they are attending. When the click on the event they are able to see the last 10 people who have also signed up for that event. The "event feed" is also subscribe-able via RSS so a constituent could be instantly notified (the ttl is 10 minutes) when someone new signs up for the event. We haven't used this whole lot yet but as we get closer to spring that'll change. 

So those are a few of our more recent projects. We have a few big ones cooking right now and I'll be sure to showcase them once we've put them into the wild. I'm not making the source code for any of the above projects available but if you have any questions I'd be happy to answer them. 

Lastly, 18 Jan marks the 1 year anniversary of my first blog post. I hope I'm providing NetCommunity users with valuable information. Over the last year the NetCommunity developer network has exploded and I only hope that continues. Thanks for reading. 

Tuesday, January 6, 2009

2008 Online Giving Trends

The subject is a bit off topic from what I usually write but I found it pretty interesting.

Steve MacLaughlin at Blackbaud did a short blog post on some highlights on the 2008 giving trends Blackbaud has observed. It's worth taking a look.