SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


«««12345»»»

Geocoding with SQL Server 2005 Expand / Collapse
Author
Message
Posted Wednesday, May 03, 2006 7:52 AM


SSCarpal Tunnel

SSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal Tunnel

Group: General Forum Members
Last Login: Yesterday @ 8:55 AM
Points: 4,976, Visits: 3,925

Thank you for your time.

It took me a while, but finaly I got the winform app to work ...

Address = Address.Replace(" ", "+")
Dim myWebSvc As New System.Net.WebClient
Dim myWebProxy As New System.Net.WebProxy("xxx.yyy.zzz.aaa", myport)
'use current users credentials
myWebProxy.Credentials = System.Net.CredentialCache.DefaultCredentials
myWebSvc.Proxy = myWebProxy
'Build the URL
URL = "http://api.local.yahoo.com/MapsService/V1/geocode?appid=" + appid + "&street=" + Address + "&city=" + City + "&state=" + State

'Get the XML response
Dim xmlread As New XmlTextReader(myWebSvc.OpenRead(URL))

 

Now, can I just port this code into the CLR-proc for sql2005 ?



Don't drive faster than your guardian angel can fly ...
but keeping both feet on the ground won't get you anywhere

Very usefull HowTo for forums:
- How to post Performance Problems
- How to post data/code to get the best help
Post #277152
Posted Wednesday, May 03, 2006 7:54 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Friday, August 21, 2009 12:15 PM
Points: 55, Visits: 138
Theoretically.....we'll see. Give it a try.
Post #277154
Posted Wednesday, May 03, 2006 8:08 AM


SSCarpal Tunnel

SSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal Tunnel

Group: General Forum Members
Last Login: Yesterday @ 8:55 AM
Points: 4,976, Visits: 3,925

It works just fine now !!

Imports System

Imports System.Data

Imports System.Data.SqlClient

Imports System.Data.SqlTypes

Imports Microsoft.SqlServer.Server

Imports System.Net

Imports System.Xml

Imports System.Xml.XPath

Imports System.Text

'Zie http://www.sqlservercentral.com/columnists/cBunch/geocodingwithsqlserver2005_printversion.asp

Partial Public Class StoredProcedures

<Microsoft.SqlServer.Server.SqlProcedure()> _

Public Shared Sub SPCLR_Geocode(ByVal Address As String, ByVal City As String, ByVal State As String, ByVal appid As String)

'Declare string for URL

Dim URL As String = "_empty_"

'Yahoo test appid ""

Try

'Replace the spaces in your Address with the plus sign to build the URL needed for Yahoo

Address = Address.Replace(" ", "+")

'Provide webservice and proxy and credentials

Dim myWebSvc As New System.Net.WebClient

Dim myWebProxy As New System.Net.WebProxy("xxx.yyy.zzz.aaa", myport)

'use current users credentials

myWebProxy.Credentials = System.Net.CredentialCache.DefaultCredentials

myWebSvc.Proxy = myWebProxy

 

'Build the URL

URL = "http://api.local.yahoo.com/MapsService/V1/geocode?appid=" + appid + "&street=" + Address + "&city=" + City + "&state=" + State

' http://api.local.yahoo.com/MapsService/V1/geocode?appid=myyahooappid&street=8&city=seattle&state=wa

'

'Get the XML response

Dim xmlread As New XmlTextReader(myWebSvc.OpenRead(URL))

' Create the record and specify the metadata for the columns.

Dim record As New SqlDataRecord(New SqlMetaData("Latitude", System.Data.SqlDbType.VarChar, 100), New SqlMetaData("Longitude", System.Data.SqlDbType.VarChar, 100), New SqlMetaData("GeoCodedAddress", System.Data.SqlDbType.VarChar, 100), New SqlMetaData("GeoCodedCity", System.Data.SqlDbType.VarChar, 100), New SqlMetaData("GeoCodedState", System.Data.SqlDbType.VarChar, 2), New SqlMetaData("GeoCodedZip", System.Data.SqlDbType.VarChar, 10), New SqlMetaData("GeoCodedCountry", System.Data.SqlDbType.VarChar, 10), New SqlMetaData("Precision", System.Data.SqlDbType.VarChar, 100), New SqlMetaData("Warning", System.Data.SqlDbType.VarChar, 100))

'Create a Dataset to hold the results

Dim ds As New DataSet()

'Fill the DS

ds.ReadXml(xmlread)

' Mark the begining of the result-set.

SqlContext.Pipe.SendResultsStart(record)

For Each Rij As DataRow In ds.Tables(0).Rows

If Rij.ItemArray.Length.Equals(9) Then

record.SetString(0, Rij.ItemArray(0).ToString())

record.SetString(1, Rij.ItemArray(1).ToString())

record.SetString(2, Rij.ItemArray(2).ToString())

record.SetString(3, Rij.ItemArray(3).ToString())

record.SetString(4, Rij.ItemArray(4).ToString())

record.SetString(5, Rij.ItemArray(5).ToString())

record.SetString(6, Rij.ItemArray(6).ToString())

record.SetString(7, Rij.ItemArray(7).ToString())

record.SetString(8, Rij.ItemArray(8).ToString())

'If the response did not include an error, then the array length is 0-6

Else

record.SetString(0, Rij.ItemArray(0).ToString())

record.SetString(1, Rij.ItemArray(1).ToString())

record.SetString(2, Rij.ItemArray(2).ToString())

record.SetString(3, Rij.ItemArray(3).ToString())

record.SetString(4, Rij.ItemArray(4).ToString())

record.SetString(5, Rij.ItemArray(5).ToString())

record.SetString(6, Rij.ItemArray(6).ToString())

record.SetString(7, "Precision Good")

record.SetString(8, "No Error")

End If

 

SqlContext.Pipe.SendResultsRow(record)

Next

' Mark the end of the result-set.

SqlContext.Pipe.SendResultsEnd()

Catch ex As Exception

Throw New Exception(String.Format("AT_Error with url [{0}]", URL.ToString), ex)

End Try

End Sub

End Class

 

 

Deploy and

EXECUTE SPCLR_Geocode '8','Seattle','WA','MyYahooAppId'

 

Thank your for your support !



Don't drive faster than your guardian angel can fly ...
but keeping both feet on the ground won't get you anywhere

Very usefull HowTo for forums:
- How to post Performance Problems
- How to post data/code to get the best help
Post #277182
Posted Wednesday, May 10, 2006 4:13 PM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Tuesday, May 12, 2009 9:46 AM
Points: 14, Visits: 40

I recoded it as a TVF function. I modified the logic to return multiple matches, and did some code changes.

I used it to geocode 500 addresses, and it found 4 that returned multiple matches (3 of them at the address level). While it doesn't happen often, and the accuracy of the first match is probably good enough for most applications, if you want high accuracy, you should analyze all the matches returned.

Some examples of how to call it (the first 3 return multiple matches)

select * from dbo.fncGeocodeByAddress ( '245 Union Ave', 'Pueblo', 'CO', '81003', 'YahooDemo' )
select * from dbo.fncGeocodeByAddress ( '7920 Stone Creek Dr.', 'Chanhassen', 'MN', '55317', 'YahooDemo' )
select * from dbo.fncGeocodeByLocation ( 'Springfield', 'YahooDemo' )
select * from dbo.fncGeocodeByAddress ( '100 East 42nd Street', 'new york', 'NY', Null, 'YahooDemo' )
select * from dbo.fncGeocodeByAddress ( '701 First Street', 'Sunnyvale', 'CA', Null, 'YahooDemo' )

By making it a TVF, it is more flexible, and you can call it in a select using CROSS APPLY

insert tmpAddressGeocode
  (
      [AddressGuid]
      ,[Latitude]
      ,[Longitude]
      ,[Address]
      ,[City]
      ,[State]
      ,[Zip]
      ,[Country]
      ,[Precision]
      ,[Warning]
  )
select top 100
  addr.AddressGuid,
  gc.*
from
  dbo.AddressTbl addr with (NoLock)
  left join tmpAddressGeocode alldone with (NoLock)
    on alldone.AddressGuid = addr.AddressGuid
  cross apply dbo.fncGeocodeByAddress (
    addr.StreetAddress1 + coalesce (' ' + StreetAddress2,''),
    addr.CityName,
    addr.State,
    addr.Zip,
    'YahooDemo' ) gc
where
  alldone.AddressGuid is Null

SQL Installation

CREATE FUNCTION [dbo].[fncGeocodeByLocation](@Location nvarchar(4000), @appid nvarchar(100))
RETURNS TABLE (
 [Latitude] float Null,
 [Longitude] float Null,
 [Address] nvarchar(80) Null,
 [City] nvarchar(35) Null,
 [State] nchar(2) Null,
 [Zip] nvarchar(10) Null,
 [Country] nvarchar(35) Null,
 [Precision] nvarchar(10) Null,
 [Warning] nvarchar(1024) Null
)
AS EXTERNAL NAME [YahooGCSQL].[sqlservercentral.Geocoding.YGeocode].[GeocodeByLocation]
GO

CREATE FUNCTION [dbo].[fncGeocodeByAddress](
  @Address [nvarchar](4000),
  @City [nvarchar](4000),
  @State [nvarchar](4000),
  @Zip [nvarchar](10),
  @appid nvarchar(100)
)
RETURNS TABLE (
 [Latitude] float Null,
 [Longitude] float Null,
 [Address] nvarchar(80) Null,
 [City] nvarchar(35) Null,
 [State] nchar(2) Null,
 [Zip] nvarchar(10) Null,
 [Country] nvarchar(35) Null,
 [Precision] nvarchar(10) Null,
 [Warning] nvarchar(1024) Null
)
AS EXTERNAL NAME [YahooGCSQL].[sqlservercentral.Geocoding.YGeocode].[GeocodeByAddress]
GO

Function Assembly Code

using System;
using System.Collections;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.Web;
using System.Xml;
using System.Xml.XPath;
using System.Text;
using Microsoft.SqlServer.Server;
namespace sqlservercentral.Geocoding
{
  public sealed class YGeocode
  {
    /// <summary>
    /// Prevent instances by making constructor private
    /// </summary>
    private YGeocode()
    {
    }
    /// <summary>
    /// Query Yahoo Geocode API, and return an enumeration of results. Have to provide City and State or Zip or Location. Data is extracted for SQL by FillRow
    /// </summary>
    /// <param name="AppId">Yahoo application ID</param>
    /// <param name="StreetAddress">Street Address</param>
    /// <param name="City">City name</param>
    /// <param name="State">US State 2 letter abbreviation</param>
    /// <param name="Zip">Zip Code</param>
    /// <param name="Location">Free form address/location</param>
    /// <returns>Enumeration of addresses found</returns>
    private static IEnumerable GeocodeStart(SqlString AppId, SqlString StreetAddress, SqlString City, SqlString State, SqlString Zip, SqlString Location)
    {
      // Based on
http://www.sqlservercentral.com/columnists/cBunch/geocodingwithsqlserver2005.asp
      try
      {
        // Build the URL
        StringBuilder sb = new StringBuilder();
        sb.Append(AppId);
        if (!StreetAddress.IsNull)
        {
          sb.Append("&street=");
          sb.Append(StreetAddress);
        }
        if (!City.IsNull)
        {
          sb.Append("&city=");
          sb.Append(City);
        }
        if (!State.IsNull)
        {
          sb.Append("&state=");
          sb.Append(State);
        }
        if (!Zip.IsNull)
        {
          sb.Append("&zip=");
          sb.Append(Zip);
        }
        if (!Location.IsNull)
        {
          sb.Append("&location=");
          sb.Append(Location);
        }
        // Replace spaces and # with +
        // string Address = System.Web.HttpUtility.UrlEncode(StreetAddress);
        // It seems that HttpUtility.UrlEncode is not available here (Compact Framework?), so we have to do it "manually"
        // If need be, could use the implementation available at
http://www.devx.com/vb2themax/Tip/19352 (just check for the = and & )
        sb.Replace('#', '+');
        sb.Replace(' ', '+');
        // Prepend the API URL
        sb.Insert(0, "
http://api.local.yahoo.com/MapsService/V1/geocode?appid=");
        String URL = sb.ToString();
        // Get the XML response
        XmlTextReader xmlread = new XmlTextReader(URL);
        // Create a Dataset to hold the results
        DataSet ds = new DataSet();
        // Fill the DS
        ds.ReadXml(xmlread);
        // Return rows of data
        return ds.Tables[0].Rows;
      }
      catch (Exception ex)
      {
        // Catch any errors here
        ;
      }
      return null;
    }
    [SqlFunction(FillRowMethodName = "FillRow",
      TableDefinition="[Latitude] float Null, [Longitude] float Null, [Address] nvarchar(80) Null, [City] nvarchar(35) Null, [State] nchar(2) Null, [Zip] nvarchar(10) Null, [Country] nvarchar(35) Null, [Precision] nvarchar(15) Null, [Warning] nvarchar(1024) Null")]
    public static IEnumerable GeocodeByAddress(SqlString StreetAddress, SqlString City, SqlString State, SqlString Zip, SqlString AppId)
    {
      return GeocodeStart(AppId, StreetAddress, City, State, Zip, null);
    }
    [SqlFunction(FillRowMethodName = "FillRow",
      TableDefinition = "[Latitude] float Null, [Longitude] float Null, [Address] nvarchar(80) Null, [City] nvarchar(35) Null, [State] nchar(2) Null, [Zip] nvarchar(10) Null, [Country] nvarchar(35) Null, [Precision] nvarchar(15) Null, [Warning] nvarchar(1024) Null")]
    public static IEnumerable GeocodeByLocation(SqlString Location, SqlString AppId)
    {
      return GeocodeStart(AppId, null, null, null, null, Location);
    }
    [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Design", "CA1021:AvoidOutParameters")]
    public static void FillRow(object obj,
      out SqlDouble Latitude, out SqlDouble Longitude,
      out string Address, out string City, out string State, out string Zip, out string Country,
      out string Precision, out string Warning )
    {
      // XML elements and attributes are case sensitive.
      // Access by name to protect against API changes. It is slower, but not so bad compared with the cost of calling an Internet API
      DataRow row = (DataRow)obj;
      if (!String.IsNullOrEmpty(row["Latitude"].ToString()))
        Latitude = Convert.ToDouble(row["Latitude"].ToString());
      else
        Latitude = SqlDouble.Null;
      if (!String.IsNullOrEmpty(row["Longitude"].ToString()))
        Longitude = Convert.ToDouble(row["Longitude"].ToString());
      else
        Longitude = SqlDouble.Null;
      Address = (!String.IsNullOrEmpty(row["Address"].ToString())) ? row["Address"].ToString() : null;
      City = (!String.IsNullOrEmpty(row["City"].ToString())) ? row["City"].ToString() : null;
      State = (!String.IsNullOrEmpty(row["State"].ToString())) ? row["State"].ToString() : null;
      Zip = (!String.IsNullOrEmpty(row["Zip"].ToString())) ? row["Zip"].ToString() : null;
      Country = (!String.IsNullOrEmpty(row["Country"].ToString())) ? row["Country"].ToString() : null;
      // Better way to detect precision/warning columns w/o going back up to the columns collection? w/o resorting to exception trapping? w/o assuming indexes?
      // Alternate way = Precision = (row.ItemArray.Length >= 8) && !String.IsNullOrEmpty(row["precision"].ToString()) ? row["precision"].ToString() : null;
      Precision = (row.Table.Columns.Contains("precision")) && !String.IsNullOrEmpty(row["precision"].ToString()) ? row["precision"].ToString() : null;
      Warning = (row.Table.Columns.Contains("warning")) && !String.IsNullOrEmpty(row["warning"].ToString()) ? row["warning"].ToString() : null;
    }
  }
};
 



Post #279093
Posted Thursday, May 11, 2006 4:37 PM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Friday, August 21, 2009 12:15 PM
Points: 55, Visits: 138
Great addition! Thanks!
Post #279449
Posted Thursday, February 08, 2007 9:19 PM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Yesterday @ 2:08 PM
Points: 2, Visits: 10
I've just found this tutorial today, and it's awesome! I've contacted Corey directly, but I'm also throwing this one out there for anyone that might catch it. My DBA is out of town, and if I wanted to execute the Assembly and immediately stick the lat/lon into a table from within an Insert trigger for a newly created row, what would that look like?
Post #343661
Posted Friday, February 09, 2007 12:16 AM


SSCarpal Tunnel

SSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal TunnelSSCarpal Tunnel

Group: General Forum Members
Last Login: Yesterday @ 8:55 AM
Points: 4,976, Visits: 3,925

- it would look like bad code !

- keep in mind you have no control over the availability of the counterparty !

- If you put this clr-routine into a trigger, it will execute "in transaction", meaning if the clr-routine fails, your transaction will fail, so your actual insert statement will be rolled back !!!

- You might want to put this in an assynchrone process (e.g. a sqlagent job).
   maybe within the insert trigger just feedin the pk into a "trigger"-table
   on which your sqlagent job will start from to get all the geo-info
   and will report bad info.

 



Don't drive faster than your guardian angel can fly ...
but keeping both feet on the ground won't get you anywhere

Very usefull HowTo for forums:
- How to post Performance Problems
- How to post data/code to get the best help
Post #343673
Posted Friday, February 09, 2007 6:59 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Friday, August 21, 2009 12:15 PM
Points: 55, Visits: 138
alzdba - This is exactly what we do on our production system. We have another table where a bit gets set on or off based on business rules, and then a sql job comes & picks up the bits=1 and runs the CLR process. This is what I would recommend as well.
Post #343748
Posted Friday, February 09, 2007 12:37 PM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Tuesday, May 12, 2009 9:46 AM
Points: 14, Visits: 40

I also agree that making a CLR call from within a Trigger is a bit risky (your DBA probably won’t appreciate you doing this while he is away). With this assembly, I've encountered a sporadic error, presumably due to connectivity issues, that locks the assembly, and forces you to have to reinstall it. I haven't been able to reproduce it in a controlled environment to try and handle it within the assembly.

Msg 6260, Level 16, State 1, Line 1

An error occurred while getting new row from user defined Table Valued Function :

System.InvalidOperationException: Handle is not initialized.

System.InvalidOperationException:

   at System.Runtime.InteropServices.GCHandle.FromIntPtr(IntPtr value)

.

The statement has been terminated.

That aside, to handle multiple matches by averaging the geocodes, using the TSV function above, the code would look like

select
  addr.AddressGuid,
  gc.*

into #tmpAddressGeocode
from
  dbo.AddressTbl addr with (NoLock)
  cross apply dbo.fncGeocodeByAddress (
    addr.StreetAddress1 + coalesce (' ' + StreetAddress2,''),
    addr.CityName,
    addr.State,
    addr.Zip,
    'YahooDemo' ) gc
 

update addr set

  LatitudeRad = LatAvg

  , LongitudeRad = LongAvg

  , LatestGeocodeAttemptDate = getutcdate()

  , GeocodePrecision = Prec

  , GeocodeMultiCnt = GeoCnt

from (

  select

    AddressGuid,

    LatMin = Min (LatitudeRad),

    LongMin = Min (LongitudeRad),

    LatMax = Max (LatitudeRad),

    LongMax = Max (LongitudeRad),

    LatAvg = Avg (LatitudeRad),

    LongAvg = Avg (LongitudeRad),

    Prec = dbo.Concatenate (distinct geos.[Precision] ),

    GeoCnt = count (*)

  from #tmpAddressGeocode geos with (NoLock)

  group by AddressGuid

) geos

join dbo.AddressTbl addr on addr.AddressGuid = geos.AddressGuid

where

  -- 5 miles or less when multiple points

  dbo.DistanceGreatArc ( LatMin, LongMin, LatMax, LongMax ) * dbo.EarthRadiusCalc ( 1, null ) <= 10.0

 

drop table #tmpAddressGeocode

 

I’m storing other information, like when the geocode was obtained, a list of the precision levels, and how many matches were found. I'm also using a couple other CLR functions. dbo.Concatenate is included in the SQL CLR programming samples, and dbo.DistanceGreatArc is used to calculate the distance (there are multiple implementations you can find on the Internet, either as a CLR or SQL function). The check for the radius to be within 5 miles is a bit redundant, and you could omit it.

 

If you’re geocoding a considerable number of records, you probably want to mange the temp table a bit more, and use a top to limit the number of geocodes you perform on each run (see earlier post).




Post #343916
Posted Wednesday, February 21, 2007 9:04 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Tuesday, February 27, 2007 3:58 AM
Points: 1, Visits: 1
I have followed the tutorial and everything went according to plan but when i execute the SP, it says Command Completed Successfully but I dont get any rows returned. I'm using the same test data as in the tutorial and I've tried other addresses too.

Any ideas whats wrong or where I should start troubleshooting?

Thanks
Post #346415
« Prev Topic | Next Topic »

«««12345»»»

Permissions Expand / Collapse