Emerillon: Geocoding Names

Current search bar on emerillon is limited:

Current Emerillon Search with GeoNames


I want emerillon to find also streets!

Emerillon search with GeoNames and OSM

And in this post I explain how I’ve done it

Geocoding is the process of finding associated geographic coordinates from other geographic data, such as street addresses, or zip codes.
— Wikipedia Geocoding Page

Let’s go for it!

GeoNames and OSM Nominatim WebServices

In an older post I talked about emerillon and how much I like it. Using this program you notice that it provides geocoding on the search bar using GeoNames web service.

— Enschede search with Geonames
http://ws.geonames.org/search?q=Enschede&maxRows=10

<geonames style="MEDIUM">
   <totalResultsCount>55</totalResultsCount>
   <geoname>
      <toponymName>Enschede</toponymName>
      <name>Enschede</name>
      <lat>52.21833</lat>
      <lng>6.89583</lng>
      <geonameId>2756071</geonameId>
      <countryCode>NL</countryCode>
      <countryName>Netherlands</countryName>
      <fcl>P</fcl>
      <fcode>PPL</fcode>
   </geoname>
   (...)
</geonames>

But GeoNames is not the only geocoding service on the block, I’ll try the same using openstreetmaps Nominatim.

— Enschede search with Open Street Maps Nominatim:
http://nominatim.openstreetmap.org/search?q=Enschede&format=xml

<searchresults 
   timestamp="Thu, 29 Mar 12 21:16:04 +0100" 
   attribution="Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0."
   querystring="Enschede" polygon="false"
   exclude_place_ids="250780,128894028,128949748,20442594,124640085,124640065,20430454,250450,34264043" 
   more_url="http://nominatim.openstreetmap.org/search?format=xml&
   exclude_place_ids=250780,128894028,128949748,20442594,124640085,124640065,20430454,250450,34264043&
   accept-language=en-us,en;q=0.5&q=Enschede">
   <place 
      place_id="250780" osm_type="node" 
      osm_id="45753522" place_rank="16" 
      boundingbox="52.20796798706,52.227971801758,6.8836185836792,6.9036190605164" 
      lat="52.2179694" lon="6.8936189" 
      display_name="Enschede, Regio Twente, Overijssel, The Netherlands, Europe" 
      class="place" type="city" icon="http://nominatim.openstreetmap.org/images/mapicons/poi_place_city.p.20.png"/>
   (...)
</searchresults>

Now a little bit more precise. I’m searching for Hengelosestraat, a street in the city of Enschede.

— With Geonames
http://ws.geonames.org/search?q=Hengelosestraat&maxRows=2

<geonames style="MEDIUM">
   <totalResultsCount>0</totalResultsCount>
</geonames>

— With OSM
http://nominatim.openstreetmap.org/search?q=Hengelosestraat&format=xml

<searchresults timestamp="Sun, 22 Apr 12 16:18:10 +0100" attribution="Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0." querystring="Hengelosestraat" polygon="false" exclude_place_ids="126410125,125653405,126638464,127016981,126944027,126611860,126594997,126638730,126629003,126617923" more_url="http://nominatim.openstreetmap.org/search?format=xml&exclude_place_ids=126410125,125653405,126638464,127016981,126944027,126611860,126594997,126638730,126629003,126617923&accept-language=en-us,en;q=0.5&q=Hengelosestraat">
   
    <place place_id="126410125" osm_type="way" osm_id="144351134" place_rank="26" boundingbox="52.2234916687012,52.2235488891602,6.88805961608887,6.88870525360107" lat="52.2235281294069" lon="6.88838181409509" display_name="Hengelosestraat, Enschede, Regio Twente, Overijssel, 7511JS, The Netherlands" class="highway" type="secondary"/>
(...)
</searchresults>

First comment: OpenStreetMap gives a lot of output, does not look as clean as GeoNames.
Second comment: OpenSteetMap gives results at street level.

Not to say that GeoNames is not as good as osm, simply it serves a different purpose.

Emerillon

Currently Emerillon only uses GeoNames, but I feel natural that it should also use OSM Nominatim. It already use osm maps, so why not osm geocoding?

>>General View

The Search plugin from emerillon is the responsible for querying the geonames webservice, it has a lot of functions but only two of my interest:

search_address Query GeoNames using librest
result_cb Parse the result, fill a GtkListStore and fill the MarkerLayer

Seems fairly easy to extend it to:

search_address_gn Query GeoNames using librest
result_cb_gn Parse the result, fill a GtkListStore and fill the MarkerLayer
search_address_osm Query Open Street Map Nominalis using librest
result_cb_osm Parse the result, fill a GtkListStore and fill the MarkerLayer

I’ll call the two services in an asynchronous fashion. Once finished the same function “result_cb” will be triggered.

/*In function search_address_osm*/
rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error)

/*In function search_address_cb*/
rest_proxy_call_async (priv->call_gn,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_gn),
        plugin,
        &error)

Result_cb controls that both calls have finished (in a way or another). When a call finishes sets its boolean flag to true, when both flags are true the parsing of the result starts.
When all the results are added, the extent of the map is set to show all the markers.

static void
result_cb (RestProxyCall *call,
           GError *error,
           GObject *weak_object,
           SearchPlugin *plugin)
{
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  if (call == priv->call_osm){
    priv->finished_osm = TRUE;
  }
  else if(call == priv->call_gn){
    priv->finished_gn = TRUE;
  }

  if (priv->finished_gn && priv->finished_osm){
    result_cb_osm(priv->call_osm,NULL,NULL,plugin);
    result_cb_gn(priv->call_gn,NULL,NULL,plugin);

    champlain_view_ensure_visible (priv->map_view,
      champlain_layer_get_bounding_box((ChamplainLayer*)priv->layer),
      FALSE);

    priv->finished_gn = FALSE;
    priv->finished_osm = FALSE;
  }
}

>>Using LibRest to call the services

First the full code is shown, then split and explained in more detail.

The full function to call the OSM Nominalis service is the following:

static void
search_address_osm (SearchPlugin *plugin){
  const gchar *query;
  gchar *locale;
  gchar lang[2];
  GError *error = NULL;
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  query = gtk_entry_get_text (GTK_ENTRY (plugin->priv->search_entry));
  locale = setlocale (LC_MESSAGES, NULL);
  g_utf8_strncpy (lang, locale, 2);
  gtk_list_store_clear (GTK_LIST_STORE (priv->model));

  if (priv->proxy_osm == NULL)
    priv->proxy_osm = rest_proxy_new ("http://nominatim.openstreetmap.org/", FALSE);

  /* Cancel previous call */
  if (priv->call_osm)
    g_object_unref (priv->call_osm);

  priv->call_osm = rest_proxy_new_call (priv->proxy_osm);

  rest_proxy_set_user_agent (priv->proxy_osm, "Emerillon/"VERSION);

  rest_proxy_call_set_function (priv->call_osm, "search");
  rest_proxy_call_set_method (priv->call_osm, "GET");
  rest_proxy_call_add_params (priv->call_osm,
      "q", query,
      "format", "xml",
      NULL);

  if (!rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error))
    {
      g_error ("Cannot make call: %s", error->message);
      g_error_free (error);
    }
}

There are 3 things to do:

  • Retrieve the query string from the input form
  • Prepare the call
  • Call and bind the result function

Retrieve the query string
Easy as pie, isn’t it?:

query = gtk_entry_get_text (GTK_ENTRY (plugin->priv->search_entry));

Prepare the call:

 if (priv->proxy_osm == NULL)
    priv->proxy_osm = rest_proxy_new ("http://nominatim.openstreetmap.org/", FALSE);

  /* Cancel previous call */
  if (priv->call_osm)
    g_object_unref (priv->call_osm);

  priv->call_osm = rest_proxy_new_call (priv->proxy_osm);

  rest_proxy_set_user_agent (priv->proxy_osm, "Emerillon/"VERSION);

  rest_proxy_call_set_function (priv->call_osm, "search");
  rest_proxy_call_set_method (priv->call_osm, "GET");
  rest_proxy_call_add_params (priv->call_osm,
      "q", query,
      "format", "xml",
      NULL);

Initially there is a check on the proxy_osm variable, if it’s not initialized I create a new rest_proxy [line 14].
Second, maybe there’s a call still working, cancel it (just in case) [lines 18,19].
After that I prepare a new call [line 21], set the user agent [line 23], the function [line 25], the method [line 26] and the parameters (followed by NULL, as the documentation dictates) [lines 25 to 30].

Bind and Call
There are two services that I want to query asynchronously.

rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error)

The function rest_proxy_call_async will do that. I call with priv->call_osm. The callback function is result_cb (that will take care of the two async petitions). The third parameter is the object where the call will be tied. plugin is the search plugin structure reference to be passed to the callback function. Finally if there’s an error it will be assigned to the error variable.
In the full code this call is inside a conditional check for controlling possible errors.

>>Parsing the result

When the service returns a result the program has to parse it and store the results.

static void
result_cb_osm (RestProxyCall *call,
           GError *error,
           GObject *weak_object,
           SearchPlugin *plugin)
{
  const gchar *answer;
  gint len;
  guint i;
  RestXmlParser *parser;
  RestXmlNode *root, *n;
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  answer = rest_proxy_call_get_payload (call);
  len = rest_proxy_call_get_payload_length (call);
  parser = rest_xml_parser_new ();

  root = rest_xml_parser_parse_from_data (parser, answer, len);

  /* Extract the result count */
  n = rest_xml_node_find (root, "place");

  if (n==NULL)
    {
      GtkTreeIter iter;

      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, 0,
                          COL_SYMBOL, "OS",
                          COL_NAME, _("No result found"),
                          COL_DISPLAY_NAME, _("<b>No result found</b>"),
                          COL_MARKER, NULL,
                          -1);

      if (root)
        rest_xml_node_unref (root);
      return;
    }

  n = rest_xml_node_find (root, "place");
  i = 1;

  while (n)
    {
      const gchar *name, *country, *lon, *lat;
      GtkTreeIter iter;
      ChamplainLabel *marker;
      gchar *symbol, *display_name, *escaped_name;
      gfloat flon, flat;

      name = rest_xml_node_get_attr (n, "display_name");
      if (!name)
        {
          n = n->next;
          continue;
        }

      country = rest_xml_node_get_attr (n, "type");
      if (!country)
        {
          n = n->next;
          continue;
        }

      lon = rest_xml_node_get_attr (n, "lon");
      if (!lon)
        {
          n = n->next;
          continue;
        }

      lat = rest_xml_node_get_attr (n, "lat");
      if (!lat)
        {
          n = n->next;
          continue;
        }

      symbol = g_strdup_printf ("OS%d", i);
      escaped_name = g_markup_escape_text (name, -1);
      if (country)
        display_name = g_strdup_printf ("%s\n<small>%s</small>", escaped_name, country);
      else
        display_name = g_strdup_printf ("%s\n", escaped_name);

      /* Create the marker */
      flon = g_strtod (lon, NULL);
      flat = g_strtod (lat, NULL);
      marker = CHAMPLAIN_LABEL(champlain_label_new());
      champlain_label_set_text (marker, symbol);
      champlain_location_set_location (CHAMPLAIN_LOCATION(marker),
          flat,
          flon);
      champlain_marker_layer_add_marker (priv->layer, CHAMPLAIN_MARKER(marker));
      /* Create the row item */
      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, i,
                          COL_SYMBOL, symbol,
                          COL_NAME, name,
                          COL_DISPLAY_NAME, display_name,
                          COL_MARKER, marker,
                          COL_LAT, flat,
                          COL_LON, flon,
                          -1);

      g_free (symbol);
      g_free (display_name);
      g_free (escaped_name);

      n = n->next;
      i++;
    }

  rest_xml_node_unref (root);
}

Ok, a lot of code, but looking carefully there’s nothing difficult. For every result:

  • Parse the attributes
  • Create a ChamplainMarker
  • Add result to the list

Check Result
This is done only once and server the purpose of informing the user if there’s no result:

n = rest_xml_node_find (root, "place");

if (n==NULL)
    {
      GtkTreeIter iter;

      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, 0,
                          COL_SYMBOL, "OS",
                          COL_NAME, _("No result found"),
                          COL_DISPLAY_NAME, _("<b>No result found</b>"),
                          COL_MARKER, NULL,
                          -1);

      if (root)
        rest_xml_node_unref (root);
      return;
    }

If there’s no result n equals to NULL, and there’s no need to create any marker. The list is populated with a “No Found” entry.

Parse the attributes
Each of the attributes has to be parsed from the result. The following code shows how to obtain the name attribute. The same is done with all the other attributes.
If one of the results is missing one if the attributes that result is skipped.

name = rest_xml_node_get_attr (n, "display_name");
if (!name)
  {
    n = n->next;
    continue;
  }

Create a ChamplainMarker
When all the attributes are assigned to a variable and correctly cast (note that the result of rest_xml_node_attr is gchar*) creating and adding the marker is a matter of 4 function calls:

marker = CHAMPLAIN_LABEL(champlain_label_new());
champlain_label_set_text (marker, symbol);
champlain_location_set_location (CHAMPLAIN_LOCATION(marker),
    flat,
    flon);
champlain_marker_layer_add_marker (priv->layer, CHAMPLAIN_MARKER(marker));

Add result to the list
The last step is to add the result to the GtkTreeModel of the plugin:

gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                    COL_ORDER, i,
                    COL_SYMBOL, symbol,
                    COL_NAME, name,
                    COL_DISPLAY_NAME, display_name,
                    COL_MARKER, marker,
                    COL_LAT, flat,
                    COL_LON, flon,
                    -1);

Final Comments

The proposed patch can be seen at emerillon bugzilla page, is currently pending review.

Advertisements

2 Comments

Filed under code, emerillon, gis, Maps, webservices

2 responses to “Emerillon: Geocoding Names

  1. krlmlr

    This is great, but your patch seems to be in review for more than a year now. Is anyone maintaining Emerillon anyway?

    • kxtells

      Yes, most of Emerillon patches are pending for review.

      A year ago I had some spare time (hence the blog), Emerillon is nice so I created some patches for it and expected a review that never came.

      I exchanged some mails with the two mantainers at the time, If they could not work on it, I could get the task and do it myself. One of them was ok
      with it, but not the other, and he simply told me that he would review the patches… I am still waiting for that… Most of the accepted patches are just translation updates, and after some time of the same story, I got bored of it.

      It is a pity.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s