Emerillon: Geocoding Names

Current search bar on emerillon is limited:

I want emerillon to find also streets!

And in this post I explain how I’ve done it

Geocoding is the process of finding associated geographic coordinates from other geographic data, such as street addresses, or zip codes.
— Wikipedia Geocoding Page

Let’s go for it!

GeoNames and OSM Nominatim WebServices

In an older post I talked about emerillon and how much I like it. Using this program you notice that it provides geocoding on the search bar using GeoNames web service.

— Enschede search with Geonames
http://ws.geonames.org/search?q=Enschede&maxRows=10

<geonames style="MEDIUM">
   <totalResultsCount>55</totalResultsCount>
   <geoname>
      <toponymName>Enschede</toponymName>
      <name>Enschede</name>
      <lat>52.21833</lat>
      <lng>6.89583</lng>
      <geonameId>2756071</geonameId>
      <countryCode>NL</countryCode>
      <countryName>Netherlands</countryName>
      <fcl>P</fcl>
      <fcode>PPL</fcode>
   </geoname>
   (...)
</geonames>

But GeoNames is not the only geocoding service on the block, I’ll try the same using openstreetmaps Nominatim.

— Enschede search with Open Street Maps Nominatim:
http://nominatim.openstreetmap.org/search?q=Enschede&format=xml

<searchresults 
   timestamp="Thu, 29 Mar 12 21:16:04 +0100" 
   attribution="Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0."
   querystring="Enschede" polygon="false"
   exclude_place_ids="250780,128894028,128949748,20442594,124640085,124640065,20430454,250450,34264043" 
   more_url="http://nominatim.openstreetmap.org/search?format=xml&
   exclude_place_ids=250780,128894028,128949748,20442594,124640085,124640065,20430454,250450,34264043&
   accept-language=en-us,en;q=0.5&q=Enschede">
   <place 
      place_id="250780" osm_type="node" 
      osm_id="45753522" place_rank="16" 
      boundingbox="52.20796798706,52.227971801758,6.8836185836792,6.9036190605164" 
      lat="52.2179694" lon="6.8936189" 
      display_name="Enschede, Regio Twente, Overijssel, The Netherlands, Europe" 
      class="place" type="city" icon="http://nominatim.openstreetmap.org/images/mapicons/poi_place_city.p.20.png"/>
   (...)
</searchresults>

Now a little bit more precise. I’m searching for Hengelosestraat, a street in the city of Enschede.

— With Geonames
http://ws.geonames.org/search?q=Hengelosestraat&maxRows=2

<geonames style="MEDIUM">
   <totalResultsCount>0</totalResultsCount>
</geonames>

— With OSM
http://nominatim.openstreetmap.org/search?q=Hengelosestraat&format=xml

<searchresults timestamp="Sun, 22 Apr 12 16:18:10 +0100" attribution="Data Copyright OpenStreetMap Contributors, Some Rights Reserved. CC-BY-SA 2.0." querystring="Hengelosestraat" polygon="false" exclude_place_ids="126410125,125653405,126638464,127016981,126944027,126611860,126594997,126638730,126629003,126617923" more_url="http://nominatim.openstreetmap.org/search?format=xml&exclude_place_ids=126410125,125653405,126638464,127016981,126944027,126611860,126594997,126638730,126629003,126617923&accept-language=en-us,en;q=0.5&q=Hengelosestraat">
   
    <place place_id="126410125" osm_type="way" osm_id="144351134" place_rank="26" boundingbox="52.2234916687012,52.2235488891602,6.88805961608887,6.88870525360107" lat="52.2235281294069" lon="6.88838181409509" display_name="Hengelosestraat, Enschede, Regio Twente, Overijssel, 7511JS, The Netherlands" class="highway" type="secondary"/>
(...)
</searchresults>

First comment: OpenStreetMap gives a lot of output, does not look as clean as GeoNames.
Second comment: OpenSteetMap gives results at street level.

Not to say that GeoNames is not as good as osm, simply it serves a different purpose.

Emerillon

Currently Emerillon only uses GeoNames, but I feel natural that it should also use OSM Nominatim. It already use osm maps, so why not osm geocoding?

>>General View

The Search plugin from emerillon is the responsible for querying the geonames webservice, it has a lot of functions but only two of my interest:

search_address	Query GeoNames using librest
result_cb	Parse the result, fill a GtkListStore and fill the MarkerLayer

Seems fairly easy to extend it to:

search_address_gn	Query GeoNames using librest
result_cb_gn	Parse the result, fill a GtkListStore and fill the MarkerLayer
search_address_osm	Query Open Street Map Nominalis using librest
result_cb_osm	Parse the result, fill a GtkListStore and fill the MarkerLayer

I’ll call the two services in an asynchronous fashion. Once finished the same function “result_cb” will be triggered.

/*In function search_address_osm*/
rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error)

/*In function search_address_cb*/
rest_proxy_call_async (priv->call_gn,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_gn),
        plugin,
        &error)

Result_cb controls that both calls have finished (in a way or another). When a call finishes sets its boolean flag to true, when both flags are true the parsing of the result starts.
When all the results are added, the extent of the map is set to show all the markers.

static void
result_cb (RestProxyCall *call,
           GError *error,
           GObject *weak_object,
           SearchPlugin *plugin)
{
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  if (call == priv->call_osm){
    priv->finished_osm = TRUE;
  }
  else if(call == priv->call_gn){
    priv->finished_gn = TRUE;
  }

  if (priv->finished_gn && priv->finished_osm){
    result_cb_osm(priv->call_osm,NULL,NULL,plugin);
    result_cb_gn(priv->call_gn,NULL,NULL,plugin);

    champlain_view_ensure_visible (priv->map_view,
      champlain_layer_get_bounding_box((ChamplainLayer*)priv->layer),
      FALSE);

    priv->finished_gn = FALSE;
    priv->finished_osm = FALSE;
  }
}

>>Using LibRest to call the services

First the full code is shown, then split and explained in more detail.

The full function to call the OSM Nominalis service is the following:

static void
search_address_osm (SearchPlugin *plugin){
  const gchar *query;
  gchar *locale;
  gchar lang[2];
  GError *error = NULL;
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  query = gtk_entry_get_text (GTK_ENTRY (plugin->priv->search_entry));
  locale = setlocale (LC_MESSAGES, NULL);
  g_utf8_strncpy (lang, locale, 2);
  gtk_list_store_clear (GTK_LIST_STORE (priv->model));

  if (priv->proxy_osm == NULL)
    priv->proxy_osm = rest_proxy_new ("http://nominatim.openstreetmap.org/", FALSE);

  /* Cancel previous call */
  if (priv->call_osm)
    g_object_unref (priv->call_osm);

  priv->call_osm = rest_proxy_new_call (priv->proxy_osm);

  rest_proxy_set_user_agent (priv->proxy_osm, "Emerillon/"VERSION);

  rest_proxy_call_set_function (priv->call_osm, "search");
  rest_proxy_call_set_method (priv->call_osm, "GET");
  rest_proxy_call_add_params (priv->call_osm,
      "q", query,
      "format", "xml",
      NULL);

  if (!rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error))
    {
      g_error ("Cannot make call: %s", error->message);
      g_error_free (error);
    }
}

There are 3 things to do:

• Retrieve the query string from the input form
• Prepare the call
• Call and bind the result function

Retrieve the query string
Easy as pie, isn’t it?:

query = gtk_entry_get_text (GTK_ENTRY (plugin->priv->search_entry));

Prepare the call:

 if (priv->proxy_osm == NULL)
    priv->proxy_osm = rest_proxy_new ("http://nominatim.openstreetmap.org/", FALSE);

  /* Cancel previous call */
  if (priv->call_osm)
    g_object_unref (priv->call_osm);

  priv->call_osm = rest_proxy_new_call (priv->proxy_osm);

  rest_proxy_set_user_agent (priv->proxy_osm, "Emerillon/"VERSION);

  rest_proxy_call_set_function (priv->call_osm, "search");
  rest_proxy_call_set_method (priv->call_osm, "GET");
  rest_proxy_call_add_params (priv->call_osm,
      "q", query,
      "format", "xml",
      NULL);

Initially there is a check on the proxy_osm variable, if it’s not initialized I create a new rest_proxy [line 14].
Second, maybe there’s a call still working, cancel it (just in case) [lines 18,19].
After that I prepare a new call [line 21], set the user agent [line 23], the function [line 25], the method [line 26] and the parameters (followed by NULL, as the documentation dictates) [lines 25 to 30].

Bind and Call
There are two services that I want to query asynchronously.

rest_proxy_call_async (priv->call_osm,
        (RestProxyCallAsyncCallback) result_cb,
        G_OBJECT (priv->proxy_osm),
        plugin,
        &error)

The function rest_proxy_call_async will do that. I call with priv->call_osm. The callback function is result_cb (that will take care of the two async petitions). The third parameter is the object where the call will be tied. plugin is the search plugin structure reference to be passed to the callback function. Finally if there’s an error it will be assigned to the error variable.
In the full code this call is inside a conditional check for controlling possible errors.

>>Parsing the result

When the service returns a result the program has to parse it and store the results.

static void
result_cb_osm (RestProxyCall *call,
           GError *error,
           GObject *weak_object,
           SearchPlugin *plugin)
{
  const gchar *answer;
  gint len;
  guint i;
  RestXmlParser *parser;
  RestXmlNode *root, *n;
  SearchPluginPrivate *priv = SEARCH_PLUGIN (plugin)->priv;

  answer = rest_proxy_call_get_payload (call);
  len = rest_proxy_call_get_payload_length (call);
  parser = rest_xml_parser_new ();

  root = rest_xml_parser_parse_from_data (parser, answer, len);

  /* Extract the result count */
  n = rest_xml_node_find (root, "place");

  if (n==NULL)
    {
      GtkTreeIter iter;

      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, 0,
                          COL_SYMBOL, "OS",
                          COL_NAME, _("No result found"),
                          COL_DISPLAY_NAME, _("<b>No result found</b>"),
                          COL_MARKER, NULL,
                          -1);

      if (root)
        rest_xml_node_unref (root);
      return;
    }

  n = rest_xml_node_find (root, "place");
  i = 1;

  while (n)
    {
      const gchar *name, *country, *lon, *lat;
      GtkTreeIter iter;
      ChamplainLabel *marker;
      gchar *symbol, *display_name, *escaped_name;
      gfloat flon, flat;

      name = rest_xml_node_get_attr (n, "display_name");
      if (!name)
        {
          n = n->next;
          continue;
        }

      country = rest_xml_node_get_attr (n, "type");
      if (!country)
        {
          n = n->next;
          continue;
        }

      lon = rest_xml_node_get_attr (n, "lon");
      if (!lon)
        {
          n = n->next;
          continue;
        }

      lat = rest_xml_node_get_attr (n, "lat");
      if (!lat)
        {
          n = n->next;
          continue;
        }

      symbol = g_strdup_printf ("OS%d", i);
      escaped_name = g_markup_escape_text (name, -1);
      if (country)
        display_name = g_strdup_printf ("%s\n<small>%s</small>", escaped_name, country);
      else
        display_name = g_strdup_printf ("%s\n", escaped_name);

      /* Create the marker */
      flon = g_strtod (lon, NULL);
      flat = g_strtod (lat, NULL);
      marker = CHAMPLAIN_LABEL(champlain_label_new());
      champlain_label_set_text (marker, symbol);
      champlain_location_set_location (CHAMPLAIN_LOCATION(marker),
          flat,
          flon);
      champlain_marker_layer_add_marker (priv->layer, CHAMPLAIN_MARKER(marker));
      /* Create the row item */
      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, i,
                          COL_SYMBOL, symbol,
                          COL_NAME, name,
                          COL_DISPLAY_NAME, display_name,
                          COL_MARKER, marker,
                          COL_LAT, flat,
                          COL_LON, flon,
                          -1);

      g_free (symbol);
      g_free (display_name);
      g_free (escaped_name);

      n = n->next;
      i++;
    }

  rest_xml_node_unref (root);
}

Ok, a lot of code, but looking carefully there’s nothing difficult. For every result:

• Parse the attributes
• Create a ChamplainMarker
• Add result to the list

Check Result
This is done only once and server the purpose of informing the user if there’s no result:

n = rest_xml_node_find (root, "place");

if (n==NULL)
    {
      GtkTreeIter iter;

      gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
      gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                          COL_ORDER, 0,
                          COL_SYMBOL, "OS",
                          COL_NAME, _("No result found"),
                          COL_DISPLAY_NAME, _("<b>No result found</b>"),
                          COL_MARKER, NULL,
                          -1);

      if (root)
        rest_xml_node_unref (root);
      return;
    }

If there’s no result n equals to NULL, and there’s no need to create any marker. The list is populated with a “No Found” entry.

Parse the attributes
Each of the attributes has to be parsed from the result. The following code shows how to obtain the name attribute. The same is done with all the other attributes.
If one of the results is missing one if the attributes that result is skipped.

name = rest_xml_node_get_attr (n, "display_name");
if (!name)
  {
    n = n->next;
    continue;
  }

Create a ChamplainMarker
When all the attributes are assigned to a variable and correctly cast (note that the result of rest_xml_node_attr is gchar*) creating and adding the marker is a matter of 4 function calls:

marker = CHAMPLAIN_LABEL(champlain_label_new());
champlain_label_set_text (marker, symbol);
champlain_location_set_location (CHAMPLAIN_LOCATION(marker),
    flat,
    flon);
champlain_marker_layer_add_marker (priv->layer, CHAMPLAIN_MARKER(marker));

Add result to the list
The last step is to add the result to the GtkTreeModel of the plugin:

gtk_list_store_append (GTK_LIST_STORE (priv->model), &iter);
gtk_list_store_set (GTK_LIST_STORE (priv->model), &iter,
                    COL_ORDER, i,
                    COL_SYMBOL, symbol,
                    COL_NAME, name,
                    COL_DISPLAY_NAME, display_name,
                    COL_MARKER, marker,
                    COL_LAT, flat,
                    COL_LON, flon,
                    -1);

Final Comments

The proposed patch can be seen at emerillon bugzilla page, is currently pending review.