petewarden/geodict

A simple Python library/tool for pulling location information from unstructured text

geodict's Issues

Trouble with the cities table

I'm seeing something strange in the cities table; it looks as though a lot of cities that are in the source data are missing from the populated geodict database, possibly getting clobbered on import.

Take Brooklyn, for example. In worldcitiespop.csv, grep finds 49 entries for 'brooklyn' (42 of which are in the US); in the geodict database, there are five entries for 'brooklyn', only one of which is in the US (and the US entry is in Alabama). The same seems to be true of other US cities like Rochester and Boston, each of which is found only once in the US (and in an alphabetically early state like AL or CA). Are the others getting clobbered on import? Or am I maybe making a mistake in looking through the database (not much experience with MySQL here).

The SQL query I'm using is:

SELECT city, country, region_code, population, lat, lon FROM cities WHERE city = 'Brooklyn';

Other things that might be relevant:

1. The populate_database.py script produces two errors when I run it:
   ./populate_database.py:49: Warning: Data truncated for column 'last_word' at row 1
   (city, country, region_code, population, lat, lon, last_word))

   ./populate_database.py:49: Warning: Data truncated for column 'city' at row 1
   (city, country, region_code, population, lat, lon, last_word))

2. populate_database.py won't work at all unless I first create the geodict database by hand, even though it looks as though the script is meant to handle that.

3. System info:

   uname -a

   Darwin wilkens-imac.wustl.edu 10.7.0 Darwin Kernel Version 10.7.0: Sat Jan 29 15:17:16 PST 2011; root:xnu-1504.9.37~1/RELEASE_I386 i386

   mysql --version

   mysql Ver 14.14 Distrib 5.1.56, for apple-darwin10.3.0 (i386) using readline 5.1

Any other info I can provide? Happy to do any kind of debugging that might help. Thanks!

Populate regions table

Hi Pete,

What an interesting work. I wanted to try your tool with my own data but I can't move forward with the population of the database. Population of cities and countries works fine, but regions table stay empty. Script populate_database.py flags me just a warning : "./populate_database.py:42: Warning: Data truncated for column 'city' at row 1 (city, country, region_code, population, lat, lon))"

Have you any idea?
Thank you.

Python Module

Could you add a blank __init__.py file into your folder? I'm importing geodict as a module and have to create the file manually.

Thanks for the awesome script!

Language Support

Hey Pete, I would like to use your geodict library with a german text. Is there some way to get a language specific .csv file?

regards, Mirko

Showing previous data

Hi

import geodict_lib
lis = ["I live in India with love", "DACH and the Netherlands Klarna Checkout purchases, Nordics Klarna Checkout purchases, and Klarna Checkout for In-App Purchases"]
for li in lis:
print geodict_lib.find_locations_in_text(li)

I tried with the above code, but for for the text, it shows INDIA as country instead of Netherlands in the second element of the text.

Thanks,
Viswesh M