Add admin2 and timezone datasets, expose featurecode, fix caching, index city names (closes #6)

[cvl01]

Adds two GeoNames datasets, exposes the feature code, fixes two caching bugs and closes #6.

Data files are gitignored, so this needs make dl && make json to build.

New datasets

• admin2Codes.txt → get_admin2_codes(), keyed by the concatenated code <countrycode>.<admin1code>.<admin2code> as the GeoNames readme describes, e. g. NL.11.0599. 47592 records.
• timeZones.txt → get_timezones() keyed by IANA id, plus get_timezones_by_country() which takes a case insensitive ISO alpha-2 code and returns the country's zones sorted by id. The header row names its offset columns after the current year, so the parser matches on the first column instead.

City records gain admin2code to complete the composite key, and get_admin1_by_city() / get_admin2_by_city() do the join. Both return None rather than building a partial key such as 'NL.' when a city has no admin1code, which affects roughly 8700 of the 235156 cities in the 500 population dataset.

The resolved admin1 name is deliberately not denormalised onto city records: it grew cities500.json by 12 MB for something a dict lookup already gives you.

Expose featurecode

The cities datasets carried it in column 8 and the build discarded it. It distinguishes a capital (PPLC) or administrative seat (PPLA–PPLA5) from an ordinary populated place (PPL), and is searchable via search_cities(attribute='featurecode').

It also explains something otherwise invisible: cities15000.txt contains 45 places with population below 15000, all PPLC or PPLG, because GeoNames includes seats of government regardless of size. The rule population > threshold or featurecode in seats reproduces every cities file from cities500.txt exactly, with the seat codes accumulating as the threshold drops:

file	rule	result
cities15000	pop > 15000 or PPLC, PPLG	exact, 34078
cities5000	pop > 5000 or + PPLA	exact, 69577
cities1000	pop > 1000 or + PPLA2, PPLA3	exact, 170584

featureclass is not stored, as it is P for every record in these datasets.

Two caching bugs

• _load_data() never cached. It returned the parsed data without storing it, so every getter re-read and re-parsed its file. Three consecutive get_cities() calls on the 500 population dataset cost ~0.45 s each and gc.cities stayed None forever. search_cities() paid that on every search. It is now free after the first call.
• get_cities_by_name() could return the wrong dataset's results. cities_by_names was a class attribute keyed by city name alone, while the results depend on min_city_population. An instance created after one with a smaller dataset was served the other's results: GeonamesCache(500).get_cities_by_name('Springfield') returned 8 records instead of 24. Now per instance.

Both have regression tests that were verified to fail against the previous implementation.

Closes #6: index city names

get_cities_by_name() scanned the whole dataset for every unseen name, at 4 ms per name on the default dataset and 31 ms on the 500 population one. It now builds a name → records index on first call. Looking up 500 distinct names on the 500 population dataset went from 5.3 s to 0.08 s.

The issue suggested generators. They do not help here: results are memoised and a generator cannot be re-iterated, and the cost was the scan, not building the result list.

Breaking: get_cities_by_name() now returns a list of city records where it returned a list of single-entry dicts keyed by geonameid. Callers that unwrapped with list(d)[0] should read city['geonameid'], which the records already carry. Unknown names return [] instead of an empty list of wrappers. The old shape was also what made an index expensive, at 68 MB against 27 MB for plain references. get_cities_by_names() exposes the index itself.

This wants a major version bump; I left __version__ alone as that is your release flow's call.

Gzip the bundled data

Installed package size drops from 203 MB to 36 MB at no cost in load time, since the saved I/O offsets decompression (first get_cities() on the 500 population dataset: 0.31 s either way). Wheel and sdist size are unchanged at ~35 MB because those were already deflate compressed, so this is a disk win rather than a bandwidth one.

bin/ scripts keep writing plain JSON into datasets/; the new bin/compress_data.py gzips it into geonamescache/data/ as the last step of make json, with mtime=0 so identical input gives identical output. Verified by building a wheel and using it from a clean virtualenv. A test asserts no stale plain .json is left in the data directory, since it would be shipped and silently ignored.

I did look at deduplicating the four nested cities files using the rule above, which would take 36 MB down to ~15 MB, and decided against it: it would make a min_city_population=15000 user parse all 235k cities instead of 34k, a 5x slower first load, to save 21 MB of disk.

Docs and tests

The README gains a data formats section documenting the return value of every method. Every example in it was executed against the built data rather than written from memory, which caught a wrong one: search_cities('Rotterdam') returns Rotterdam and Hoogvliet, both NL, because it searches alternatenames by default and Hoogvliet's alternate name is "Hoogvliet Rotterdam". You need attribute='name' for the two actual Rotterdams.

37 tests pass, mypy is clean. Two pre-existing issues are untouched: tests/test_data.py::test_continents fails because continents.json needs GEONAMES_USER and an API call, and ruff flags the import order in geonamescache/mappers.py.

Also fixed along the way: get_us_counties() had no return annotation, _load_data() was annotated -> dict while us_counties.json is a list, and CitySearchAttribute was missing admin2code.

🤖 Generated with Claude Code