Jan 28
WOEIDs in Twitter's Local Trends
When we were designing the API for Local Trends on Twitter, we needed a way to represent a "place" in a permanent and language-independent way -- we didn't want to be caught in internationalization issues, and we didn't want to be caught in using an identifier that may change over time. What we landed on was using Yahoo!'s Where On Earth IDs, or WOEIDs, as our representation.
WOEIDs, in summary (you can read in more detail here), are 32-bit identifiers that are "unique and non-repetitive" — if a location is assigned a WOEID, the WOEID assigned is never changed, and that particular WOEID is never given to another location. In addition, a WOEID has certain properties such as an implied hierarchy (a particular city is in a particular state, for example), and a taxonomy of categories (there is a "language" to categorize something as a "town" or a "suburb"). Finally, there is a standard endpoint to query to get more information about a place. A program that wanted to get data about San Francisco, CA would first determine that it has a WOEID of 2487956, and with that query http://where.yahooapis.com/v1/place/2487956 (to use that URL, you need to register for a Yahoo! application ID first).
This is all great because it means that Twitter doesn't have to come up with our own naming schema, which we would need to maintain and expose. It also means that our Local Trends system is interoperable with anybody else who is building a system on top of WOEIDs — you could easily mash-up Flickr's places with our Trends!
If you want to look at our Trends API, then check out trends/available endpoint, as that will let you know what locations we are exposing trending information for. With those WOEIDs, you can then hit up trends/location to get the actual trending data.
There are two niceties to the API, that I just want to bump people towards: pass in a latitude/longitude when querying trends/available, and remember that there is a hierarchy of WOEIDs. If you pass in a lat and a long parameter to trends/available, then all the locations that are returned are sorted by their haversine distance from the coordinate passed in. Application developers can use this to help get trends from "around where you are".
And second, like I mentioned above, WOEIDs form a hierarchy (that's mostly correct). Here is the hierarchy of the locations that we have as of today:
1 ("Terra")
|---- 23424775 ("Canada" - Country)
|---- 23424803 ("Ireland" - Country)
|---- 23424975 ("United Kingdom" - Country)
| \---- 24554868 ("England" - State)
| \---- 23416974 ("Greater London" - County)
| \---- 44418 ("London" - Town)
|---- 23424900 ("Mexico" - Country)
|---- 23424768 ("Brazil" - Country)
| \---- 2344868 ("Sao Paulo" - State)
| \---- 12582314 ("São Paulo" - County)
| \---- 455827 ("Sao Paulo" - Town)
\---- 23424977 ("United States" - Country)
|---- 2347572 ("Illinois" - State)
| \---- 12588093 ("Cook" - County)
| \---- 2379574 ("Chicago" - Town)
|---- 2347567 ("District of Columbia" - State)
| \---- 12587802 ("District of Columbia" - County)
| \---- 2514815 ("Washington" - Town)
|---- 2347606 ("Washington" - State)
| \---- 12590456 ("King" - County)
| \---- 2490383 ("Seattle" - Town)
|---- 2347579 ("Maryland" - State)
| \---- 12588679 ("Baltimore City" - County)
| \---- 2358820 ("Baltimore" - Town)
|---- 2347563 ("California" - State)
| |---- 12587707 ("San Francisco" - County)
| | \---- 2487956 ("San Francisco" - Town)
| \---- 12587688 ("Los Angeles" - County)
| \---- 2442047 ("Los Angeles" - Town)
|---- 2347580 ("Massachusetts" - State)
| \---- 12588712 ("Suffolk" - County)
| \---- 2367105 ("Boston" - Town)
|---- 2347591 ("New York" - State)
| \---- 2459115 ("New York" - Town)
|---- 2347569 ("Georgia" - State)
| \---- 12587929 ("Fulton" - County)
| \---- 2357024 ("Atlanta" - Town)
|---- 2347602 ("Texas" - State)
| |---- 12590226 ("Tarrant" - County)
| | \---- 2406080 ("Fort Worth" - Town)
| |---- 12590107 ("Harris" - County)
| | \---- 2424766 ("Houston" - Town)
| |---- 12590063 ("Dallas" - County)
| | \---- 2388929 ("Dallas" - Town)
| \---- 12590021 ("Bexar" - County)
| \---- 2487796 ("San Antonio" - Town)
\---- 2347597 ("Pennsylvania" - State)
\---- 12589778 ("Philadelphia" - County)
\---- 2471217 ("Philadelphia" - Town)
Right now, even though we don't expose this information using trends/available, you could ask for any of those WOEIDs, and we'll choose the nearest trend location (or locations!) that we have data for.
And, of course, we have a few more things in the pipeline...
how about passing a west/south/east/north bounding box (rather than a single lat/lng point) to get the *best fitting* WOEID? the world is rather lumpy and haversine distance will not work so well around irregularly shaped countries.
here’s the OpenSearch-Geo standard approach:
http://www.opensearch.org/Specifications/OpenSearch/Extensions/Geo/1.0/Draft_1#The_.22box.22_parameter
actually, using a west/south/east/north bounding box query on /trends/location would also be really useful for map-based apps (like GeoMeme http://www.geome.me) which use a viewport of arbitrary size, from a whole country down to a street corner. it would still be possible to infer all the language and location hierarchy data that you mention.
please keep us posted on that pipeline!
but, the bounding box notion is a great idea - i'll stick that in the pipeline!