/** * @article Publishing an app to Google * Play * * @since March 22, 2012 * @package Mobile Apps * * @tags Android, * jQuery Mobile, * scrapers * */
Fast forward to this week. I’m working on a pretty fun canvas app using fabric.js that I plan on releasing as both a Facebook app and a mobile app (will do Android first, then worry about iOS later). While I was stuck on some annoying text-rendering bugs in fabric, I decided to try out the process of packaging and publishing an app, so as to be prepared when I got to that stage.
For an app, I dug up an old scraper project that I worked on a long time ago. I have a scraper up on scraperwiki.com to scrape all the entries from pantheon.org. Its a nice geeky wiki site with sentence or two definitions of gods, heroes, and monsters from like every pantheon in history. Unfortunately, its somewhat abandoned (hasn’t been updated since 2009 or so), hard-to-navigate, and in some places desperately needing to be fleshed out with source material, images, or whatnot. Still, its the best source for this information I could find, and I figured what better way to motivate people to contribute to a project than to do something that gets its info out to more people.
My scraper was about 100 lines of PHP, done with the Simple_HTML_Dom class. Was a bit of a pain to write, if only because the source pages were a little awkward to figure out how to parse. It went fairly smoothly though. My scraper pulled in 5380 records, the site says it has 7296. Either there are a lot of duplicates on the site, or some pages didn’t follow the same markup and my scraper missed them, or some pages on the site aren’t linked to from anywhere and my scraper coudn’t find them. Oh well. I still managed to get a lot of data. It’ll do for government work.
I decided to be completely lazy when it came to building the app. If anyone actually uses it, I can always go back and do it right. So I just built a one-page app in jQm that queries the datastore at scraperwiki. (Is that against the Terms of Service? Surprisingly, I don’t think so. That site is pretty awesome.) So, on opening the app, it immediately makes two calls to the scraperwiki api, to generate the list of categories (about 300b) and to pre-fetch the names of all entries (about 26K). I suppose I could get rid of that larger call, but it seemed to make the app run much faster, so I left it in. Then on choosing a record to view, the app makes another call to scraperwiki to retrieve the record.
How would this be done correctly? I guess it would be faster to just include the entire sqlite database with the app. I didn’t go that route because one, getting phonegap to handle large database creation looked like a huge chore (there are some promising libraries out there that handle that, like html5sql.js, which I will try out when I go back to update the app); and two, adding a 3MB sql resource with my app would triple the size of my app, and really, this is some basic stuff… I can’t justify taking up that much disk space for something that should be next to nothing in size.
I seriously put next to no effort into design here. Took the default jQm theme, threw in a little Trajan font and some free stock photo of parchment for background, and put it all together. Easy enough, if you don’t mind looking like every other site out there. Luckily, the transitions built into jQm work pretty well for my purpose.
Next post I’ll tell the story of packaging and publishing the app. Right now I’m still waiting for Google to approve my developer account.
Related, if you want to get into using jQuery Mobile, you might want to check out my boilerplate for jQuery Mobile WordPress themes. Its really a bare minimum, but it should get you started on the framework.