Side Project Journal: A City Comparison Web Scraper

It was a bright morning. Jackie and I were daydreaming about cities that we could live in together, just for the fun of it. What ensued was a flurry of Google searches and data comparison.

I’ve been learning the basics of the Ruby programming language. It’s a versatile and flexible weapon, a beautiful development experience, complete with power. While web frameworks (like Rails) is what Ruby is famously known for, I was on the lookout for a useful but fun project to test my skills with pure Ruby.

With that backdrop, I had a lightbulb moment. 💡

What if I could scrape the city data website I was looking at in order to compare data across multiple different cities, and only extract the data that I cared about?

That’s where the idea for City Compr came from. Yes, the name is super lame, and it’s dumb that I left out vowels in the word “Compare,” but I made a split second decision and I’m rolling with it.

As a program that prints out data to the terminal, I was able to leverage Ruby to scrape websites dynamically, and collect data on any city or town in the US.

In a couple days, I cranked on this. I learned to:

Want to check out the project? It’s open source on Github.