Have you ever wondered if something you’re about to buy will go on a sale, or if it was on sale before? Believe it or not, I do that all the time because I hate overpaying. One day, I was at a local Canadian Tire store and noticed a price drop on something I bought a few weeks ago. Unfortunately it was a real bummer that I couldn’t get a price match due to their 2-week price match policy.

Later that night I was determined not to make the same mistake again. I started designing a price tracking tool that can watch prices, and notify me when it goes on sale. After inspecting their website, I found these XML files having a list of all products and store pages. This seemed like a good data entrypoint. There were three product sitemaps and one store sitemap.

  • https://www.canadiantire.ca/sitemap_Product-en_CA-CAD-1.xml
  • https://www.canadiantire.ca/sitemap_Product-en_CA-CAD-2.xml
  • https://www.canadiantire.ca/sitemap_Product-en_CA-CAD-3.xml
  • https://www.canadiantire.ca/sitemap_Store-en_CA-CAD.xml

My initial goal was to create a tool that would allow me to:

  1. Tracks prices of products over time.
  2. Logs price changes in a database for historical analysis.
  3. Notifies when prices change.

With this information, I was excited to start building the tool to keep an eye on deals and analyze trends.

Tech Stacks

  1. Ruby on Rails Rails is my go-to framework for app development. It’s designed to be highly productive for individual developers with minimum resources.

  2. SQLite I hadn’t used SQLite much in the past becase I was under the misconception that it was only suitable for testing and development. However, this video completely changed my perspective. While it’s the default database for new Rails projects, I typically switch to MySQL or PostgreSQL without fully understanding the implications. This time, I wanted to give SQLite a try and see how it performed in a production environment. The best part I noticed is that the database exists as a file on disk and runs within the application process.

  3. Solid Queue I’d used Sidekiq for job processing in previous Rails projects, but I wanted to try the new ActiveJob adapter, Solid Queue. It’s a relatively new system in the Rails ecosystem. The main idea is to use the project’s existing database for queuing, eliminating the need for an in-memory database like Redis. It seemed like a perfect fit for this project. I had a feeling that combining Solid Queue with SQLite would further simplify the tech stacks.

  4. Selenium The website renders content dynamically, so a headless browser like Selenium was necessary to capture all data I need. It’s great for handling dynamic content generated by JavaScript.

  5. Docker For consistent development and production environments, Docker was a good choice for bundling application and its dependencies.

First Release

The first version is quite simple, limited to tracking just a few products I regularly buy. But I’m happy with how well it works and how quickly I was able to build it.

email