The Blog of Xangis

I wrote an application to harvest historical stock data from Yahoo.  It’s an interesting piece of homebrew and has allowed me to build up a database of 3 gigabytes of historical data so far using SQL Server Express 2008.  Since Yahoo! data is specifically “for non-commercial use” I can’t exactly sell the database to anyone, but it’s handy to have for calculations and drawing charts.

Here’s a screenshot of the app (click to enlarge):

It can grab data in an few different ways.

1. Can import CSV files downloaded from Yahoo or Google finance.
2. Can batch load a space-separated or newline-separated list of stock symbols.
3. Can update all symbols already in the database with a specific number of days of data.

I started by pasting in big lists of symbols and batch importing. This takes a long time since there’s a lot of data to load, but once the bulk of data is loaded, I can hit the “update all” button once a week to keep the data current.

In addition, the “Populate Symbols” button will go through all symbols and fill a table containing the company name and exchange that it’s traded on, which is something you won’t necessarily know already if all you haveis a symbol.

I’ve written a simple charting application that can use information in this database to draw charts.  I’ll post about it sometime later, but I’m gradually adding features and it’s turning out to be a pretty neat app.

One of the possible goals for this importer is to add export capabilities and support for more database types (like PostgreSQL) and auto-creation of databases on first import.  Either way it works pretty well for my purposes at the moment.