Category Archives: Search Engines

Search engine related posts.

Quora Answer: How would you find the websites to build a search index from scratch?

I originally wrote this as an answer to a question on Quora. It depends on the scale. If you just want to experiment with web crawling and build a basic search index, it’s common to start with the Alexa top million websites, which can be downloaded in a CSV file via S3 at: http://s3.amazonaws.com/alexa-static/top-1m.csv.zip The top […]

Setting Up a Redash Dashboard

This was originally posted on wbsrch.com. It is reproduced here to preserve history. The more WbSrch evolves, the more it becomes necessary to keep track of a bunch of metrics. Until now we’ve been using a mix of simple report pages and raw SQL queries. It has worked well enough, but not having a clean […]

An Experiment with Project Wonderful

This was originally posted on wbsrch.com. It is reproduced here to preserve history. I’m always looking for new and efficient ways to let people know about WbSrch. That’s why I decided to try advertising with Project Wonderful. Project Wonderful was built as a banner ad network for web comics. That doesn’t mean you can only […]

Analysis of Search Engine Crowdfunding Campaigns on IndieGoGo

This was originally posted on wbsrch.com. It is reproduced here to preserve history. In the process of researching crowdfunding campaigns, I searched IndieGoGo for search engine pitches. I found 22 attempts to fund “actual search engines”. Here is a list (with links to the IndieGoGo campaign): TheNet101 Thumbar Jixty.com Xense Iyiyes Aspinosa Rexyo Asim Shah […]

AdSense Alternatives for Startups and Small Websites

This was originally posted on wbsrch.com. It is reproduced here to preserve history. In starting WbSrch, a search competitor to Google, I knew that at some point Google would find a way to “invite us to leave” AdSense. The Terms of Service make it clear that it is incompatible with a search engine (can’t have ads on […]

Why I Decided to Build a Search Engine (And You Should Too)

This was originally posted on wbsrch.com. It is reproduced here to preserve history. I’ve always wanted to build something big, but never had a burning desire to create any one specific thing. Instead I built lots of little things – small desktop apps, weekend websites, etc. It wasn’t until AltaVista shut down that I realized that the world […]