Category: Search Engines

Search engine related posts.

The WbSrch Experiment

Off-and-on over the last 8 years I’ve worked on an independent search engine called WbSrch. It made it as far as being as good as the late-1990s search engines, which is great, because the original goal was to build something much like Altavista. That was my first “main” search engine. At one point I tried

Smarthost.net: Ultimately Just a Waste of Time

Smarthost.net has some nice “storage server” deals with some very configurable options. If you want a VPS with 1TB of disk space, their offerings are pretty attractive. For my search engine, I need hosting with a good chunk of disk space in order to hold the index. It doesn’t need to be fast storage, and

Media.net Didn’t Work For Me

I received a message that my site was disapproved today, so they’re the first ones to fail out of my newest ad network comparison experiment. Looking closer at their policies I see this (bold added by me): “Our program has been designed for sites with premium content. Sites that promote, contain, or link directly to

WbSrch Online Again

I found a way to get WbSrch online inexpensively, through a combination of code optimizations and an inexpensive high-disk-space internet provider. It doesn’t need fast SSD storage to serve the index data, so it works just fine on a mechanical hard drive, and it’s easier to get a lot of space inexpensively with on of

PageRank Lives: OpenPageRank by Domcop

In the early days of Google, PageRank was a very important piece of information about a website. It let you know the general authority level of a site and how well it would tend to rank against similar content on another site. The PageRank toolbar, released in 2000, became an important tool in the SEO

Top-Level Domain Popularity

In a crawl of just over 32 million pages, this is the number of domains that I discovered for each top-level domain (TLD). The “Known Domains” is the number of domains with that extension that were found in links, while the “Crawled Domains” is the number of domains where pages were retrieved from. Extension Known

WbSrch Offline Again

I put the WbSrch search engine back online in March of 2018. I spent a lot of time improving it over the 16 months, but it’s the sort of thing that always manages to demand more time and energy. It’s time to stop giving it either — though it’s grown and improved a lot, it’s

WbSrch Online Again

A while back I open-sourced the code for the WbSrch search engine. It’s online now in a much-reduced form at wbsrch.com. It’s not the full search engine. Far from it. It’s just a tiny database of about 10,000 or so URLs to demo the source code, but it’s possible you’ll actually find what you’re looking

2018 Is The Year That Twitter Ceases To Be Relevant

2018 is the year that Twitter ceases to be relevant. It’s already stopped being relevant for me. I’ve stopped using it, and have deleted all of my tweets. As a user, it’s just not worth it. It’s a miserable experience, made much worse by the userbase being made up primarily of Russian bots posing as

Interesting But Not A Business: The Story of the WbSrch Search Engine

I write this after having just shut down my almost-startup, the WbSrch search engine. I started working on WbSrch for “fun” in the fall of 2013. AltaVista, my favorite search engine from “back in the day” had shut down that summer. Nostalgia combined with annoyance at how bad/annoying/intrusive/evil Google had become convinced me to try

Setting Up a Redash Dashboard

This was originally posted on wbsrch.com. It is reproduced here to preserve history. The more WbSrch evolves, the more it becomes necessary to keep track of a bunch of metrics. Until now we’ve been using a mix of simple report pages and raw SQL queries. It has worked well enough, but not having a clean

An Experiment with Project Wonderful

This was originally posted on wbsrch.com. It is reproduced here to preserve history. I’m always looking for new and efficient ways to let people know about WbSrch. That’s why I decided to try advertising with Project Wonderful. Project Wonderful was built as a banner ad network for web comics. That doesn’t mean you can only

Analysis of Search Engine Crowdfunding Campaigns on IndieGoGo

This was originally posted on wbsrch.com. It is reproduced here to preserve history. In the process of researching crowdfunding campaigns, I searched IndieGoGo for search engine pitches. I found 22 attempts to fund “actual search engines”. Here is a list (with links to the IndieGoGo campaign): TheNet101 Thumbar Jixty.com Xense Iyiyes Aspinosa Rexyo Asim Shah

AdSense Alternatives for Startups and Small Websites

This was originally posted on wbsrch.com. It is reproduced here to preserve history. In starting WbSrch, a search competitor to Google, I knew that at some point Google would find a way to “invite us to leave” AdSense. The Terms of Service make it clear that it is incompatible with a search engine (can’t have ads on

Why I Decided to Build a Search Engine (And You Should Too)

This was originally posted on wbsrch.com. It is reproduced here to preserve history. I’ve always wanted to build something big, but never had a burning desire to create any one specific thing. Instead I built lots of little things – small desktop apps, weekend websites, etc. It wasn’t until AltaVista shut down that I realized that the world