Yearly Archives: 2014

Quora Answer: I know almost nothing about the stock market but have strong programming skills. Will I be able to make profitable trading software?

I originally wrote this as an answer to a question on Quora.

The world is littered with the empty wallets of engineers that approached the stock market as a math, algorithmic, or engineering problem.

The largest investment banks all have algorithmic trading programs, and they often lose large chunks of money. And sometimes they make large chunks of money. You might be able to compete in the gaps, but they have what amounts to infinitely deep pockets to hire the absolute best people and give them the highest-quality tools and set up servers that have sub-millisecond market access to systems you couldn’t begin to afford the connection fees for.

Something that makes money by trading algorithmically is one of the worst forms of parasite. Like a tapeworm, it removes nutrients from the market and offers nothing in return. But creating something like that is an interesting intellectual problem.

You might be able to build profitable trading software, but if you make it your life’s work, it’s entirely possible that you’ll end up with nothing to show at the end of the journey.

Quora Answer: Before joining as a CTO and only coder of an early stage startup, what knowledge should I have?

I originally wrote this as an answer to a question on Quora.

Full question:

“And, which kind of questions should I make to the initial founders?
I am not only talking about technical skills, all kind of knowledge matters.”

Knowledge

You should already know most of what you need to know to build version 1.0 of the product. You’ll learn the rest as you go.

You should also know how to deploy it, secure it, and manage it.

You should know how to document what you’re doing, at least enough that a smart person could pick up and maintain it later.

You should know enough to plan for disaster recovery. That’s not too hard at the early stage — just don’t keep everything on one computer in one building.

You should also know how to evaluate, onboard, and delegate tasks and projects to future hires.

You should be VERY good at estimating how long it will take to develop a new feature.

You should be comfortable with networking, presenting at user groups, and going out into the world to get to know other CTOs, developers, and people in your industry. If things work out you’ll be hiring these people someday.

You should be good at researching the competition and finding out what strategies they are using.

You should be good at the technical aspects of sales/marketing. A/B testing, mailing list management, promotional codes, etc. None of this is hard, but you should expect to do things that make it easier for the non-technical people to get the product tested and sold.

Questions to Ask

You should dig VERY deeply on what validation they’ve done and who they already have on the hook as potential users, investors, and beta testers.

You should find out whether they have the skills to do their job. Their main jobs are:

– Getting people to care about the product enough to buy. (selling).

– Getting people to the top of the sales funnel in the first place (marketing).

– Getting product-market fit (customer validation).

– Getting the resources the company needs (fundraising).

You need to find out whether they’ve done the work to validate their idea and whether they have people interested in at least trying (or better yet, buying) once it’s built.

You also need to find out how much of version 1.0 they’ve already designed. Find out what feature set they’ve designed, and be sure to ask how much they’ve validated each feature or aspect of the design.

You also need to find out what their expectations are and how rewards and equity are to be split up — what will your reward be for your efforts? What’s the existing equity split?

Quora Answer: What did Steve Jobs mean by programming ‘teaches you how to think’?

I originally wrote this as an answer to a question on Quora.

It forces you to figure out how to solve a problem in a way that can be explained to a computer.

By doing this you learn to break down a problem into its components, how to define the problem using constraints (you have to figure out what’s going to go in and what’s going to come out of your code), you have to be precise, and you have to have a full understanding of all of the components of the problem and its solution in order to solve the problem.

This define and disassemble approach to problem solving is something that is very applicable to problems across many domains, especially science, engineering, and mathematics.

Quora Answer: When people at tech companies or startups in Silicon Valley refer to “cultural fit” with a company, what exactly do they mean?

I originally wrote this as an answer to a question on Quora.

It’s code for “is this person like us”. That almost always means that the company lacks diversity in any form. Most of the employees tend to look the same, have similar backgrounds, and similar interests. In tech that usually means white males under 40, but that’s not always the case. It could be all Indian males under 40.

While this may lead to increased harmony and fewer conflicts, it’s also a terrible form of laziness. Companies that emphasize cultural fit will tend to be terribly deficient in HR and likely be inept at interacting with people who aren’t just like them (including customers). It’s basically a cop-out for those who can’t be bothered to make the effort to be inclusive and learn to work with people who are different.

Most places that emphasize cultural fit bear a striking resemblance to fraternity houses. That’s not my thing and I’d rather work with grownups who make an effort to function within the context of the rest of the world rather than creating their own isolated little bubbles of groupthink. I also learn more from people who aren’t just like me.

Quora Answer: What are some ways that programming was better in the past?

I originally wrote this as an answer to a question on Quora.

In the “bad old days” the actual programming work wasn’t much different, just worse. Monitors were smaller and lower-resolution, chairs more uncomfortable, you had to spend a lot more time waiting for the computer to finish what it was doing, hardware was much more expensive, tools were barely capable, there weren’t pre-existing libraries for much of anything, open source wasn’t a thing, and there weren’t good marketplaces for selling what you created.

There was no internet, no StackOverflow, and no communities to go to for help. You did not have a Meetup group in your town where you could talk to other people working with similar technologies.

The one thing that was better in some ways is that fewer of the “interesting” problems had been solved yet, and many of the solutions and applications that were built were created by a single person with an itch to solve a particular problem.

Though sometimes I wish I could go back to a time where Java did not exist yet, programming really was awful in the old times. The same goes for the web. Sometimes it feels like it would be nice to go back to a time before it was hypercommercialized and mainstream – a time before the popup ad existed and before you couldn’t avoid being inundated with inane celebrity gossip on sites like Twitter and Facebook – but no. It was nearly useless back then.

Quora Answer: Which career is more fun, networking or programming?

I originally wrote this as an answer to a question on Quora.

Full question:

“Say networking is a blanket, for IT, systems administration, and network security. We’ll say programming is basically web app development. Can anyone with experience in either of these fields give some pros and cons about what they do and do not like?

I have basic knowledge in web development and none in networking, but I have recently taken a liking to getting involved with network security, and I think I would like systems administration more than programming so I am curious as to your opinions.”

I’ve done both, and still do both.

I almost always prefer building software, but there are almost no days where it’s easy. The big draw for me is that it’s creative work and is a form of artistic expression, more in the way that literature is than painting. I also find that my level of enjoyment is inversely proportional to the number of people involved in the process.

Both paths rely on having a great deal of knowledge. While you can look up specifics, having enough background to understand the specifics takes a lot of  learning and experience. Once you know the material well, networking and system administration are usually easier and less stressful, requiring a lot of thoroughness, attention to detail, and keeping track of things.

Both rely on having a set of tools that you rely heavily on. In software, it’s your IDE, debugging tools, libraries, frameworks. In systems it’s a set of utilities, commands, scanning tools, or even hardware tools.

What I dislike about what you put in the umbrella of “networking” is that if you’re good at it, it can get to be pretty boring. When systems are running well, you’ve earned your pay, but it’s when things are breaking that it gets most interesting. And stressful. And you get the least recognition for your efforts. Though ages ago I was an MCSE and worked with Microsoft products regularly, I’ve grown into being a Linux bigot and can’t stand working with non-open-source systems. Luckily I’m in a position where I don’t have to, but you may not have a say in what things you end up working with.  Same goes for programming, really.

One place where it gets interesting is when you combine the two, and that’s what I refer to as DevOps. It’s system administration AND programming. This involves working with tools that create and configure servers, deploy things, set up security, and perform all sorts of IT administration tasks, but instead of being done by a team of 20 IT staff, they can be done by one person. Things like Chef, Puppet, Vagrant, Salt Stack, Ansible, Fabric, Capistrano, and Nagios are popular and worth exploring.

It’s a lot harder to find a “sane” work environment in programming. Almost nobody who hasn’t done it appreciates how incredibly difficult it can be. Douglas Crockford said that “computer programs are the most complex things Humans make”. Being able to manage an environment like that where people aren’t just cogs in a machine takes a special kind of person, and unspecial managers abound. Lots of little things can add up to make a chaotic and stressful environment. It’s often more “interesting” in product companies, but those are a small part of programming. Most programming is writing “boring business apps”, like order entry systems, sales report generators, accounting tools, and other business-specific pieces. It’s no less challenging, but often less stressful and more stable than building software products. The challenges are just different, and one’s not better than the other. I’m a product person, you may not be.

Before making a decision, I’d recommend exploring the world of DevOps first. Not only is it a growing trend and career path, it’ll give you a taste of the administration side of things and you may fall in love with it. Or you may like every part except for the programming and go full IT.

Quora Answer: What is the best way to convince a founder that agile is not the best way to do everything for an effective startup?

I originally wrote this as an answer to a question on Quora.

Good luck with that. At this point “agile” should be considered a psychological disorder in need of treatment for people who cling to it with a death grip no matter how badly it fails in their organization. Let’s hope your founder is not one of those.

Find out why they think it’s so great. It’s probably because they’ve heard that it makes developers more effective and causes you to ship higher-quality software more often. In many cases and in many organizations this is true, and they’ve probably worked somewhere where that was the case. I’ve seen it work better for larger organizations than small ones.

In places where it doesn’t work, it just adds busy work and friction to an already-challenging process. Often trotted out is the No True Scotsman fallacy — that says that if agile is failing for you, you’re not doing agile right. No, in some places it’s just doesn’t work. If you’re a smart person that it doesn’t work for, it doesn’t mean that you’re broken. You just work differently and it takes a different approach to be effective.

In many startups, adhering to some arbitrary set of rules will make people less productive than just caring about the quality of the product, doing the best you can, and making sure you’re doing the right things. Yes, you need source control and an issue tracking system and have to have your design goals written down somewhere and understood by everyone involved. It’s not because you need to keep track of whether someone is working — that’s always obvious — but you need to have enough structure to know what you’re doing now, and doing next.

Some people use agile as a crutch and excuse to not actively manage their teams. It is NOT POSSIBLE to put your development team on some sort of auto-pilot method treadmill and expect an awesome product. The builders need to talk to the business people ALL THE TIME. If this really is a proper startup, every single day you know more about how to build what you’re building and what things are or are not possible or easy/difficult, and you’ll have to revise your development plan constantly. If you’ve gone more than a week without any priorities changing, it may be possible that your company has exactly the right laser focus on exactly the right thing. More likely is that the business is not learning enough about its customer’s needs fast enough. And, in some places two or three week sprints are just too slow and cumbersome.

It might also be possible that you’ve not given it enough of a chance, but I bet you wouldn’t be asking this question if you hadn’t already given it an earnest try.

Quora Answer: Is it rude to change a co-worker’s badly written code?

I originally wrote this as an answer to a question on Quora.

Full question:

“I joined a company as a new grad and currently working with two mid level and one senior designer. It’s a chip design company. One of my co-worker’s code is very badly written and I am pretty sure there are many other good ways to implement the same thing. Badly written code bugs me a lot and I’m tempted to change/improve it. Will it look bad if I go ahead and change his code? Or should I ask his permission beforehand ? I’m not sure what will be a good move as I’m comparatively new in industry. I don’t wanna look arrogant!”

Yes. It’s also rude to write bad code.

The best way to handle that is a code review. It’s not necessary to have anything formal – just spend a few minutes going over the code together so you can point out what had issues and why.

Just changing the code without communication will cause two problems: The original writer is likely to resent it, and the original writer won’t learn anything and will keep doing things the same way. There’s also a risk that you don’t fully understand the intent or scope of the changes and your “fixes” might break something.

Don’t make it adversarial or approach it in a way that puts the other developer on the defensive. That will make them non-receptive to the conversation, and the goal is to improve the code and the developer both now and in the future.

Analysis of Search Engine Crowdfunding Campaigns on IndieGoGo

This was originally posted on wbsrch.com. It is reproduced here to preserve history.

In the process of researching crowdfunding campaigns, I searched IndieGoGo for search engine pitches. I found 22 attempts to fund “actual search engines”.

Here is a list (with links to the IndieGoGo campaign):

TheNet101
Thumbar
Jixty.com
Xense
Iyiyes
Aspinosa
Rexyo
Asim Shah (unnamed project)
iSearchonline.tk
Slikk
ISearch2Help
MeSeek
Vexed Inc
Personalized Curated Mobile Search (no official name, so calling it PCMS)
Qrate
reSEARCH
Fedge No (unnamed project)
Crackerror
Aglepie
QuickVu
Chronologically
Nintag

Some of these have launched campaigns more than once, but I’m only counting them once. I’m also not counting niche and vertical search, only general search engines.

Aggregate Statistics

Project Asked Pledged Backers Date Comments FB Twitter G+ Pitch Quality
TheNet101 500 0 0 2014-03 1 0 0 0 C+
Thumbar 350000 1050 3 2013-12 2 2 183 0 B
Jixty.com 25000 25 1 2012-10 0 4 0 0 C-
Xense 2000000 0 0 Ongoing 2 0 0 0 D+
Iyiyes 250000 0 0 2012-09 0 0 0 0 C-
Aspinosa 35000 0 0 2012-03 0 0 0 0 C+
Rexyo 250000 0 0 2012-09 0 0 0 0 D*
Asim Shah 1500 40 8 2012-08 6 28 1 8 C-
iSearchonline 5000 0 0 2012-12 0 0 0 0 C+
Slikk 100000 40 2 2013-12 2 26 298 2 B
ISearch2Help 550 0 0 2013-05 0 0 0 0 D-
MeSeek 75000 77 5 2014-03 6 1000 517 0 A-*
Vexed Inc 500 0 0 2013-08 3 4 0 4 C-
PCMS 200000 0 0 2013-11 2 672 0 0 B-
Qrate 8500 105 5 2012-12 2 16 1 0 B+
reSEARCH 600000 0 0 2012-12 1 2 7 0 B
Fedge No 20000 0 0 2012-05 1 0 1 0 C-
Crackerror 5000 1 1 2014-07 4 13 1 1 D+
Aglepie 5000 0 0 2011-01 0 0 0 0 C-
QuickVu 867943 100 1 2013-11 4 2 0 0 D
Chronologically 3000 0 0 2013-09 2 0 0 0 D+
Nintag 100000 0 0 2013-03 0 0 0 0 C
* Video missing (deleted from YouTube)
** Some of these are denominated in GBP. I didn’t really pay attention to which, but it doesn’t change the numbers meaningfully, since they’re all nearly zero.

Statistics Summary

No campaigns were fully funded.

None with zero Facebook shares had any pledges.

Only four were shared on Google+ and of those, only 3 had pledges. As always, G+ is not relevant unless you’re a Google employee.

Only four had more than one person listed as being on the team.

14 of the 22 had no backers at all.

9 of the 22 asked for a six-figure or higher sum.

14 projects were not shared on Twitter. Of those, only 2 had pledges.

The biggest pledge was $1050. Of that $1025 was by someone related to (same last name as) the pitcher.

$64 average per campaign, or $19 not counting the bid by a relative.

I tried to find out what became of these pitches and whether they continued after the
failed campaign. Some point to domain parking pages, some to sites
not related to search at all, and at least one points to a malware
site. The best part about the malware site was that a popup said that my “Ubuntu needs
updating” and the update showed up as being for “Ubuntu by Microsoft, Inc.”
Hilarious. Can’t find it again or I’d share a screenshot.

Postmortem

Here are the ones I was able to find anything about:

Nintag (gone as of 2015-09) is a Nigerian search engine. If you search for the words Yoruba, Igbo, or Lagos you
find real results, most of which are based in Nigeria. If you search
for the word “cheese” you get zero results. I guess they don’t
have cheese in Nigeria, and that makes me a bit sad for them. Even though the results are somewhat questionable to a non-Nigerian,
they appear to be at least partially accomplishing their mission.
That’s good because Nigeria should have its own search engine. They
certainly have enough people to serve. I wish them well.

Jixty (gone as of 2015-09) exists. It’s not obvious at first glance, but based on the search results for
“pants” being essentially identical to Google, it looks like they’re a front end to Google (a google Custom Search Engine).

Iyiyes exists. It, too, appears to be a front-end to a Google custom search. I don’t
know why sites bother if a big bag of nothing is all they’re going to bring to the table. Google’s already doing Google.

MeSeek.com is also interesting. They had the strongest social effort and
scored the highest in my opinion of the different pitches, though
nobody had an “A” rating (but they could have had an amazing video presentation). MeSeek appears to let you rate search
results, but the results look like they came from Google (I can’t be sure). They have
news, horoscopes, and weather, a publisher program, stock quotes, an
advertising platform that only appears to show ads for MeSeek, and
even an article directory, too. You can even change your background on their site. It’s
also in the Alexa top million (around 200,000 as I write this). The campaign was started by a fella inamed Charles Forell, and I’d like to have a chat with him to see what he’s up to.

TheNet101 is also an interesting result. They’re a meta-search engine that includes results from Google, Bing, Blippex, Wikipedia, Blekko, Yelp, Archive.org, and Faroo.

Wait, what’s Blippex?

Blippex.org is a search engine that ranks results based on user engagement determined by using a browser plugin that measures “dwell time”. It’s a very Alexa-like system, one I understand well since I built one to gather data for Alytik, but it adds a bit more.

However, it appears that Blippex is the walking dead. Despite search being online and showing a URL count of 29.3 million, there haven’t been any blog posts since October 2013 and no Github activity since October 2013. This seems like an interesting idea that has been shelved, but not shut down yet. Neat, though, that you can change the search results by adjusting how much the dwell time and age metrics affect ranking.

Thank you, TheNet101.com, for introducing me a new and interesting search project, even if it may not still be active. Sorry your crowdfunding campaign didn’t work out.

Conclusions

In most of these pitches, the pitcher didn’t claim any particular domain knowledge required to build a search again, let alone claim software development skill. Combined with a scarcity of working demos, it’s unsurprising that none of these were funded.

Social sharing helps, and no sharing at all is pretty much a guarantee of failure.

Saying you are going to beat Google does not help. I wonder why.

IndieGoGo also doesn’t seem to be the best choice for funding a search engine.

Update September 2015: noted dead links

AdSense Alternatives for Startups and Small Websites

This was originally posted on wbsrch.com. It is reproduced here to preserve history.

In starting WbSrch, a search competitor to Google, I knew that at some point Google would find a way to “invite us to leave” AdSense. The Terms of Service make it clear that it is incompatible with a search engine (can’t have ads on pages that link to adult content, gambling, etc.)

That day came a little over a month ago when I received a message that ads were no longer running on the site because Google discovered a violation of their TOS in one of the result pages for a particular adult-oriented search term.

Sure, I could remove the offending link from the search results page (which I did because it also didn’t fit with the WbSrch inclusion policy), but that sort of thing would be sure to happen again. Around one sixth of the URLs on the web are porn, so it’s virtually impossible to exclude it all. Be very skeptical of anyone who claims they’re able to block all porn.

The Advertising Options

From my research, these are the notable companies that do online advertising:

Bidvertiser
Qadabra
Affinity
Infolinks
Advertising.com
Adversal
PulsePoint
Conversant (formerly ValueClick)
Clicksor
AdBlade
AdSide
Vibrant Media
Yahoo/Bing Ads (formerly Media.net)
Link Worth
Tribal Fusion
Kontera

Contacting the Advertisers

I looked into all of them, eliminating those that
require massive traffic volume to get started or have a reputation for
spreading malware.

These are the ones I tried to contact (at the end of April) asking whether their service would be compatible with WbSrch:

Qadabra
Chitika
Yahoo/Bing (formerly Media.net)
Conversant Media
Kontera
Bidvertiser
Affinity.com
Infolinks

I asked the same question of every site:

Hello,

I run a small but growing search engine at http://wbsrch.com.

I would like to know whether your service would be appropriate for use as the advertising provider for this search engine.

WbSrch.com indexes and links to most of the internet. We try to
exclude adult and other “icky” sites from the index, but that’s not
possible to do with an automated crawler. This means that at any given
time there will be links to things we don’t want to index per our policy
(http://wbsrch.com/policy/),
but that will eventually be removed. None of this content is hosted on
our site, but it is linked to depending on the search phrase used.

The search engine has indexes in 25 different languages, though most traffic is for the English-language index.

Given the nature of search engines, would WbSrch.com be compatible with your advertising platform?

What follows are the responses to this message and the action I took based on the responses.

Outright Failures

Bidvertiser had a broken captcha on their contact form, so I couldn’t contact them. Their policy says that they don’t allow linking to some content types, so
they probably would have said no.

Bing did not have a contact form. They might now. I think
they are still in alpha/beta/whatever. Even so, they’re still a
competitor, so not something wise to use long-term.

Non-Responses

Chitika never responded to my inquiry.

Conversant Media never responded to my inquiry.

Affinity.com never responded to my inquiry.

Responses

Qadabra responded the fastest, saying that they were totally
compatible with search engines and that they already had some search engine
customers. The message had a friendly tone.

Kontera was the second response. They said they they are not
compatible with search engines, but they were polite about it.

Infolinks replied three days later (on a Sunday) with a
fairly rude message that said “our quality assurance team found that your site
does not meet our publisher criteria” and “We at Infolinks
have the responsibility to keep our advertising environment up to certain
standards to ensure the success of Infolinks for our
publishers, advertisers and those viewing our ads.” OK, that’s fine if you don’t
want to work with a new site, but don’t be rude about it. At least now I know they’re too special and important to ever do business with.

The Winner

Based on these responses I went with Qadabra. They also said
that they work with traffic in all languages. Great!

Setup was easy, and ads started working immediately. I had a
few glitches with some ads behaving strangely, but it was a minor thing. Every
time I contacted them they were very helpful and friendly.

You don’t really get control over the types of ads that are
shown. Most of what I saw were ads for video games and the occasional ad for Russian
brides.

I did not enable any of their rich media ads, just banners,
so I have no experience with those. I know they earn more, but I’m generally opposed to popups, popovers, flyouts, videos, and things that make noise. If I visit a site that uses them I’m less likely to return.

Qadabra revenue was significantly less than AdSense, earning about
one sixth as much per thousand impressions. Their system documentation says
that they optimize it over time, so if I gave them a longer trial period, income
would probably go up.

Now that WbSrch has switched to SSL-only (inspired by Reset the Net), I can’t use
Qadabra. They don’t have SSL support, so even if their ads
were enabled, they wouldn’t load. If they add that I’ll consider using them
again, if not for WbSrch then for other sites.

I like the people at Qadabra, and I’m happy with their tech
support, but this experiment has ended after only one month, and there don’t
appear to be any other reasonable alternatives.

Qadabra is relatively new, created in 2011, so
they are still polishing their game. If you want a reasonable AdSense
alternative for lower-traffic sites and don’t require SSL, I recommend them.

The long-term plan always was to build an ad platform
internally to use with WbSrch. Not finding a platform that is a perfect fit for
us is just another motivating factor.

For now, I’m just going to focus on improving the search engine so we’re in a better position to monetize it later on. Since traffic has been increasing by around 100% per month for the last few months, it shouldn’t be too long.

New Search Engine Launches From Oregon

Reprint of a press release originally published on PRWeb at https://www.prweb.com/releases/2014/05/prweb11876315.htm.

A new search engine has launched, and it aims to take back the web from corporate search interests.

WbSrch.com is the first general English-language web search engine created since Blekko was launched in 2010. WbSrch has indexes available in 25 languages. It was written using open source tools and offers an alternative to all-knowing, user-tracking data warehouses that invisibly privilege certain services and social network marketing above more relevant content. With major engines coming under increased scrutiny by antitrust and consumer protection organizations, it has never been more important to have more competition in the world of search engines.

WbSrch doesn’t aim to be the biggest or the fastest. It does things its own way and offers search results that are not driven by popularity or social media. It gives more weight to original, useful, human-created content and doesn’t track individuals or personalize search results based on what it thinks it knows about users.

WbSrch’s philosophy is that content creators should build things for real people, not to please algorithms. Big-search algorithm changes can affect even major businesses when it’s decided that they’re benefiting from so-called “unnatural links.” But linking is what the web is about. “Nofollow” tags negate the very nature of links. Consequences like that are inevitable, though, when there are too few options in the world of search.

Quite simply, the power of search needs to be spread out so that a system that every nation’s economy depends on isn’t concentrated in the hands of so few. WbSrch is a move in that direction.

About Championix, LLC

Championix is a web software company focusing on search and analytics. It was formed in 2014 to help free users and webmasters from the stifling influence of a really small search market. Its first products, the WbSrch search engine and the Alytik Web Analytics Plugin, were released in 2014.

Why I Decided to Build a Search Engine (And You Should Too)

This was originally posted on wbsrch.com. It is reproduced here to preserve history.

I’ve always wanted to build something big, but never had a burning desire to create any one specific thing. Instead I built lots of little things – small desktop apps, weekend websites, etc.

It wasn’t until AltaVista shut down that I realized that the world needs another search engine. Not just one, but dozens.

Almost all of the search greats have been shut down, bought and shelved, or replaced their engines with Google or Bing. The only real competition we have in English-language search is between Google and Bing, and Bing has been accused of copying Google’s results.

Bear with me for a bit. What I’m about to say will probably sound a bit tinfoil-hat. It’s all just speculation, but there are so many billions of dollars involved that at least some of this sounds plausible.

A search engine is the gateway to the world’s information. If you control
the gateway, you control the information. Knowledge is power, and
gatekeeping is big money. At this point, the leading gatekeeper has too
much money and power and the effects are causing real harm to businesses
and the world economy.

Each time Google changes its algorithm, hundreds or even thousands of businesses are damaged or destroyed. That’s the nature of godlike power – even if you’re just trying to use a little bit, there are side effects.

“Combatting spam” is the main reason cited for their algorithm changes. If you look
at it, much of what Google classifies as spam could just as easily be
classified as “things that don’t make us enough money”. They’re a public
company. They’re required to maximize revenue. To do otherwise would
expose them to shareholder lawsuits.

It’s easy to see the incentive for demoting sites that have ads that earn you 4 cents per
click in favor of sites that earn 12 you cents per click, or of doing
things that wipe out competing advertising companies. There has been an
ongoing war against “selling links”. That’s called advertising. What is
Google Adsense? It’s an ad service that sells links. Convenient, though,
that competing ad service text-link-ads.com was a casualty of this war on paid links, isn’t it?

Other things also seem suspicious, like the war on guest blogging. Is this Google being jealous of people finding other sites without going through their search engine?

Now we have another wave of chaos, which some people have referred to as “breaking the internet”.
Google added a “disavow links” tool and has many webmasters afraid to
link or be linked to for fear of an “unnatural link penalty”. I get one
or two emails a day asking to remove a link to a website because the
webmaster is afraid Google might not approve. Google has said (or
really, implied) that you don’t need to have links removed, but there’s
one benefit to less linking on the web: You’re less likely to find a
site without going through Google if there are no other links to it.

While this is going on, Google is also adding more direct answers to
searches. Things that you might click on a link to find before now show
up directly as answers so you never have to leave Google’s site. A
question like “How tall is the Eiffel Tower?” could have led you to an
exploration of lots of wonderful information about the Eiffel Tower. Now
it just gives you a fact, and you can go on to the next search. This
means Google is driving less traffic to websites and everyone who isn’t
Google is suffering, slowly but surely getting less traffic.

All of this sounds very much like anti-competitive practices to me, the kind
that you get hit with a regulatory hammer for. Even if only some of it
is true or intentional, it’s a dangerous abuse of power that needs to be
investigated. It’s fine if you want to be the biggest and best search
engine, but don’t be evil.

Why do they get away with it? Because webmasters and users let them. Websites are rarely built for Humans
anymore. Instead, they’re written for the Googlebot with Humans as an
afterthought. That’s why Demand Media was so successful – they designed
and created their content for Google search.

The world needs more search engines. That much power should not be concentrated in one
company’s hands. Next time you search, consider using something other
than the market leader. Bing, Blekko, DuckDuckGo, WbSrch, and Gigablast
are all options. I wish there were even more. If you have the skills,
now has never been a more important time to start building. It’ll take a
while to build something good, but it’s needed. Just don’t try to do it
the way Cuil did – trying to index more pages than Google and running
out of money before they figured out how to be useful. Find a way to be
useful to people, and focus on that.