Yearly Archives: 2015

Setting Up a Redash Dashboard

This was originally posted on wbsrch.com. It is reproduced here to preserve history.

The more WbSrch evolves, the more it becomes necessary to keep track of a bunch of metrics.

Until now we’ve been using a mix of simple report pages and raw SQL queries. It has worked well enough, but not having a clean way to track things in a single place is a nuisance.

That’s why I was happy to discover the redash.io open source project. It’s a query tool meant to be used for setting up business intelligence dashboards and it works with a wide range of databases.

No stranger to code, I tried to check out the GitHub source and get it running on my local machine. It didn’t quite work out. They have a bootstrap script, and it had some trouble with my particular system setup (it fell over when it came to configuring local database users).

But they also have EC2 AMI images you can launch to get running in AWS. I fired up an Amazon micro instance on the free tier and had the app running in seconds. It only took some minor configuration to get set up with my SSL certificate, and I was ready to go.

Connecting my three PostgreSQL databases was easy and the clean interface made it easy to find the query editor. After running a few queries I had the feel for how things worked well enough to save them. It also lets you set a refresh interval on your queries so you can have data refresh daily, hourly, or whatever. Results are cached so you’re not taxing your database gathering totals every page load.

After you have a few queries, you can start adding them to a dashboard as panels. You just select the query name, the visualization type (you get table by default, but can add graphs and charts in the query builder), and the widget size.

This is a dashboard that I built to keep track of the search traffic and index state for the Somali-language version of WbSrch:

I created dashboards for each supported language plus an overall meta-dashboard. It was fairly quick, taking about a day to set up 35 dashboards and about 200 queries.

Luckily the interface is pretty good, because once you have the software set up, that’s where the documentation ends. You can figure out most things with experimentation (trial-and-error), but it would be very helpful to have a few getting started tutorials, or at the least an explanation of how the various visualizations work.

A micro EC2 instance may stumble if you have some large queries (selecting an entire table is a bad idea, don’t do it), or a lot of things refreshing, but it kept up pretty well.

WbSrch, the Independent Search Engine, Expands

Reprint of a press release originally published on PRWeb at https://www.prweb.com/releases/2015/11/prweb13073007.htm.

WbSrch, an independent search engine based in Oregon, has expanded its data center, growing from a single dedicated server to three.

Founder Jason Champion had this to say about the expansion:

“We’ve grown enough that a single server no longer meets our needs. Tripling our footprint will allow us to continue growing and improving our algorithms throughout 2016.

We chose colocating servers over cloud hosting like EC2 because we’re running a very RAM-intensive operation. Most cloud hosting companies make their profits on memory, so once you go beyond a certain point, it’s much less expensive to host your own servers.

Our hosting provider, Opus Interactive, has been great. Their price, quality of service, and reliability has enabled us to continue improving our algorithms without worrying too much about infrastructure costs, and they have plenty of rack space available if we need to grow quickly.”

Started in 2013 and launched in 2014, WbSrch is still quite young, with an index of 10 million pages grouped into more than 30 languages. Traffic has been steadily increasing as the index grows and algorithms improve.

Rather than trying to crawl and index every page on the web, WbSrch aims to build quality results by weighing what is excluded just as heavily as what is included. More than 500,000 domains have been excluded, based primarily on content language and a few other criteria. The number of crawled pages, indexed keywords, and excluded sites is published on the site.

An Experiment with Project Wonderful

This was originally posted on wbsrch.com. It is reproduced here to preserve history.

I’m always looking for new and efficient ways to let people know about WbSrch. That’s why I decided to try advertising with Project Wonderful.

Project Wonderful was built as a banner ad network for web comics.

That doesn’t mean you can only advertise web comics or that advertising can only be placed on web comic sites, but that’s its core demographic.

As a trial, I ran an ad for WbSrch on a few sites that seemed like they’d have people who would be interested in trying out a new search engine. That means other search engines, SEO sites, and literature sites. I also wanted to find out whether webcomic readers were a good target audience.

I deposited $100, and after spending about $70, I think I have a pretty good idea of what works and what doesn’t.

If you want really fine-grained control over your campaigns and ad spending, this is the perfect network for you. You know exactly what you’re going to spend per day on a site, and you can bid on traffic on a per-region basis. Their regions are US, Canada, Europe, and Everywhere Else.

The search functionality is amazing. You can search for gaming sites that have traffic that is at least 50% from Germany and has between 100 and 10000 page views per day, for example.

As a publisher, you can set per-region bid minimums and can auto-approve bids, or require manual approval. This means that you don’t have to worry about running ads for things that you’d be opposed to, so no bacon ads on a veganism site.

Results have been mixed, and I’ve learned more about the types of people who are interested in trying WbSrch.

Some takeaways:

Webcomic sites have a high number of page views, but the number of unique users tends to be a fraction of that. The same goes for SEO tools.
Blogs tend to have more unique users and fewer page views.
Literature sites are somewhere in-between.

Here are my slightly-obfuscated results:

Site	Pageviews	Unique Views	Clicks	Spend	CPM	CPC
A Major Webcomic	581953	11126	35	13.34	0.02	0.38
An SEO Site	233489	17881	141	50.54	0.22	0.36
A Poetry Site	60780	4876	34	5.64	0.09	0.17
A Dutch Site	15584	305	4	0.73	0.05	0.18
A Hungarian Site	3485	730	1	0.35	0.10	0.35
A Search Engine	2711	963	40	0.62	0.23	0.02
A Swedish Site	2315	621	0	0.39	0.17	INF
A Movie Blog	1424	477	0	0.25	0.17	INF
A Knowledge Blog	1285	967	1	0.42	0.33	0.42
A Web Directory	296	83	0	0.05	0.17	INF
A Science Blog	231	124	1	0.29	1.24	0.29

The efficiency varies by site, but some are unbeatable deals for targeted traffic. Others are pricey, but just the type of people that will spend some time searching for themselves and the things they control. Hopefully we’ll be good enough for them to come back again.

There are some sites that I’ll run ads on as long as they exist even though the traffic is low. It’s easy to convince people trying a new search engine to try another new search engine.

I also suspect that my Hungarian and Swedish translations aren’t very good. I know basic Swedish, but the Hungarian is robot-translated.

One of the limitations of Project Wonderful is that if you have a large budget, you may run out of places to advertise efficiently, and for those things that are efficient, they may not get enough traffic to satisfy your hunger (2-cent clicks from your site? I’ll buy at least 1000 per day!). I could easily see struggling to spend a $1000/day budget effectively. If you’re prepared to work on a smaller scale, there is probably no better place to test-run ads because their data and reporting is good and you can learn a lot from your experiments. They also have enough fine-grained control that you can iterate and learn quickly.

$70 is hardly enough to get the full measure of an ad network, but I think I was able to get some useful data out of this experiment. Try Project Wonderful, you may just find it wonderful for your project, especially if your project plays well to webcomic audiences.

Arcade Tokens

These arcade (and other) tokens were originally posted on stampscoinsnotes.com.

Quora Answer: Are analog synthesizers overrated?

I originally wrote this as an answer to a question on Quora.

To put it simply, yes. Analog synthesizers are absolutely overrated. I’m referring specifically to subtractive synthesizers.

The differences in sound quality really ARE quite minor, so don’t feel bad if you can’t much tell them apart.

That’s not to say they’re not useful. They certainly are, and I say this as someone who owns more than one. They each have their own distinctive sound, but analog sounds are quite limited. I don’t mean you can’t create a wide variety of sounds with them, but rather that the type of sounds you can create don’t vary much from one analog synthesizer to another.

Most of the love (and idolatry) for vintage analog gear comes from how groundbreaking and iconic certain synthesizers are and how important they were for music of their time and the amount of joy the music created with them has brought to listeners. Just like most vintage cars aren’t actually very good cars, the historical and nostalgia aspects of vintage analog synthesizers affect their value greatly.

From a music maker’s perspective, the analog sound is worth it to a certain degree, but vintage analog not as much. 30-40 year old synthesizers break all the time, and you’ll be hard-pressed to find a single player or collector who hasn’t spent inordinate amounts of time and/or money on repairs. Good luck touring with an old Jupiter or CS. Just use or create the sounds you like and don’t worry about how they came into existence. The nice things about most modern synthesizer workstations is that they all have a massive library of sample-based sounds (including the sounds of vintage analog), one or more kinds of virtual analog synthesis, and the ability to load samples. Find a friend who likes vintage gear and sample some of it, because a working sampler or virtual analog sounds far better than a broken analog.

In a way it’s good that there’s an analog fad going on right now. New analog gear is being created and can substitute very well for decaying vintage gear. A prophet 6 is a great buy when you can get it new and it has much better capabilities and stability than a prophet 5, not the least of which is a large amount of patch memory.

Xangis.com

Software and Music Mad Scientist