The Bing Experiment

01 Jul 2013

Observations from 100 Days of Bing

Intrigued by the Bing It On challenge, in March 2013 I switched from Google to Bing for search. The challenge itself is a false indicator of what service is better. The search results shown in BingItOn have no user context.

Adding Context

Since the essence of search is relevance and without understanding the users context it is difficult to be relevant. Therefore a more accurate experiment would require me to use Bing as my primary search engine for 100 days. Allow Bing to learn from my previous searches, build context and consequently enable more relevant searching.

A search engine that understands who you are, can tailor results for you. For Google, understanding who you are is done by linking your past history via your Google Account. Millions of Google Search users are logged are logged into their Google account because of Gmail and Youtube.

Bing cleverly integrated with Facebook. It gave Bing social search features with a platform that Google doesn’t support, but more importantly it solved Bing’s context problem. Facebook is the gateway to a billion active users and their interests, ‘likes’, location and friends.

Without linking your Facebook account it would be impossible to make a fair Bing to Google comparison. BingItOn strips all context from your search making Google search essentially as clueless about your personal preference as Bing Search. A better experiment would mean signing into Bing with my Facebook account and giving it 100 days to learn about me and be relevant.

Observations

For over 15 years, Google has trained users into using search as the gateway to everything. Here I break down a handful of different types of searches.

Round 1: Bookmark Searches

Searches where you know that answer or the page you’re looking for are essentially ‘bookmark’ searches. I want the Wikipedia article on netflix but I don’t search through Wikipedia, I search on google and go to the wikipedia article on netflix. Little to no user context is needed for a ‘bookmark’ search and Bing and Google perform perfectly in such scenarios.

Google 1 : Bing 1

Google owns Zagat, Bing partners with Yelp. Bing has a neat heat map feature that gives you a better feel for density than Googles pins but I have yet to feel unsatisfied with Bing’s local search tools. +1 for both.

maps

Google 2 : Bing 2

Searching Bing for programming related queries is painful. Interesting after even 100 days, Bing fails to grasp the context of terms. When I query express orm I get the clothing store Express, ‘sinatra models’ links to Frank Sinatra and not the Ruby Micro-Framework. Bing also has a peculiar affinity towards Microsoft technology. After 4 consecutive ‘ruby’ queries, if I drop the word ‘ruby’ from the query I get C# results.

This has proved extremely frustrating, but it is even worse when Bing points to relevant yet bad links. For example, Bing may link to forums discussing the problem I am looking to solve but these links have no resolution or lead me down the wrong path. The same Google search yields the perfect answer on more reputable sites like stackoverflow.com. With Bing my time is wasted reading and browsing several pages before getting the correct one. All things considered my programming productivity has dropped 30%, because I rely so heavily on Google for documentation and problem solving.

This is worse than lacking a feature, this is a -1 for Bing.

Google 3: Bing 1

I personally have few friends on Google Plus and follow a handful of strangers. Seldom have I found social search features useful on Google. Bing’s Facebook integrated social search proves equally worthless, nobody gets any points.

Google 3 : Bing 1

Googles tight integration with Drive and Gmail made searching for my files and emails super easy. Searching for my flight information is a breeze. Bing doesn’t offer any of these right as of July 2013. Outlook.com and SkyDrive integration would be great feature to add.

google personalize results

Google 4 : Bing 1

Not every person we search online is ‘famous’ so to speak. Google and Bing do a great job of find the non-famous people but surprisingly Bing trumps Google with its Linkedin, Yelp, Twitter and Klout integration.

Bings ‘snapshots’ for ‘Oprah’ and ‘Ferrari’ are expected but it is incredible when they do with developers, and professors and even a snapshot of yourself.

snapshots

Google 5 : Bing 2

For some searches, you have no idea what you are looking for, or how to look for it. For example, when I was buying and electric drum set, I wanted to know if the noise would disturb my neighbors.

What do I search Bing for?

  • Bing.com: roland noise drum Set - All about Roland Drums, nothing about noise
  • Bing.com: electric drum noise - Irrelevant
  • Bing.com: electric drum noise levels - Not Helpful Yahoo Answer
  • Bing.com: electric drum noise db - Irrelevant
  • Google.com: electric drum noise - YouTube video Titled ‘How Electric Drums Sound Without Amplification’

Google saves the day. Here are a handful of other blind searches that Bing failed to answer adequately that let me to fall back on Google

jetblue add points retroactive - Google Search
library not found for -lX11 - Google Search
chord algorithm explained - Google Search
how to set gdb bteakpoint with signal - Google Search
level up is down to half its head count - Google Search
libasync documentation - Google Search
how to forward declare a typedef - Google Search
anonymous functions in ansi c - Google Search
pthread joinable thread can't access memory - Google Search
osx sdl waitevent not responding - Google Search

Google 5 : Bing 2

Statistics

Since I changed the default search provider in Chrome from Google to Bing, I could look at my browsing history at chrome://history to revisit every bing search I have made, interestingly it is cataloged locally as a Sqlite3 database.

# make sure Chrome is closed
$ sqlite3 ~/Library/Application\ Support/Google/Chrome/Default/History 

Turns out that I made 2802 Bing searches. From Bing I visited 3984 sites averaging about 1.42 clicks per query.

I had to resort to Google 252 times. Thats 1 Google search for every 10 Bing searches, most of which were programming related searches. Consequently Google.com has become my 4th most typed url.

UrlTyped Count
http://www.jimmyr.com/612
http://news.ycombinator.com/266
https://news.ycombinator.com/107
http://google.com/101
https://github.com/97

In case you’re interested in how I queried this,

-- How Many Visits Since March 31st 2013

SELECT count(*) FROM urls
WHERE last_visited_time > 13009251222752164 
      AND url   LIKE "%q=%"
      AND title LIKE "%- Google%"; -- change to Bing for bing searches

-- Get 5 Most Typed In Sites

SELECT url, typed_count FROM urls ORDER BY typed_count DESC LIMIT 5;

-- Pages Visited From from Bing Searches

SELECT count(*) FROM urls, visits src, visits dest 
WHERE urls.last_visit_time > 13009251222752164 
      AND urls.title LIKE "%- Bing"
      AND urls.url LIKE "%q=%"
      AND urls.id = src.url
      AND src.id = dest.from_visit;



Future

After 100 days immersed in Bing search I am frustrated and less productive. Considering Bing is 4 years old and catching up to 15 years of Google glory, frustrating and less productive are not that bad.

The only leg up Bing has on Google is its Twitter, Linkedin, Yelp and Klout integration.

There are something things Bing can easily do in the near future,

  • Outlook.com, Office 365 and SkyDrive integration
  • Bing as a platform for developers and Azure
  • Deeper integration with Twitter, Facebook and Linkedin

However none of these things will improve ‘true’ search relevance. To improve that Bing needs more users, more clicks, more feedback. More relevance breeds more users. Its almost a chicken or egg problem, except Microsoft can pay and power their way towards more users.

Bing is becoming a formidible developer platform for Search, Maps, OCR and Translation. Its APIS are less restricting or cheaper than Google, because Microsoft is willing to subsidize this. Bing integration in Windows 8.1 will also guarantee Bing tens of millions of users. All these will mean more data for Bing and more relevance in the long term.

For me however, I miss Google and so June 30th will be my last day of Bing for 2013 at least. I might revisit it again in 2014.