Dear Whereis,

Working with large clients, for a large digital agency and part of a large advertising agency I get to take part on many pitches to large clients. A recent pitch we did was to Whereis, the mapping data company which is part of Telstra. During the investigation into the website I came across the sub domain site www.poweredby.whereis.com - which is the site dedicated to selling their maps and services. It must have been built around late August, early September 2008.

Not a bad looking site, a little bit old school and not polished like the main domain. The homepage has a low pagerank but sub-pages have good solid pageranks.

The biggest revelation was that this site was not listed in Google. A site command revealed no pages were indexed in Google at all
site:www.poweredby.whereis.com

My first thought was “how tragic” .. not so good for business to not be found in Google. My second thought, rightly or wrongly was “I guess they must have planned it” (?). It was the second thought that spawned a whole bunch of questions in my mind about why a brand would want to not have their site indexed in Google and then questions about whether there are other brands and websites out there that knowingly do the same thing? There are sure to be sites out there that unknowingly are in a similar situation though, that’s for sure - maybe a site moved from a development environment to a live server with a robots.txt file disallowing indexing, maybe noindex tags in the html .. they have to exist, surely, but do companies ever want to block search engines from their normal website content? - maybe they do sometimes!

Anyway, Mr/Miss Whereis .. you have a basic SEO problem. You have a robots.txt file in your root folder which is configured to not index your whole sub domain website www.poweredby.whereis.com

The file can be seen at http://poweredby.whereis.com/robots.txt