Peter's Blog

I've been using statcounter to track visitors to this site for a few years now. While my site5 hosting also provides awstats I prefer StatCounter as it uses javascript for the tracking and so is less likely to track bots and other rubbish.

The free statcounter service used to show the details of the last 100 visitors to a site. My site gets around 1000 visitors a day so 100 visitors is maybe two hours worth. Statcounter have now increased this limit to 500 visitors which is more like half a day and consequently more interesting.

Currently the most popular page is this one about using T_hunderbird with M_icrosoft exchange (see what I did there? don't want this page getting those hits). This page about f_ence sprayers has had a lot of traffic recently, it being spring and folk wanting to paint their fences. There don't appear to be many public forums for people to air their f_ence spraying grievances and this has become one.

Remember the search engine wars? Here is a status report:

If I were yahoo I'd drop the exclamation mark. Microsoft are nowhere to be seen.

I get lots of World of Warcraft related hits, lets look at some recent wow related searches:

• how to get skill up in weapons faster in wow
• wow daily quest
• wow g11 g15
• good way to earn sara armor
• winterfall flight path
• tanaris to ungoro crater foot
• world of warcraft fast firing ammo bag
• good prot aoe spots
• how to use the g15 in wow
• titan panel download patch 2.1
• heroic tactics

(that last one was wide of the mark).

It makes me wish all this info was nicely laid out and not in the heavy stream of consciousness format.

The warcraft stuff has boosted the page rank of this site to four which increased the number of visitors a day by about three or four hundred.

I'm not getting a kickback from Statcounter but despite that I still recommend them.

Thanks for coming.

Summary of site statistics for 2006, courtesy of Statcounter.

186,756 visitors! That's two or three football stadiums. An awful lot never come back though Still, I don't visit THEIR sites either.

This site wouldn't load. Looking in the status bar it was trying to contact Google Analytics to get a javascript file. This affected Firefox but not IE which loaded fine.

I commented Google Analytics out of the page and now it is working again.

I don't use Google Analytics a lot, I still prefer statcounter for my web statistics because of the detail it gives me. Google Analytics is more about pie charts for marketing people. Statcounter tells me what google searches people are using to find my site and that is much more interesting. Statcounter also gives graphs and averages and these tally pretty much with Google Analytics.

I'll leave just statcounter installed for now, I don't need both, it slows down page loading, google analytics is in the header and brings the whole site down if the server goes, statcounter is at the end of the html and most of the page still displays even if statcounter isn't responding.

Moral: don't link to unreliable servers in your page header.

Since I started with statcounter very nearly a year ago, it tells me my site has served up 100,214 pages to 72,521 visitors.

I have quickly become bored with Google Analytics: I think it's appeal is more for marketing departments and powerpoint/pie chart enthusiasts. I like the raw detail that statcounter gives me in an easily accessable way. I can see what people are searching for and sometimes it even inspires me to update articles to be more useful.

I still think their counting is buggy: returning visitors seems unreliable, as if page refreshing counts as a return visit, so I take it with a pinch of salt and follow the trends. The numbers are roughly on a par with google analytics.

Statcounter is easier to install, it can go anywhere on the page, Google Analytics has to go in the header. For drupal this means editing the page template rather than sticking it in a block.

I have set up Google Analytics on this site. This is a very sophisticated looking system for tracking web site visits. Google have set it up to help their advertisers count their money but it is free if your site has less than 5 million page views/month which should suffice for me.

The analytics site is still a bit flaky and overloaded and it is refusing to validate the code on this site. I'll give it till tomorrow to settle down and I'll try again.

It looks to be more biased towards grand overviews than statcounter which is good for analysing individual visits: especially the free version that only tracks the last 100 page views.

Apart from the current flakiness, the analytics reports are only updated every few hours (like awstats I suppose) whereas statcounter is real time.

UPDATE: although the site still tells me the tracker is not installed properly and the link to an example installation is broken, the tracker is working and I can see the stats. So far it looks very sophisticated for a free (to me) service: it looks professional and expensive. Here is the geographical chart:

This shows a tooltip of each town as you pass the mouse over it (the analytics page, not the screenshot above) whereas Statcounter just gives a list of countries. However, in Statcounter I can drill into the visits from each country to see which pages they have been looking at.

I now have a new metric to worry about: depth of visit, i.e. people looking at more than one page

Analytics and Statcounter may complement each other: analytics for charts, Statcounter for the details of who is doing what. I could carry on using both, it's all a matter of how quickly I want pages to load and how much I trust both lots of servers to keep up with mine.

I was studying the statcounter logs for this site and lamenting how the country info only lists the last 100 page views. Because one visitor can look at 50 pages, their country will appear to take 50% of traffic, giving a distorted view of proceedings.

So I decided to update my awstats config with GeoIP to give long term country information. GeoIP is a library from MaxMind that converts IP addresses to countries. Country lookup is free, you can pay for higher resolution lookup (city etc) if you have the $$$.

I installed it as follows:

• Installed the MaxMind GeoIP C Library and perl library from here
• I downloaded and installed the country database.
• I let these configure and install themselves. This may lead to an unholy mix with my debian apt setup but it is more likely to work than if I start messing with it.
• I went into webmin and enabled the GeoIP plugin. A word of recommendation here, webmin does a great job of managing multiple sites with awstats. In fact webmin does a great job of most things.

There is an old version of GeoIP that comes as a debian package but it installs in a debian way and was not picked up by the MaxMind Perl module so I installed the MaxMind stuff by the book.

It seems to be running, it is showing visitors from germany and stuff but I need to let it run for longer and see if the number of visitors from unknown countries goes down (it probably doesn't add country info to old records only new ones).

This may end up simply telling me where the bots that make up a large number of my visitors come from: so far this month this site has used 1.15G of bandwidth. Accursed Inktomi Slurp (which I think is yahoo) has taken 177Mb of this. Googlebot has taken 77M but given me 7073 visitors against 182. Slurp is a good name. It has probably uploaded all the text on the site 100 times. I don't change it that much.

The number of visitors to my site has increased by about 40% in the last two days. Yesterday I got an all-time record of 509 unique visitors (according to statcounter). It's about a level that I can quote without feeling embarrassed. According to statcounters 'came from' thing they mostly come from google.

I must admit I don't totally trust statcounter. Today it shows this in 'keyword analysis':

4  7.41%	onenote firefox
3  5.56%	thunderbird taskbar notifier
3  5.56%	vnc ubuntu
2  3.70%	ubuntu mp3

This would imply that four different people came from google searching for 'onenote firefox'. If I drill down into 'onenote firefox' it only shows one ip address with eight entries. What gives ? How many 'unique user visitors' is this supposed to be? Why would someone come here from the same search four times?

I still trust statcounter not to show me hits from bots, crawlers, comment spammers and whatever. According to awstats I got 649 visitors. Also, in the awstats referrer list out of the top 25 entries this month only 4 are referrer spams links with obscene url's, the others come from techy sites or searches from other than google/msn/yahoo (I am on the statcounter free program which only shows details for the last 100 hits, awstats is unlimited).

This is almost credibility. Thanks for reading.

Another awTags feature is pinging technorati with tags (or something like that). From my statcounter logs I see I am already getting more traffic from there.

I never did get around to trying to install awstats. I've been using Statcounter but I fancied trying awstats with reverse DNS turned on. I can't do this on my Site5 host as they don't like reverse DNS. I didn't install it on Gentoo as that looked like big time hastle.

I realised today that installing awstats under Ubuntu should be as simple as installing the awstats package and it almost is. I can install it on my home server, download my Site5 access logs there and let awstats format them up.

Here are the steps I had to take to install it:

• Install awstats package
• Edit a file called /etc/awstats/awstats.hostname.conf where hostname is the hostname. Put something like this in it:

  LogFile="/var/log/apache/access.log"
  LogFormat=1
  DNSLookup=1
  DirData="/var/cache/awstats/"
  DirCgi="/cgi-bin"
  DirIcons="/icon"
  SiteDomain="hostname"
  AllowToUpdateStatsFromBrowser=1
  AllowFullYearView=3
  

• Make a directory called /var/cache and chmod it 777 so it can be used from the web server
• Copy icons to web directory:

  cp -r /usr/share/awstats/icon /var/www/icon
  

• Run this to update databases:

  /usr/lib/cgi-bin/awstats.pl -config=hostname -update
  

• In your web browser, go to the url:

  http://hostname/cgi-bin/awstats.pl?config=hostname
  

• Study the stats in quiet awe
• Edit crontab to update stats automatically every night:

  crontab -e
  0 1 * * * /usr/lib/cgi-bin/awstats.pl -config=hostname -update
  

I've been keeping an eye on my visitor logs to see how much my domain name problems have effected my traffic. According to Statcounter they had been climbing but yesterday there is a sudden dip. The Awstats logs provided by Site5 show no such dip.

I've seen a number of such dips in the Statcounter logs: their servers do not appear to be the most reliable. This is not a big complaint, I use them for free, more of a lamentation. Their professional service is too expensive for my simple ego brushing needs, $9 a month, but if I was paying that I would not want drop-outs approximately once a week.

The main advantage of Statcounter for me is that it counts visitors who have javascript enabled so it is essentially counting human beings rather than crawlers and referrer spam bots. It is also easy to set it up to ignore my own IP address. The Drupal statistics module does not have this feature but I could simply use phpmyadmin or another generic mysql database report generation tool to filter the drupal logs in any way I desire. The statistics module does list external referrers in reverse chronological order so it is useful for updating .htaccess referrer exclusion lists.

I'm one step further along in the Domain Name Saga, I have heard from Nominet and they have transferred my tag for bisiand.me.uk to 123-reg. I have told them to start the transfer and I am waiting for them to confirm.

Google progress is interesting (to me), petersblog.org is getting hits in it's own right for newer content, particularly regarding the v547 phone (which out of about 261,000 results, google places on the first page at number 8 ), and the crazy frog ringtone. The old content is getting odd visits but that may be through random dns successes for bisiand.me.uk.

I was getting about 100 visitors/day on bisiand.me.uk, I'm currently at 31 today (according to Statcounter) and it looks like either petersblog.org or statcounter went down a while last night as traffic dropped to zero.

According to StatCounter I am now getting more visitors from the UK than the US: 39 vs 25 out of the last 100.

I noticed today that when I go to google.com I am automatically redirected to www.google.co.uk. Could it be that this is happening to a large number of UK googlers and I am thereby appearing higher up in their search results? I also have a record high number of visitors overall today, a day when most people have gone back to work after the holiday.

Then again, I am getting lots of hits for digital cameras and N T H e l l.

I was looking through my StatCounter logs looking at referrals from google. These give the search terms and it's interesting to see what people are searching for when they find my site. I noticed that someone in Australia was searching for 'StatCounter' itself. I checked with a google search and was surprised to find my post came third on the first page of results, after the StatCounter site at #1 and #2. This is odd, my pagerank is 2 (www.bisiand.me.uk, for some reason bisiand.me.uk is only 1) so why are my ramblings showing up? Does nobody else mention StatCounter?

I started a game, find random blog entry, pick a couple of keywords and do a google search. I found that out of about 5 goes, my blog came up in the first three pages of google results. Even something broad like 'Drupal' puts me on the second page.

So:

• do I only write about obscure topics that nobody else writes about
• does google indeed favour blogs over all those sites full of random search results and price comparisons
• it was a straw pole
• I chose the right keywords
• the googlebot recognises me as an authority on these things
• all the above
• all the first five but mainly the first one and very little of the fifth

I cannot get my post about getting msn desktop search to list exe files to appear on the first few pages of google. exe hits too many program names.

Right now my page loads are all taking a long time because the StatCounter server is not responding. Normally it's Technorati that drags me down.

I've subscribed to StatCounter which is a service for gathering statistics for web traffic. It's a hit counter with very nice tracking facilities. You don't have to have a hit counter displayed on your page: I've hidden mine, it's for me to know how fewmany visitors I have had. It shows things like:

• Where in the world visitors are connecting from (hey there, how are things in China?), their ISP's etc.
• Which pages they have looked at (how they a going around the site)
• How long they spend looking at each page
• Easily set it up to ignore my own accesses

It does the kind of things that awstats, webstats, Drupal statistics etc cannot do without reverse DNS lookups etc.

It relies on the visitor's browser downloading a little bitmap and running some javascript. I think this is good as it means it will not record the 300 odd bots (including damn referrer spam) that suck my bandwidth every month, I can tell which visitors are humans. If I want to look at bot activity I can analyse the raw logs. Awstats and Drupal ststistics are still there for me.

Best of all, it's free if you have less than 1000 visitors a day.