Peter's Blog

I've been using statcounter to track visitors to this site for a few years now. While my site5 hosting also provides awstats I prefer StatCounter as it uses javascript for the tracking and so is less likely to track bots and other rubbish.

The free statcounter service used to show the details of the last 100 visitors to a site. My site gets around 1000 visitors a day so 100 visitors is maybe two hours worth. Statcounter have now increased this limit to 500 visitors which is more like half a day and consequently more interesting.

Currently the most popular page is this one about using T_hunderbird with M_icrosoft exchange (see what I did there? don't want this page getting those hits). This page about f_ence sprayers has had a lot of traffic recently, it being spring and folk wanting to paint their fences. There don't appear to be many public forums for people to air their f_ence spraying grievances and this has become one.

Remember the search engine wars? Here is a status report:

If I were yahoo I'd drop the exclamation mark. Microsoft are nowhere to be seen.

I get lots of World of Warcraft related hits, lets look at some recent wow related searches:

• how to get skill up in weapons faster in wow
• wow daily quest
• wow g11 g15
• good way to earn sara armor
• winterfall flight path
• tanaris to ungoro crater foot
• world of warcraft fast firing ammo bag
• good prot aoe spots
• how to use the g15 in wow
• titan panel download patch 2.1
• heroic tactics

(that last one was wide of the mark).

It makes me wish all this info was nicely laid out and not in the heavy stream of consciousness format.

The warcraft stuff has boosted the page rank of this site to four which increased the number of visitors a day by about three or four hundred.

I'm not getting a kickback from Statcounter but despite that I still recommend them.

Thanks for coming.

Summary of site statistics for 2006, courtesy of Statcounter.

186,756 visitors! That's two or three football stadiums. An awful lot never come back though Still, I don't visit THEIR sites either.

This site wouldn't load. Looking in the status bar it was trying to contact Google Analytics to get a javascript file. This affected Firefox but not IE which loaded fine.

I commented Google Analytics out of the page and now it is working again.

I don't use Google Analytics a lot, I still prefer statcounter for my web statistics because of the detail it gives me. Google Analytics is more about pie charts for marketing people. Statcounter tells me what google searches people are using to find my site and that is much more interesting. Statcounter also gives graphs and averages and these tally pretty much with Google Analytics.

I'll leave just statcounter installed for now, I don't need both, it slows down page loading, google analytics is in the header and brings the whole site down if the server goes, statcounter is at the end of the html and most of the page still displays even if statcounter isn't responding.

Moral: don't link to unreliable servers in your page header.

Since I started with statcounter very nearly a year ago, it tells me my site has served up 100,214 pages to 72,521 visitors.

I have quickly become bored with Google Analytics: I think it's appeal is more for marketing departments and powerpoint/pie chart enthusiasts. I like the raw detail that statcounter gives me in an easily accessable way. I can see what people are searching for and sometimes it even inspires me to update articles to be more useful.

I still think their counting is buggy: returning visitors seems unreliable, as if page refreshing counts as a return visit, so I take it with a pinch of salt and follow the trends. The numbers are roughly on a par with google analytics.

Statcounter is easier to install, it can go anywhere on the page, Google Analytics has to go in the header. For drupal this means editing the page template rather than sticking it in a block.

I have set up Google Analytics on this site. This is a very sophisticated looking system for tracking web site visits. Google have set it up to help their advertisers count their money but it is free if your site has less than 5 million page views/month which should suffice for me.

The analytics site is still a bit flaky and overloaded and it is refusing to validate the code on this site. I'll give it till tomorrow to settle down and I'll try again.

It looks to be more biased towards grand overviews than statcounter which is good for analysing individual visits: especially the free version that only tracks the last 100 page views.

Apart from the current flakiness, the analytics reports are only updated every few hours (like awstats I suppose) whereas statcounter is real time.

UPDATE: although the site still tells me the tracker is not installed properly and the link to an example installation is broken, the tracker is working and I can see the stats. So far it looks very sophisticated for a free (to me) service: it looks professional and expensive. Here is the geographical chart:

This shows a tooltip of each town as you pass the mouse over it (the analytics page, not the screenshot above) whereas Statcounter just gives a list of countries. However, in Statcounter I can drill into the visits from each country to see which pages they have been looking at.

I now have a new metric to worry about: depth of visit, i.e. people looking at more than one page

Analytics and Statcounter may complement each other: analytics for charts, Statcounter for the details of who is doing what. I could carry on using both, it's all a matter of how quickly I want pages to load and how much I trust both lots of servers to keep up with mine.

I was studying the statcounter logs for this site and lamenting how the country info only lists the last 100 page views. Because one visitor can look at 50 pages, their country will appear to take 50% of traffic, giving a distorted view of proceedings.

So I decided to update my awstats config with GeoIP to give long term country information. GeoIP is a library from MaxMind that converts IP addresses to countries. Country lookup is free, you can pay for higher resolution lookup (city etc) if you have the $$$.

I installed it as follows:

• Installed the MaxMind GeoIP C Library and perl library from here
• I downloaded and installed the country database.
• I let these configure and install themselves. This may lead to an unholy mix with my debian apt setup but it is more likely to work than if I start messing with it.
• I went into webmin and enabled the GeoIP plugin. A word of recommendation here, webmin does a great job of managing multiple sites with awstats. In fact webmin does a great job of most things.

There is an old version of GeoIP that comes as a debian package but it installs in a debian way and was not picked up by the MaxMind Perl module so I installed the MaxMind stuff by the book.

It seems to be running, it is showing visitors from germany and stuff but I need to let it run for longer and see if the number of visitors from unknown countries goes down (it probably doesn't add country info to old records only new ones).

This may end up simply telling me where the bots that make up a large number of my visitors come from: so far this month this site has used 1.15G of bandwidth. Accursed Inktomi Slurp (which I think is yahoo) has taken 177Mb of this. Googlebot has taken 77M but given me 7073 visitors against 182. Slurp is a good name. It has probably uploaded all the text on the site 100 times. I don't change it that much.

The number of visitors to my site has increased by about 40% in the last two days. Yesterday I got an all-time record of 509 unique visitors (according to statcounter). It's about a level that I can quote without feeling embarrassed. According to statcounters 'came from' thing they mostly come from google.

I must admit I don't totally trust statcounter. Today it shows this in 'keyword analysis':

4  7.41%	onenote firefox
3  5.56%	thunderbird taskbar notifier
3  5.56%	vnc ubuntu
2  3.70%	ubuntu mp3

This would imply that four different people came from google searching for 'onenote firefox'. If I drill down into 'onenote firefox' it only shows one ip address with eight entries. What gives ? How many 'unique user visitors' is this supposed to be? Why would someone come here from the same search four times?

I still trust statcounter not to show me hits from bots, crawlers, comment spammers and whatever. According to awstats I got 649 visitors. Also, in the awstats referrer list out of the top 25 entries this month only 4 are referrer spams links with obscene url's, the others come from techy sites or searches from other than google/msn/yahoo (I am on the statcounter free program which only shows details for the last 100 hits, awstats is unlimited).

This is almost credibility. Thanks for reading.

Another awTags feature is pinging technorati with tags (or something like that). From my statcounter logs I see I am already getting more traffic from there.

I never did get around to trying to install awstats. I've been using Statcounter but I fancied trying awstats with reverse DNS turned on. I can't do this on my Site5 host as they don't like reverse DNS. I didn't install it on Gentoo as that looked like big time hastle.

I realised today that installing awstats under Ubuntu should be as simple as installing the awstats package and it almost is. I can install it on my home server, download my Site5 access logs there and let awstats format them up.

Here are the steps I had to take to install it:

• Install awstats package
• Edit a file called /etc/awstats/awstats.hostname.conf where hostname is the hostname. Put something like this in it:

  LogFile="/var/log/apache/access.log"
  LogFormat=1
  DNSLookup=1
  DirData="/var/cache/awstats/"
  DirCgi="/cgi-bin"
  DirIcons="/icon"
  SiteDomain="hostname"
  AllowToUpdateStatsFromBrowser=1
  AllowFullYearView=3
  

• Make a directory called /var/cache and chmod it 777 so it can be used from the web server
• Copy icons to web directory:

  cp -r /usr/share/awstats/icon /var/www/icon
  

• Run this to update databases:

  /usr/lib/cgi-bin/awstats.pl -config=hostname -update
  

• In your web browser, go to the url:

  http://hostname/cgi-bin/awstats.pl?config=hostname
  

• Study the stats in quiet awe
• Edit crontab to update stats automatically every night:

  crontab -e
  0 1 * * * /usr/lib/cgi-bin/awstats.pl -config=hostname -update
  

I've been keeping an eye on my visitor logs to see how much my domain name problems have effected my traffic. According to Statcounter they had been climbing but yesterday there is a sudden dip. The Awstats logs provided by Site5 show no such dip.

I've seen a number of such dips in the Statcounter logs: their servers do not appear to be the most reliable. This is not a big complaint, I use them for free, more of a lamentation. Their professional service is too expensive for my simple ego brushing needs, $9 a month, but if I was paying that I would not want drop-outs approximately once a week.

The main advantage of Statcounter for me is that it counts visitors who have javascript enabled so it is essentially counting human beings rather than crawlers and referrer spam bots. It is also easy to set it up to ignore my own IP address. The Drupal statistics module does not have this feature but I could simply use phpmyadmin or another generic mysql database report generation tool to filter the drupal logs in any way I desire. The statistics module does list external referrers in reverse chronological order so it is useful for updating .htaccess referrer exclusion lists.