My Basic 3 Pronged Approach To Website Analytics

Posted by



The other day a non-technical friend of mine told me how his webhost shut him off because he was using too much bandwidth. I was pretty surprised because they allocate him hundreds of GB per month and his website does not get much traffic.

He was dumbfounded. He loaded up his Google Analytics account and showed me that he was only getting about 50-100 unique visitors a day.

But his webhost said that he burned through over 200GB in less than a week!

I asked him if he used any other sort of analytical package, and he did, but they were all Javascript based.

So here is something important that people need to understand. Javascript tracking packages like Google Analytics are very narrow in their scope of what is actually going on. They can only process actions for web browsers that are actually running the Javascript.

In my friend’s case he had a 500MB video file that was being linked to from a very popular internet forum. Because his Google Analytics code wasn’t being executed he had no idea these people were stealing his bandwidth.

So how did we figure it out? We processed his raw access_log files with webalizer. Every major webhost is going to have webalizer (or some variant) available and that will show you MUCH more (in some ways) about what’s going on with your site than Google Analytics can.

We put webalizer on the log file and in a matter of minutes we were able to see that the forum was the top refer to his website… and we could also see that the file had been downloaded enough to use up all of his bandwidth.

Let me show you another example… this one a bit more practical that I guarantee affects everyone reading.

Webalizer shows me that the biggest use of bandwidth for all of shoemoney.com is from an IP from a Yahoo! address.

Yahoo SUcks

In fact 9 of the top 15 biggest bandwidth users for ShoeMoney.com are all Yahoo! IPs.

Interesting sidenote:

Yahoo bots use up more then 5% of the total bandwidth for ShoeMoney.com but bring in less then 1% of the traffic.

Now I am not saying you should completely rely on a log analyzer like webalizer either though. It can’t show you things like bounce rates, time spent on page, browser stats (size of window) and other vital marketing information.

I highly recommend a 3 pronged approach to basic web analytics:

Google Analytics
– Great overall view of your website visitors. Can report vital marketing information and goal tracking.

Webalizer – Awesome for getting to the gut of your users.

Google Webmaster Central – Excellent tool that Google put out which shows you exactly what their Google Bot is reporting back to them. It tells you if you have broken links, non indexable content, non reachable content and tons of other great stuff.

64 thoughts on “My Basic 3 Pronged Approach To Website Analytics

  1. Chris Pontine

    I’m a big fan of Google Analytics just because of certain features it offers and as well the console setup. I feel it has a feel for your first time user and of course your user that is looking for key data in fields you may not know of going in as a rookie. I’ll probably have to check out Webalizer to see what great goodies it can offer me to better utlize my data. Thanks for the 3 pronger, I’ll just make sure to point it away from my eyes.

  2. War Wizard

    Those are great tips and a gentle reminder to watch your stats – another nice, economical real time tool is Clicky – great for blogs.

  3. Paul

    Useful information, I didnt know about Webalizer until now or the info that search engine bots use so much bandwidth.

  4. jtGraphic

    That’s the same approach I use. The real value of Webalizer is the ability to see bots hitting your server. Analytics doesn’t tend to pick up bots because they don’t parse Javascript. I’m a little old school, but I also like to store stats via PHP in a database, so I know total number of page loads, etc.

  5. Davor Gasparevic @ Ebooks blogspot

    I’ve seen these screenshots on many places but I never actually knew what it really was (webalizer)

    I use Google Analytics, but I realize it is not enough because most of the time my Adsense impressions and Google Analytics impressions don’t match.

    This post really goes to my bookmarks right away

  6. Web Design Lou

    Hey Shoe, great article!

    Let me know, ae you considering droppinmg the yahoo bot?

    Also, with regards to the screenshot all I see is IP’s, do youi simply run those IP’s through a checker to see the referring website?

    I just can’t seem to analyse that data for my own benefit.

    All the best,
    Lou Sparx

  7. Free Classifieds Blog

    I always put the video files in Amazon S3 and create a cloud front distribution. Cloudberry explorer is an excellent free tool to manage your S3 buckets and objects (files and folders) and to create cloud front distribution. You can also use CNAME using your own domain to hide amazon colud front URL.

  8. tom

    Thanks for the tips shoe. So if that video file being lined was say 5mb instead of 500mb would that have affected the bandwidth allotment much?

  9. Travis Lusk

    Yeah you definitely have to run more than one stats package on your site.

    The javascript methods are better at tracking pageviews, uniques, geo stuff, etc.

    The log based ones are better at tracking system resource usage.

    I think most good publishers are probably running 3 or more stats tracking systems.

    It’s also good because you can identify discrepancies between the different tools and try to figure out which is more accurate.

  10. Wynne

    Google analytics is insane. It has a huge number of features. It boggles my brain to be honest.

    However, the one real detractor (for me at least) is that I found that it dropped quite a significant chunk of traffic referral data.

  11. Morgan Thomas

    wow interesting I usually just use awstats to check stuff but that goes through the log files to I presume so there isn’t a need to use webalizer.

    1. jtGraphic

      Yeah – AWStats is basically the same thing. I think the interface looks better, but Webalizer seems to have better information.

  12. Wynne

    I think google analytics has amazing depth of functionality. The only problem that I found with it when I used to use it a lot, was that it was missing big chunks of traffic referrer data.

  13. Jack

    You can stop hot linking by putting this code in .htaccess file, in this example for video’s.
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} !^$
    RewriteCond %{HTTP_REFERER} !^http://(www\.)?yoursite.com(/)?.*$ [NC]
    RewriteRule \.(flv|swf|png|bmp)$ – [F]

    They get a 403 error if the request comes other site than yoursite.com.

  14. Ann

    Personally, I use free services such as statcounter.com and sitemeter.com. I find Google Analytics time consuming: I have to make too many clicks to see the data I’m after.

    And the time I spend with analytics can hurt my bottem line.

  15. Ann

    Personally, I use free services such as statcounter.com and sitemeter.com. I find Google Analytics time consuming: I have to make too many clicks to see the data I’m after.

    And the time I spend with analytics can hurt my bottom line.

  16. Krazza

    Yeah nice post, but how can I see Webalizer reports for all my domains at once.? Thats where Mr Google wins all the time, they make things easier.

    I manage 50+ sites, going in to each site backend is a chore I only do that when the alarm bells ring.

    And as someone else mentioned who in their right mind hosts videos on a low end server anyway.?

  17. Learn Affiliate Marketing

    Lol I would of never figured that out yeah its always good to check out your stats, and analytics to see where your traffic is coming from. They were stealing his bandwidth huh? Good thing you guys figured it out!

  18. Dina

    Hi, I am new to blogging, thanks for this good tips, anyway i signed up in google analytics, but somehow it doesn’t work? I have configured google analytics plugin correctly, i wonder if it happens to anyone else?

  19. AnnieP78

    Interesting and very useful information. I don’t have a site, but I’m planning to get one pretty soon. I didn’t know Yahoo bots use up a lot of site bandwidth. Is there any other site that does that?

  20. sandy

    hey wow! I’ve never really looked at my analytics information that way before. You really opened my eyes on this one. Im gonna look at my analtyics information more carefully from now on.

  21. Justin Hitt

    Webalizer can also be used to extract keywords that Google Analytics doesn’t see, especially from internal search engines from Niche sites. Commonly overlooked, but powerful tool when you learn how to use it.

  22. Jennifer Moore

    You know, I really think my colleagues over at Artfire.com could benefit from this information, so I’m going to post a link to this entry on our forums.

    Heck, the people BEHIND Artfire might even find it useful.

    Thanks again for such great info!

  23. author wanglili

    Hi, shoe:

    Thanks for the information.

    I dont know why I can Not Verify my Google Webmaster Central . I did it according to the instruciton from Google itself.

    I want to use Webalizer as well. Would you please kindly leave me a detailed url which I can download directly? I believe it could help many others who are like me—non-technical bloggers?

    thanks.

  24. David Koh

    Just wanted to say that you guys should try out Reinvigorate and Woopra. I have been a beta tester for both these tracking systems..and both are very good real time systems. Reinvigorate is still in beta, but Woopra is out. Woopra is THE most detailed and innovative web tracking system I have ever seen..I highly recommend you try the free package.

    David.

  25. Mikko

    I really don’t see linking to a video as “stealing the bandwith”. It just is simply stupid to host huge videos on your own web site…

  26. divemaster

    Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a little bit, but other than that, this is excellent blog. An excellent read. I will definitely be back.

  27. divemaster

    I have read a few good stuff here. Definitely worth bookmarking for revisiting. I wonder how much effort you put to make such a great informative website.

  28. sportster transmission

    I was just seeking this info for a while. After six hours of continuous Googleing, at last I got it in your web site. I wonder what’s the lack of Google strategy that don’t rank this kind of informative web sites in top of the list.

  29. fitness exercise workout at home

    I wanted to put you that little word in order to thank you yet again for those magnificent thoughts you have provided in this article. This has been simply incredibly generous of people like you to deliver publicly exactly what a lot of people would have offered for sale as an e-book to earn some profit for their own end, specifically seeing that you could possibly have done it in case you decided. The principles likewise acted to be a good way to recognize that the rest have the identical zeal much like my very own to understand whole lot more with regard to this matter. I am sure there are a lot more pleasurable moments up front for many who view your site.

  30. echte amateure

    Just want to say your article is as surprising. The clearness in your post is just cool and i could assume you’re an expert on this subject. Well with your permission let me to grab your feed to keep up to date with forthcoming post. Thanks a million and please continue the gratifying work.

  31. Evan Sorhaindo

    Is the home alarm system you are talking about a wirless system? And I have heard the celular based monitoring services are a lot safer as there are no phone lines that can be cut. Great site, great range of content.

  32. Emo Hair Girl

    Very nice post. I simply stumbled upon your blog and wished to say that I have really loved browsing your weblog posts. After all I’ll be subscribing in your feed and I am hoping you write again very soon!

  33. Success Attraction in my Life

    Thank you for pointing this issue out, regarding web analytics. I think another point to take note of is that some web hosts talk about unlimited bandwidth, but I have since found that to not always be so cut and dry. I am continuously monitoring my sites as I have quite a number of videos and I want to see how far I can push the download issue, because if you are on a shared host, I think they may well shut you down, or limit you if your users are affecting other websites on the same server. In some cases, it may pay to use external video server companies to eliminate that problem.

  34. SEO Sheffield

    I use Google Analytics for my sites and my clients sites but I really do only ‘scrape the surface’ – note to self, buy a good ‘Google Analytics’ book and really learn what it can do!

    Any book recommendations?

Comments are closed.