The other day a non-technical friend of mine told me how his webhost shut him off because he was using too much bandwidth. I was pretty surprised because they allocate him hundreds of GB per month and his website does not get much traffic.

He was dumbfounded. He loaded up his Google Analytics account and showed me that he was only getting about 50-100 unique visitors a day.

But his webhost said that he burned through over 200GB in less than a week!

I asked him if he used any other sort of analytical package, and he did, but they were all Javascript based.

So here is something important that people need to understand. Javascript tracking packages like Google Analytics are very narrow in their scope of what is actually going on. They can only process actions for web browsers that are actually running the Javascript.

In my friend’s case he had a 500MB video file that was being linked to from a very popular internet forum. Because his Google Analytics code wasn’t being executed he had no idea these people were stealing his bandwidth.

So how did we figure it out? We processed his raw access_log files with webalizer. Every major webhost is going to have webalizer (or some variant) available and that will show you MUCH more (in some ways) about what’s going on with your site than Google Analytics can.

We put webalizer on the log file and in a matter of minutes we were able to see that the forum was the top refer to his website… and we could also see that the file had been downloaded enough to use up all of his bandwidth.

Let me show you another example… this one a bit more practical that I guarantee affects everyone reading.

Webalizer shows me that the biggest use of bandwidth for all of shoemoney.com is from an IP from a Yahoo! address.

Yahoo SUcks

In fact 9 of the top 15 biggest bandwidth users for ShoeMoney.com are all Yahoo! IPs.

Interesting sidenote:

Yahoo bots use up more then 5% of the total bandwidth for ShoeMoney.com but bring in less then 1% of the traffic.

Now I am not saying you should completely rely on a log analyzer like webalizer either though. It can’t show you things like bounce rates, time spent on page, browser stats (size of window) and other vital marketing information.

I highly recommend a 3 pronged approach to basic web analytics:

Google Analytics
– Great overall view of your website visitors. Can report vital marketing information and goal tracking.

Webalizer – Awesome for getting to the gut of your users.

Google Webmaster Central – Excellent tool that Google put out which shows you exactly what their Google Bot is reporting back to them. It tells you if you have broken links, non indexable content, non reachable content and tons of other great stuff.

By Jeremy Schoemaker

Jeremy "ShoeMoney" Schoemaker is the founder & CEO of ShoeMoney Media Group, and to date has sold 6 companies and done over 10 million in affiliate revenue. In 2013 Jeremy released his #1 International Best selling Autobiography titled "Nothing's Changed But My Change" - The ShoeMoney Story. You can read more about Jeremy on his wikipedia page here.

64 thoughts on “My Basic 3 Pronged Approach To Website Analytics”
  1. I’m a big fan of Google Analytics just because of certain features it offers and as well the console setup. I feel it has a feel for your first time user and of course your user that is looking for key data in fields you may not know of going in as a rookie. I’ll probably have to check out Webalizer to see what great goodies it can offer me to better utlize my data. Thanks for the 3 pronger, I’ll just make sure to point it away from my eyes.

  2. Those are great tips and a gentle reminder to watch your stats – another nice, economical real time tool is Clicky – great for blogs.

  3. Useful information, I didnt know about Webalizer until now or the info that search engine bots use so much bandwidth.

  4. Google Webmaster Tools are amazing! It always helps me to stumble upon an interesting niche when a longtail post comes up as a high ranking.

  5. That’s the same approach I use. The real value of Webalizer is the ability to see bots hitting your server. Analytics doesn’t tend to pick up bots because they don’t parse Javascript. I’m a little old school, but I also like to store stats via PHP in a database, so I know total number of page loads, etc.

  6. The moral of the story is…don’t host massive video files on non-premium hosting :.)

  7. Don’t forget about server logs either. You’ll find some interesting data there of you dig a little bit.

    1. That’s basically what Webalizer does – it parses logs and builds graphs – they’re extremely detailed…

  8. I’ve seen these screenshots on many places but I never actually knew what it really was (webalizer)

    I use Google Analytics, but I realize it is not enough because most of the time my Adsense impressions and Google Analytics impressions don’t match.

    This post really goes to my bookmarks right away

  9. Hey Shoe, great article!

    Let me know, ae you considering droppinmg the yahoo bot?

    Also, with regards to the screenshot all I see is IP’s, do youi simply run those IP’s through a checker to see the referring website?

    I just can’t seem to analyse that data for my own benefit.

    All the best,
    Lou Sparx

  10. I always put the video files in Amazon S3 and create a cloud front distribution. Cloudberry explorer is an excellent free tool to manage your S3 buckets and objects (files and folders) and to create cloud front distribution. You can also use CNAME using your own domain to hide amazon colud front URL.

  11. I totally overlooked Google Webmaster Central as a stat tool. Right now I’m using G.Analytics and will definitely take a closer look at webalizer. Thanks

  12. Thanks for the tips shoe. So if that video file being lined was say 5mb instead of 500mb would that have affected the bandwidth allotment much?

  13. Yeah you definitely have to run more than one stats package on your site.

    The javascript methods are better at tracking pageviews, uniques, geo stuff, etc.

    The log based ones are better at tracking system resource usage.

    I think most good publishers are probably running 3 or more stats tracking systems.

    It’s also good because you can identify discrepancies between the different tools and try to figure out which is more accurate.

  14. Google analytics is insane. It has a huge number of features. It boggles my brain to be honest.

    However, the one real detractor (for me at least) is that I found that it dropped quite a significant chunk of traffic referral data.

  15. wow interesting I usually just use awstats to check stuff but that goes through the log files to I presume so there isn’t a need to use webalizer.

    1. Yeah – AWStats is basically the same thing. I think the interface looks better, but Webalizer seems to have better information.

  16. I think google analytics has amazing depth of functionality. The only problem that I found with it when I used to use it a lot, was that it was missing big chunks of traffic referrer data.

  17. Most people take analytics stats as gospel and don’t bother checking raw log files.

  18. You can stop hot linking by putting this code in .htaccess file, in this example for video’s.
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} !^$
    RewriteCond %{HTTP_REFERER} !^http://(www\.)?yoursite.com(/)?.*$ [NC]
    RewriteRule \.(flv|swf|png|bmp)$ – [F]

    They get a 403 error if the request comes other site than yoursite.com.

  19. Enlightening post as usual, Shoe. I never bother to check my Webalizer, never really know what’s the use. But, now I do. So, thanks.

  20. About that video thing; it might also be done purposely to drive the owner of the video out of business especially if he is being charged on a per bandwidth usage basis.

  21. Woow, that means we should look at our logs to see what exactly happends. Great post!

  22. Personally, I use free services such as statcounter.com and sitemeter.com. I find Google Analytics time consuming: I have to make too many clicks to see the data I’m after.

    And the time I spend with analytics can hurt my bottem line.

  23. Personally, I use free services such as statcounter.com and sitemeter.com. I find Google Analytics time consuming: I have to make too many clicks to see the data I’m after.

    And the time I spend with analytics can hurt my bottom line.

  24. Yeah nice post, but how can I see Webalizer reports for all my domains at once.? Thats where Mr Google wins all the time, they make things easier.

    I manage 50+ sites, going in to each site backend is a chore I only do that when the alarm bells ring.

    And as someone else mentioned who in their right mind hosts videos on a low end server anyway.?

  25. I thought that Google Analytics would be more comprehensive. Thank you for this artigo.Eu’ll follow your tips.

  26. Lol I would of never figured that out yeah its always good to check out your stats, and analytics to see where your traffic is coming from. They were stealing his bandwidth huh? Good thing you guys figured it out!

  27. Hi, I am new to blogging, thanks for this good tips, anyway i signed up in google analytics, but somehow it doesn’t work? I have configured google analytics plugin correctly, i wonder if it happens to anyone else?

  28. This is really great info! Thanks for it!

    I DO find GA sorely lacking, so now I have some new tools to look at.

    Awesome!

  29. Interesting and very useful information. I don’t have a site, but I’m planning to get one pretty soon. I didn’t know Yahoo bots use up a lot of site bandwidth. Is there any other site that does that?

  30. hey wow! I’ve never really looked at my analytics information that way before. You really opened my eyes on this one. Im gonna look at my analtyics information more carefully from now on.

  31. Webalizer can also be used to extract keywords that Google Analytics doesn’t see, especially from internal search engines from Niche sites. Commonly overlooked, but powerful tool when you learn how to use it.

  32. You know, I really think my colleagues over at Artfire.com could benefit from this information, so I’m going to post a link to this entry on our forums.

    Heck, the people BEHIND Artfire might even find it useful.

    Thanks again for such great info!

  33. I thought I’d place a message here to guide you the simplest way to create money by using dating affiliate along with free methods. We were able to make 3 to 5 thousand dollars a month following the methods in this totally free guide, http://bit.ly/5kmonthguide – click to see.

  34. Hi, shoe:

    Thanks for the information.

    I dont know why I can Not Verify my Google Webmaster Central . I did it according to the instruciton from Google itself.

    I want to use Webalizer as well. Would you please kindly leave me a detailed url which I can download directly? I believe it could help many others who are like me—non-technical bloggers?

    thanks.

  35. Just wanted to say that you guys should try out Reinvigorate and Woopra. I have been a beta tester for both these tracking systems..and both are very good real time systems. Reinvigorate is still in beta, but Woopra is out. Woopra is THE most detailed and innovative web tracking system I have ever seen..I highly recommend you try the free package.

    David.

  36. I only use google analytics…but I will take a gander at the others to see what kind of value I can pull from them.

  37. I really don’t see linking to a video as “stealing the bandwith”. It just is simply stupid to host huge videos on your own web site…

  38. Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a little bit, but other than that, this is excellent blog. An excellent read. I will definitely be back.

  39. I have read a few good stuff here. Definitely worth bookmarking for revisiting. I wonder how much effort you put to make such a great informative website.

  40. I was just seeking this info for a while. After six hours of continuous Googleing, at last I got it in your web site. I wonder what’s the lack of Google strategy that don’t rank this kind of informative web sites in top of the list.

  41. I wanted to put you that little word in order to thank you yet again for those magnificent thoughts you have provided in this article. This has been simply incredibly generous of people like you to deliver publicly exactly what a lot of people would have offered for sale as an e-book to earn some profit for their own end, specifically seeing that you could possibly have done it in case you decided. The principles likewise acted to be a good way to recognize that the rest have the identical zeal much like my very own to understand whole lot more with regard to this matter. I am sure there are a lot more pleasurable moments up front for many who view your site.

  42. Just want to say your article is as surprising. The clearness in your post is just cool and i could assume you’re an expert on this subject. Well with your permission let me to grab your feed to keep up to date with forthcoming post. Thanks a million and please continue the gratifying work.

  43. award-winning! I think it’s worth to leave a piece of words here!

  44. Is the home alarm system you are talking about a wirless system? And I have heard the celular based monitoring services are a lot safer as there are no phone lines that can be cut. Great site, great range of content.

  45. Very nice post. I simply stumbled upon your blog and wished to say that I have really loved browsing your weblog posts. After all I’ll be subscribing in your feed and I am hoping you write again very soon!

  46. Thank you for pointing this issue out, regarding web analytics. I think another point to take note of is that some web hosts talk about unlimited bandwidth, but I have since found that to not always be so cut and dry. I am continuously monitoring my sites as I have quite a number of videos and I want to see how far I can push the download issue, because if you are on a shared host, I think they may well shut you down, or limit you if your users are affecting other websites on the same server. In some cases, it may pay to use external video server companies to eliminate that problem.

  47. Didn’t realize that about google analytics – are there any desktop tools out there that will analyze my webalizer logs?

  48. I use Google Analytics for my sites and my clients sites but I really do only ‘scrape the surface’ – note to self, buy a good ‘Google Analytics’ book and really learn what it can do!

    Any book recommendations?

Comments are closed.