Topic awaiting preservation: Server Logs vs Database for tracking stats....

Hi all, i know i posted a question about tracking site stats not long back, and someone said to use the logs. I did some calculations and worked out that database size wouldnt be an issue if you were to do it that way.

One problem i am currently having is log files - i have hosting limits in terms of available space and although most websites themselves dont take up much room, the log files seem to quickly become megs in size! I had to delete a whole bunch just to get back within my limits


Obviously this poses a problem if i wanted to keep history and use them for tracking stats. So is database more efficient at storing this data, i honestly thought raw data in a log file would take up less space, but i may be wrong!

Anyone have any idea? Thanks

Do you need the raw data itself, or could you just compile a webpage from the logs and cut it off monthly, yearly, or whatnot?

I'd talk to your hosting provider. Normally log files don't count against your server space. And should be cleaned automatically.

Second. Most of the log file data is extraneous I'd look at writing a small script that runs daily that parses your logfiles for the data you want to keep and store that in a file and then clean up the rest periodically.



.:[ Never resist a perfect moment ]:.

So i guess if i do that i might as well just collect data straight into the database? Essentially its doing the same thing..

Unfortunately it seems the log files do count towards server space, actually i have noticed this for alot of my hosting providers.

Nada: I really require the raw data so that i can provide more detailed/complex reports kinda thing.

There a few things I would do in this case. I would try to establish a cron job (if you have shell access or a good panel) to dump your log file folder to a different (maybe home) server each day/week/etc. Then you could have an additional script compile the data once daily so noone would ever have to deal with low bandwidth.
As Bitdamage mentioned, though, hosting providers should offer you free space for logs. The best idea may be to pull up your roots and head to a new host. I see you are located in Austrailia, so I can't mention any hosts there of high quality.

Yeh, well im pretty happy with this host for the most part - even tho i often go over the limit they dont seem to care so it hasnt been a real problem.

Alot of my sites are on windows hosting, only a few on linux at the moment so cron jobs are only available for those.

I could just make a script that runs occasionally when people visit the website heh, which deletes the log files and sets a flag or something. Bit dodgey but ill see.