Frequently Asked Questions
We have made AWStats, a graphical site statistics package, available for our members. Setup instructions for AWStats have been moved to our Wiki.
Yes they do. If you use log files, you should regularly rotate them in our member interface.
They are located in the /home/logs directory, as seen from ssh and SFTP, or the /logs directory as seen from FTP.
Most FTP clients start in the /public directory, so you would need to first change to the root directory (/) before you will see this.
There are three primary kinds of log files available at NearlyFreeSpeech.NET:
All three are (if enabled) found in the /home/logs directory of your site.
Access logs are the most common type. Each line in the access log represents one access to your website. It contains a variety of information, like what IP address accessed your site, what they accessed, what web browser they used, etc. This is the type of log file that is most commonly used by analytics programs like AWStats and analog.
Error logs contain information about problems with your site. Although they contain lots of generic information, like records of attempts to access pages that don't exist, they are most important to web developers and people with complex software loaded on their site. Whenever software running on a site has a problem, it should (but doesn't always) say something about the problem in the error log. Finding that message can save you fruitless hours of wondering what the heck is wrong with your site.
If you ask for help troubleshooting a problem, on our forum for example, the first thing people will typically ask is whether you have your site's error log enabled and, assuming you do, what it says. To save time, it's often helpful to make sure it's enabled, say so in your question, and include any relevant log entries from around the time the problem occurred.
The third type, rewrite logs, are a bit more special-purpose. Rewrite logs track internal changes made to URLs by Apache. They're generally only useful when debugging these changes if you've set up your own rewriting rules, because the time it takes to write rewrite logs to our disk can slow your site to a crawl if you use a lot of them. If you don't know what a rewrite is or why you'd want to log them, you should definitely leave this one turned off.
As long as the system is writing to a log file, you cannot modify or erase it. Once you rotate the log file and the system starts on a new one, you can delete the old one if you wish. You can read your log files even if they have not been rotated, you just can't delete them while a server is still using them.
You can access and delete your old log files via FTP by looking in the "/logs" directory of your site. Via ssh or SFTP, this is the /home/logs directory. (But you'll still have to rotate them before you can do anything but look.)
If you've disabled logging, the option to rotate them will not be available to you. Enable your logs, rotate them, disable them, and then delete them.
If you have log files enabled, you can ask our system to rotate them from the Site Information page for your site in our member interface.
When you ask to rotate your logs, our system waits for an opportune moment and then does it. Your access_log file, for instance, becomes access_log.old and a new access_log is created. It may not be right away, but it usually takes just a few minutes.
If there was already an access_log.old, that file gets a name based on the timestamp of the logfile, such as access_log.20160828. At this point, any compression you have selected will be applied. However, as of August 2016, compression is generally not necessary as the filesystem already incorporates transparent LZ4 compression. If you do not want your log files to be doubly compressed, you can disable log compression on the Site Information panel.
Here are the steps for enabling log files:
If you're planning to measure any kind of statistics, enable the access log. For best results, enable it a week or so before measuring your statistics, as data from a longer period of time will produce more informative reports.
To find and debug problems with your site, enable the error log. If you ask for help with an error message received from your site, we will often ask what your error log says about it. If you do not have your error log enabled, we may not be able to help you diagnose some problems.
We also provide a rewrite log which can be enabled in the same way, but may be set to varying levels of verbosity. Do not enable the rewrite log unless you are having problems with RewriteRule directives in your .htaccess file, and disable it as soon as you fix them. Using the rewrite log will dramatically slow down your site.
Because they take up space.
304 is the HTTP status response for "not modified." It is used in our log files to indicate that the logged request was actually served by some other part of our network. Usually only simple files, like static graphics and files, are eligible for this type of handling. Dynamic content, such as PHP and CGI scripts, will typically be served (and logged) in its entirety unless your application generates appropriate cache-eligible headers and 304 responses.
This code can also appear for other reasons, such as if someone accesses your site through a web cache.
This is largely due to requests filled by other parts of the network, which may appear in your access log as 304 responses with a size of 0 bytes, or if served entirely from memory, may not appear in the access log at all.
The number you see for bytes with 200 response codes should match the size of the named file, unless it is a PHP or CGI script. However, every object served regardless of origin has some overhead (HTTP headers) that is also not reflected in the access log.
There is no arbitrary limit on how many old log files there are. You will need to remove old log files manually when you get tired of them.
Our log files are based on the Apache-standard "combined" log file format, which is a superset of the "common" format. This format is good for web log analyzing software. The fields, in order, are:
clientip - username [time] "request" status bytes "referer" "useragent"
18.104.22.168 - - [22/Nov/2013:22:32:01 -0700] "GET /index.html HTTP/1.0" 200 0 "-" "Wget/1.9.1"
The meaning of each field is shown below:
More information about Apache log file formats is available on the Apache site.
The .old log files are never automatically compressed. If you rotate again, it will be renamed based on its last-modified date and only then will it be automatically compressed.
This is for two reasons. First, log entries come from many different parts of our network so there needs to be a "cooling off" period after rotating to make sure all parts of the network have switched to the new log. Otherwise, stuff might still be logged while compression is happening, and it would get lost.
Also, many people like to run analysis against a static log file. That way, if you change an analyzer setting and run it again, or if you want to do more than one thing to the log, you're working with a consistent set of logs. After log files are compressed, they're a lot harder to analyze, so keeping the .old file around often makes log analyzing much easier.
You're free to remove the .old log file yourself if you like, although we recommend waiting until you see log entries appearing in the new log file. If you rename or compress it yourself, it will not be auto-rotated in the future.
The error log is a "stream of consciousness" output of all the various programs you run on your site. As such, it does not have a specific format, other than that the most recent errors are at the bottom.
Nothing extraordinary. The change will only apply to newly compressed log files. Your old ones will be left as-is. If you want to update them to your newly chosen format, it is possible to do this from the ssh command line.