Frequently Asked Questions

We have made AWStats, a graphical site statistics package, available for our members. Setup instructions for AWStats have been moved to our Wiki.

Yes they do. If you use log files, you should regularly rotate them in our member interface.

They are located in the /home/logs directory, as seen from ssh and SFTP, or the /logs directory as seen from FTP.

Most FTP clients start in the /public directory, so you would need to first change to the root directory (/) before you will see this.

There are four kinds of log files available at NearlyFreeSpeech.NET:

• Access log
• Error log
• Rewrite log
• Daemon log

These logs (if enabled) found in the /home/logs directory of your site.

Access log (/home/logs/access_log)

Each line in the access log represents one access to your website. It contains information, like what IP address accessed your site, what path they accessed, what web browser they used, etc. Analytics programs like AWStats use this type of log file. The access log can take up a lot of space, so it is disabled by default to save you money.

Error log (/home/logs/error_log)

This contains information about problems with your site. Although it records generic problems, like records of attempts to access pages that don't exist, it is most important to web developers and people with complex software loaded on their site. Whenever software running on a site has a problem, it should (but doesn't always) write something about it in the error log. Finding that message can save you fruitless hours of wondering what the heck is wrong with your site. For sites created after 2021-01-30, the error log is enabled by default.

Rewrite log (/home/logs/rewrite_log)

Rewrite logs track internal changes made to URLs by Apache. They're generally only useful if you've set up your own rewriting rules, and they aren't working as you expect. Rewrite logs are disabled by default. Enabling the rewrite log can tank the performance of your site. Enable it only if you are having rewrite-related problems, and turn it off as soon as you fix them.

Daemon log (/home/logs/daemon_example.log)

If you are running a server type that allows daemon processes, each daemon process gets its own log file (replacing example with the daemon's tag), that receives its standard output and error.

As long as the system is writing to a log file, you cannot modify or erase it. Once you rotate the log file and the system starts on a new one, you can delete the old one if you wish. You can read your log files even if they have not been rotated, you just can't delete them while a server is still using them.

You can access and delete your old log files via ssh or SFTP by looking in the /home/logs directory. (But you'll still have to rotate them before you can do anything but look.)

If you've disabled logging, the option to rotate them will not be available to you. Enable your logs, rotate them, disable them, and then delete them.

If you have log files enabled, you can ask our system to rotate them from the Site Information page for your site in our member interface.

When you ask to rotate your logs, our system waits for an opportune moment and then does it. Your access_log file, for instance, becomes access_log.old, and a new access_log is created. It may not be right away, but it usually takes just a few minutes. If there was already an access_log.old, that file gets a name based on the timestamp of the log file, such as access_log.20160828.

Before August 2016, non-transparent compression options were available. Now, the filesystem already incorporates transparent LZ4 compression, which does a great job on log files while keeping them easy to access and analyze. If you previously had compression enabled, it will continue to work, but you can deactivate it on the Site Information panel.

Here are the steps for enabling log files:

1. Visit the Sites panel.
2. Choose the site's "Short Name" to load the Site Information page.
3. Choose "Enabled" for the access or error logs (or both).
4. Choose "Save Changes."
5. Wait a few minutes for the change to take effect.

If you're planning to measure any kind of statistics, enable the access log. For best results, enable it a week or so before measuring your statistics, as data from a longer period of time will produce more informative reports.

To find and debug problems with your site, enable the error log. If you ask for help with an error message received from your site, we will often ask what your error log says about it. If you do not have your error log enabled, we may not be able to help you diagnose some problems.

We also provide a rewrite log which can be enabled in the same way, but may be set to varying levels of verbosity. Do not enable the rewrite log unless you are having problems with RewriteRule directives in your .htaccess file, and disable it as soon as you fix them. Using the rewrite log will dramatically slow down your site.

Because they take up space.

304 is the HTTP status response for "not modified." It is used in our log files to indicate that the logged request was actually served by some other part of our network. Usually only simple files, like static graphics and files, are eligible for this type of handling. Dynamic content, such as PHP and CGI scripts, will typically be served (and logged) in its entirety unless your application generates appropriate cache-eligible headers and 304 responses.

This code can also appear for other reasons, such as if someone accesses your site through a web cache.

This is largely due to requests filled by other parts of the network, which may appear in your access log as 304 responses with a size of 0 bytes, or if served entirely from memory, may not appear in the access log at all.

The number you see for bytes with 200 response codes should match the size of the named file unless it is a PHP or CGI script. However, every object served regardless of origin has some overhead (HTTP headers) that is also not reflected in the access log.

There is no arbitrary limit on how many old log files there are. You will need to remove old log files manually when you get tired of them.

Our log files are based on the Apache-standard "combined" log file format, which is a superset of the "common" format. This format is good for web log analyzing software. The fields, in order, are:

clientip - username [time] "request" status bytes "referer" "useragent"
1.2.3.4 - - [22/Nov/2013:22:32:01 -0700] "GET /index.html HTTP/1.0" 200 0 "-" "Wget/1.9.1"

The meaning of each field is shown below:

clientip

The IP address of the machine that requested the page. If you get a "-" in this field, it may mean that your site performs HTTP accesses on itself, or that we had to test your site with some internal application.

dash

This field used to be used, but isn't anymore. Lots of log analyzers still expect it to be there, so it always appears as a dash (-).

username

If you use Basic HTTP Authentication, the authenticated username will appear here. If you don't, it will be a dash (-).

time

The time in the format DD/Mon/Year:HH:MM:SS tzone

request

The request method, requested URI, and protocol, like GET /index.html HTTP/1.0.

status

The return code. The most common codes are 200 (ok), 304 (served elsewhere), 404 (file not found), and 500 (internal server error - a script error or a problem with our server).

bytes

The number of raw bytes of content served. This number does not include HTTP headers, transfer encoding, or TCP/IP protocol overhead.

referer

The page that contained the link to the requested resource. You can use this to see who is linking to you, and weblog analyzers use it to map your site. "referer" is not a typo, although it is spelled incorrectly. It's actually in the published standard that way.

useragent

The web browser sends this to identify itself. Lots of log analyzers can graph this.

More information about Apache log file formats is available on the Apache site.

The .old log files are never automatically compressed. If you rotate again, it will be renamed based on its last-modified date and only then will it be automatically compressed.

This is for two reasons. First, log entries come from many different parts of our network so there needs to be a "cooling off" period after rotating to make sure all parts of the network have switched to the new log. Otherwise, stuff might still be logged while compression is happening, and it would get lost.

Also, many people like to run analysis against a static log file. That way, if you change an analyzer setting and run it again, or if you want to do more than one thing to the log, you're working with a consistent set of logs. After log files are compressed, they're a lot harder to analyze, so keeping the .old file around often makes log analyzing much easier.

You're free to remove the .old log file yourself if you like, although we recommend waiting until you see log entries appearing in the new log file. If you rename or compress it yourself, it will not be auto-rotated in the future.

The error log is a "stream of consciousness" output of all the various programs you run on your site. As such, it does not have a specific format, other than that the most recent errors are at the bottom.

Nothing extraordinary. The change will only apply to newly compressed log files. Your old ones will be left as-is. If you want to update them to your newly chosen format, it is possible to do this from the ssh command line.