WikiSysop: /* Example of a log entry */

2025-09-03T11:31:44Z

Example of a log entry

← Older revision		Revision as of 13:31, 3 September 2025
Line 7:		Line 7:

	== Example of a log entry ==		== Example of a log entry ==
	This is one line split in 3, so it can be seen.		This is one line split in 3, so it can be seen. If your screen is not wide enough, the lines may break further.
	<span style="color:#FF0000">52d3ccde.dynamic-ip.k-net.dk</span> - - [07/Mar/2017:19:48:44 +0100] "GET <span style="color:#00FF00">/teaching/index.php?title=-&action=raw&gen=js&useskin=monobook</span> HTTP/1.1"		<span style="color:#FF0000">52d3ccde.dynamic-ip.k-net.dk</span> - - [07/Mar/2017:19:48:44 +0100] "GET <span style="color:#00FF00">/teaching/index.php?title=-&action=raw&gen=js&useskin=monobook</span> HTTP/1.1"
	200 290 "http://wiki.bio.dtu.dk/teaching/index.php/Course27617Spring2017"		200 290 "http://wiki.bio.dtu.dk/teaching/index.php/Course27617Spring2017"

WikiSysop: Created page with "NOTOC == Files used in example == [https://teaching.healthtech.dtu.dk/material/22116/apachewiki.log Log from web server] == Web server statistics == It is of interest to the BOSS to see how many visitors a web site has and how many web pages has been seen. So therefore someone (you) has to make some statistics from the web server log file. == Example of a log entry == This is one line split in 3, so it can be seen. 52d3ccde.dynamic-ip.k..."

2025-09-03T11:30:47Z

Created page with "__NOTOC__ == Files used in example == [https://teaching.healthtech.dtu.dk/material/22116/apachewiki.log Log from web server] == Web server statistics == It is of interest to the BOSS to see how many visitors a web site has and how many web pages has been seen. So therefore someone (you) has to make some statistics from the web server log file. == Example of a log entry == This is one line split in 3, so it can be seen. 52d3ccde.dynamic-ip.k..."

New page

__NOTOC__
== Files used in example ==
[https://teaching.healthtech.dtu.dk/material/22116/apachewiki.log Log from web server]

== Web server statistics ==
It is of interest to the BOSS to see how many visitors a web site has and how many web pages has been seen. So therefore someone (you) has to make some statistics from the web server log file.

== Example of a log entry ==
This is one line split in 3, so it can be seen.
52d3ccde.dynamic-ip.k-net.dk - - [07/Mar/2017:19:48:44 +0100] "GET /teaching/index.php?title=-&action=raw&gen=js&useskin=monobook HTTP/1.1"
200 290 "http://wiki.bio.dtu.dk/teaching/index.php/Course27617Spring2017"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/602.3.12 (KHTML, like Gecko) Version/10.0.2 Safari/602.3.12"
In this example, we are only interested in the red part , which is the IP-number or host name of the visitor, and the green part, which is the file served to the visitor.
The problem is that the web is filled with search engines indexing the web sites, and they don't count as humans - yet. So they must be excluded from your statistics. Most of the indexing robots look at a file called /robots.txt, for what they are allowed to index. So here is a way to identify the non-humans.
Also, many web pages uses pictures, meaning that the web server also serves picture files to the visitors. These pictures do not count as a page view (since they are being part of a page). Picture files mostly have the extensions .gif, .jpg, .png and .jpeg, so they can also be identified.

== The program ==
<pre>
#!/usr/bin/python3
# makes statistics on a apache webserver log file
import sys

# get file
try:
logfile = open('apachewiki.log', 'r')
except IOError as err:
print('There seems to be a problem with the file:', str(err))
sys.exit(1)

# Dict for counting page views
hosts = dict()
# Set of search engine crawlers
crawler = set()

for line in logfile:
field = line.split()
# Is this a crawler
if field[6] == '/robots.txt':
crawler.add(field[0])
# Is this a pic?
if not field[6].endswith(('.gif','.jpg','.png','.jpeg')):
# Must be a page view then
if field[0] in hosts:
hosts[field[0]] += 1
else:
hosts[field[0]] = 1

logfile.close()

# Remove page views made by crawlers
crawlCount = 0
for item in crawler:
crawlCount += hosts[item]
del hosts[item]

# Sort page views by size
# Print top ten
print('Top 10 viewers')
for host in sorted(hosts.keys(), reverse=True, key=hosts.get)[:10]:
print(host, hosts[host])
# Get total page views
total = 0
for val in list(hosts.values()):
total += val
print('Unique visitors:', len(hosts))
print('Unique crawlers:', len(crawler))
print('Total visitor page views:', total)
print('Total crawler page views:', crawlCount)
</pre>

Example code - Dicts - Revision history

WikiSysop: /* Example of a log entry */