HostOnNet Blog


How to analyze Apache access logs using Scalp

What is Scalp ??

scalp

Scalp! is a log analyzer for the Apache web server that aims to look for security problems. The main idea is to look through huge log files and extract the possible attacks that have been sent through HTTP/GET (By default, Apache does not log the HTTP/POST variable).

Installing Scalp On Your PC

mkdir ~/programs
cd ~/programs
wget https://apache-scalp.googlecode.com/files/scalp-0.4.py
wget https://raw.githubusercontent.com/PHPIDS/PHPIDS/master/lib/IDS/default_filter.xml

That will install scalp on your PC in ~/programs folder. You don’t need to do it again as long as you don’t delete these 2 files.

Lets analyze apache log for a web site. Lets say you downloaded billing.hostonnet.com-May-2015.gz to your “Downloads” folder.

Lets uncompress the log file.

cd ~/Downloads
gunzip billing.hostonnet.com-May-2015.gz

Run Scalp, that will go through Apache access log and generate report.

python ~/programs/scalp-0.4.py  -f ~/programs/default_filter.xml -o ./scalp-output --html -l billing.hostonnet.com-May-2015

This will create report in HTML format inside scalp-output folder. Open it in browser, go though the results.

Example

$ python ~/programs/scalp-0.4.py  -f ~/programs/default_filter.xml -o ./scalp-output --html -l manage.hostonnet.com-Feb-2015
The directory %s doesn't exist, scalp will try to create it
Loading XML file '/home/melbin/programs/default_filter.xml'...
Processing the file 'manage.hostonnet.com-Feb-2015'...
Scalp results:
   Processed 26134 lines over 26134
   Found 758 attack patterns in 11.480543 s
Generating output in ./scalp-output/manage.hostonnet.com-Feb-2015_scalp_*
$

Posted in Apache