Last update: 28 nov. 2007

Purpose of py0f

p0f can guess the OS of all the computers that are trying to connect to yours by simply listening to the network. At home, with an ADSL connection, the log files that p0f produces grow very quickly so that they are difficult to analyse (I get about 3 Mb for 50 hours). py0f helps this task by providing basic statistics on who tried to connect to who: OS origin, destination port, etc.

With py0f you can easily evaluate the Internet threat (malware installed on Windows® machines) or detect IP's of hackers intruded into Linux web servers for example. The following figure, made with matplotlib, shows the proportion of origin OS'es and destination ports on my machine:

pie charts

These indicators clearly shows the nature of the main Internet threat and which ports to protect more than others. py0f also draws a figure with the 10 most active IP attempts per hour and the machine connection attempts frequency:

bar charts

These indicators give an idea of the malware activity evolution, in this case it increases. Finally a text output is always available for more details. Go to the end of this post to see a sample.

Download

py0f is released under the GPL 2.0 license. It was tested on Linux only[1]. You are free to download, install, use, modify and distribute the full Python package:

py0f-0.1.0.tar.gz

(MD5 sum d5aab6d9a7abb00528aad0760e301d5d)

Use the developer version if you intend to modify the code and want to repackage it later:

py0f-cvs-20071128.tar.bz2

(MD5 sum 6b08278ac23c74138d8d6abf04158c92)

Thanks to Epydoc a technical documentation is available in Html and Pdf format. The user documentation is not written yet :-( but you can invoke the online help to get information on optional arguments.

If you get interesting statistics by your own, let me know!

Installation

Uncompress the archive and install the Python module as root:

# tar xzf py0f-0.1.0.tar.gz
# cd py0f-0.1.0/
# python setup.py install

You may also make a link to the main script (recommended). Just adapt the python version in this command:

# ln -s /usr/local/lib/python2.4/site-packages/py0f/p0f-stats.py /usr/local/bin/p0f-stats

You're done!

Typical use

First generate a set of p0f log files. For example, if your p0f log directory is ~/p0f, you may run this command each time you login:

$ sudo p0f -U -t -p > ~/p0f/p0f-$(date +%Y-%m-%d-%Hh%Mmin).log

You need root access to p0f to run it since it has to access the whole network activity. Alternatively, you could set p0f as a service which automatically starts a new log file at each boot. You can quickly setup such service on Debian and Debian-like systems using my own p0fd service (MD5 sum 3c16d07d7d7891897a78ed1bbaa9cf9b). Don't forget to set logrotate in this case to limit disk space use.

Once you have collected enough p0f log files, you can compute some statistics. The typical command to run is:

$ p0f-stats ~/p0f/ --reject-from-ip *my-IP* -g

The -g option activates graphical output and requires matplotlib to be installed. Always reject your own IP as emitter since you would otherwise take into account your own web browser or email client connections! You can try the -h option to get the inline help:

$ p0f-stats -h

Good analyses!

Text output sample

Here is a session output sample performed over about 20 days and about 50 hours of log recording. The 6636 rejected attempts come from my own machine as 16330 other ones were validated for a mean frequency of one attempt every 11 seconds.

Loaded 16330 attempts while ignored 6636 attempts
Total recording time: 50:05:22
Average frequency: 1 connection every 11.0s
* OS stats:
       [9468/16330]    58.0%   Windows XP Pro SP1, 2000 SP3
       [4616/16330]    28.3%   Windows 2000 SP2+, XP SP1 (seldom 98 4.10.2222)
       [1664/16330]    10.2%   Windows 2000 SP4, XP SP1
       [175/16330]     1.1%    Windows XP SP1, 2000 SP3 (2)
       [141/16330]     0.9%    Windows XP/2000 (RFC1323, w+, no tstamp) [GENERIC]
       [66/16330]      0.4%    Windows XP Pro SP1, 2000 SP3 (NAT!)
       [65/16330]      0.4%    Windows XP (RFC1323, w+) [GENERIC]
       [59/16330]      0.4%    Windows XP/2000 while downloading (leak!)
       [20/16330]      0.1%    Windows 2000 SP4, XP SP 1 (2)
       [17/16330]      0.1%    Windows XP, 2000 SP2+
       [15/16330]      0.1%    Windows XP/2000 [GENERIC]
       [6/16330]       0.0%    Windows 2000 SP4, XP SP1 (firewall!)
       [4/16330]       0.0%    Windows 98 [GENERIC]
       [3/16330]       0.0%    Windows 98 (4)
       [3/16330]       0.0%    Windows XP/2000 (RFC1323 no tstamp) [GENERIC]
       [2/16330]       0.0%    Linux 2.4/2.6 \<= 2.6.7 (up: 2726 hrs)
       [2/16330]       0.0%    Windows XP SP1, 2000 SP4 (3)
       [2/16330]       0.0%    Windows SP3
       [1/16330]       0.0%    Linux 2.4/2.6 \<= 2.6.7 (up: 1934 hrs)
       [1/16330]       0.0%    Linux 2.4/2.6 \<= 2.6.7 (up: 522 hrs)
* source IP stats:
       [241/16330]     1.5%    82.231.xxx.xxx
       [203/16330]     1.2%    82.231.xxx.xxx
       [199/16330]     1.2%    82.231.xxx.xxx
       [198/16330]     1.2%    82.231.xxx.xxx
       [192/16330]     1.2%    82.231.xxx.xxx
       [191/16330]     1.2%    82.231.xxx.xxx
       [190/16330]     1.2%    82.231.xxx.xxx
       [182/16330]     1.1%    82.231.xxx.xxx
       [176/16330]     1.1%    82.231.xxx.xxx
       [174/16330]     1.1%    82.231.xxx.xxx
       [172/16330]     1.1%    82.231.xxx.xxx
       [169/16330]     1.0%    82.231.xxx.xxx
       [167/16330]     1.0%    82.231.xxx.xxx
       [13876/16330]   85.0%   \< threshold or unknown
* destination IP stats:
       [16330/16330]   100.0%  82.231.xxx.xxx
* destination port stats:
       [8437/16330]    51.7%   microsoft-ds - Microsoft Naked CIFS (445)
       [4575/16330]    28.0%   loc-srv - epmap Location Service (135)
       [3038/16330]    18.6%   netbios-ssn - NETBIOS session service (139)
       [198/16330]     1.2%    ms-sql-s - Microsoft SQL Server (1433)
       [61/16330]      0.4%    www - http WorldWideWeb HTTP (80)
       [6/16330]       0.0%    radmin-port - RAdmin Port (4899)
       [4/16330]       0.0%    smtp - mail (25)
       [3/16330]       0.0%    ssh - SSH Remote Login Protocol (22)
       [2/16330]       0.0%    socks - socks proxy server (1080)
       [1/16330]       0.0%    pop3 - pop-3 POP version 3 (110)
       [5/16330]       0.0%    \< threshold or unknown

Notes

[1] Sorry I don't care of Windows® any more, but this should work on it too! Please let me know if you successfully used py0f under Windows.