Since my NFDump/DFSen Upgrade post many moons ago,
I am once again doing an install for analyzing Netflow, this time on a server running Open vSwitch (OVS).
NFDump is a netflow collector. NFSen analyzes the netflow entries captured by NFDump.
Installing NFDump is easy using Debian's package:
apt install nfdump
systemctl stop nfdump
systemctl disable nfdump
The last two lines stop and disable the default service. Service control is provided with NFSen.
From a visualization perspective, there are a number of possibilities:
From an NFSen perspective, here are the steps I used on a Debian Bullseye installation:
apt install wget less vim
apt install php rrdtool librrd-dev librrds-perl php-rrd
apt install libmailtools-perl
apt install libsocket6-perl
apt install git
cd /usr/src
git clone https://github.com/p-alik/nfsen.git
cd nfsen
cp etc/nfsen-dist.conf etc/nfsen.conf
vim etc/nfsen.conf
./install.pl ./etc/nfsen.conf
/usr/local/bin/nfsen start
diff nfsen-dist.conf nfsen.conf
These are the changes to the nfsen.conf file:
# diff etc/nfsen-dist.conf etc/nfsen.conf
21c21
< $BASEDIR = "/data/nfsen";
---
> $BASEDIR = "/usr/local/nfsen";
25c25
< $BINDIR="${BASEDIR}/bin";
---
> $BINDIR="/usr/local/bin";
33c33
< $CONFDIR="${BASEDIR}/etc";
---
> $CONFDIR="/etc/nfsen";
39c39
< $HTMLDIR = "/var/www/nfsen/";
---
> $HTMLDIR = "/var/www/html/nfsen/";
47c47
< $VARDIR="${BASEDIR}/var";
---
> $VARDIR="/var/local/nfsen";
50c50
< # $PIDDIR="$VARDIR/run";
---
> $PIDDIR="/run/nfsen";
53c53
< # FILTERDIR="${VARDIR}/filters";
---
> $FILTERDIR="${VARDIR}/filters";
57c57
< # FORMATDIR="${VARDIR}/fmt";
---
> $FORMATDIR="${VARDIR}/fmt";
63c63
< $PROFILESTATDIR="${BASEDIR}/profiles-stat";
---
> $PROFILESTATDIR="${VARDIR}/profiles/stat";
67c67
< $PROFILEDATADIR="${BASEDIR}/profiles-data";
---
> $PROFILEDATADIR="${VARDIR}/flows";
79c79
< $PREFIX = '/usr/local/bin';
---
> $PREFIX = '/usr/bin';
83c83
< # $COMMSOCKET = "$PIDDIR/nfsen.comm";
---
> $COMMSOCKET = "$PIDDIR/nfsen.comm";
91c91
< $USER = "netflow";
---
> $USER = "www-data";
95,96c95,96
< $WWWUSER = "www";
< $WWWGROUP = "www";
---
> $WWWUSER = "www-data";
> $WWWGROUP = "www-data";
115a116
> #
128c129
< $SUBDIRLAYOUT = 1;
---
> $SUBDIRLAYOUT = 7;
174a176,180
> #%sources = (
> # 'upstream1' => { 'port' => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
> # 'peer1' => { 'port' => '9996', 'IP' => '172.16.17.18' },
> # 'peer2' => { 'port' => '9996', 'IP' => '172.16.17.19' },
> #);
176,178c182
< 'upstream1' => { 'port' => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
< 'peer1' => { 'port' => '9996', 'IP' => '172.16.17.18' },
< 'peer2' => { 'port' => '9996', 'IP' => '172.16.17.19' },
---
> 'host01' => { 'port' => '9995', 'col' => '#0000ff', 'type' => 'netflow' }
For the the OVS flow generator, a couple of blog entries helped:
The command to add the netflow exporter:
ovs-vsctl -- set Bridge ovsbr0 netflow=@nf0 \
-- --id=@nf0 create NetFlow targets=\[\"10.55.50.222:9995\"\] \
active-timeout=30 add_id_to_interface=false
The address and port are in square brackets which provide the ability to add additional destinations (separated by commas). Some additional options 'engine_id=10 engine_type=20' with suitable changes in parameters to distinguish exporters.
ovs-vsctl man page provides more information.
This is used to clear the entry:
ovs-vsctl clear Bridge ovsbr0 netflow
A few other commands:
# show interface numbers used in the export:
ovs-ofctl show ovsbr0
# show the netflow settings:
ovs-vsctl list netflow
After about 15 minutes of collecting, I decided to take a look at my cameras to see what they were doing by using a query in the NFSen interface:
net 10.55.60.0/24 and not (host 10.55.60.101 or host 10.55.90.10)
The cameras are in a block, and I wanted to exclude other known devices with which they were communicating. I end up with a considerable number of unknown external connections:
** nfdump -M /var/local/nfsen/flows/live/host01 -T -r 2020-12-27/nfcapd.202012272340 -c 20
nfdump filter:
net 10.55.60.0/24 and not (host 10.55.60.101 or host 10.55.90.10)
Date first seen Event XEvent Proto Src IP Addr:Port Dst IP Addr:Port X-Src IP Addr:Port X-Dst IP Addr:Port In Byte Out Byte
2020-12-27 23:39:33.886 INVALID Ignore UDP 3.94.153.121:58200 -> 10.55.60.34:51531 0.0.0.0:0 -> 0.0.0.0:0 245 0
2020-12-27 23:39:59.032 INVALID Ignore TCP 10.55.60.22:58219 -> 10.55.50.202:53 0.0.0.0:0 -> 0.0.0.0:0 272 0
2020-12-27 23:39:59.032 INVALID Ignore TCP 10.55.50.202:53 -> 10.55.60.22:58219 0.0.0.0:0 -> 0.0.0.0:0 140 0
2020-12-27 23:39:59.031 INVALID Ignore TCP 10.55.60.22:58219 -> 10.55.50.202:53 0.0.0.0:0 -> 0.0.0.0:0 288 0
2020-12-27 23:39:59.032 INVALID Ignore TCP 10.55.50.202:53 -> 10.55.60.22:58219 0.0.0.0:0 -> 0.0.0.0:0 140 0
2020-12-27 23:39:57.435 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8810 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:57.435 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8814 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:57.437 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8815 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:57.435 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8811 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:57.435 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8812 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:57.435 INVALID Ignore UDP 10.55.60.21:32370 -> 54.209.127.50:8813 0.0.0.0:0 -> 0.0.0.0:0 1015 0
2020-12-27 23:39:11.095 INVALID Ignore TCP 54.89.234.2:443 -> 10.55.60.22:45018 0.0.0.0:0 -> 0.0.0.0:0 124 0
2020-12-27 23:39:43.887 INVALID Ignore UDP 3.94.153.121:58200 -> 10.55.60.35:54634 0.0.0.0:0 -> 0.0.0.0:0 245 0
2020-12-27 23:39:13.857 INVALID Ignore TCP 34.201.217.96:443 -> 10.55.60.21:33279 0.0.0.0:0 -> 0.0.0.0:0 123 0
2020-12-27 23:40:11.002 INVALID Ignore TCP 10.55.60.22:45018 -> 54.89.234.2:443 0.0.0.0:0 -> 0.0.0.0:0 201 0
2020-12-27 23:38:53.780 INVALID Ignore UDP 10.55.60.35:54634 -> 3.94.153.121:58200 0.0.0.0:0 -> 0.0.0.0:0 690 0
2020-12-27 23:38:53.792 INVALID Ignore UDP 10.55.60.34:51531 -> 3.94.153.121:58200 0.0.0.0:0 -> 0.0.0.0:0 690 0
2020-12-27 23:38:53.896 INVALID Ignore UDP 10.55.60.33:37548 -> 3.94.153.121:58200 0.0.0.0:0 -> 0.0.0.0:0 690 0
2020-12-27 23:40:13.777 INVALID Ignore TCP 10.55.60.21:33279 -> 34.201.217.96:443 0.0.0.0:0 -> 0.0.0.0:0 200 0
2020-12-27 23:38:54.309 INVALID Ignore UDP 10.55.60.31:50770 -> 3.94.153.121:58200 0.0.0.0:0 -> 0.0.0.0:0 690 0
Summary: total flows: 20, total bytes: 10828, total packets: 62, avg bps: 956, avg pps: 0, avg bpp: 174
Time window: 2020-12-27 23:37:28 - 2020-12-27 23:44:58
Total flows processed: 20, Blocks skipped: 0, Bytes read: 240428
Sys: 0.000s flows/second: 20661.2 Wall: 0.000s flows/second: 34188.0
Time to put in some firewall entries.
