Networks

Since my NFDump/DFSen Upgrade post many moons ago, I am once again doing an install for analyzing Netflow, this time on a server running Open vSwitch (OVS).

NFDump is a netflow collector. NFSen analyzes the netflow entries captured by NFDump.

Installing NFDump is easy using Debian's package:

apt install nfdump
systemctl stop nfdump
systemctl disable nfdump

The last two lines stop and disable the default service. Service control is provided with NFSen.

From a visualization perspective, there are a number of possibilities:

• NFSen Documentation is the main starting page for the code at SourceForge.
• NFSen on Github is p-alik's unofficial updated version of the original repository - my preferred location as it has some primary bug fixes and stays with the times
• mbolli / nfsen-ng - a modernized version of NFSen but doesn't appear to have the flexible query capability
• robcowart / elastiflow - I saw a link to this from Netflow collection and visualization with Elastiflow - which may have not relationship to NFDump / NFSen but seems promising none-the-less. Installation notes are at Installing Elastiflow flow monitoring solution.

From an NFSen perspective, here are the steps I used on a Debian Bullseye installation:

apt install wget less vim
apt install php rrdtool  librrd-dev librrds-perl php-rrd
apt install libmailtools-perl
apt install libsocket6-perl
apt install git
cd /usr/src
git clone https://github.com/p-alik/nfsen.git
cd nfsen
cp etc/nfsen-dist.conf etc/nfsen.conf
vim etc/nfsen.conf
./install.pl ./etc/nfsen.conf
/usr/local/bin/nfsen start
diff nfsen-dist.conf nfsen.conf

These are the changes to the nfsen.conf file: Continue reading "Netflow with NFDUMP, NFSEN and Open vSwitch and..." »

From a networking list serve:

Question:

Can anyone suggest tools, techniques and helpful contacts for backtracking spoofed packets? At the moment someone is forging TCP syns from my address block. I'm getting the syn/ack and icmp unreachable backscatter. Enough that my service provider briefly classified it a DDOS. I'd love to find the culprit.

Response:

It's not complete, but if you're receiving the ICMP net/port unreachable backscatter it should include a portion of the original packet. This might provide some insight into the TTL left on TCP the packet when it reached its destination which could provide a rough radius that you would need to look at. Also, if the TTL is constant it would support the idea that one or few hosts are spoofing your address block, but if the TTL varies widely it might indicate that many bots are spoofing your address block.

You might check looking glass tools or something like https://radar.qrator.net to see if someone is not only spoofing your address range, but has gone farther and has hijacked it.

Quite some ago, there was a discussion around [ovs-dev] Creating a libopenvswitch-dev package, back in 2016. Since then libraries have been updated for C++ inclusion, plus other improvements.

From somewhere, I don't have the reference, maybe my own blog, but since I have this in a note on my desktop, I'll include it here:

• Build a string formatted as the “ovs-ofctl” command would do, for example “table=1,cookie=0xdeadbeef,in_port=1,actions=resubmit(,2)”
• Pass this to parser_ofp_flow_mod_str(). The parser will magically parse the string in to a rich struct.
• Pass the struct to ofputil_encode_flow_mod(). This returns a struct with a serialized OpenFlow message buffer.
• Send this buffer on the wire either via a socket you manage yourself or let OpenVswitch do it for you.

• OpenWebRX -for listening to On-Line SDRs using a web browser. It is a web based server and interface for remotely accessing RTL-SDRs and SDRPlay's. Another link: Open Source SDR Web App for Everyone
• NYC Mesh - community-owned network for fast, affordable and fair access to the Internet - Most of the devices we use, such as a LiteBeam or NanoStation, are self-contained so they have an antenna, radio and ethernet router all in one.

To build a wpa_supplicant configuration file:

# wpa_passphrase testap >> /etc/wpa_supplicant/testap.conf
  a_password
# cat /etc/wpa_supplicant/testap.conf
  # reading passphrase from stdin
  network={
          ssid="testap"
          #psk="a_password"
          psk=1b97a5c02076ac80739263fb33db49c7c50d3199202872e06097e7baf305f35e
  }

Additional info at How to connect to a WPA/WPA2 WiFi network using Linux command line.

From a mailing list:

One way to avoid knowing or typing the specific LL address is to ping the all-nodes link scoped multicast address, and looking for the address you're after in the response.

E.g. under Linux, the following command will show all IPv6 link local addresses on the link (assuming they reply to an ICMPv6 echo request) that respond within 1 second:

# ping -w 1 -I wlp6s0 ff02::1
ping6: Warning: source address might be selected on device other than wlp6s0.
PING ff02::1(ff02::1) from :: wlp6s0: 56 data bytes
64 bytes from fe80::e54:15ff:fe66:3b9a%wlp6s0: icmp_seq=1 ttl=64 time=0.074 ms
64 bytes from fe80::aca7:4aff:fed2:a7dd%wlp6s0: icmp_seq=1 ttl=64 time=3.47 ms (DUP!)
64 bytes from fe80::400:55ff:fe40:5%wlp6s0: icmp_seq=1 ttl=64 time=3.51 ms (DUP!)
64 bytes from fe80::9e8e:cdff:fe0e:6709%wlp6s0: icmp_seq=1 ttl=64 time=216 ms (DUP!)
64 bytes from fe80::9e8e:cdff:fe0e:66ff%wlp6s0: icmp_seq=1 ttl=64 time=221 ms (DUP!)
64 bytes from fe80::fe90:d8d2:7ec5:3c8%wlp6s0: icmp_seq=1 ttl=64 time=221 ms (DUP!)
64 bytes from fe80::a65d:36ff:fe40:e31c%wlp6s0: icmp_seq=1 ttl=64 time=250 ms (DUP!)

--- ff02::1 ping statistics ---
1 packets transmitted, 1 received, +6 duplicates, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.074/130.707/249.971/111.633 ms

Comment:

Can you write the code which tells you when you have the no IPv4 “thing” without trying the IPv4 thing on a network which by definition should not have the IPv4 thing where you want people not to try the IPv4 thing?

Response:

We've been here before - I don't think there is any way to write an algorithm that can differentiate between "I can see IPv4 chatter and I should join in because this network is completely IPv6 only yet" and "I can see IPv4 chatter and I shouldn't join in because this network is supposed to be IPv6 only but some other nodes don't understand that". If a key part of the algorithm is "there aren't other IPv4 nodes chattering" then you only need one device on the network to prevent everything else shutting down IPv4.

Because ovsdb is a generic database tool, some one may have build other schemas. But for a native open vswitch install, this is the default state:

# ovsdb-client list-dbs
Open_vSwitch
_Server

It is possible to add a tcp connection to gain access to the database:

# ovs-appctl -t ovsdb-server ovsdb-server/add-remote ptcp:6640
# netstat -antp |grep ovs
tcp        0      0 0.0.0.0:6640            0.0.0.0:*               LISTEN      746/ovsdb-server
tcp        0      0 127.0.0.1:57424         127.0.0.1:6633          ESTABLISHED 397/ovs-vswitchd

But on linux, there is an existing linux stream available:

# lsof|grep ovsdb|grep STREAM
ovsdb-ser   746                   root    5u     unix 0x000000001740f722        0t0   59425581 /var/run/openvswitch/db.sock type=STREAM
ovsdb-ser   746                   root   16u     unix 0x0000000021edd23e        0t0      12914 /var/run/openvswitch/db.sock type=STREAM
ovsdb-ser   746                   root   19u     unix 0x000000001b392a81        0t0      20287 /var/run/openvswitch/ovsdb-server.746.ctl type=STREAM

# ovs-vsctl list open_vswitch
_uuid               : 8cfeb1c0-34ca-4f91-a162-7e9aef141344
bridges             : [e65f5e97-36ec-45e3-b83e-64129dae61d3]
cur_cfg             : 65
datapath_types      : [netdev, system]
db_version          : "7.16.1"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname="nuc8i7hvk01.burkholder.net", rundir="/var/run/openvswitch", system-id="02700cd5-96f5-4b4e-90b4-5e67fe06abf7"}
iface_types         : [erspan, geneve, gre, internal, "ip6erspan", "ip6gre", lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 65
other_config        : {}
ovs_version         : "2.10.1"
ssl                 : []
statistics          : {}
system_type         : debian
system_version      : []

# ovs-appctl dpif/show
system@ovs-system: hit:50661581 missed:207452
  ovsbr0:
    eno1 5/3: (system)
    enp5s0 2/1: (system)
    ovsbr0 65534/2: (internal)
    veth-nvpn-v90 6/6: (system)
    vlan110 4/5: (internal)
    vlan90 3/4: (internal)

# ovs-appctl dpctl/show -s
system@ovs-system:
  lookups: hit:50661616 missed:207458 lost:0
  flows: 6
  masks: hit:211426886 total:10 hit/pkt:4.16
  port 0: ovs-system (internal)
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:1428 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:0  TX bytes:177489 (173.3 KiB)
  port 1: enp5s0
    RX packets:19439356 errors:0 dropped:0 overruns:0 frame:0
    TX packets:30374161 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:10608677166 (9.9 GiB)  TX bytes:29950322693 (27.9 GiB)
  port 2: ovsbr0 (internal)
    RX packets:333032 errors:0 dropped:29829 overruns:0 frame:0
    TX packets:1428 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:23060625 (22.0 MiB)  TX bytes:177489 (173.3 KiB)
  port 3: eno1
    RX packets:6037565 errors:0 dropped:0 overruns:0 frame:0
    TX packets:5719808 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:4726094728 (4.4 GiB)  TX bytes:3277870100 (3.1 GiB)
  port 4: vlan90 (internal)
    RX packets:0 errors:0 dropped:274558 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:0  TX bytes:0
  port 5: vlan110 (internal)
    RX packets:0 errors:0 dropped:274101 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:0  TX bytes:0
  port 6: veth-nvpn-v90
    RX packets:5000388 errors:0 dropped:0 overruns:0 frame:0
    TX packets:5788139 errors:0 dropped:0 aborted:0 carrier:0
    collisions:0
    RX bytes:3055006377 (2.8 GiB)  TX bytes:4647166781 (4.3 GiB)

IPv6 Transition and Coexistence IPv6 - only and IPv4 as - a - Service

You can use Jool for both 464XLAT and just NAT64.

Question:

Thanks... that is what I don't understand: Why is NAT64 such a difficult concept to put into routers and firewalls? We already do NAT with IPv4 just fine.

I feel like IPv6 adoption would be much faster if there was a transition mechanism other than dual stacking.

Think: Corporate offices. Rather than renumbering everything inside, they just turn on NAT64 and now they can begin a slow and controlled transition.

Answer:

This document, which is already in the IESG review, may answer your question: draft-ietf-v6ops-transition-ipv4aas

Also take a look into this one: draft-ietf-v6ops-nat64-deployment.

Remember that if your enterprise network has apps that use literals, or they don't support IPv6, you still need dual-stack in the LANs, but access IPv6-only is just fine.

Other comments:

It’s not s difficult concept but you need to remember NAT44 breaks stuff and NAT64/NAT46 breaks more stuff.

Dual stacking is SIMPLE. REALLY. Turn on IPv6 with the M bit set and configure the DHCPv6 server. If you don’t need that level of control of address assignments leave the M bit off. 99.99% of your machines will just add a second address to the interface without you having to do anything more.

Getting to IPv6 resources from IPv4 address is a *much* harder problem that getting to IPv4 resources from IPv6 which is what you are describing here with the “no renumber everything as they already have a IPv4 address” requirement. NAT64 allows IPv6 devices to get to legacy IPv4 servers. To allow IPv4 devices to get to IPv6 servers you need to map the IPv6 addresses you want to talk to in to a pool of IPv4 addresses and push that mapping to a NAT46 (not NAT64) device.

Go dual stack then, once IPv6 is stable, turn off IPv4 if you want to be single stacked. You are then no longer dependent on the services you want want to access continuing to be offered over IPv4. 464XLAT will only work as a stop gap for IPv4 clients while services are offered over IPv4. After ~20 years of IPv6 being available (Windows XP had IPv6 support and it was not the first major OS to have it) just turn on IPv6.

Some ideas from a mailing list:

quick and dirty:

jens@screen:~$ dig nanog.org @8.8.8.8 | grep "Query time"
;; Query time: 16 msec
jens@screen:~$ dig nanog.org @1.1.1.1 | grep "Query time"
;; Query time: 3 msec

Indeed. For instance, the delay depends wether the cache it hot or cold (measuring response time for an authoritative server is easier).

also could use ripe atlas ...

Which embeds clients for ICMP Echo, DNS, NTP, TLS, arbitrary TCP (with some hacks), and, with serious limitations, HTTP.

use dig -u to get microsecond resolution, e.g.

$ dig -u @131.111.8.42 nanog.org | grep time:
;; Query time: 611 usec

I recommend that eyeball networks don't run any external recursive server for optimal CDN performance. Yes, some CDNs support other methods, but not all. If not all do, then the requirement remains. See On Firefox Moving DNS to a Third Party

freeware GRC tool for benchmarking DNS performance

• PacketLife's Armory - encryption, exploitation, intrusion detection, forensics, vulnerability scanning, system monitoring, packet crafting, spoofing, ...
• IpCisco's Network Tools - packet analyzers, security audit, simulators, intrusion detection, ...
• packages found in Debian Linux for generating packets and sending, useful for testing openflow controllers: packeth, ostinato, netexpect, tgn (simpler netexpect), sendip, 4g8 (l2 interception)

• Telegeography.com
• Hurricane Electric
• Cablemap
• Submarine Cable Map

... from the mailing list:

//You can verify that OVS supports deletion by cookie specification by checking these tests:

 932: ofproto.at:1746    ofproto - del flows based on cookie
 933: ofproto.at:1767    ofproto - del flows based on cookie mask

// Run the tests using “make check” like this:
sudo make check TESTSUITEFLAGS='932-933' -C _gcc

//Look at what is being tested here:
tests/ofproto.at

Maybe you want to check what Ryu is sending the switch

http://www.openvswitch.org//support/dist-docs/ovs-ofctl.8.pdf

“OpenFlow Switch Monitoring Commands:
monitor switch [miss-len] [invalid_ttl] [watch:[spec...]]
Connects to switch and prints to the console all OpenFlow messages received. Usually, switch
should specify the name of a bridge in the ovs−vswitchd database.“

eg) ovs-ofctl monitor br0 65534 invalid_ttl --detach --no-chdir --pidfile 2> ofctl_monitor.log

• Routing In Fat Trees (rift): addresses the demands of routing in Clos and Fat-Tree networks via a mixture of both link-state and distance-vector techniques colloquially described as 'link-state towards the spine and distance vector towards the leafs'. RIFT uses this hybrid approach to focus on networks with regular topologies with a high degree of connectivity, a defined directionality, and large scale.
• Network Virtualization Overlay Solution using EVPN explores the various tunnel encapsulation options over IP and their impact on the EVPN control-plane and procedures
• Interconnect Solution for EVPN Overlay networks

For Android:

wifianalyzer for Android, shows what station IDs are on what channels, handles 2.4g and 5G connections, etc. Doesn't provide mapping, just shows "from where I am right know, what channels have which stations as what strengths?"

Nanobeam:

NanoBeam® AC is a directional antenna/radio integrated unit and is intended as a point to point or point-to-multipoint WISP client radio. The one feature you can get from it very cheaply is a directional, 2x2 MIMO 5.x GHz band spectrum analyzer that sees things *which are not 802.11 or wifi based. sample images

Highly useful for tracking down a specific source of non-wifi 5 GHz band interference. There's all sorts of random consumer grade things people can buy and introduce into an environment which do not broadcast MAC addresses or SSIDs, and do not show up on purely 802.11(abgn/ac) based tools.

It will of course also see hidden SSIDs and standard+non-standard 802.11abgn(ac) emitters.

There are also 2.4 GHz versions of similar products which will let you find non-802.11 emitters in the 2300 to 2500 MHz band. At $79 a lot less expensive than a "real" spectrum analyzer.

You can get DC PoE injectors for them which will connect to a Makita drill battery if you want to make it portable and wander around with a laptop.