Raymond P. Burkholder - Things I Do

In looking for open source routing and open flow solutions, I came across Brian Linkletter's site called Open-Source Routing and Network Simulation. I never realised that there were so many network simulators out there. His site specializes in evaluating many of them.

Two reviews stand out. The first is DNS and BIND demonstration using the Cloonix network emulator which provides a mechanism for experimenting with the DNS protocol and the various open-source implementations of DNS including BIND.

The second interesting network emulator is the OFNet SDN network emulator. it comes prepackaged as an .ova file. The article indicates that it comes with the Floodlight and Beacon controllers. I wonder if Ryu could be used? Anyway, based upon the article, the emulator has visual tools for watching OpenFlow messages, evaluating topologies, and show traffic patterns. Further into the article, the author installs OpenDaylight, so it does look like other controllers can be trialled.

I have this page in my browser as an interesting project, called VillageTelco, developed by a group of people for the 'more remote places' of this planet. The have developed low power gear and used open source tools to build "easy-to-use, scalable, wireless local, DIY, telephone company toolkit". It works on a mesh principle, and they call their devices "Mesh Potatoes".

When looking at kernel commits, I saw BATMAN, and was wondering what it was. That led to open-mesh, which lead to their projects page, which linked to the VillageTelco project.

Related to IP Telephony, in an open-source world, I also came across Kamailio:(successor of former OpenSER and SER) is an Open Source SIP Server ... used to build large platforms for VoIP and realtime communications – presence, WebRTC, Instant messaging and other applications. On their web site:

Among the powerful features: asynchronous TCP, UDP and SCTP, secure communication via TLS for VoIP (voice, video, text); WebSocket support for WebRTC; IPv4 and IPv6; SIMPLE instant messaging and presence with embedded XCAP server and MSRP relay; asynchronous operations; IMS extensions for VoLTE; ENUM; DID and least cost routing; load balancing; routing fail-over; accounting, authentication and authorization; support for many backend systems such as MySQL, Postgres, Oracle, Radius, LDAP, Redis, Cassandra, MongoDB, Memcached; Json and XMLRPC control interface, SNMP monitoring.

A related SIP site is intel.org, which is home to the SIP Express Router, SIP Express Media Server, and SIP Express Router Web.

CSRP is a captive solution making use of the above products. I can't vouch for the product, just saw it in passing, which lead to the above resources.

Continuing along on the series of articles on HDMI Encoding, I came across some additional boards which provide various indirect ways of getting HDMI to a CPU which can perform the encoding.

Pender Electronic Design has a HDMI2CSI: 4K HDMI to CSI Interface for NVIDIA TX1 Evalboard. This is a card which rides on the TX1 as a mezzanine card, takes two channels of HDMI in, and uses the Toshiba TC358840 to convert to CSI-2 signals for input to the TX1. A TX2 could probably be used as well. Audio channels are included. I encountered the description in a blog post. An additional blog post talks about Open Source Driver for HDMI2CSI Module Release. Kernel and drivers are available on github InES-HPMM.

Auvidea carries a number of NVIDIA Jetson TX1 carrier boards and CSI-2 Bridges. The bridge boards are single HDMI-in to CSI-2 interfaces. These have bee used atop the Raspberry Pi devices. These modules use the Toshiba TC358743XBG.

LWN has an article about the Linux Driver for Toshiba TC358743 HDMI to CSI-2 bridge.

From the HDF5 mailing list:

A little announcement, like I think potentially people in the HDF5 community could be interested in that.

We developed at Blue Brain Project / Human Brain Project a simple, modern, header-only C++ interface to the libhdf5. Mainly to overcome the current limitations of the actual HDF5 C++ interface ( no parallel HDF5, no thread safety, not modern C++ friendly )

github.com/BlueBrain/HighFive

It is Open Source and here is a non-explicit list of features :

• It supports both serial and parallel HDF5 with C++ ( contrary to the official bindings that disable c++ bindings when compiled with parallel mode )
• It does not require any compilation, and is embeddable in any project ( Boost Software License ).
• It supports automatic type mapping with C++ types and STL containers : std::vector, std::string, std::array, etc...
• It supports Boost MultiArray and Boost UBLAS for Multi-dimentional dataset and Matrix load/save to/from HDF5 datasets.
• It is design to be modern C++11 / C++14 friendly
• It aims to be minimalsit and does not have any other dependencies than libhdf5 itself
• It does not support the integrality of the libhdf5 API for now, but If you have any interest or comment about it, let us know.

Adrien Devresse - Blue Brain Project / Human Brain Project

A question on the mailing list:

I'm interested in how you handle thread safety. I've only had a quick look, but looking at e.g. SliceTraits::read, I can't see any lock being taken before the call to H5Dread (nor around the surrounding C++ code). Do you assume the HDF5 library is compiled in thread safe mode? And is are the wrappers themselves thread safe? -- Elvis Stansvik

An answer to the question:

For now, you get the thread safety provided by the HDF5 library yes. If libhdf5 has been compiled with thread safety, then HighFive is thread safe. If not, you need to do the locking yourself unfortunately. All the wrappers around libhdf5 provided by HighFive are thread-safe.

When I first superficially looked at the Nvidia TX2 card i noticed two things: a) HDMI port, and b) video encoding SDK. First though: oh cool, I can encode inbound HDMI signals. On second inspection, nope. That is an outbound HDMI port.

Hmm.. how does one get video into the unit? Well, it does have the CSI2 ports. Then the next search is: anything having to do with HDMI to CSI2 signal conversion?

One of the search results is a Raspberry Pi forum discussion of a HDMI to CSI adapter board using a TC358743 camera bridge. The 746 version.

As a side bar to that forum discussion, one salient point is that many of the initial Pi devices have only two of four possible 1 gbps lanes: "not going to work easily with a A, B, B+, or B2, particularly as they all only expose 2-lane CSI-2". Another form article about HDMI to CSI-2 via TC358743 on kernel 4.1 using more recent Pi versions and a auvidea B101 HDMI to CSI-2 bridge.

The Nvidia Jetson TX1/TX2 handle four lanes.

The Toshiba TC358840 is a follow on part to the TC358743 and able to handle 4K Ultra HD.

There is much software out there which can be used for decoding video streams. But the question is, how does one go about encoding a stream? Can it be done in software? Can it be done in hardware? How best to go about it? What resources are required? I asked those questions, and this is what I've put together so far for information.

For offline encoding of video, software can be used. Some thing like the opensource staxrip github based solution could be used. This is billed as a "multiformat video encoding application" with x264, x265, and XVID capabilities and looks like it incorporates some how the usage of the Nvidia encoding libraries, which might include the use of the Nvidia TX1/TX2 Jetson's for real time use. But... used .NET 4.6.1, so looks to be a Windows solution. It does reference other opensource tools like VapourSynth which is a application/plugin/library for video manipulation. It is self-purported to be inspired by AviSynth which is a video post-production tool.

Another multiplatform tool is HandBrake, which is an opensource video transcoder. It also handles h.265, h.264 plus other formats.

H.265, also known as the HEVC (High Efficiency Video Coding) compression standard, has an opensource, GPL2 licensed, c++ implementation maintained at x265.

From a hardware perspective, in my previous blog entry, I mentioned the Nvidia Jetson TX1/TX2. With the built in GPU and libraries, I believe it can do real time encoding and network distribution.

From another hardware perspective, my primary intent is to be able to take HDMI and extend it over IP, hence the existence of 'HDMI over IP extenders'. danman's blog talks about reverse engineering the transmitter end of a HDMIoverIP extender. The comments in that post have much useful info as well. With one stand out being apantac extender with docs. The gist of the article indicates that the encoded video stream is multicast. Which means that software or hardware devices on the network can pick up the stream and present it. VLC is used as one example. Digital Signage -- here we come! The ip extender concept has info in a Hacker News thread. Here is Lenkeng's site where some of the extenders are referenced. A picture showing the multicast capabilities.

The article also refers to the HDbitT technology. The HDbitT site referred to InfoComm 2016 where AV-over-IP steals the show. Seems to be a show to visit to see what is current for XoverIP. Beyond that, the only other useful info on HDbitT was an HDMI Version Comparison.

Talking about shows and media, I managed to land on AVNetwork which includes AVTechnology, SCN (Systems Contractor News), and Digital Signage Magazine. Finally, sources for current information on stuff-over-ip, including boardroom mics over Dante! And PoE speakers running Dante!

rivendellaudio: radio broadcast automation solution, with facilities fro the acquisition, management, scheduling and playout of audio content.

With digital audio/video streams available on the network, the next obvious step would be to mix and switch. One open source project I found is voctomix, which is written in Python and uses GStreamer, and is designed for interactive live mixing of incoming video streams. Github has timvideos streaming system based upon the concept.

streamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.

Snowmix is another live video mixer supporting: overlaying and mixing video, images, texts and graphic elements as well as suporting mixing audio sources. Snowmix offers animation, simple and advanced scripting, clipping and pattern gradient blending.

Given a mixed stream, then there is OpenBroadcaster: professional read IPTV On-Air studio software. Something for DigitalSignage!

While on the subject of video, there is Pitivi for non-linear editing, amongst others.

The nvidia Jetson TX2 GPU card is coming out soon, or may already be available. I would like to use that for video encoding to transport camera and hdmi information across a network. It is able to perform H.265 encode/decode processing. A comparison of Jetson TX2 vs TX1. There is a native Nvidia Video CODEC SDK.

It is possible to connect up to three camera heads to the board. Leopard Imaging has a TX1/TX2 Camera kit to do so.

ConnectTech has a carrier for Jetson TX2 and Jetson TX1 cards for enhancing i/o port accessibility.

More products from nvidia ecosystem partners.

Developer kits and news can be found at developer.nvidia.com. Another reference to nvidia embedded development. Nvidia has made various development tools available as a download.

MyzharBot is a tracked robot based upon the TX1. It appears to use a StereoLabs stereo camera, hence the use of the TX1.

• Jetson/Cameras
• Tegra/Resources
• Linux For Tegra Archive
• TX2/TX1 Comparison
• Jetson TK1 - lots of info
• Jetson TX2
• Jetson Hacks - GStreamer xvimage, which leads to the concept of GStreamer, which is an effective tool for video distribution. Plus links to drones, and other interesting things.

Khadas VIM: credit card sized 64 bit quad-core ARM cortex-A53 with Broadcom AP6212/AP6255 Wifi/Bluetooth and HDMI.

NVIDIA Jetson Modules: "his is an AI supercomputer on a module, powered by NVIDIA Pascal™ architecture. Best of all, it packs this performance into a small, power-efficient form factor that’s ideal for intelligent edge devices like robots, drones, smart cameras, and portable medical devices. It supports all the features of the Jetson TX1 module while enabling bigger, more complex deep neural networks."

HiKey960: development platform is based around the Huawei Kirin 960 octa-core ARM

From the ovs-dev mailing list:

There are cases where the default gateway of a interface is in a different subnet than its IP address. Linux allows such configuration. For e.g, one could set the IP address of a Linux interface as 172.16.1.2/32 and then give it a default gateway of 172.16.1.1. This can be done for e.g. by running the following commands.

ifconfig eth0 172.16.1.2 netmask 255.255.255.255 broadcast 172.16.1.2 
route add 172.16.1.1 dev eth0 
route add default gw 172.16.1.1

The above configuration is what google cloud uses for its VMs.

In OVN static routes, we currently have the ability to specify the router port via which the packet needs to be pushed out to reach a next hop. But when support for IPv6 was added, we only allowed nexthops to be in the same subnet as one of the router's IP addresses.

This commit relaxes that restriction. When a outport is specified in static routes and when a nexthop is in a different subnet than any of the router IP addresses, we will assume that it is reachable from the first IP address of the router. Since this is a corner case, we just go with the first IP address. If it turns out that there are more cases, we can let users choose the IP address via which the destination is reachable.

It’s usually best to leverage some sort of “smart” DNS to handle CNAME distribution, giving you the ability to weight your CNAME distribution vs. only using one CDN all the time, or prefer different CDNs in various global regions. I’ve had decent experience with Dyn here, but Route53 has all the features you’d want as well. If possible, write tooling towards your DNS provider’s API to automate your failovers.

Weight your distribution such that you never have one CDN turned off completely; you’ll want a small trickle of user traffic hitting every CDN so that the caches won’t be cold when you switch over to it.

Make sure you have a distributed metrics service (ThousandEyes, WebMetrics, et al) testing your CDNs individually as well as the external hostname.

Stay away from HTML- or Header-munging features when possible; stick with feature sets that are common (and implementable in similar ways) across your providers. (Similar advice goes for multi-vendor *anything*, TBH)

Some folks just let Cedexis do the work: www.cedexis.com

From the Linux NetDev List:

Running VXLANs over IPv6 link-local addresses allows to use them as a drop-in replacement for VLANs, avoiding to allocate additional outer IP addresses to run the VXLAN over.

The first patch is basically a bugfix, not allowing to use link-local addresses without specifying an interface; it doesn't seem important enough for net/stable though (and without the second patch, allowing to specify link-local addresses at all does not result in a working configuration anyways). The second patch then actually makes VXLAN over link-local IPv6 work by passing interface indices at the right places.

The final patch lifts the limitation of not allowing multiple VXLANs with the same VNI and port, as long as link-local IPv6 addresses are used and different interfaces are specified. Again, this brings VXLAN a bit closer to VLANs feature-wise.

ip needs some additional examples, ... from the netdev mailing list:

excellent opportunity for an aspiring writer / user to make a contribution to improve iproute2

git clone
git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
cd iproute2
--make changes--
commit, send to netdev.

From the ova-dev mailing list:

OVN will generate the IPv6 address for a logical port if requested using the IPv6 prefix and the MAC address (as IEEE EUI64 identifier). To generate the IPv6 address, CMS should define the IPv6 prefix in the 'Logical_switch.other_config:ipv6_prefix' column.

If the CMS wants to make use of ovn ipam it can now provide a list of IPv4 addresses and a range of IPv4 addresses which will be excluded from the dynamic address assignment. To support this, a new option 'exclude_ips' is added in the Logical_switch.other_config column.

Eg. ovn-nbctl set Logical_switch sw0
other_config:exclude_ips="10.0.0.2 10.0.0.30..10.0.0.40"

The present code, uses hash maps to store the assigned IP addresses. In order to support this option, this patch has refactored the IPAM assignment. It now uses a bitmap to manage the IP assignment with each bit in the bitmap representing an IPv4 address.

This patch also clears the 'Logical_switch_port.dynamic_addresses' if the CMS has cleared 'dynamic' address assignment request.

From the morning paper :

Chronix: Long term storage and retrieval technology for anomaly detection in operational dataLautenschlager et al., FAST 2017

Chronic (http://www.chronix.io/ ) is a time-series database optimised to support anomaly detection. It supports a multi-dimensional generic time series data model and has built-in high level functions for time series operations. Chronix also a scheme called "Date-Delta-Compaction" (DDC) for timestamps as well as data compression.

From the ipv6 mailing list:

Can anyone tell how will a DHCPv6 subscriber learn about the default gateway it has to contact for the subscriber traffic?

A DHCPv6 subscriber is supposed to learn about default gateway from Router Advertisement.

You need to read RFC4861 to understand the relevant details of RA.

Also, can you point me to the document where it is mentioned that a DHCPv6 subscriber has to use RS-RA for resolving gateway address?

Section 7.2.2 of RFC6434 explains this (but the wording could be clearer, perhaps). The basic point is that RFC4861 is effectively mandatory and DHCPv6 is optional.