Entries by Raymond P. Burkholder

As a user with far too many FireFox open tabs, memory seems to disappear far too quickly. Knowing which of my many tabs contributes the most to the memory consumption would be helpful. But it seems as though Firefox doesn't make troubleshooting for that very easy. Here are a few tools which might help in narrowing my focus to determine granular memory consumption.

With recent versions of Firefox, there are some built-in tools which help to diagnose cpu and memory issues:

• about:performance - shows memory and cpu consumption
• about:memory?verbose - shows detailed memory consumption - not easily to digest
• about:cache?device=memory

Add-ons which provided in-tab memory consumption have been obsoleted. But one add-on which itemizes tabs: Tab Stats

Limiting the amount of RAM a program can use shows how on a Linux system to use cgroups to limit an application's memory footprint. The old 'ulimit' command is deprecated, and may not work.

Regarding the about:memory tool, more details can be found at Limit memory usage per Tab without blocking things? which points to about:memory and Memory.

Based upon How to Reduce Firefox Memory Usage using about:config it is possible to reduce memory caching with:

• 'about:config' in the address bar
• right click and create a 'new' -> 'integer'
• variable name is: browser.cache.memory.capacity (which is maximum RAM in KB) (further details at Browser.cache.memory.capacity) via About:config entries.

$ date
Thu May  9 09:47:03 MDT 2019
$ TZ=Z date -d '2 hours ago' +%FT%TZ
2019-05-09T13:47:04Z

I saw a pre-release of "Linux Observability with BPF" by David Calavera and Lorenzo Fontana. Well, I'd say it is a release prior to pre-release. Some good information it has, but teases the details.

This post is about what it takes to get the first example functional: the things I saw, didn't see, and kludges I put in place. With the understanding and first step in place, it can be cleaned up and additional progress can be now proceed.

I have a mixed Debian Buster/Testing/Sid workstation installation. A messy system due to the vagaries of getting Linux Stretch/Buster installed onto a new Intel NUC nuc8i7hvk01 when firmware and display drivers were just getting rolled out. Kernel updates have made progress. Given the inclination, a nice simple clean Buster rebuild will probably work this time around.

But enough of that. Here is what I needed to do for building a simple eBPF program, building a simple installer, and getting it to install, pass the verifier, and achieve a successful conclusion. Continue reading "eBPF Basics" »

It seems that in the last day or so, Mozilla forgot to renew an intermediate certificate in their FireFox browser. The result is that the browser has mass-disabled add-ons.

One temporary solution someone mentioned (which may only work on a Dev version, and/or Linux version):

Simple temporary fix is to browse to "about:config" and change the value for "xpinstall.signatures.required" to false. Well, that worked for me, anyway. When Mozilla fixes whatever the issue is, I'll set it back to true.

The 'studies based' alternate solution:

You can turn on the Studies option under Privacy & Security for a little while, then check about:studies and you should see one or two in there regarding the xpi verification/signing. Once you have those two studies, you can disable Studies again.

Mozilla has an Update Regarding Add-ons in Firefox, but does not apply to Firefox ESR (my Debian Linux install) or FireFox for Android.

Another link about resolution: Certificate issue causing add-ons to be disabled or fail to install

In this case, the three switches are special purpose appliances running Linux and containing a series of interchangeable network ports. In this session, I am going to sketch out some of the subsystems and open-source tools I've used to build a redundant/resilient router/firewall/monitoring solution on these appliances. In addition, I will describe how I was able to fully automate the build of the solution. The result being a fully documented and repeatable build. The design allows any one of the three switches to be rebuilt while the other two handle production. The end result is a template for building cost-effective, horizontally-scalable virtualization solutions.

Slide deck for the presentation has been posted at Cloud Monitors Cloud.

Raymond Burkholder is the founder of One Unified Net Limited. This has allowed him to take on a variety of challenges in network design, software development, and systems automation. His most recent stint was in Bermuda where he consulted on infrastructure for re-insurance companies, transitioned to handling day to day technical operations of an ISP, and moved on to optimizing the performance of the network, storage, monitoring, and virtualization infrastructure of a data center. He is now back in Canada taking a brief respite while looking for his next opportunity.

Raymond has a personal blog at https://blog.raymond.burkholder.net

Place 800

800 - 6 Ave. S.W.
Plus-15 Conference Room

There is $2 parking after 16:00 one block north-east of the meeting location, in the underground parkade at McDougall Centre.

5:30 PM, Tuesday, April 23, 2019

Snacks at 17:30. Meeting begins at 18:00.

Attendance is free for CUUG members, or $10 (cash only) at the door for non-CUUG members.

As a continuation of my other article incorporating CMake as a makefile generator, here are some references I used for getting a handle on how to build a set of CMakeLists.txt files to manage a project containing a library shared by two applications.

• cmake.org tutorial
• Reference Guide - direct from cmake.org
• CMake Community Wiki
• wxWiki - using CMake to include wxWidgets in projects - see below for sample
• stack overflow - Questions tagged [cmake] - the popular entries
• stack overflow - Multiple sub-projects depending on the same library - primary source for skeleton entries for a project with inter-dependent apps and libraries.
• stack overflow - CMake: Static library with multiple projects - Normally, when CMake projects have to cooperate, it's beneficial to make them part of the same buildsystem using add_subdirectory(). That way, CMake can track dependency between their targets, automatically resolve references to logical target names etc.
• Typical Linux Project using CMake - goes into detail for a project with libraries and applications, including external dependencis
• CMake Dependencies Done Right - mechanisms to evolve a CMake file - depend on defining a proper dependency tree for all libraries and executables
• stack overflow - CMake: Project structure with unit tests
• Basic CMake C++ project structure - includes an example of ExternalProject_Add - also with style notes in one of the answers
• stack overflow - CMake Setting up Release and Debug version and Flags 'cmake -DCMAKE_BUILD_TYPE=Release .. ', 'cmake -DCMAKE_BUILD_TYPE=Debug .. '
• How to configure a C/C++ project with Eclipse and CMake - manual mechanism when not using the wizard from the gui - talks about debug/release versions - something which isn't intuitively obvious when using the gui wizard for importing a CMake progject
• It's Time To Do CMake Right - Pablo Arias - Build Requirements vs Usage Requirements - defines public, private, interface plus additional concepts
• Mirko Kiefer's Blog - CMake by Example - a few twists on the simple CMake files with a focus on the 'install' concept

wxWidgets inclusion with CMake

project(myGreatProject)
cmake_minimum_required(VERSION 2.
aux_source_directory(. SRC_LIST)

set(wxWidgets_CONFIGURATION mswu)
find_package(wxWidgets COMPONENTS core base REQUIRED)
include(${wxWidgets_USE_FILE})

add_executable(${PROJECT_NAME} ${SRC_LIST})
target_link_libraries(${PROJECT_NAME} ${wxWidgets_LIBRARIES})

Random note:

${CMAKE_SOURCE_DIR}, ${CMAKE_BINARY_DIR}

• How to use GPG with YubiKey (bonus: WSL)
• Yubico - YubiKey 5C with NFC
• OpenBSD Journa - Using a Yubikey as smartcard for SSH public key authentication (referenced from Hacker News

I've been using NetBeans for quite some time as a GUI based IDE for building C++ projects. Recently, the project has been migrated over to Apache for management. In the transition time frame, NetBeans has progressed from 8.2 to 9.0 to the current 10.0 incarnation. The 8.2 version was the last which had an integral C++ development environment.

Via jumping through a few hoops, it is possible to run the c++ development environment in v10.0. The draw back is that what ever was available in 8.2 remains as 'current'. Which means the build environment is stuck at C++13.

By finding a development/beta version of 8.2 , I was able to obtain C++17 capability, but at a cost of stability of the IDE. Development of this branch has been discontinued since, I think, early 2018. I do not know if any behind-the-scenes work is happening somewhere.

So... it is time to try something else. Prior to NetBeans, I had been using Eclipse as an IDE. But, way back then, I had switched to NetBeans as Eclipse appeared to be lagging and unstable. The tables have turned. Eclipse appears to be progressing well now as a C++ IDE, far surpassing NetBean's capability.

For building, NetBeans generated makefiles directly. Eclipse can do the same. But Eclipse is also able to work with CMakeLists.txt files. Probably a boon as well as a bane, but as other projects are CMake based, I felt it might be worth a try to try.

So I dived in. I fought for a while to convert my make environment over to CMake. Boost libraries, with v1.69.0 can cause some troubles with CMake find_package abilities. The basic trick is to add the following for builds for v1.69 and possibly for v1.68:

set(CMAKE_CXX_COMPILER_ARCHITECTURE_ID, "x64")
set(Boost_ARCHITECTURE "-x64")

A primary issue undermining my successful transition was that I was using boost::log::trivial. No matter what I tried, it seems as though a CMake initiated make just wouldn't find the symbols. A message of the kind:

undefined reference to `boost::log::v2s_mt_posix::trivial::logger::get()'

The solution was simple. I had already performed the needed resolution in the old IDE. But forgot to migrate it to the new CMake file.

But as a note for posterity, adding the following to a c++ build line (which also incorporates the link step), will forward options to 'ld', the linker.

-Wl,--trace -Wl,--verbose

These options tell ld to emit additional troubleshooting information. But where to put these flags? I used the following to show the command expansions, which I could then copy and paste and change for testing (I found the flag in CMakeCache.txt):

build_dir$ cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE ..

The result was that I was indeed successfully resolving the boost_log library inclusion. But why wasn't it accessing the symbols?

The realization occurred after inspecting the library directly, and there is was, the symbol, in all its glory, different from the one for which the linker was originally searching (variations are discussed on StackOverFlow:

# nm -gC libboost_log-gcc8-mt-x64-1_69.so.1.69.0|grep trivial
0000000000099a80 T boost::log::v2_mt_posix::trivial::logger::get()

Well the solution is to place the following in to the CMakeLists.txt file:

target_compile_definitions(${PROJECT_NAME} PUBLIC BOOST_LOG_DYN_LINK )

This forces the c++ build to use the dynamic boost::log library rather than the static file. The symbol then changes to

`boost::log::v2_mt_posix::trivial::logger::get()'

which can fe found in the dynamic link file. And all is well.

As a side note, when using the CMAKE_VERBOSE_MAKEFILE expansion, with a few tweaks I was able to take a '/usr/bin/c++' invocation and run it instead as '/usr/bin/ld'. But this doesn't work very well due to many missing libraries, which the linking form of '/usr/bin/c++' automatically provides.

The link command is found in CMakeFiles/${program_name}.dir/link.txt along with other build products.

Somehow I landed at Programmer migration patterns, which to cut to the chase, says that programmers will end up in JavaScript, Rust, Java, or Go.

In the blog entry, TypeScript is mentioned, which 'kinda' a successor to JavaScript, but not really, but 'kinda' a meta-JavaScript. And by happenstance, as I 'grew up' around Turbo Pascal and Delphi (designed by Anders Hejlsberg), I found out TypeScript development is headed by the same fellow. Small world. JavaScript/TypeScript gets some strong type checking, so my own conversion from C++ may result.

In the Wikipedia entry, as a 'see also' there is a link to Dart which I came across just the other day. It is another Google construct (as in, Go is a Google construct), which appears to be a JavaScript wanna-be replacement, as it, too, compiles to JavaScript. But, I think it is less mainstream.

But Dart seems to appear in conjunction with Google Fuchsia, which wants to be an alternative MicroKernel operating system. Flutter appears to be an IDE for the development work for Android and iOS, of which Dart is included as the development language. Dr. Xu Zhongxing's speech: Fuchsia OS Introduction introduces the why's and wherefore's of the OS design thinking. It might be an interesting container management tool, which in ways, is what Chrome already is.

Have you ever wanted to change the kernel/distribution of a running server but you don't have access to boot media or pxe mechanisms? I had come across a solution many moons ago, but lost the link to how it could be accomplished.

The basic mechanism relies on being able to SSH into a server, remain in a console environment, build a ram based environment, unmount and rebuild the drive environment, then reboot from the modified drive environment.

Now, within the space of a week, I came across two links which describe, or at least of scripted explanations to the process:

• reload.sh found on github with 192 stars (as of this writing), and links to the next one as it's source of influence
• takeover.sh, also found on github but with 5808 stars (as of this writing), and has some Hacker News background -- "A script to completely take over a running Linux system remotely, allowing you to log into an in-memory rescue environment, unmount the original root filesystem, and do anything you want, all without rebooting. Replace one distro with another without touching a physical console."

I have not yet personally evaluated these scripts for my own use. But I will be sure to run the procedure in a protected, reproducible test environment prior to attempting the procedure in something resembling a production environment.

Somewhere in the Globe and Mail:

All things being equal – such as revenue, number of employees – the value of a product company that owns intellectual property (IP) is four times that of a similar service company. Generally speaking, an investor will value a service-based company at one to two times its revenue, while product-based companies are valued at four to 10 times.

• 6-Digit NTP Synchronized Network Clock with IEEE 802.3af PoE - LED digital wall clocks that synchronize over an Ethernet network to an accurate time reference. [with a starting point of A Guide To GPS NTP Servers For Network Time Synchronization]
• GPS NTP Network Time Server (TM1000A) - The TimeMachines GPS NTP Network Time Server is a simple to use GPS based time server that supplies accurate time for all computers and time keeping devices on the network.
• Nothing to see, but something to read: Network Synchronization: Public NTP Servers vs. GPS NTP Servers

Or not see:

• LeoNTP Network Master Clock / Time Server is a Stratum 1 Master Clock / Time server with GPS synchronised clock source designed from the ground up with the sole purpose of providing a cost effective but highly accurate and extremely high performing networked time server.

From the Debian bugs mailing list here is an interesting result of using non-native code in emulation:

1) Install qemu-system-arm.

2) Unpack kernel .deb, locate vmlinuz-4.9.0-8-armmp.
Or search your just-built zImage.

3) Run qemu (no root required):

  qemu-system-arm -M virt -nographic -kernel vmlinuz-4.9.0-8-armmp

If you see a kernel panic - the outcome is positive, the kernel is OK.
If it just stays there eating 100% CPU - the outcome is negative.

The above is used to sus out some issues with the linux-image-4.9.0-8-armmp-lpae:armhf kernel on Banana PI and Lamobo R1, with the notes in Bug#922478. More talk of the bug on Hacker News.

The more you try to hide, the more attention you attract. -- Anon - the movie

Anonymizing one's self is difficult. Technology companies are coming up with surprising ways to bypass the cloak of 'leave me alone' we attempt to wrap around ourselves. From a news group, here is an interesting mechanism describing the continuing escalation:

The statement of purpose:

To augment what many already have said here - it's a tie between IP address and household address and name/surname that should not be public, the address itself is mostly harmless and it doesn't matter if it's static or dynamic. Tracing gets done on L7 and above anyway

The statement of bypass:

In the current state of the web, if a user gets a new address, the various web surveillance mechanisms will associate it with the previous user identity and context after maybe a dozen web queries or two. That gets better if the user has a strong defensive posture, blocks web trackers, etc. But it just takes one cookie, or one login, or some other inference, and poof, the surveillance databases know that new_address_X is the same user as old_address_Y.

The case of the household is even worse. It takes only one cookie, login, etc, and the surveillance databases will learn that new_prefix_P is the same household as old_prefix_Q. Even you run Ghostery or Privacy Badger or the latest release of Firefox, you won't escape if you share a prefix with someone in your household who doesn't. Or with one of those surveillance devices that masquerade as thermostats or light switches or voice assistants.

So that's pretty bleak. If you want defense, you probably want to allocate different /64 prefixes to different devices, and change them really often. You may not need to change them for every devices -- no point providing privacy to the surveillance thermostat. But for your laptop, your tablet or your phone, they to change before the surveillance advertisers see you access a dozen web pages or two.

-- Christian Huitema

To help the discussion of how matching identifying information can be quite easy, take the following test with your browser:

Panopticlick - is your browser safe against tracking?

To finish things off, another quote from Anon - the movie:

It is not that I have something to hide, it is that I have nothing I want you to see

Building from Android Source Code -- Android Open Source Project

The repo init command doesn't like some forms of NAT. A command like:

$ repo init  --depth 1 --no-clone-bundle  -u https://android.googlesource.com/platform/manifest

may stick at the following with 100% cpu:

 * [new tag]         v1.9.5     -> v1.9.5
 * [new tag]         v1.9.6     -> v1.9.6
Get https://android.googlesource.com/platform/manifest

When performing a debug, this is the stage where it is stuck (no idea how to fix it, ended up bypassing NAT):

$ repo --trace init  --depth 1 --no-clone-bundle  -u https://android.googlesource.com/platform/manifest
Geta https://gerrit.googlesource.com/git-repo
remote: Counting objects: 142, done
remote: Total 4158 (delta 2038), reused 4158 (delta 2038)
Receiving objects: 100% (4158/4158), 4.69 MiB | 519.00 KiB/s, done.
Resolving deltas: 100% (2038/2038), done.
From https://gerrit.googlesource.com/git-repo
 * [new branch]      maint      -> origin/maint
 * [new branch]      master     -> origin/master
 * [new branch]      stable     -> origin/stable
 * [new tag]         v1.0       -> v1.0
 * [new tag]         v1.0.1     -> v1.0.1
 * [new tag]         v1.0.2     -> v1.0.2
 * [new tag]         v1.0.3     -> v1.0.3
...
 * [new tag]         v1.8.2     -> v1.8.2
 * [new tag]         v1.9.0     -> v1.9.0
 * [new tag]         v1.9.1     -> v1.9.1
 * [new tag]         v1.9.2     -> v1.9.2
 * [new tag]         v1.9.3     -> v1.9.3
 * [new tag]         v1.9.4     -> v1.9.4
 * [new tag]         v1.9.5     -> v1.9.5
 * [new tag]         v1.9.6     -> v1.9.6
: git --version 1>| 2>|
: parsing /home/rpb/.gitconfig
Get https://android.googlesource.com/platform/manifest
: export GIT_DIR=/home/rpb/aosp/.repo/manifests.git
: git init 1>| 2>|
: git config --file /home/rpb/aosp/.repo/manifests.git/config --null --list 1>| 2>|
: git config --file /home/rpb/aosp/.repo/manifests.git/config --replace-all filter.lfs.smudge git-lfs smudge --skip -- %f 1>| 2>|
: git config --file /home/rpb/aosp/.repo/manifests.git/config --unset-all core.bare 1>| 2>|
: git config --file /home/rpb/aosp/.repo/manifests.git/config --replace-all remote.origin.url https://android.googlesource.com/platform/manifest 1>| 2>|
: git config --file /home/rpb/aosp/.repo/manifests.git/config --replace-all remote.origin.fetch +refs/heads/*:refs/remotes/origin/* 1>| 2>|
: git fetch --progress origin --tags +refs/heads/*:refs/remotes/origin/* +refs/heads/master:refs/remotes/origin/master 1>| 2>|
^Caborted by user

A few different sync command styles from AOSP repo sync takes too long:

$ repo sync  -f --force-sync --no-clone-bundle --no-tags -j$(nproc --all)
$ repo sync -c --no-tags --no-clone-bundle -j2
$ repo sync -c --no-clone-bundle

where -j2 is number of parallel processes to use, the other parameters require some description.

Android for Enterprise

• Android for enterprise in Platform Developers
• Build for Enterprise - overview
• Enterprise - Device Administration
• Android Enterprise Recommended
• Android Enterprise devices and service providers
• DevicePolicyManager - in developers