C++

• CMake: the Good, the Bad, the Weird - ... strongly recommend for any developers who are learning or struggling with CMake to purchase Professional CMake -- I have found it very helpful in explaining things where most other resources haven't, and it is consistently updated with new major versions of CMake.

From #include <rules>, via hacker news, two compile time options to consider:

• use the preprocessor output (cl /E, gcc -E)
• use the include output (cl /showIncludes, gcc -M), gather the codebase statistics (average size after preprocessing, most included header files, header files with largest payload, etc.)

I've been doing this backwards, but I don't understand why though:

The header file named after the source should be included first (to catch errors in the header)

HackerNews had an article about Boehm-Demers-Weiser conservative C/C++ Garbage Collector which leads to A garbage collector for C and C++.

It can be used in garbage collection mode or leak detection mode.

The garbage collector distribution includes a C string (cord) package that provides for fast concatenation and substring operations on long strings. A simple curses- and win32-based editor that represents the entire file as a cord is included as a sample application. From Wikipedia:

Boehm GC is also distributed with a C string handling library called cords. This is similar to ropes in C++ (trees of constant small arrays), but instead of using reference counting for proper deallocation, it relies on garbage collection to free objects. Cords are good at handling very large texts, modifications to them in the middle, slicing, concatenating, and keeping history of changes (undo/redo functionality).

Code can be found at github - The Boehm-Demers-Weiser conservative C/C++ Garbage Collector (bdwgc, also known as bdw-gc, boehm-gc, libgc)

HPX -- An open source C++ Standard Library for Parallelism and Concurrency

To achieve scalability with today's heterogeneous HPC resources, we need a dramatic shift in our thinking; MPI+X is not enough. Asynchronous Many Task (AMT) runtime systems break down the global barriers imposed by the Bulk Synchronous Programming model. HPX is an open-source, C++ Standards compliant AMT runtime system that is developed by a diverse international community of collaborators called The Ste||ar Group. HPX provides features which allow application developers to naturally use key design patterns, such as overlapping communication and computation, decentralizing of control flow, oversubscribing execution resources and sending work to data instead of data to work. The Ste||ar Group comprises physicists, engineers, and computer scientists; men and women from many different institutions and affiliations, and over a dozen different countries. We are committed to advancing the development of scalable parallel applications by providing a platform for collaborating and exchanging ideas. In this paper, we give a detailed description of the features HPX provides and how they help achieve scalability and programmability, a list of applications of HPX including two large NSF funded collaborations (STORM, for storm surge forecasting; and STAR (OctoTiger) an astro-physics project which runs at 96.8% parallel efficiency on 643,280 cores), and we end with a description of how HPX and the Ste||ar Group fit into the open source community.

TimeGraphs: Graph-based Temporal Reasoning

Many real-world systems exhibit temporal, dynamic behaviors, which are captured as time series of complex agent interactions. To perform temporal reasoning, current methods primarily encode temporal dynamics through simple sequence-based models. However, in general these models fail to efficiently capture the full spectrum of rich dynamics in the input, since the dynamics is not uniformly distributed. In particular, relevant information might be harder to extract and computing power is wasted for processing all individual timesteps, even if they contain no significant changes or no new information. Here we propose TimeGraphs, a novel approach that characterizes dynamic interactions as a hierarchical temporal graph, diverging from traditional sequential representations. Our approach models the interactions using a compact graph-based representation, enabling adaptive reasoning across diverse time scales. Adopting a self-supervised method, TimeGraphs constructs a multi-level event hierarchy from a temporal input, which is then used to efficiently reason about the unevenly distributed dynamics. This construction process is scalable and incremental to accommodate streaming data. We evaluate TimeGraphs on multiple datasets with complex, dynamic agent interactions, including a football simulator, the Resistance game, and the MOMA human activity dataset. The results demonstrate both robustness and efficiency of TimeGraphs on a range of temporal reasoning tasks. Our approach obtains state-of-the-art performance and leads to a performance increase of up to 12.2% on event prediction and recognition tasks over current approaches. Our experiments further demonstrate a wide array of capabilities including zero-shot generalization, robustness in case of data sparsity, and adaptability to streaming data flow.

how to do a dynamic parser in X3?

The presentation went well, but as always with a large surface area library like Spirit it went on for too long IMO (2 hrs). This was similar to what happened when I presented Spirit 2.x almost 10 yrs ago. If I were to present on this again, I would split the topic into two talks:

• A background on recursive variant datatypes and visitor pattern and dipping into the Fusion library for adapting arbitrary application structures to Fusion tuples.
• Spirit X3 parsing w/annotations and error handling.

I try to present from real code in the IDE as much as possible so that we're looking at real code. I almost always write my own examples with the library I'm presenting and put them on github. The exception is when the library comes with an example that is everything I want to talk about .

I cribbed quite a bit from the spirit x3 fun example presented by Michael Caisse a few years back, but I did extend the AST to handle imaginary numbers.

Visual Studio Code, by default, when using the clangd language server, will automatically insert headers files for types which may have already been declared.

To disable this, go into the configuration options for clangd, and in 'clangd:arguments' add:

--header-insertion=never

Source: Stop VSCode from adding redundant headers

When using Cern's ROOT Data Analysis Framework, there are notes about possible compile time enhancements. It is best to take a look on which cpu variant the compiler will generate code, and for which target cpu the code will run. They need to be compatible.

AVX-512 is an interesting Wikipedia page describing the various cpu variant incarnations and their specialized instruction subsets.

Here are a few command line examples for examining what the gcc compiler sees as being available.

$ gcc -march=native -Q --help=target | grep march
  -march=                               skylake
  Known valid arguments for -march= option:

$ echo | gcc -dM -E - -march=native
  ... large quantity of flags ...

gcc x86 cpu options cmake command:

add_definitions(-march=native)

Even though this optimizes instruction use, it has to be used carefully. It will probably cause side effects with compiling code on one version of cpu and when copying the executables to a different machine, with possibly a different cpu variant.

Reference: How to see which flags -march=native will activate?

Tutorial: When to Write Which Special Member - "When explaining someone the rules behind the special member functions and when you need to write which one, there is this diagram that is always brought up. I don’t think the diagram is particularly useful for that, however."

Tutorial: Managing Compiler Warnings with CMake - " But how do you manage the very compiler-specific flags in CMake? How do you prevent your header files from leaking warnings into other projects?"

Pipeflow: An Efficient Task-Parallel Pipeline Programming Framework using Modern C++

Pipeline is a fundamental parallel programming pattern. Mainstream pipeline programming frameworks count on data abstractions to perform pipeline scheduling. This design is convenient for data-centric pipeline applications but inefficient for algorithms that only exploit task parallelism in pipeline. As a result, we introduce a new task-parallel pipeline programming framework called Pipeflow. Pipeflow does not design yet another data abstraction but focuses on the pipeline scheduling itself, enabling more efficient implementation of task-parallel pipeline algorithms than existing frameworks. We have evaluated Pipeflow on both micro-benchmarks and real-world applications. As an example, Pipeflow outperforms oneTBB 24% and 10% faster in a VLSI placement and a timing analysis workloads that adopt pipeline parallelism to speed up runtimes, respectively.

Since none of this is registered in github, here is my alternate mechanism for 'staring' these items:

• FreeCAD - open-source parametric 3D modeler made primarily to design real-life objects of any size. Parametric modeling allows you to easily modify your design by going back into your model history and changing its parameters. You get modern Finite Element Analysis (FEA) tools, experimental CFD, dedicated BIM, Geodata or CAM/CNC workbenches, a robot simulation module that allows you to study robot movements and many more features.
• cppyy: Automatic Python-C++ bindings - an automatic, run-time, Python-C++ bindings generator, for calling C++ from Python and Python from C++
• Open CASCADE Technology, The Open Source 3D Modeling Libraries - is a software development kit (SDK) intended for development of applications dealing with 3D CAD data or requiring industrial 3D capabilities. It includes a set of C++ class libraries providing services for 3D surface and solid modeling, CAD data exchange, and visualization. tutorial
• OpenStudio is a cross-platform (Windows, Mac, and Linux) collection of software tools to support whole building energy modeling using EnergyPlus and advanced daylight analysis using Radiance.
• Topologic is a software development kit and plug-in that enables logical, hierarchical and topological representation of spaces and entities

Some C++ libraries I've encountered for managing currency manipulation:

• vpiotr / decimal_for_cpp - this is the one I use for my currency work, is header based, succinct, and easy to use.
• I was wondering if boost.multiprecision could be used, and came up with mariusbancila / moneycpp - a bit too much for me as it also handles currency codes. There is a a blog entry: moneycpp – a C++ library for handling monetary values.
• General Decimal Arithmetic goes into some detail about the related IEEE 754 standard, as well as a link to a C language implementation.

Continuing on in the search for a responsive IDE for building large C++ codesets with CMake, the next stop is Visual Studio Code.

The initial install supplies a basic IDE editing environment.

Usability occurs when extensions are loaded. I have the following installed (versions are as of this writing):

• C/C++ for Visual Studio Code, v0.26, by Microsoft <== some sites say to disable this, but it is required by the debug function
• CMake, v0.0.17, by twxs, which provides CMake language support
• CMake Tools, v1.2.3, by Microsoft, which provides the build capability
• vscode-clangd, v0.0.19, is the clang language server, which provides code completion and symbol tables

Prior to installation of the extensions, on Debian, I installed the following packages first (in addition to the standard compilers):

• cmake
• clangd
• clangd-9

reddit - PROTIP: Use clangd instead of the official C/C++ extension suggests adding the following to .vscode/settings.json:

"C_Cpp.autocomplete": "Disabled",
"C_Cpp.formatting": "Disabled",
"C_Cpp.errorSquiggles": "Disabled",
"C_Cpp.intelliSenseEngine": "Disabled",

In addition, I have a more recent version of clangd, so in .vscode/settings.json, I also have:

"clangd.path": "/usr/lib/llvm-9/bin/clangd",

My over all file looks like:

{
  "clangd.path": "/usr/lib/llvm-9/bin/clangd",
  "C_Cpp.autocomplete": "Disabled",
  "C_Cpp.formatting": "Disabled",
  "C_Cpp.errorSquiggles": "Disabled",
  "C_Cpp.intelliSenseEngine": "Disabled",
  "C_Cpp.default.intelliSenseMode": "clang-x64",
  "editor.tabSize": 2
}

In the root directory of the project, a link is required (the setting "compileCommands" in .vscode/c_cpp_properties.json does not appear to be used by the clangd module) :

ln -s build/compile_commands.json .

2023/11/05 - Or add the following to .vscode/settings.json (which causes the file to be created and referenced directly in the basic workspace directory):

"cmake.copyCompileCommands": "${workspaceFolder}/compile_commands.json"

As a note, if compile_commands.json is not provided, then the following flag needs to be passed to CMake (seems to be defaulted "on" in current Visual Studio Code):

-DCMAKE_EXPORT_COMPILE_COMMANDS=1

Documentation for clangd is found at Getting started with clangd.

As a continuation of my other article incorporating CMake as a makefile generator, here are some references I used for getting a handle on how to build a set of CMakeLists.txt files to manage a project containing a library shared by two applications.

• Tutorial: Managing Compiler Warnings with CMake - But how do you manage the very compiler-specific flags in CMake? How do you prevent your header files from leaking warnings into other projects?
• cmake.org tutorial
• Reference Guide - direct from cmake.org
• CMake Community Wiki
• wxWiki - using CMake to include wxWidgets in projects - see below for sample
• stack overflow - Questions tagged [cmake] - the popular entries
• stack overflow - Multiple sub-projects depending on the same library - primary source for skeleton entries for a project with inter-dependent apps and libraries.
• stack overflow - CMake: Static library with multiple projects - Normally, when CMake projects have to cooperate, it's beneficial to make them part of the same buildsystem using add_subdirectory(). That way, CMake can track dependency between their targets, automatically resolve references to logical target names etc.
• Typical Linux Project using CMake - goes into detail for a project with libraries and applications, including external dependencis
• CMake Dependencies Done Right - mechanisms to evolve a CMake file - depend on defining a proper dependency tree for all libraries and executables
• stack overflow - CMake: Project structure with unit tests
• Basic CMake C++ project structure - includes an example of ExternalProject_Add - also with style notes in one of the answers
• stack overflow - CMake Setting up Release and Debug version and Flags 'cmake -DCMAKE_BUILD_TYPE=Release .. ', 'cmake -DCMAKE_BUILD_TYPE=Debug .. '
• How to configure a C/C++ project with Eclipse and CMake - manual mechanism when not using the wizard from the gui - talks about debug/release versions - something which isn't intuitively obvious when using the gui wizard for importing a CMake progject
• It's Time To Do CMake Right - Pablo Arias - Build Requirements vs Usage Requirements - defines public, private, interface plus additional concepts
• Mirko Kiefer's Blog - CMake by Example - a few twists on the simple CMake files with a focus on the 'install' concept
• CMake Examples on github. There is something to be learned from each example [added 2019/0917]
• CMake – Properties and Options - conditionals - which shares an example with CMAKE compile options
• dendisuhubdy / libtorch_examples - has an example of how to use custom commands [could be used with my libs-build]
• 2020/02/25 CMake is not a build system - brief walk through of key features.
• 2021/05/23 Effective Modern CMake - many 'thou shalts', but few examples on what it means, other than that, effective starting point on the research journey

wxWidgets inclusion with CMake

project(myGreatProject)
cmake_minimum_required(VERSION 2.
aux_source_directory(. SRC_LIST)

set(wxWidgets_CONFIGURATION mswu)
find_package(wxWidgets COMPONENTS core base REQUIRED)
include(${wxWidgets_USE_FILE})

add_executable(${PROJECT_NAME} ${SRC_LIST})
target_link_libraries(${PROJECT_NAME} ${wxWidgets_LIBRARIES})

Random note:

${CMAKE_SOURCE_DIR}, ${CMAKE_BINARY_DIR}

Generator for ninja:

cmake -G Ninja.

I've been using NetBeans for quite some time as a GUI based IDE for building C++ projects. Recently, the project has been migrated over to Apache for management. In the transition time frame, NetBeans has progressed from 8.2 to 9.0 to the current 10.0 incarnation. The 8.2 version was the last which had an integral C++ development environment.

Via jumping through a few hoops, it is possible to run the c++ development environment in v10.0. The draw back is that what ever was available in 8.2 remains as 'current'. Which means the build environment is stuck at C++13.

By finding a development/beta version of 8.2 , I was able to obtain C++17 capability, but at a cost of stability of the IDE. Development of this branch has been discontinued since, I think, early 2018. I do not know if any behind-the-scenes work is happening somewhere.

So... it is time to try something else. Prior to NetBeans, I had been using Eclipse as an IDE. But, way back then, I had switched to NetBeans as Eclipse appeared to be lagging and unstable. The tables have turned. Eclipse appears to be progressing well now as a C++ IDE, far surpassing NetBean's capability.

For building, NetBeans generated makefiles directly. Eclipse can do the same. But Eclipse is also able to work with CMakeLists.txt files. Probably a boon as well as a bane, but as other projects are CMake based, I felt it might be worth a try to try.

So I dived in. I fought for a while to convert my make environment over to CMake. Boost libraries, with v1.69.0 can cause some troubles with CMake find_package abilities. The basic trick is to add the following for builds for v1.69 and possibly for v1.68:

set(CMAKE_CXX_COMPILER_ARCHITECTURE_ID, "x64")
set(Boost_ARCHITECTURE "-x64")

A primary issue undermining my successful transition was that I was using boost::log::trivial. No matter what I tried, it seems as though a CMake initiated make just wouldn't find the symbols. A message of the kind:

undefined reference to `boost::log::v2s_mt_posix::trivial::logger::get()'

The solution was simple. I had already performed the needed resolution in the old IDE. But forgot to migrate it to the new CMake file.

But as a note for posterity, adding the following to a c++ build line (which also incorporates the link step), will forward options to 'ld', the linker.

-Wl,--trace -Wl,--verbose

These options tell ld to emit additional troubleshooting information. But where to put these flags? I used the following to show the command expansions, which I could then copy and paste and change for testing (I found the flag in CMakeCache.txt):

build_dir$ cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE ..

The result was that I was indeed successfully resolving the boost_log library inclusion. But why wasn't it accessing the symbols?

The realization occurred after inspecting the library directly, and there is was, the symbol, in all its glory, different from the one for which the linker was originally searching (variations are discussed on StackOverFlow:

# nm -gC libboost_log-gcc8-mt-x64-1_69.so.1.69.0|grep trivial
0000000000099a80 T boost::log::v2_mt_posix::trivial::logger::get()

Well the solution is to place the following in to the CMakeLists.txt file:

target_compile_definitions(${PROJECT_NAME} PUBLIC BOOST_LOG_DYN_LINK )

This forces the c++ build to use the dynamic boost::log library rather than the static file. The symbol then changes to

`boost::log::v2_mt_posix::trivial::logger::get()'

which can fe found in the dynamic link file. And all is well.

As a side note, when using the CMAKE_VERBOSE_MAKEFILE expansion, with a few tweaks I was able to take a '/usr/bin/c++' invocation and run it instead as '/usr/bin/ld'. But this doesn't work very well due to many missing libraries, which the linking form of '/usr/bin/c++' automatically provides.

The link command is found in CMakeFiles/${program_name}.dir/link.txt along with other build products.

I've been building an Android app off and on. It uses NDK to delivery audio mixing and synthesis capabilities. There is functionality I would like to add which exists in third party libraries such as Boost and libsodium. There are a bunch of pieces necessary to make it all work: tooling, directory locations, options, ...

To start, Boost has quite a number of symbols, and even with improvements in Android Studio, which as I write this, is at v3.1.2, Android Studio will consume much CPU for many minutes while it builds symbol tables.

To help make this go a bit faster, it would probably be good to have the project installed on fast drives like SSD or NVME drives. In addition, if you have memory to spare, providing memory to Android Studio benefits the process. These are changes I made to my bin/studio.vmoptions file:

-Xms1024m
-Xmx4096m
-XX:ReservedCodeCacheSize=1024m

Some say the above doesn't work. An alternate location is in "Help" -> "Edit custom VM options" , which will create studio64.vmoptions, and put something similar in place:

-Xms1024m
-Xmx4096m
-XX:MaxPermSize=700m
-XX:ReservedCodeCacheSize=480m
-XX:+UseCompressedOops

Another suggested setting from Speed Up Gradle Build In Android Studio is "File" -> "Settings" -> "Gradle", and select "Offline Work".

Running 'invalidate caches' from the main File menu helped trial some changes in configuration. With three sets of boost include files (a set of files per each of the three architectures, it took about 12minutes for the Symbol Build to happen. I think for Boost, I will just use a single Include directory and list accordingly in the CMakeLists.txt file, rather than one for each architecture.

Changing anything in build.gradle, causes a total resync of everything, including rebuilding the Symbols. Uggh.

Reading in Reduce the APK size , I wonder if these Symbol Build issues have to with classes.dex file, as mentioned under the heading "Avoid enumerations". [in looking, it seems to be about MB in size, so that may not be the issue]

I'm not sure if it is really required, but Android Studio: Background task running indefinitely suggested putting the following into /etc/sysctl.conf file and restarting with 'sudo sysctl -p':

# Increase the limit for Android Studio
fs.inotify.max_user_watches = 500000

For the really desperate, more suggestions are found at Speed up your slow Android Studio.

When I had just one architecture, x86, the initial symbol build took a number of minutes. When I bumped it up to three architectures, 'x86', 'arm64-v8a', 'armeabi-v7a', building symbols seemed to take for ever. I had to add additional changes. In gradle.properties (in Gradle Scripts via Project window), I added:

org.gradle.daemon=true

I've scripted much of what I've learned, with the latest version available in github - rburkholder/android-ndk-build-libraries. Since the script is designed to run from within the libsodium directory and from within the boost directory, you will need to change into the directory of the library you wish to build, then run an absolute or relative path to run the script. BASE and PROJECT will need to be defined. You will probably want to read through the script first to understand the implied directory locations for the library, the build tools, the target android project, and the options available for the command line. Only some of the Boost libraries are built in this script. Additional ones can be easily added.

No matter what settings I applied to the boost shared library build, I was unable to remove the boost version codes at the end of the filename. There must be an option, but I havn't spotted it yet. So I build that library as a static library and link it it. Even changing 'version_type=linux' to 'version_type=none' in libtool.m4 in section 'linux*)' did not work. Even the patch suggested at NDK: Support for prebuild libs with full sonames didn't seem to work. But then I got to thinking, that it would be better to run with statically linked libraries. Since multiple architectures are packaged together, multiple architectures of the libraries will tend to bloat the package, where static libraries are reduced down to only the parts required. But having said that, for reference sake, there may be a solution to the Boost issue via an article at Build boost without version number suffixes where they suggest commenting out some or all of the following in boostcpp.jam:

    if $(type) = SHARED_LIB &&
      ! [ $(property-set).get  ] in windows cygwin darwin aix &&
      ! [ $(property-set).get  ] in pgi
    {
        result = $(result).$(BOOST_VERSION)  ;
    }

One of the next libraries to build will be ffmpeg, with a good write up at How to integrate pre-built libraries in case of the FFmpeg.

Via a suggestion at How to build Boost 1.64.0 libraries (static, shared) for ARM, MIPS, x86, +64-bit variants (using clang), it is possible to view the file type of the built libraries:

$ ar -t libboost_thread.a 
thread.o
once.o
future.o
$ ar -x libboost_thread.a  
$ ls -alt
total 1352
drwxr-sr-x 1 rpb rpb    498 May 29 16:34 .
-rw-r--r-- 1 rpb rpb  12080 May 29 16:34 future.o
-rw-r--r-- 1 rpb rpb   4184 May 29 16:34 once.o
-rw-r--r-- 1 rpb rpb 308220 May 29 16:34 thread.o
-rw-r--r-- 1 rpb rpb 359568 May 29 16:32 libboost_thread.a
$ file future.o
future.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped

A possible optimization for the boost libraries is to run 'ranlib' on them prior to copying over to the final project. This code would need to be modified to work with the directory structures and destinations in the script, but is recorded here for posterity. From the man page, ranlib "enerates an index to the contents of an archive and stores it in the archive".

echo "Running ranlib on libraries..."
libs=$(find "bin.v2/libs/" -name '*.a')
for lib in $libs; do
  "$toolchain/bin/arm-linux-androideabi-ranlib" "$lib"
done

Some miscellaneous references for working with CMake and NDK:

• Getting Started with the NDK -
• Standalone Toolchains - the example lower in the page forms the basis of my script
• Android NDK Native APIs - Table 1 has an excellent summary of key native API support introduced by Android version, with the remainder of the page describing the libraries at each API step.
• ABI Management - primary ABIs are armeabi, armeabi-v7a, arm64-v8a, x86, x86_64.
• Configure CMake - the basics of what goes into the CMakeLists.txt file
• Configure Build Variants - what goes into the gradle.build file
• Link Gradle to Your Native Library -
• CMake tutorial -
• CMake Reference Documentation -