Raymond P. Burkholder - Things I Do

For trying out some LSTM Machine Learning algorithms with my TradeFrame Algorithmic Trading Library, I wanted to install LibTorch with NVidia/Cuda support for hardware accelerating learning.

I used Installing C++ Distributions of PyTorch as a starting point. However, their example is CPU based. My desire is for a Cuda based installation. This meant going to the CUDA Zone and start the Download process. My configuration options were: Linux, x86_64, Debian, 12, deb (local).

Using the "deb (local)" with a complete file seemed to be the only way to ensure all components were available.

The steps, as of this writing, were:

wget https://developer.download.nvidia.com/compute/cuda/12.9.0/local_installers/cuda-repo-debian12-12-9-local_12.9.0-575.51.03-1_amd64.deb
sudo dpkg -i cuda-repo-debian12-12-9-local_12.9.0-575.51.03-1_amd64.deb
sudo cp /var/cuda-repo-debian12-12-9-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-9

Since I have the nvidia binary video driver installed, I chose the non-nouveau flavour of Cuda Drivers. This could be run prior to the previous steps.

sudo apt-get install -y cuda-drivers

There is also the NVIDIA CUDA Installation Guide for Linux for further information.

The PyTorch LibTorch library can be downloaded from PyTorch Start Locally. Choose the C++/Java option with Cuda 12.8 (as of this writing). An appropriate link is presented. Download and expand the file into a development directory. LibTorch doesn't have at this moment a build for Cuda 12.9, but is referenced as 12.8.

The most recent can be found at https://download.pytorch.org/libtorch/nightly/cu128/libtorch-shared-with-deps-latest.zip.

It is probably advised to NOT use the Debian package, as it may be out of date: pytorch-cuda.

To test out the installation, I then created a subdirectory containing a couple of files. The first is the test code example-app.cpp:

#include <torch/torch.h>
#include <iostream>

int main() {
  torch::Tensor tensor = torch::rand({2, 3});
  std::cout << tensor << std::endl;
}

The second file is the CMakeLists.txt file. This is my version:

cmake_minimum_required(VERSION 3.18 FATAL_ERROR)

cmake_policy(SET CMP0104 NEW)
cmake_policy(SET CMP0105 NEW)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
set(CMAKE_CUDA_STANDARD 17)

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 17)

Then to build the example:

mkdir build
cmake \
  -DCMAKE_PREFIX_PATH=~/data/projects/libs-build/libtorch \
  -DCMAKE_CUDA_ARCHITECTURES=native \
  -DCMAKE_BUILD_TYPE=DEBUG \
  -DCMAKE_CUDA_COMPILER=/etc/alternatives/cuda/bin/nvcc \
  -Dnvtx3_dir=/usr/local/cuda/targets/x86_64-linux/include/nvtx3  \
  ..
make
./example-app

Notes:

• PREFIX_PATH points to the directory of your expanded libtorch download
• CMAKE_CUDA_ARCHITECTURES provides a 'native' cuda solution, the build process will determine the specific gpu for which to build
• CMAKE_BUILD_TYPE can be DEBUG or RELEASE
• CMAKE_CUDA_COMPILER needs to be set, by using /etc/alternatives, these are softlinks to the version you desire (as were installed by the cuda installation)
• nvtx3_dir is required, as the current libtorch library seems to still refer to nvtx and not nvtx3

If you get output along the lines of:

-- Automatic GPU detection failed. Building for common architectures.
-- Autodetected CUDA architecture(s): 5.0;8.0;8.6;8.9;9.0;9.0a;10.0;10.0a;10.1a;12.0;12.0a

Even though you may have the nvidia driver installed:

$ dpkg -l |grep nvidia-driv
ii  nvidia-driver              575.51.03-1 amd64  NVIDIA metapackage
ii  nvidia-driver-cuda         575.51.03-1 amd64  NVIDIA driver CUDA integration components
ii  nvidia-driver-libs:amd64   575.51.03-1 amd64  NVIDIA metapackage (OpenGL/GLX/EGL/GLES libraries)

See if the nouveau driver is installed.

$ lsmod |grep nouveau

If so, then run these commands to enable the nvidia driver and to blacklist the nouveau driver and reboot:

sudo mv /etc/modprobe.d/nvidia.conf.dpkg-new  /etc/modprobe.d/nvidia.conf
sudo update-initramfs -u

My system has two RTX 4070 cards, and can be verified with (an extract is shown with important parts, noticing that the nvidia driver is properly shown):

$ sudo lshw -c video
  *-display
    product: Arrow Lake-S [Intel Graphics]
    configuration: depth=32 driver=i915 latency=0 mode=3840x2160 resolution=3840,2160 visual=truecolor xres=3840 yres=2160
  *-display
    product: AD103 [GeForce RTX 4070]
    configuration: driver=nvidia latency=0
  *-display
    product: AD103 [GeForce RTX 4070]
    configuration: driver=nvidia latency=0

Therefore, the output of my cmake process will include gpu specific selections:

-- Autodetected CUDA architecture(s):  8.9 8.9
-- Added CUDA NVCC flags for: -gencode;arch=compute_89,code=sm_89

And running the generated binary results in valid output:

$ ./example-app
0.7141  0.9744  0.3179
0.7794  0.9281  0.7529
[ CPUFloatType{2,3} ]

A Deep Learning Approach to Anomaly Detection in High-Frequency Trading Data

his paper proposes an algorithm based on a staged sliding window Transformer architecture to detect abnormal behaviors in the microstructure of the foreign exchange market, focusing on high-frequency EUR/USD trading data. The method captures multi-scale temporal features through a staged sliding window, extracts global and local dependencies by combining the self-attention mechanism and weighted attention mechanism of the Transformer, and uses a classifier to identify abnormal events. Experimental results on a real high-frequency dataset containing order book depth, spread, and trading volume show that the proposed method significantly outperforms traditional machine learning (such as decision trees and random forests) and deep learning methods (such as MLP, CNN, RNN, LSTM) in terms of accuracy (0.93), F1-Score (0.91), and AUC-ROC (0.95). Ablation experiments verify the contribution of each component, and the visualization of order book depth and anomaly detection further reveals the effectiveness of the model under complex market dynamics. Despite the false positive problem, the model still provides important support for market supervision. In the future, noise processing can be optimized and extended to other markets to improve generalization and real-time performance.

Trading with CNN fed by LSTM

This project addresses the challenge of automated stock trading, where traditional methods and direct reinforce- ment learning (RL) struggle with market noise, complex- ity, and generalization. Our proposed solution is an in- tegrated deep learning framework combining a Convolu- tional Neural Network (CNN) to identify patterns in tech- nical indicators formatted as images, a Long Short-Term Memory (LSTM) network to capture temporal dependen- cies across both price history and technical indicators, and a Deep Q-Network (DQN) agent which learns the optimal trading policy (buy, sell, hold) based on the features ex- tracted by the CNN and LSTM. The CNN and LSTM act as sophisticated feature extractors, feeding processed in- formation to the DQN, which learns the optimal trading policy (buy, sell, hold) through RL. We trained and evalu- ated this model on historical daily stock data, using distinct periods for training, testing, and validation. Performance was assessed by comparing the agent’s returns and risk on out-of-sample test data against baseline strategies, includ- ing passive buy-and-hold approaches. This analysis, along with insights gained from explainability techniques into the agent’s decision-making process, aimed to demonstrate the effectiveness of combining specialized deep learning archi- tectures, document challenges encountered, and potentially uncover learned market insights

Predicting Stock Prices Using Permutation Decision Trees And Strategic Trailing

Qe explore the application of Permutation Decision Trees (PDT) and strategic trailing for predicting stock market movements and executing profitable trades in the Indian stock market. We focus on high-frequency data using 5-minute candlesticks for the top 50 stocks listed in the NIFTY 50 index. We implement a trading strategy that aims to buy stocks at lower prices and sell them at higher prices, capitalizing on short-term market fluctuations. Due to regulatory constraints in India, short selling is not considered in our strategy. The model incorporates various technical indicators and employs hyperparameters such as the trailing stop-loss value and support thresholds to manage risk effectively. Our results indicate that the proposed trading bot has the potential to outperform the market average and yield returns higher than the risk-free rate offered by 10-year Indian government bonds. We trained and tested data on a 60 day dataset provided by Yahoo Finance. Specifically, 12 days for testing and 48 days for training. Our bot based on permutation decision tree achieved a profit of 1.3468% over a 12-day testing period, where as a bot based on LSTM gave a return of 0.1238% over a 12-day testing period and a bot based on RNN gave a return of 0.3096% over a 12-day testing period. All of the bots outperform the buy-and-hold strategy, which resulted in a loss of 2.2508%

Deep Q-Network (DQN) multi-agent reinforcement learning (MARL) for Stock Trading

This project addresses the challenge of automated stock trading, where traditional methods and direct reinforcement learning (RL) struggle with market noise, complexity, and generalization. Our proposed solution is an integrated deep learning framework combining a Convolutional Neural Network (CNN) to identify patterns in technical indicators formatted as images, a Long Short-Term Memory (LSTM) network to capture temporal dependencies across both price history and technical indicators, and a Deep Q-Network (DQN) agent which learns the optimal trading policy (buy, sell, hold) based on the features extracted by the CNN and LSTM.

Reinforcement Learning (RL) has experienced significant advancement over the past decade, prompting a growing interest in applications within finance. This survey critically evaluates 167 publications, exploring diverse RL applications and frameworks in finance. Financial markets, marked by their complexity, multi-agent nature, information asymmetry, and inherent randomness, serve as an intriguing test-bed for RL. Traditional finance offers certain solutions, and RL advances these with a more dynamic approach, incorporating machine learning methods, including transfer learning, meta-learning, and multi-agent solutions. This survey dissects key RL components through the lens of Quantitative Finance. We uncover emerging themes, propose areas for future research, and critique the strengths and weaknesses of existing methods.

LLM4FTS: Enhancing Large Language Models for Financial Time Series Prediction

Predicting financial time series presents significant challenges due to inherent low signal-to-noise ratios and intricate temporal patterns. Traditional machine learning models exhibit limitations in this forecasting task constrained by their restricted model capacity. Recent advances in large language models (LLMs), with their greatly expanded parameter spaces, demonstrate promising potential for modeling complex dependencies in temporal sequences. However, existing LLM-based approaches typically focus on fixed-length patch analysis due to the Transformer architecture, ignoring market data's multi-scale pattern characteristics. In this study, we propose $LLM4FTS$, a novel framework that enhances LLM capabilities for temporal sequence modeling through learnable patch segmentation and dynamic wavelet convolution modules. Specifically,we first employ K-means++ clustering based on DTW distance to identify scale-invariant patterns in market data. Building upon pattern recognition results, we introduce adaptive patch segmentation that partitions temporal sequences while preserving maximal pattern integrity. To accommodate time-varying frequency characteristics, we devise a dynamic wavelet convolution module that emulates discrete wavelet transformation with enhanced flexibility in capturing time-frequency features. These three modules work together to improve large language model's ability to handle scale-invariant patterns in financial time series. Extensive experiments on real-world financial datasets substantiate the framework's efficacy, demonstrating superior performance in capturing complex market patterns and achieving state-of-the-art results in stock return prediction. The successful deployment in practical trading systems confirms its real-world applicability, representing a significant advancement in LLM applications for financial forecasting.

I picked up an Intel Z890 chipset based Asus ProArt Z890-CREATOR WIFI motherboard. It has a relatively new Wifi card:

# lspci |grep -i wi-fi
85:00.0 Network controller: Intel Corporation Wi-Fi 7(802.11be) AX1775*/AX1790*/BE20*/BE401/BE1750* 2x2 (rev 1a)

With the Trixie based install, Bluetooth mice have a few problems with connecting and maintaining connection. Maybe these dmesg fail messages might point to a solution:

[    2.806746] iwlwifi 0000:85:00.0: firmware: failed to load iwlwifi-gl-c0-fm-c0-93.ucode (-2)
[    2.806750] iwlwifi 0000:85:00.0: firmware: failed to load iwlwifi-gl-c0-fm-c0-93.ucode (-2)
[    2.806751] iwlwifi 0000:85:00.0: Direct firmware load for iwlwifi-gl-c0-fm-c0-93.ucode failed with error -2
[    2.811678] iwlwifi 0000:85:00.0: firmware: failed to load iwl-debug-yoyo.bin (-2)
[    2.811681] iwlwifi 0000:85:00.0: firmware: failed to load iwl-debug-yoyo.bin (-2)

Linux Wireless Documentation suggests an updated firmware package from linux-firmware.git, and then an installation with:

# cp iwlwifi-*.{ucode,pnvm} /lib/firmware/

The following may be required to load the firmware during boot:

# update-initramfs -u -k all

I also upgraded the kernel from 6.12.6 to 6.12.19

/debian/pool/non-free-firmware/f/firmware-nonfree has latest packaged firmware, which may not be as recent as the git repository.

I used this to attempt to load a more recent firmware, but the kernel module seems to be limiting it to v92, and v93 is not available, for some reason. I hope the 6.13.8 series kernel, when it comes out of experimental might alleviate that.

# ln -s /lib/firmware/iwlwifi-gl-c0-fm-c0-96.ucode /lib/firmware/iwlwifi-gl-c0-fm-c0-93.ucode

Intel Wireless WiFi Link, Wireless-N, Advanced-N, Ultimate-N devices - iwlwifi mentions:

You may try to switch power management scheme from balanced (default) into active by adding the following to /etc/modprobe.d/iwlmvm.conf and reboot:

options iwlmvm power_scheme=1

The following command will identify the firmware files for which a driver is looking. In this case, the driver looks for -93, but it isn't in the firmware files:

# modinfo iwlwifi |grep firmware
...
firmware:       iwlwifi-gl-c0-fm-c0.pnvm
firmware:       iwlwifi-gl-c0-fm-c0-93.ucode
...
parm:           fw_restart:restart firmware in case of error (default true) (bool)

Additional references:

• gentoo linux iwlwifi
• kernel/git/firmware/linux-firmware-main.tar.gz

Two security vulnerabilities have been discovered in the OpenSSH secure networking utility suite that, if successfully exploited, could result in an active machine-in-the-middle (MitM) and a denial-of-service (DoS) attack, respectively, under certain conditions.

The vulnerabilities, detailed by the Qualys Threat Research Unit (TRU), are listed below -

• CVE-2025-26465 - The OpenSSH client contains a logic error between versions 6.8p1 to 9.9p1 (inclusive) that makes it vulnerable to an active MitM attack if the VerifyHostKeyDNS option is enabled, allowing a malicious interloper to impersonate a legitimate server when a client attempts to connect to it (Introduced in December 2014)
• CVE-2025-26466 - The OpenSSH client and server are vulnerable to a pre-authentication DoS attack between versions 9.5p1 to 9.9p1 (inclusive) that causes memory and CPU consumption (Introduced in August 2023)

Reddit has an elegant solution to the sound issue. Simply convert .mp3 files to .wav files. The .mp3 files in the TWS sound directory can be manipulated this way:

ffmpeg -i FILE.mp3 -acodec pcm_s16le -ar 44100 FILE.wav
# or as a oneliner for all at once
find . -iname '*.mp3' -exec sh -c 'ffmpeg -i $1 -acodec pcm_s16le -ar 44100 $1.wav' sh {} ';'

The sounds directory has voice based sounds. For the actual sound oriented sounds, you'll need to supply your own, or pull from the original distribution and convert. An example extraction from the article:

jar xf ~/Jts/1019/jars/jts4launch-1019.jar $(jar tf ~/Jts/1019/jars/jts4launch-1019.jar | grep '\.mp3$')

At the beginning of February, Lets Encrypt sent out an email saying they were no longer sending emails prior to certificate expiration. That means I'm going to have to track my own certificate expiration dates.

As an example command line to get things started:

</dev/null  openssl s_client -connect ounl.bm:443 -servername www.ounl.bm  | \
  openssl x509 -text -noout | grep 'Not After'

• </dev/null - send an empty connection (otherwise the client will not disconnect)
• -text - expands the content of the certificate
• -noout - does not show the encoded certificate

Some other links if I want to do this programmatically:

• cseelye/ssl.cpp - git gist to extract contents, notably a way to extract the expiry date/time
• RaymiiOrg/openssl-modern-cpp - Modern C++ OpenSSL examples
• libs/beast/example/websocket/server/coro-ssl/websocket_server_coro_ssl.cpp - maybe this beast example has the basics of acquiring the certificate, then the other code snippets can be used to decode it
• Boost ASIO SSL - this is probably an easier way to acquire a certificate and then use the other code segments to decode it

git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux/
git checkout v6.1.113

cp /boot/config-$(uname -r) .config
make oldconfig
scripts/config --set-str SYSTEM_REVISION "$(git rev-parse --short HEAD)"
scripts/config --set-str SYSTEM_SERIAL "$(date +%s)"

git revert COMMITID
export CONCURRENCY_LEVEL=$(nproc)
fakeroot make -j$(nproc) deb-pkg LOCALVERSION=-revertCOMMITID
git reset HEAD^ --hard

On my ProxMox machine, I need a more dynamic way of handling containers rather than being limited to the LXC flavour offered by ProxMox. There was a question on reddit to which I supplied an answer about why to use full virtualization for Kubernetes vs running in a container. In summary:

• proxmox is a virtualization engine which enables LXC containers and QEMU/KVM guest virtual machines
• LXC containers are compartmentalized and share direct access to the kernel
• containers are used so as not to pollute your based proxmox installation with more packages and runtimes
• typically, you don't want to nest containers inside containers
• virtual machines run their own kernel/operating system, and are more secure (failure/security) and independent
• since k8s manages docker containers (not LXC containers), and you don't want to run containers of any fashion in containers of another fashion (docker inside lxc), you run a virtual machine with k8s to keep k8s packages and runtime separate from the core proxmox runtime environment
• hence the container inside virtual machine on hypervisor platform
• for security and compartmentalization

I built a basic LXC management container to take a look at talosctl, then used these instructions to perform a few tests:

apt update
apt upgrade
apt install --no-install-recommends curl
# based upon https://www.talos.dev/v1.9/talos-guides/install/talosctl/
curl -sL https://talos.dev/install | sh

# based upon https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

I ultimately removed this container, as I started reading the Getting Started Guide. But the above did generate the information that the lastest talosctl version is v1.9.1, which is useful for generating the iso image required.

.... more to come

Reference:

• Talos Getting Started
• construct/download images for control plane and worker nodes
• discusses qemu guest agent
• recommended resources

A couple months ago, the printer ink in my HP OfficeJet Pro 8620 ran low on most of the ink cartridges. Replacing all those cartridges would cost a significant fraction of the replacement cost of a new printer/scanner. I went looking for some cartridges which might cost less. I did find some on Amazon, installed them, and was successful in printing.

A month or two later, I needed to print again (Yes, I don't print all the time). At this point, the printer display showed:

The printhead appears to be missing, not detected, or
incorrectly installed.
Error Code 0xc19a0023

HP has been pretty loud about not wanting non-branded cartridges installed in their printers - it obviously reduces their own recurring revenue model. It may or may not be a conspiracy that the printer enters this mode on purpose, or it could be that the printhead suddenly failed. No one but HP knows with these kind of opaque error messages and lack of any suitable diagnostics.

After expressing this problem to the third party cartridge supplier, they are sending some new cartridges which they say will make this problem go away. hmm... We'll see.

So, now that I have an over-designed scanner/paper-weight, it is time to go look for alternative solutions.

Given this whole lock-in with cartridges, I'm not in the mood for anyone's cartridges. They seem costly for no added reason other than some additional profit. Fortunately, there are various brands and models on the market which utilize reservoirs - probably due to consumer demand for just the reasons I no longer like/appreciate priinter cartridges.

I went shopping for some non-HP brand. The Best All-in-One Printers for 2025 seemed to be a good starting point. I ended up looking at the Canon models. There are GX20xx, GX40xx, GX60xx all-in-one models. Even at the higher end models, Canon can't seem to make a decent all-in-one model with 'everything' in it. After comparing a bunch of them, I decided on the GX2020, as it seemed to be a newer update, had an ADF for scan and a duplexer for print. Flat scan can only go to about 8.5" x 11", unlike the HP which could do larger.

The next challenge is working with Debian Linux, and having the printer/scanner on a wifi subnet different from my workstation. Typical printer locator utilities utterly fail in this scenario.

For installing the printer, they do have a package which can be installed with the Debian package manager: cnijfilter2-6.70-1-deb.tar.gz. This can be installed with (remember the service restart):

sudo dpkg -i cnijfilter2_6.80-1_amd64.deb
sudo service cups restart

Cups printer utilities could then be used to specify the printer by ip address and set it with the PASSTHRU printer queue mode.

The scanner mode wasn't as 'easy' to configure. After installing the Canon package for the scanner, i had to find a Debian Wiki for SaneOverNetwork which said to modify the /etc/sane.d/pixma.conf file and add a line like (there are examples in the file):

bjnp://ip-address

However, the following error message occurred:

# scanimage -L
[17:44:27.140613] [bjnp] udp_command: ERROR - no data received (timeout = 10000)
[17:44:27.140719] [bjnp] bjnp_init_device_structure: Cannot read mac address, skipping this scanner

When looking for more assistance, there are references to ThierryHFR / scangearmp2. This is someone who has reverse engineered Canon's protocols, wrapped their binaries, and built something which Canon should have done in the first place.

A word of warning, don't try and build your own from the repository. The instructions aren't so clear, and maybe incorrect. Instead, go directly to the linked repositories, download, and install what you need. Do this for both the print and the scanner drivers.

I did a look at the files in the package:

dpkg -c scangearmp2_4.80-1_amd64.deb

There is a file called /etc/sane.d/canon_pixma.conf which provides the ability of providing the ip address of the printer (which I set statically in dhcp based upon the printer's mac address).

My scanner programs, like gscan2pdf, worked right away.

Are there printer / scanners with better Linux support, and with the 'ultimate' combination of ADF/Flatbed scanning with Colour printing with duplex capability?

• CMake: the Good, the Bad, the Weird - ... strongly recommend for any developers who are learning or struggling with CMake to purchase Professional CMake -- I have found it very helpful in explaining things where most other resources haven't, and it is consistently updated with new major versions of CMake.

Practical Evaluation of Low-Frequency Vibration Energy Harvesting for Creating Green RFID and IoT Devices

One of the main limitations for the development and deployment of many Green Radio Frequency Identification (RFID) and Internet of Things (IoT) systems is the access to energy sources. In this aspect batteries are the main option to be used in energy constrained scenarios, but their use is limited to certain cases, either because of the constraints imposed by a reduced-form factor, their limited lifespan, or the characteristics of the environment itself (e.g. operating temperature, risk of burning, need for fast response, sudden voltage variations). In this regard, supercapacitors present an interesting alternative for the previously mentioned type of environment, although, due to their short-term capacity, they must be combined with an alternative energy supply mechanism. Energy harvesting mechanisms, in conjunction with ultra-low-power electronics, supercapacitors and various methods to improve the efficiency of communications, have enabled the emergence of battery-less passive electronic devices such as sensors, actuators or transmitters. This paper presents a novel analysis of the performance of an energy harvesting system based on vibrations for Green RFID and IoT applications in the field of maritime transport. The results show that the proposed system allows for charging half of a 1.2 F supercapacitor in about 72 minutes, providing a stable current of around 210 uA and a power output of 0.38 mW.

Artificial Neural Network and Deep Learning: Fundamentals and Theory

"Artificial Neural Network and Deep Learning: Fundamentals and Theory" offers a comprehensive exploration of the foundational principles and advanced methodologies in neural networks and deep learning. This book begins with essential concepts in descriptive statistics and probability theory, laying a solid groundwork for understanding data and probability distributions. As the reader progresses, they are introduced to matrix calculus and gradient optimization, crucial for training and fine-tuning neural networks. The book delves into multilayer feed-forward neural networks, explaining their architecture, training processes, and the backpropagation algorithm. Key challenges in neural network optimization, such as activation function saturation, vanishing and exploding gradients, and weight initialization, are thoroughly discussed. The text covers various learning rate schedules and adaptive algorithms, providing strategies to optimize the training process. Techniques for generalization and hyperparameter tuning, including Bayesian optimization and Gaussian processes, are also presented to enhance model performance and prevent overfitting. Advanced activation functions are explored in detail, categorized into sigmoid-based, ReLU-based, ELU-based, miscellaneous, non-standard, and combined types. Each activation function is examined for its properties and applications, offering readers a deep understanding of their impact on neural network behavior. The final chapter introduces complex-valued neural networks, discussing complex numbers, functions, and visualizations, as well as complex calculus and backpropagation algorithms. This book equips readers with the knowledge and skills necessary to design, and optimize advanced neural network models, contributing to the ongoing advancements in artificial intelligence.