sdlamp2/docs/TOOLCHAIN.md

# Cross-Compilation Toolchain

This document describes the current Docker/Buildroot cross-compilation toolchain, the architecture mismatch with the target device, and possible approaches to fix it.

## Current Toolchain Architecture

### Pipeline Overview

The toolchain is built via a Docker container defined in `docker/`:

1. **`docker/Dockerfile`** — Debian Buster (EOL) base image, installs build dependencies (bc, build-essential, cmake, git, etc.), copies support scripts, runs `build-toolchain.sh`, sources `setup-env.sh` in `.bashrc`
2. **`docker/support/build-toolchain.sh`** — Downloads Buildroot 2017.11, applies patches, loads `rg35xx-buildroot-2017.11.config`, runs `make world`. If a pre-built `rg35xx-toolchain.tar.xz` exists, extracts that instead of rebuilding.
3. **`docker/support/install-toolchain.sh`** — Copies host tools from `buildroot/output/host/usr/` to `/opt/rg35xx-toolchain/usr/`, copies `hwcap.h` into the sysroot, runs `relocate-sdk.sh` to fix embedded paths.
4. **`docker/support/setup-env.sh`** — Sets environment variables:
   ```sh
   export PATH="/opt/rg35xx-toolchain/usr/bin:${PATH}:/opt/rg35xx-toolchain/usr/arm-buildroot-linux-gnueabihf/sysroot/bin"
   export CROSS_COMPILE=/opt/rg35xx-toolchain/usr/bin/arm-buildroot-linux-gnueabihf-
   export PREFIX=/opt/rg35xx-toolchain/usr/arm-buildroot-linux-gnueabihf/sysroot/usr
   export UNION_PLATFORM=rg35xx
   ```
5. **`docker/support/package-toolchain.sh`** — Archives `/opt/rg35xx-toolchain` as `rg35xx-toolchain.tar.xz` for caching.
6. **`docker/support/relocate-sdk.sh`** — Rewrites hardcoded paths in SDK text files so the toolchain can live at a different filesystem location.
7. **`docker/Makefile`** — `make shell` builds the Docker image (tag `rg35xx-toolchain`) and drops into an interactive bash shell with `./workspace` mounted at `/root/workspace`.

### How the Main Makefile Uses It

The project `Makefile` detects `PREFIX` to switch modes:

- **Cross-compile** (PREFIX set): uses `-I$(PREFIX)/include` and `-L$(PREFIX)/lib` directly.
- **Native** (no PREFIX): uses `pkg-config` to find SDL2/FFmpeg.

`CROSS_COMPILE` is prepended to `gcc` to select the cross-compiler (e.g. `arm-buildroot-linux-gnueabihf-gcc`).

## The Problem

The Buildroot config targets **32-bit ARM Cortex-A9** but the Anbernic RG35XX runs a **64-bit aarch64 userland**.

The cross-compiled binary is `ELF 32-bit LSB executable, ARM, EABI5` while the device only has 64-bit aarch64 shared libraries. The binary cannot run — the dynamic linker and every `.so` it needs are the wrong architecture.

Key mismatched settings in the config:

| Setting | Current Value | Needed |
|---------|--------------|--------|
| `BR2_arm` | `y` (32-bit ARM) | `BR2_aarch64=y` |
| `BR2_cortex_a9` | `y` | Cortex-A53 or similar |
| `BR2_ARM_EABIHF` | `y` | LP64 ABI |
| `BR2_GCC_TARGET_ABI` | `aapcs-linux` | `lp64` |
| `BR2_GCC_TARGET_CPU` | `cortex-a9` | `cortex-a53` |
| `BR2_GCC_TARGET_FPU` | `neon` | (implicit in AArch64) |
| Sysroot triple | `arm-buildroot-linux-gnueabihf` | `aarch64-buildroot-linux-gnu` |

## Buildroot Config Details

**File:** `docker/support/rg35xx-buildroot-2017.11.config`

### Architecture

```
BR2_arm=y
BR2_cortex_a9=y
BR2_ARM_EABIHF=y
BR2_ARM_FPU_NEON=y
BR2_ARM_INSTRUCTIONS_THUMB2=y
```

### Toolchain Versions

| Component | Version |
|-----------|---------|
| Buildroot | 2017.11 |
| GCC | 7.2.0 (`BR2_GCC_VERSION_7_X`) |
| Binutils | 2.28.1 (`BR2_BINUTILS_VERSION_2_28_X`) |
| Kernel headers | 3.10.108 (`BR2_KERNEL_HEADERS_3_10`) |
| glibc | 2.28 (`BR2_TOOLCHAIN_BUILDROOT_GLIBC`) |

### Packages Enabled

- **FFmpeg** — all encoders, decoders, muxers, demuxers, parsers, protocols, filters; nonfree codecs enabled
- **SDL2** — with KMS/DRM support, SDL2_image (PNG), SDL2_mixer, SDL2_net, SDL2_TTF
- **SDL 1.2** (legacy) — with framebuffer backend, SDL_image, SDL_TTF
- **ALSA** — PCM, mixer, raw MIDI, HWDEP, sequencer plugins
- **Graphics** — FreeType, libjpeg (SIMD), libpng, libdrm, zlib
- **Utilities** — evtest, tree

### Other Settings

- Optimization: size (`BR2_OPTIMIZE_S`)
- LTO and OpenMP enabled
- No stack smashing protection (`BR2_SSP_NONE`)
- Shared libraries only (`BR2_SHARED_LIBS`)
- Extra libs: libasan (`BR2_TOOLCHAIN_EXTRA_LIBS="libasan"`)

## Patches Applied

All patches live in `docker/support/patches/`.

### glibc 2.28 Compatibility Fixes

Three packages fail to build against glibc 2.28 because `_IO_ftrylockfile` was removed from the private API:

- **bison** (`package/bison/110-glibc-change-work-around.patch`) — replaces `_IO_ftrylockfile` checks with `_IO_EOF_SEEN`, adds `_IO_IN_BACKUP` macro
- **m4** (`package/m4/000-fix-fseeko.patch`) — same pattern across fflush.c, fpending.c, fpurge.c, freadahead.c, freading.c, fseeko.c, stdio-impl.h
- **e2fsprogs** (`package/e2fsprogs/000-fix-gcc-conflict.patch`) — renames `copy_file_range()` to `copy_file_chunk()` to avoid conflict with the new glibc function of the same name

### SDL 1.2 RG35XX Patches

- **`package/sdl/0003-rg35xx-sdlk-additions.patch`** — adds 9 RG35XX-specific scancodes and key mappings (KATAKANA, HIRAGANA, HENKAN, POWER, etc.) to SDL 1.2's framebuffer input driver. **Not relevant for SDL2.**
- **`package/sdl/0004-modernize-SDL_FBCON_DONT_CLEAR.patch`** — changes framebuffer clearing behavior: skips palette restoration and pixel clearing when `dontClearPixels` is set or `SDL_DOUBLEBUF` is used. **Not relevant for SDL2.**

### Buildroot Framework Patch

- **`toolchain-expose-BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS-for-all-toolchain-types-2017.11.1.diff`** — backports `BR2_TOOLCHAIN_EXTRA_LIBS` config option so it works with the internal Buildroot toolchain (not just external). Required for the `libasan` extra lib.

## Possible Approaches

### A: Replace Buildroot with Debian Multiarch (Recommended)

Use `apt` with `dpkg --add-architecture arm64` inside a Debian Docker container to install aarch64 cross-compiler and libraries directly from Debian repos.

**Pros:**
- Modern toolchain (GCC 12+, recent glibc, recent FFmpeg/SDL2)
- No Buildroot config to maintain, no patches needed
- Fast setup — apt install instead of multi-hour Buildroot build
- Libraries match what's on the device (Debian-based firmware)

**Cons:**
- Library versions must match the device's firmware (or ship them)
- Less control over exact build options

**Outline:**
```dockerfile
FROM debian:bookworm
RUN dpkg --add-architecture arm64 && apt-get update && \
    apt-get install -y gcc-aarch64-linux-gnu \
    libsdl2-dev:arm64 libsdl2-image-dev:arm64 \
    libavformat-dev:arm64 libavcodec-dev:arm64 \
    libavutil-dev:arm64 libswresample-dev:arm64
```

### B: Fix Buildroot Config for aarch64

Stay with Buildroot but update the defconfig to target aarch64 instead of ARM.

**Pros:**
- Minimal conceptual change — same pipeline, just different arch settings
- Full control over every library version

**Cons:**
- Buildroot 2017.11 is ancient and may not have good aarch64 support
- Would likely need to upgrade Buildroot to a modern release (2024.x)
- Upgrading Buildroot may break existing patches or require new ones
- Multi-hour rebuild from scratch
- Still maintaining a custom Buildroot config long-term

**Key config changes needed:**
```
BR2_aarch64=y
# Remove all BR2_arm, BR2_cortex_a9, BR2_ARM_* settings
# Update kernel headers to match device
```

### C: Skip Docker, Cross-Compile on Host Directly

Install `aarch64-linux-gnu-gcc` and aarch64 libraries on the host system (e.g. Arch Linux) without Docker.

**Pros:**
- Simplest setup — just `pacman -S aarch64-linux-gnu-gcc` and aarch64 libs
- No Docker overhead
- Fast iteration

**Cons:**
- Not reproducible across machines without documenting exact packages
- Harder to match device library versions
- Host distro package availability varies (Arch has good cross-compile support, others may not)