Skip to content

Commit 1fadb9f

Browse files
ckagererckagerer
committed
feat: add persist-ccache feature
1 parent 051588e commit 1fadb9f

6 files changed

Lines changed: 352 additions & 0 deletions

File tree

‎.github/workflows/test.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ jobs:
1818
- clang
1919
- persist-shell-history
2020
- persist-pre-commit-cache
21+
- persist-ccache
2122
baseImage:
2223
- alpine:3.20
2324
- alpine:latest
@@ -51,6 +52,7 @@ jobs:
5152
- clang
5253
- persist-shell-history
5354
- persist-pre-commit-cache
55+
- persist-ccache
5456
steps:
5557
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
5658

‎src/persist-ccache/README.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# Persist ccache compiler cache (persist-ccache)
2+
3+
Retain the ccache compiler cache directory across container rebuilds to speed up C/C++ compilation by reusing cached compilation results.
4+
5+
## What this feature does
6+
7+
This feature persists the ccache cache directory across container rebuilds:
8+
9+
- `~/.cache/ccache/` – ccache compiler cache
10+
11+
The cache is mounted as a persistent Docker volume that is shared across all users and workspaces, allowing cache reuse across container rebuilds.
12+
13+
## How it works
14+
15+
### Build phase (as root)
16+
17+
The `install.sh` script creates a Docker volume and mount point:
18+
- `/.persist-ccache` – Volume for ccache cache
19+
20+
The directory is created with permissions `777` so all users can access it.
21+
22+
### User initialization phase (in user context)
23+
24+
The `persist-ccache-init.sh` script (runs via `postCreateCommand`) performs:
25+
26+
1. **Backup of existing data**: If `~/.cache/ccache` exists as a regular directory, it is renamed to `.bak` (preserving any existing cache).
27+
2. **Symlink creation**: Creates a symlink from `~/.cache/ccache` to the persistent volume.
28+
3. **Environment configuration**: Sets `CCACHE_DIR` and `CCACHE_MAXSIZE` environment variables.
29+
4. **Shell integration**: Adds environment variables to `~/.bashrc` and `~/.zshrc` if they exist.
30+
31+
This approach ensures:
32+
- **No data loss**: Existing caches are backed up before being replaced
33+
- **Automatic recovery**: Symlinks persist across rebuilds; no repeated setup needed
34+
- **Multi-user support**: All users can access the shared volume
35+
- **Proper configuration**: Environment variables are set in the container so ccache knows where to store/find cache
36+
37+
## Example Usage
38+
39+
```json
40+
"features": {
41+
"ghcr.io/ckagerer/devcontainer-features/persist-ccache:1": {
42+
"cache_size": "5G"
43+
}
44+
}
45+
```
46+
47+
## Options
48+
49+
### `cache_size` (string, default: `"5G"`)
50+
51+
Sets the maximum size of the ccache cache. Valid formats include:
52+
- Decimal: `kB`, `MB`, `GB`, `TB` (e.g., `10GB`)
53+
- Binary: `KiB`, `MiB`, `GiB`, `TiB` (e.g., `5GiB`)
54+
55+
Examples:
56+
- `"5G"` (default) – 5 gigabytes
57+
- `"10GB"` – 10 gigabytes
58+
- `"100G"` – 100 gigabytes
59+
- `"50GiB"` – 50 gibibytes (binary)
60+
61+
### `keep_going` (boolean, default: `false`)
62+
63+
If set to `true`, the installer will not fail on errors and will attempt to continue setup. Useful for troubleshooting or environments with unusual configurations.
64+
65+
## Performance benefits
66+
67+
Typical ccache initialization includes:
68+
69+
1. Creating cache directory structure
70+
2. Configuring cache limits
71+
3. Initializing cache database
72+
73+
With persistent cache volumes, subsequent container rebuilds skip setup entirely and can immediately reuse cached compilation results. Depending on your project, this can reduce build times by 50-90% on subsequent builds.
74+
75+
## Requirements
76+
77+
- C/C++ build tools using ccache (optional; feature sets up infrastructure even if ccache is not yet installed)
78+
- Write permissions to `~/.cache/` (normally available in user context)
79+
- Docker volumes support (standard in dev container environments)
80+
81+
## Environment Variables
82+
83+
The feature automatically sets:
84+
85+
- `CCACHE_DIR=~/.cache/ccache` – Location of the cache
86+
- `CCACHE_MAXSIZE=<cache_size>` – Maximum cache size
87+
88+
These are set in user shell RC files (`.bashrc`, `.zshrc`) for persistence across shell sessions.
89+
90+
## Troubleshooting
91+
92+
### Symlinks not created
93+
94+
Check that the `persist-ccache-init.sh` script ran without errors:
95+
96+
```bash
97+
# View logs of the postCreateCommand
98+
# This is typically shown in the dev container build output
99+
```
100+
101+
### Cache still not persisting
102+
103+
Verify the volume mount is active:
104+
105+
```bash
106+
# Inside the container
107+
mount | grep persist-ccache
108+
# Should show:
109+
# devcontainer-persist-ccache on /.persist-ccache
110+
```
111+
112+
Verify environment variables are set:
113+
114+
```bash
115+
echo $CCACHE_DIR
116+
echo $CCACHE_MAXSIZE
117+
```
118+
119+
### Clearing old cache data
120+
121+
If you need to reset cached data:
122+
123+
```bash
124+
# Inside the container (as user)
125+
rm ~/.cache/ccache.bak # Remove backups if any
126+
rm -rf ~/.cache/ccache # This will remove symlink too
127+
```
128+
129+
Then rebuild the dev container to recreate fresh symlinks.
130+
131+
### Checking cache statistics
132+
133+
```bash
134+
# Show cache statistics and current configuration
135+
ccache -s
136+
137+
# Show compression statistics
138+
ccache -x
139+
140+
# Clear the cache if needed
141+
ccache -C
142+
```
143+
144+
## Related features
145+
146+
- [persist-shell-history](../persist-shell-history/) – Persist bash/zsh history across rebuilds
147+
- [persist-pre-commit-cache](../persist-pre-commit-cache/) – Persist pre-commit hook cache across rebuilds
148+
- [clang](../clang/) – Install clang/LLVM toolchain

‎src/persist-ccache/devcontainer-feature.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"name": "persist-ccache",
3+
"id": "persist-ccache",
4+
"version": "1.0.0",
5+
"description": "Persist ccache compiler cache across container rebuilds.",
6+
"documentationURL": "https://github.com/ckagerer/devcontainer-features/tree/main/src/persist-ccache",
7+
"options": {
8+
"cache_size": {
9+
"type": "string",
10+
"default": "5G",
11+
"description": "Maximum ccache size (e.g., 5G, 10GB, 50G). Default: 5G"
12+
},
13+
"keep_going": {
14+
"type": "boolean",
15+
"default": false,
16+
"description": "Ignore errors during setup and continue execution."
17+
}
18+
},
19+
"mounts": [
20+
{
21+
"source": "devcontainer-persist-ccache",
22+
"target": "/.persist-ccache",
23+
"type": "volume"
24+
}
25+
],
26+
"postCreateCommand": "/usr/local/share/persist-ccache-init.sh"
27+
}

‎src/persist-ccache/install.sh‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
#!/usr/bin/env sh
2+
3+
# Initialize ccache volume directory and configuration
4+
# This script runs as root during container build
5+
6+
if [ "${KEEP_GOING:-false}" = "true" ]; then
7+
set +e
8+
else
9+
set -e
10+
fi
11+
set -x
12+
13+
# Create cache directory with permissions for all users
14+
mkdir -p /.persist-ccache
15+
chmod 777 /.persist-ccache
16+
17+
# Generate the postCreateCommand script that runs in user context
18+
INIT_SCRIPT_PATH="/usr/local/share/persist-ccache-init.sh"
19+
20+
tee "$INIT_SCRIPT_PATH" >/dev/null <<'EOF'
21+
#!/usr/bin/env bash
22+
23+
# Initialize ccache symlinks and environment configuration
24+
# This script runs in user context (postCreateCommand)
25+
26+
set -e
27+
set -x
28+
29+
# Get CCACHE_MAXSIZE from environment, default to 5G
30+
CCACHE_MAXSIZE="${CCACHE_MAXSIZE:-5G}"
31+
32+
# Check if a path is a mount point
33+
is_mount_point() {
34+
if command -v mountpoint >/dev/null 2>&1; then
35+
mountpoint -q "$1"
36+
else
37+
# Fallback for systems without mountpoint command
38+
mount | grep -q " on $(readlink -f "$1") "
39+
fi
40+
return $?
41+
}
42+
43+
# Handle existing ~/.cache/ccache directory
44+
if [ -d ~/.cache/ccache ] && [ ! -L ~/.cache/ccache ]; then
45+
# Check if it's already a mount point (e.g., from another devcontainer feature)
46+
if is_mount_point ~/.cache/ccache 2>/dev/null; then
47+
echo "Note: ~/.cache/ccache is already mounted by another feature, skipping backup"
48+
else
49+
echo "Backing up existing ~/.cache/ccache to ~/.cache/ccache.bak"
50+
mv ~/.cache/ccache ~/.cache/ccache.bak
51+
fi
52+
fi
53+
54+
# Create symlink for ccache cache if it doesn't already exist
55+
if [ ! -L ~/.cache/ccache ] && [ ! -d ~/.cache/ccache ]; then
56+
mkdir -p ~/.cache
57+
ln -s /.persist-ccache ~/.cache/ccache
58+
echo "Created symlink: ~/.cache/ccache -> /.persist-ccache"
59+
fi
60+
61+
# Initialize ccache with max size if it hasn't been configured yet
62+
if [ ! -f ~/.cache/ccache/ccache.conf ]; then
63+
# Create ccache configuration directory
64+
mkdir -p ~/.cache/ccache
65+
66+
# Set the maximum cache size
67+
ccache -M "$CCACHE_MAXSIZE" 2>/dev/null || {
68+
echo "ccache not yet installed, configuration will be set on first use"
69+
}
70+
fi
71+
72+
# Set environment variable for ccache directory
73+
export CCACHE_DIR=~/.cache/ccache
74+
export CCACHE_MAXSIZE="${CCACHE_MAXSIZE}"
75+
76+
# Optionally add to shell RC files if they exist
77+
for rc_file in ~/.bashrc ~/.zshrc; do
78+
if [ -f "$rc_file" ]; then
79+
if ! grep -q "CCACHE_DIR" "$rc_file"; then
80+
{
81+
echo ""
82+
echo "# ccache configuration"
83+
echo "export CCACHE_DIR=~/.cache/ccache"
84+
echo "export CCACHE_MAXSIZE='${CCACHE_MAXSIZE}'"
85+
} >> "$rc_file"
86+
fi
87+
fi
88+
done
89+
90+
echo "ccache persistence initialization complete"
91+
echo " Cache directory: ~/.cache/ccache (symlink to /.persist-ccache)"
92+
echo " Max cache size: $CCACHE_MAXSIZE"
93+
EOF
94+
95+
chmod 755 "$INIT_SCRIPT_PATH"
96+
97+
echo "ccache persistence feature installation complete"

‎test/persist-ccache/scenarios.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"scenarios": [
3+
{
4+
"name": "default",
5+
"description": "Test persist-ccache with default settings (5G cache)"
6+
},
7+
{
8+
"name": "custom_size",
9+
"description": "Test persist-ccache with custom cache size",
10+
"options": {
11+
"cache_size": "10G"
12+
}
13+
}
14+
]
15+
}

‎test/persist-ccache/test.sh‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
#!/usr/bin/env bash
2+
3+
# Test script for persist-ccache feature
4+
5+
set -e
6+
7+
echo "=== Testing persist-ccache feature ==="
8+
9+
# Check if volume mount directory exists
10+
echo "Checking volume mount directory..."
11+
[ -d /.persist-ccache ] || {
12+
echo "Error: /.persist-ccache not found"
13+
exit 1
14+
}
15+
16+
# Check permissions
17+
echo "Checking directory permissions..."
18+
perms=$(stat -c '%a' /.persist-ccache)
19+
[ "$perms" = "777" ] || {
20+
echo "Error: /.persist-ccache has wrong permissions: $perms"
21+
exit 1
22+
}
23+
24+
# Create a test user cache directory structure
25+
mkdir -p ~/.cache
26+
27+
# Check if symlinks exist (after postCreateCommand runs)
28+
echo "Checking symlinks in user cache..."
29+
if [ -L ~/.cache/ccache ]; then
30+
target=$(readlink ~/.cache/ccache)
31+
[ "$target" = "/.persist-ccache" ] || {
32+
echo "Error: ccache symlink points to wrong target: $target"
33+
exit 1
34+
}
35+
echo "✓ ~/.cache/ccache symlink is correct"
36+
fi
37+
38+
# Test write access to mounted volume
39+
echo "Testing write access to mounted volume..."
40+
touch /.persist-ccache/test-file.txt || {
41+
echo "Error: Cannot write to /.persist-ccache"
42+
exit 1
43+
}
44+
rm /.persist-ccache/test-file.txt
45+
46+
# Check if environment variables are set (if shell rc files have been sourced)
47+
echo "Checking environment variable configuration..."
48+
if [ -f ~/.bashrc ] || [ -f ~/.zshrc ]; then
49+
# The init script should have added CCACHE_DIR and CCACHE_MAXSIZE
50+
# to the shell rc files
51+
for rc_file in ~/.bashrc ~/.zshrc; do
52+
if [ -f "$rc_file" ]; then
53+
if grep -q "CCACHE_DIR" "$rc_file"; then
54+
echo "✓ CCACHE_DIR found in $rc_file"
55+
fi
56+
if grep -q "CCACHE_MAXSIZE" "$rc_file"; then
57+
echo "✓ CCACHE_MAXSIZE found in $rc_file"
58+
fi
59+
fi
60+
done
61+
fi
62+
63+
echo "=== All tests passed ==="

0 commit comments

Comments
 (0)