Skip to content

ci: try disk-based cache instead of remote cache #5388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
chronark wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

ci: try disk-based cache instead of remote cache #5388

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 31 additions & 5 deletions .github/workflows/job_bazel.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,9 @@ jobs:
# Avoid downloading Bazel every time.
bazelisk-cache: true
# Store build cache per workflow.
disk-cache: false # we're already using depots remote cache
disk-cache: ${{ hashFiles('.bazelversion', '.bazelrc', 'MODULE.bazel') }}
# Share repository cache between workflows.
repository-cache: true
bazelrc: |
common --remote_cache=https://cache.depot.dev
common --remote_header=authorization=${{ secrets.DEPOT_BAZEL_CACHE_AUTHORIZATION }}
common --remote_local_fallback

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
Expand All @@ -39,3 +35,33 @@ jobs:
run: docker compose -f ./dev/docker-compose.yaml up s3 clickhouse mysql vault -d --wait
- name: Run tests
run: bazel test //... --test_output=errors

- name: Report disk usage
if: always()
run: |
BAZEL_OUTPUT_BASE=$(bazel info output_base)
BAZEL_REPOSITORY_CACHE=$(bazel info repository_cache)
echo "Bazel disk cache usage"
du -ms -t 1 ~/bazel-disk-cache/* 2>/dev/null || echo "No disk cache found"
echo "Bazel repository cache usage"
du -ms -t 1 "$BAZEL_REPOSITORY_CACHE" 2>/dev/null || echo "No repository cache found"
echo "Bazel output base usage"
du -ms -t 1 "$BAZEL_OUTPUT_BASE"
echo "Workspace usage"
du -ms -t 1 "$GITHUB_WORKSPACE"

- name: Drop large Bazel cache files
if: always()
run: |
BAZEL_REPOSITORY_CACHE=$(bazel info repository_cache)
for cache_dir in ~/bazel-disk-cache "$BAZEL_REPOSITORY_CACHE"; do
if [ -d "$cache_dir" ]; then
find "$cache_dir" -size +100M -type f -exec rm {} \;
echo "Trimmed $cache_dir"
du -ms -t 1 "$cache_dir" 2>/dev/null || echo "Cache empty"
fi
done

- name: Bazel shutdown
if: always()
run: bazel shutdown
Loading