Skip to content

Ship static busybox shell in gpu-operator image #2434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rajathagasthya wants to merge 1 commit into
base: main
Choose a base branch
from
+18 −1
Open

Ship static busybox shell in gpu-operator image #2434

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,20 @@ RUN curl -L https://codeload.github.com/NVIDIA/cuda-samples/tar.gz/refs/tags/v${
make && \
cp ${SAMPLE_NAME} /build/${SAMPLE_NAME}

# Build a static busybox layout: one binary plus applet symlinks (sh, rm,
# ln, sleep, cat, ...) so PATH-resolved commands in init-container wrappers
# and lifecycle hooks keep working on the non-*-dev* distroless base.
FROM debian:trixie-slim AS shell
RUN apt-get update \
&& apt-get install -y --no-install-recommends busybox-static \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir /busybox \
&& cp /bin/busybox /busybox/busybox \
&& /busybox/busybox --install -s /busybox
Copy link
Copy Markdown
Contributor

@cdesiniotis cdesiniotis May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this statement? What is --install actually doing?

Copy link
Copy Markdown
Contributor Author

@rajathagasthya rajathagasthya May 20, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--install creates links for all the busybox internal commands (rm, cat, etc.). The -s makes them a symlink. What we're doing here is installing those symlinks in /busybox, copying that directory into the final image (symlinks are preserved) and adding /busybox to PATH so direct command invocations are resolved correctly.

Here's what /busybox directory looks like:

$ ls -l /busybox
-rwxr-xr-x 1 root root 1975064 May 20 04:48  busybox
lrwxrwxrwx 1 root root      16 May 20 04:48  cat -> /busybox/busybox
lrwxrwxrwx 1 root root      16 May 20 04:48  chgrp -> /busybox/busybox
lrwxrwxrwx 1 root root      16 May 20 04:48  chmod -> /busybox/busybox
lrwxrwxrwx 1 root root      16 May 20 04:48  chown -> /busybox/busybox
lrwxrwxrwx 1 root root      16 May 20 04:48  chroot -> /busybox/busybox

My understanding is invocation of, say cat (which is resolved via $PATH), is dispatched as /busybox/busybox cat since busybox is a "multi-call binary".

Copy link
Copy Markdown
Contributor

@tariq1890 tariq1890 May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason we used busybox before was because it came with the distroless-dev images. Since we are moving away from them, why can't we just add a statically-built bash binary to the final image? Wouldn't bash be much simpler to use than a busybox shell?

Copy link
Copy Markdown
Contributor Author

@rajathagasthya rajathagasthya May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deployment manifests call out to coreutils like rm, cat, etc directly. We don't get those if we just add a static bash binary.

Copy link
Copy Markdown
Contributor

@tariq1890 tariq1890 May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough


# The C/C++ distroless image is used as a base since the CUDA vectorAdd
# sample application depends on C/C++ libraries.
FROM nvcr.io/nvidia/distroless/cc:v4.0.6-dev
FROM nvcr.io/nvidia/distroless/cc:v4.0.6

ENV NVIDIA_VISIBLE_DEVICES=void

Expand All @@ -89,6 +100,12 @@ LABEL description="See summary"
LABEL vsc-ref=${GIT_COMMIT}

WORKDIR /

COPY --from=shell /busybox /busybox
USER 0:0
RUN ["/busybox/ln", "-s", "/busybox/sh", "/bin/sh"]
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/busybox

COPY --from=builder /workspace/gpu-operator /usr/bin/
COPY --from=builder /workspace/manage-crds /usr/bin/
COPY --from=builder /workspace/nvidia-validator /usr/bin/
Expand Down
Loading