Infra Notes

Docker no space left on device: identify usage before pruning

Resolve Docker no space left on device safely by measuring images, containers, volumes, build cache, logs, and inodes before deleting recoverable data.

Published

ContainersDocker Disk SpaceNo Space Left on Devicedocker system dfDocker PruneContainer Logs

Problem

Docker fails to pull, build, start, or write data with no space left on device. The host may be out of bytes or inodes, while the space may belong to images, stopped containers, build cache, writable layers, logs, or volumes.

Quick answer

Measure first:

bash

df -h
df -i
docker system df -v
docker ps --size

df -h checks capacity and df -i checks inode exhaustion. Deleting one large image will not solve a filesystem that has millions of tiny files and no free inodes.

Classify Docker usage

docker system df -v separates images, containers, local volumes, and build cache. Determine which data is recoverable:

  • An unused image can usually be pulled again.
  • A stopped disposable container can usually be recreated.
  • Build cache is reproducible but removing it slows the next build.
  • A named volume may contain the only copy of application or database data.
  • Container logs may be needed for incident evidence.

Do not begin with docker system prune --all --volumes. Its scope crosses several data classes and can remove material that is difficult to reconstruct.

Recover low-risk space

Review candidates before pruning:

bash

docker image ls
docker container ls --all
docker builder prune
docker image prune
docker container prune

Each prune command prompts for confirmation and targets a narrower resource type. In production, reconcile the candidate list with deployment ownership and retention requirements first.

Treat volumes separately:

bash

docker volume ls
docker volume inspect <volume>

Only remove a volume after confirming no current or recoverable workload depends on its data and that an adequate backup exists.

Check container logs

The default local logging setup can grow when applications emit high-volume output without rotation. Inspect container logging configuration and host-level usage. Configure bounded rotation, for example through the Docker daemon or per-container logging options, according to the operational standard.

Do not truncate active log files blindly. Rotation must coordinate with the logging driver so file descriptors and accounting remain correct.

Prevent recurrence

  • Monitor filesystem bytes and inodes.
  • Configure log rotation and retention.
  • Clean build cache on CI runners with an explicit retention policy.
  • Keep stateful data in named, backed-up storage with capacity alerts.
  • Investigate unexpected image churn and oversized build contexts.
  • Reserve emergency headroom so cleanup commands can still operate.

If Docker storage is healthy but a Kubernetes image cannot start, see Kubernetes ImagePullBackOff.

References