A disk can have free space while moving toward a point where it cannot create more files. Watching only a capacity graph can hide that problem on a Kubernetes node.

A CNCF Blog case published on October 5, 2026 illustrates the tension: recorded disk usage was 83% and inode usage was 67%, yet an alert predicted inode exhaustion. The larger percentage did not identify the resource that needed attention first.

For operators of Linux nodes, the practical question is whether to wait for kubelet to clean up. Understanding bytes, inodes, image GC, and Pod eviction makes it possible to connect diagnosis with prevention.

Free bytes do not tell you whether more files will fit

An inode holds filesystem metadata such as ownership, permissions, timestamps, and data locations. The Linux kernel documentation for ext4 describes this structure separately from file size.

Creating many small files can consume many inodes without adding much data. A compressed image size therefore cannot fully describe the burden of unpacking it on a node.

In the CNCF case, investigation reached dependencies unpacked into containerd snapshots; the image contained more than 40,000 files. That is a reported case figure, not a general property of Node.js images.

Image GC at 85% and inode eviction at 5% measure different resources

Capacity-based image GC calculates image filesystem usage from bytes. The defaults are 85 for imageGCHighThresholdPercent and 80 for imageGCLowThresholdPercent, the target for reclamation. They do not trigger cleanup at 85% inode usage. See the GC implementation and configuration reference.

Linux hard eviction does include free inodes. The disk-related defaults in Kubernetes v1.37.0 are:

Kubelet first tries to reclaim unused resources; if that is insufficient, it evicts Pods. Hard eviction uses a zero-second termination grace period. Falling below 5% free inodes is therefore different from waiting for routine cleanup. The node-pressure eviction documentation explains this sequence.

Compare these upstream defaults with the effective evictionHard settings. When mergeDefaultEvictionSettings is false, partially overriding thresholds leaves unspecified signals at zero instead of inheriting defaults. Check the configuration merge rules before assuming the defaults protect a node.

Use df and du to locate the inode-consuming storage

Start by comparing bytes and inodes for the same paths. These read-only commands assume containerd stores data in /var/lib/containerd; substitute the actual storage path when it differs.

# Inspect byte capacity for root and containerd storage
df -h / /var/lib/containerd

# Inspect inode usage for the same paths
df -i / /var/lib/containerd

df -i reports inodes instead of blocks. As the GNU df manual explains, a path selects its containing filesystem. Checking only / misses containerd storage on a separate disk.

GNU coreutils provides a directory-level view with du. Reading the tree requires permissions and takes longer as file counts grow, so start with the relevant storage directory.

# Find inode-heavy directories without crossing filesystem boundaries
sudo du --inodes -xS /var/lib/containerd | sort -nr | head -n 20

--inodes counts inodes, -x excludes other filesystems, and -S excludes subdirectories from each parent total. The GNU du manual documents these options. Use the results to trace the producer of files, not as a list of snapshot directories to delete directly.

Monitor inode growth and remaining headroom separately

NodeFilesystemFilesFillingUp combines predict_linear, free-inode conditions, and read-only status in the current node_exporter mixin rules. Usage of 67% alone does not explain an alert.

Those rules also include NodeFilesystemAlmostOutOfFiles, with fixed thresholds below 5% and 3% free. Inspect the deployed rules first. Having a fixed-threshold alert does not establish that it gives enough time to act before kubelet eviction.

This expression shows current usage for a filesystem with a positive inode total, assuming node_exporter labels root as mountpoint="/":

1 - (
  node_filesystem_files_free{mountpoint="/"}
  / node_filesystem_files{mountpoint="/"}
)

A result of 0.80 means 80% used. Pair a usage threshold and persistence period with predictive alerts, choosing values that leave time to respond. Treat 80% as an example, and include the actual imagefs mountpoint when it is separate. This design keeps low headroom visible even when growth pauses and the prediction changes.

Reduce runtime files and review unused-image retention

If build dependencies dominate inode consumption, change what the image carries. Docker multi-stage builds separate build and runtime stages, transferring selected artifacts with COPY --from. An application served as static files can leave its build tools and dependency tree behind.

An application running Node.js on the server still needs its runtime dependencies. Also use .dockerignore to exclude local files such as node_modules from the build context, preventing accidental inclusion through COPY . .. Evaluate changes using unpacked file counts and node inode growth under comparable deployments, alongside compressed size.

Already-retained images require a separate decision. As additional context, kubelet supports imageMaximumGCAge for reclaiming images based on how long they remain unused, independently of disk usage. Its default is 0s, disabled. The official GC documentation also notes that restarting kubelet resets its age tracking. This is a retention control, not an inode-triggered cleanup mechanism.

Free bytes alone do not justify waiting for kubelet. Measure inodes on the actual storage filesystem, arrange alerts early enough to act before eviction, and reduce the files introduced by the next deployment. That connects node protection to changes in the workload that caused the pressure.

Source

• Title: Kubelet watches inodes. Just not until it’s an emergency.
• URL: https://www.cncf.io/blog/2026/10/05/kubelet-watches-inodes-just-not-until-its-an-emergency/