check the root filesystem is actually using the whole drive
New first section, before anything else that changes the machine, because the
swapfile and the ballast both size themselves from free disk.
Ubuntu Server's installer on its defaults gives the root logical volume a fixed
size and leaves the rest of the drive as unallocated extents in the volume group.
On a 2TB disk that is a ~100G root with nothing to indicate a problem: lsblk
shows the whole drive, df shows 100G, and the two are never seen side by side
until the day it fills. Growing a virtual disk at a provider leaves the same
shape one layer down, and so does resizing a partition without telling the
filesystem inside it.
Three layers, any of which can be the short one, so all three are measured and
printed together:
drive: 76.3GB /dev/sda
volume: 76.1GB /dev/sda1
filesystem: 76.1GB ext4, mounted at /
Seeing them in one place is most of the value. The fix is then whichever layer is
short: lvextend for free extents, growpart for a partition that stops early
(followed by pvresize and lvextend when LVM is in the way), or resize2fs alone
when only the filesystem is behind.
Only ever grows. Nothing here shrinks, creates or deletes a partition, and ext4,
xfs and btrfs all grow while mounted — so no unmount, no reboot, and a failure
part-way leaves a smaller filesystem on a larger container, which is the state it
started in.
growpart is the authority on whether a partition can move — it exits 1 with
NOCHANGE when the partition already reaches the end — but it comes from
cloud-guest-utils, which is not on every image. Installing a package purely to
ask a question is too eager, so plain arithmetic on the device sizes decides
whether it is even worth looking, and only then is growpart fetched.
A gigabyte of slack before anything is reported: a filesystem is always slightly
smaller than its container, and reporting journal and reserved-block overhead as
reclaimable space would make this section cry wolf on every machine.
Verified on this host: plain ext4 partition filling its disk, correctly reports
nothing to reclaim, and every helper returns the right device and size.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,182 @@
|
||||
#!/bin/bash
|
||||
# =============================================================================
|
||||
# machine-setup — using the whole disk
|
||||
# =============================================================================
|
||||
#
|
||||
# Definitions only, like the other lib/ files.
|
||||
#
|
||||
# ── The problem this exists for ──
|
||||
#
|
||||
# Ubuntu Server's installer, left on its defaults, creates an LVM logical volume
|
||||
# at a fixed size and leaves the rest of the disk as free extents in the volume
|
||||
# group. On a 2TB drive you get a root filesystem of around 100GB and no
|
||||
# indication anything is wrong: `lsblk` shows the whole disk, `df` shows 100G,
|
||||
# and the two are never seen side by side until the day it fills.
|
||||
#
|
||||
# The same shape turns up two other ways:
|
||||
#
|
||||
# a virtual disk grown at the hypervisor or provider, where the partition still
|
||||
# ends where it used to
|
||||
#
|
||||
# a partition that was resized without the filesystem inside it being told
|
||||
#
|
||||
# Three layers, and any one of them can be the short one:
|
||||
#
|
||||
# disk the physical or virtual device
|
||||
# container the partition, or the logical volume
|
||||
# filesystem what df reports
|
||||
#
|
||||
# So all three are measured and reported together. Seeing them in one place is
|
||||
# most of the value; the fix is usually two commands once you know which layer is
|
||||
# short.
|
||||
#
|
||||
# ── Only ever grows ──
|
||||
#
|
||||
# Nothing here shrinks anything, and nothing here creates or deletes a partition.
|
||||
# ext4, xfs and btrfs all grow while mounted, so there is no unmount and no
|
||||
# reboot, and a failure part-way leaves a smaller filesystem on a larger
|
||||
# container — which is exactly the state it started in.
|
||||
|
||||
[[ -n "${MACHINE_SETUP_DISK_LOADED:-}" ]] && return 0
|
||||
MACHINE_SETUP_DISK_LOADED=1
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# What is where
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
root_device() { findmnt -no SOURCE / 2>/dev/null; }
|
||||
root_fstype() { findmnt -no FSTYPE / 2>/dev/null; }
|
||||
|
||||
# Size of a block device in bytes.
|
||||
dev_bytes() { lsblk -bndo SIZE "$1" 2>/dev/null || echo 0; }
|
||||
|
||||
# Bytes, formatted the way df and lsblk format them.
|
||||
human_bytes() { numfmt --to=iec --suffix=B --format='%.1f' "$1" 2>/dev/null || echo "${1}B"; }
|
||||
|
||||
# Is the root filesystem on a logical volume?
|
||||
root_is_lvm() { [[ "$(lsblk -ndo TYPE "$(root_device)" 2>/dev/null)" == "lvm" ]]; }
|
||||
|
||||
# The whole disk a device ultimately sits on: /dev/sda1 -> /dev/sda, and through
|
||||
# LVM as well, since PKNAME walks one level at a time.
|
||||
parent_disk() {
|
||||
local dev="$1" name
|
||||
while true; do
|
||||
name="$(lsblk -ndo PKNAME "$dev" 2>/dev/null)"
|
||||
[[ -z "$name" ]] && break
|
||||
dev="/dev/${name}"
|
||||
done
|
||||
echo "$dev"
|
||||
}
|
||||
|
||||
# The partition immediately below a device — for LVM, the one holding the PV.
|
||||
backing_partition() {
|
||||
local dev="$1" name
|
||||
while [[ "$(lsblk -ndo TYPE "$dev" 2>/dev/null)" != "part" ]]; do
|
||||
name="$(lsblk -ndo PKNAME "$dev" 2>/dev/null)"
|
||||
[[ -z "$name" ]] && return 1
|
||||
dev="/dev/${name}"
|
||||
done
|
||||
echo "$dev"
|
||||
}
|
||||
|
||||
# Split /dev/sda1 into "/dev/sda 1" — growpart wants them as separate arguments.
|
||||
# The digits come off the end because that is where a partition number is, on
|
||||
# /dev/sda1 and /dev/nvme0n1p2 alike.
|
||||
partition_parts() {
|
||||
local part="$1" num disk
|
||||
num="${part##*[!0-9]}"
|
||||
disk="${part%"$num"}"
|
||||
disk="${disk%p}" # nvme0n1p2 -> nvme0n1
|
||||
echo "$disk $num"
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Sizes of the three layers
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
# What the filesystem itself believes it is, which is the number df reports and
|
||||
# the only one of the three that is asked of the filesystem rather than the
|
||||
# kernel's block layer.
|
||||
fs_bytes() {
|
||||
local dev="$1"
|
||||
case "$(root_fstype)" in
|
||||
ext2 | ext3 | ext4)
|
||||
local count size
|
||||
count="$(tune2fs -l "$dev" 2>/dev/null | awk -F: '/^Block count:/ { gsub(/ /, "", $2); print $2 }')"
|
||||
size="$(tune2fs -l "$dev" 2>/dev/null | awk -F: '/^Block size:/ { gsub(/ /, "", $2); print $2 }')"
|
||||
[[ -n "$count" && -n "$size" ]] && echo $((count * size)) || echo 0
|
||||
;;
|
||||
xfs | btrfs)
|
||||
# Both report through the mount rather than the device.
|
||||
echo $(($(findmnt -bno SIZE / 2>/dev/null || echo 0)))
|
||||
;;
|
||||
*) echo 0 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Unallocated extents in the volume group behind root. This is the Ubuntu
|
||||
# installer case, and the one that is invisible without asking LVM directly.
|
||||
vg_free_bytes() {
|
||||
local vg
|
||||
command -v vgs &>/dev/null || {
|
||||
echo 0
|
||||
return
|
||||
}
|
||||
vg="$(lvs --noheadings -o vg_name "$(root_device)" 2>/dev/null | tr -d ' ')"
|
||||
[[ -z "$vg" ]] && {
|
||||
echo 0
|
||||
return
|
||||
}
|
||||
vgs --noheadings --nosuffix --units b -o vg_free "$vg" 2>/dev/null | tr -d ' ' || echo 0
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Can anything be reclaimed?
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
# growpart answers this better than arithmetic on sector counts: it exits 0 when
|
||||
# it would change something and 1 with NOCHANGE when the partition already
|
||||
# reaches the end of the disk. Needs cloud-guest-utils, which is not installed by
|
||||
# default on every image.
|
||||
partition_can_grow() {
|
||||
local part="$1" disk num
|
||||
command -v growpart &>/dev/null || return 1
|
||||
read -r disk num <<<"$(partition_parts "$part")"
|
||||
growpart --dry-run "$disk" "$num" &>/dev/null
|
||||
}
|
||||
|
||||
ensure_growpart() {
|
||||
command -v growpart &>/dev/null && return 0
|
||||
info " installing cloud-guest-utils, which provides growpart"
|
||||
pkg_install_now cloud-guest-utils
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Growing
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
grow_partition() {
|
||||
local part="$1" disk num
|
||||
read -r disk num <<<"$(partition_parts "$part")"
|
||||
growpart "$disk" "$num"
|
||||
}
|
||||
|
||||
# Tell LVM the partition under the physical volume got bigger.
|
||||
grow_pv() { pvresize "$1"; }
|
||||
|
||||
# Take every free extent in the volume group.
|
||||
grow_lv() { lvextend -l +100%FREE "$(root_device)"; }
|
||||
|
||||
# Grow the filesystem into whatever room it now has. All three do this online, so
|
||||
# the root filesystem is grown while it is mounted and in use.
|
||||
grow_fs() {
|
||||
case "$(root_fstype)" in
|
||||
ext2 | ext3 | ext4) resize2fs "$(root_device)" ;;
|
||||
xfs) xfs_growfs / ;;
|
||||
btrfs) btrfs filesystem resize max / ;;
|
||||
*)
|
||||
warn "do not know how to grow a $(root_fstype) filesystem"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
@@ -24,6 +24,8 @@ source "$SCRIPT_DIR/lib/packages.sh"
|
||||
source "$SCRIPT_DIR/lib/tools.sh"
|
||||
# shellcheck source=lib/system.sh
|
||||
source "$SCRIPT_DIR/lib/system.sh"
|
||||
# shellcheck source=lib/disk.sh
|
||||
source "$SCRIPT_DIR/lib/disk.sh"
|
||||
|
||||
# Trap errors with context. Installed here rather than in lib/base.sh, because
|
||||
# that file is definitions only and a trap is a side effect on whoever sources it.
|
||||
@@ -85,7 +87,102 @@ info "Refreshing the package index..."
|
||||
pkg_refresh >/dev/null
|
||||
|
||||
# =============================================================================
|
||||
# 2. System update
|
||||
# 2. Disk space
|
||||
# =============================================================================
|
||||
#
|
||||
# First of the sections that change anything, because everything after it sizes
|
||||
# itself from free disk — the swapfile and the ballast both.
|
||||
|
||||
step "Disk space"
|
||||
if ! skip; then
|
||||
ROOT_DEV="$(root_device)"
|
||||
ROOT_DISK="$(parent_disk "$ROOT_DEV")"
|
||||
ROOT_PART="$(backing_partition "$ROOT_DEV" || true)"
|
||||
|
||||
DISK_B="$(dev_bytes "$ROOT_DISK")"
|
||||
CONT_B="$(dev_bytes "$ROOT_DEV")"
|
||||
FS_B="$(fs_bytes "$ROOT_DEV")"
|
||||
VG_FREE_B="$(vg_free_bytes)"
|
||||
|
||||
# A filesystem is always a little smaller than the thing holding it —
|
||||
# metadata, journal, reserved blocks. Only a real gap is worth reporting.
|
||||
SLACK=$((1024 * 1024 * 1024))
|
||||
|
||||
echo ""
|
||||
info "Disk space — whether the root filesystem is actually using the whole drive"
|
||||
echo " Ubuntu's installer, left on its defaults, gives the root volume a"
|
||||
echo " fixed size and leaves the rest of the drive unallocated. On a 2TB"
|
||||
echo " disk that is a 100G root and no sign anything is wrong: lsblk shows"
|
||||
echo " the whole drive, df shows 100G, and the two are never seen together"
|
||||
echo " until the day it fills up. Growing a virtual disk at the provider"
|
||||
echo " leaves the same shape."
|
||||
echo ""
|
||||
echo " drive: $(human_bytes "$DISK_B") ${ROOT_DISK}"
|
||||
echo " volume: $(human_bytes "$CONT_B") ${ROOT_DEV}"
|
||||
echo " filesystem: $(human_bytes "$FS_B") $(root_fstype), mounted at /"
|
||||
((VG_FREE_B > SLACK)) && echo " unused in LVM: $(human_bytes "$VG_FREE_B")"
|
||||
|
||||
# growpart is the authority on whether a partition can move, but installing a
|
||||
# package just to ask is too eager — the arithmetic decides whether it is even
|
||||
# worth looking.
|
||||
MIGHT_GROW=false
|
||||
((DISK_B - CONT_B > SLACK)) && MIGHT_GROW=true
|
||||
[[ -n "$ROOT_PART" ]] && $MIGHT_GROW && ensure_growpart
|
||||
|
||||
if ((VG_FREE_B > SLACK)); then
|
||||
echo ""
|
||||
echo " $(human_bytes "$VG_FREE_B") is sitting unallocated in the volume group"
|
||||
echo " the fix is lvextend, then growing the filesystem into it — both online"
|
||||
if confirm "Use it?"; then
|
||||
grow_lv && grow_fs
|
||||
ok "root filesystem is now $(human_bytes "$(fs_bytes "$ROOT_DEV")")"
|
||||
SUMMARY+=("Disk: reclaimed $(human_bytes "$VG_FREE_B") from the volume group")
|
||||
else
|
||||
warn "skipped by request"
|
||||
SUMMARY+=("Disk: SKIPPED — $(human_bytes "$VG_FREE_B") left unallocated")
|
||||
fi
|
||||
|
||||
elif [[ -n "$ROOT_PART" ]] && partition_can_grow "$ROOT_PART"; then
|
||||
echo ""
|
||||
echo " the partition stops short of the end of the drive"
|
||||
echo " this is the one step that edits the partition table — it only ever"
|
||||
echo " moves the end of ${ROOT_PART} outwards, and never touches another one"
|
||||
if confirm "Extend it?"; then
|
||||
grow_partition "$ROOT_PART"
|
||||
if root_is_lvm; then
|
||||
grow_pv "$ROOT_PART"
|
||||
grow_lv
|
||||
fi
|
||||
grow_fs
|
||||
ok "root filesystem is now $(human_bytes "$(fs_bytes "$ROOT_DEV")")"
|
||||
SUMMARY+=("Disk: partition extended, filesystem now $(human_bytes "$(fs_bytes "$ROOT_DEV")")")
|
||||
else
|
||||
warn "skipped by request"
|
||||
SUMMARY+=("Disk: SKIPPED — partition left short of the drive")
|
||||
fi
|
||||
|
||||
elif ((FS_B > 0 && CONT_B - FS_B > SLACK)); then
|
||||
echo ""
|
||||
echo " the filesystem is smaller than the volume holding it"
|
||||
if confirm "Grow it?"; then
|
||||
grow_fs
|
||||
ok "root filesystem is now $(human_bytes "$(fs_bytes "$ROOT_DEV")")"
|
||||
SUMMARY+=("Disk: filesystem grown to $(human_bytes "$(fs_bytes "$ROOT_DEV")")")
|
||||
else
|
||||
warn "skipped by request"
|
||||
SUMMARY+=("Disk: SKIPPED — filesystem left short of its volume")
|
||||
fi
|
||||
|
||||
else
|
||||
echo ""
|
||||
echo " the whole drive is in use, nothing to reclaim"
|
||||
SUMMARY+=("Disk: already using the whole drive")
|
||||
fi
|
||||
step_ok
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 3. System update
|
||||
# =============================================================================
|
||||
#
|
||||
# Its own section because it is the only thing in the script that moves versions
|
||||
@@ -125,7 +222,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 3. Core utils
|
||||
# 4. Core utils
|
||||
# =============================================================================
|
||||
#
|
||||
# What the distribution provides: the six this script would break without, and
|
||||
@@ -140,7 +237,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 4. Command-line tools
|
||||
# 5. Command-line tools
|
||||
# =============================================================================
|
||||
#
|
||||
# A different thing from core utils, and kept apart from them: upstream binaries
|
||||
@@ -158,7 +255,7 @@ fi
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# 5. Locale
|
||||
# 6. Locale
|
||||
# =============================================================================
|
||||
#
|
||||
# LOCALE in the environment overrides the default.
|
||||
@@ -196,7 +293,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 6. Timezone
|
||||
# 7. Timezone
|
||||
# =============================================================================
|
||||
#
|
||||
# TIMEZONE in the environment answers the prompt ahead of time.
|
||||
@@ -262,7 +359,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 7. Swap
|
||||
# 8. Swap
|
||||
# =============================================================================
|
||||
#
|
||||
# Disk the kernel can park cold pages on when RAM fills, so a spike costs
|
||||
@@ -316,7 +413,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 8. Emergency disk ballast
|
||||
# 9. Emergency disk ballast
|
||||
# =============================================================================
|
||||
#
|
||||
# Always offered, whatever the role — the role only decides which way the
|
||||
@@ -359,7 +456,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 9. earlyoom
|
||||
# 10. earlyoom
|
||||
# =============================================================================
|
||||
|
||||
step "earlyoom"
|
||||
@@ -394,7 +491,7 @@ if ! skip; then
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
# 10. inotify watch limit
|
||||
# 11. inotify watch limit
|
||||
# =============================================================================
|
||||
|
||||
step "inotify watch limit"
|
||||
@@ -447,7 +544,7 @@ fi
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# 23. Summary
|
||||
# 12. Summary
|
||||
# =============================================================================
|
||||
|
||||
echo ""
|
||||
|
||||
Reference in New Issue
Block a user