Do I have enough space for that?

Let’s say I have several copies of a system: it doesn’t really matter what this system is, I think this describes fairly well how many production environments are set up. If it makes you happy to have a concrete example, let’s say this is a server-based Java application.

In this system, dev is the next version the application development team is working on, prod is the version that’s in production use now, and prod.old is the previous version from production that we need to keep as a backup. We also have separate test and staging areas that we use when new versions are ready to come out of development, but before we put them into production.

When the QA team has approved the new version to move into production, we need to keep the staging system where it is (not just rename it) because we need to validate that the production version matches the staging version. In the end, the process to move a version from staging into production looks like this:

1. Delete the prod.old version
2. Rename prod to prod.old
3. Copy staging to prod

Over time, the disk starts to fill up. When space becomes a premium, how can your script detect if it has enough space to do the job so it can abort if there isn’t enough room?

How much space

Unix and Unix-like systems (such as Linux) provide two commands that help you understand how much space is being used:

• du (“disk usage”) counts how much disk space is used by a directory
• df (“disk free”) reports how much disk space is used (and is available) on a filesystem

I’ve set up a very tiny virtual disk to demonstrate how to use these commands. This disk is mounted on the /mnt directory, and I already set up the dev, test, staging, prod and prod.old directories to mimic the application environment I described:

$ pwd
/mnt/app

$ ls
dev  prod  prod.old  staging  test

We can use the du command to count the disk space used in a file or directory. I want to know how much space is used within the directory, so I’ll add the -s option to display only a summary for each directory:

$ du -s *
696 dev
682 prod
671 prod.old
693 staging
693 test

By default, these sizes are reported in 1 KB blocks, but you can put this into “human readable” form by adding the -h option:

$ du -sh *
696K    dev
682K    prod
671K    prod.old
693K    staging
693K    test

You’ll note that each new version gets larger and larger over time: dev is slightly larger than the previous version in test and staging, staging takes more space than the prod version, and prod is a little bigger than the older prod.old version.

To see how much space is available on the filesystem, we use the df command. You can specify the filesystem path like /mnt to view that filesystem, or you can use . to mean the filesystem that contains the current path. In this case, I’m running the command from /mnt/app and my very tiny disk is mounted at the /mnt path:

$ df .
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/loop0          3752  3455        41  99% /mnt

There’s not much space left, just 41 KB. When it’s time to move the new version into production, will there be enough space to do that after I delete the prod.old version, rename prod to prod.old, then copy staging to prod?

The simple version

Being good systems administrators who want to automate everything, we should create a script to perform all the actions to move a new version into production. A simple version of this script might look like this:

#!/bin/bash

cd /mnt/app

rm -rf prod.old
mv prod prod.old
cp -r staging prod

But this script doesn’t have any “smarts” to know if there will be enough space to do all of this work. We need to account for removing the old prod.old and copying the new staging. Let’s explore some options to do that:

The GNU version of the df command (which is the version available on pretty much every Linux system) lets you give a field name to specify what value you want to report on. Valid names are source, fstype, itotal, iused, iavail, ipcent, size, used, avail, pcent, file and target. I’m most interested in the avail value, because that tells me how much space is left on the filesystem. I can use that to calculate if I’ll have enough room left after migrating the new version into production.

$ df --output=avail .
Avail
   41

The df command doesn’t have an option to not display the header line, but you can use the tail command to just print the last line, containing the number. In a Bash script, we might get the value like this:

avail=$(df --output=avail . | tail -n 1)

How much space will we free up by removing the old prod.old version? We can answer that with the du command:

$ du -s prod.old
671 prod.old

To automate this in a Bash script, we only need the first field, which is the number portion. We can use the awk command to print just the number, which is useful when storing this value in a Bash script:

old=$(du -s prod.old | awk '{print $1}')

The steps are the same to determine how much space is needed to copy staging into the new prod version:

$ du -s staging
693 staging

Or, for the Bash script:

staging=$(du -s staging | awk '{print $1}')

Look at the numbers

Using du and df, we know the /mnt filesystem currently has 41 KB free. That’s not a lot. To move a new version of the application into production, we’ll first remove the prod.old directory, which should free up 671 KB, or a total of 712 KB free. Then, we’ll rename prod to prod.old and make a copy of the staging directory to become the new prod directory. Making the copy requires 693 KB, which means I’d expect to have 19 KB free at the end.

One way to do this calculation is with the expr command, which evaluates simple math expressions. Just type the calculation the way you’d enter it into a calculator:

$ expr 41 + 671 - 693
19

We can add this to the Bash script to detect if there’s enough room on the filesystem before we delete or copy anything:

#!/bin/bash

cd /mnt/app

avail=$(df --output=avail . | tail -n 1)
old=$(du -s prod.old | awk '{print $1}')
staging=$(du -s staging | awk '{print $1}')

free=$(expr $avail + $old - $staging)

if [ "$free" -le 0 ] ; then
  echo 'abort -- not enough space'
  exit 1
fi

rm -rf prod.old
mv prod prod.old
cp -r staging prod

This does all the math to determine if there’s enough free space. After the math is done, a negative or zero value indicates that there won’t be enough space left on the filesystem to do all of the work; if so, the Bash script prints an error and quits.

If I instead wanted to make sure there was a little room left on the filesystem after copying everything, like 1% so I could edit configuration files, we can compare the result to a calculation of percent. 1% is pretty easy to calculate, because we just need to divide the total filesystem size by 100. We can get the filesystem system with the size field in the df command:

$ df --output=size .
1K-blocks
     3752

Using this, we can make one final update to the script:

#!/bin/bash

cd /mnt/app

avail=$(df --output=avail . | tail -n 1)
old=$(du -s prod.old | awk '{print $1}')
staging=$(du -s staging | awk '{print $1}')

free=$(expr $avail + $old - $staging)

size=$(df --output=size . | tail -n 1)
minfree=$(expr $size / 100)

if [ "$free" -le $minfree ] ; then
  echo 'abort -- not enough space'
  exit 1
fi

rm -rf prod.old
mv prod prod.old
cp -r staging prod

In case you’re curious, while deleting and copying won’t completely fill the disk, I’d have less than 1% free after everything is done. 1% of 3752 KB is 37 KB, and the calculation for free space is 19 KB. So the script would abort and not do the migration for me.

Automate everything

System administrators should automate everything, which means using shell scripts to do the steps the same way every time you run them. You can prevent failure with a little scripting magic, and ensure that your automated process to copy a new version of a system won’t fill up the disk.