du

There’s plenty of ways to run du, but I like output to have the following characteristics

1. filter: I don’t want to see all files and folders right away - too much information.
   
2. sort output: if you have a lot of directories, it’s annoying to go through all of these to check which contains a lot of GB, and which don’t.
   
3. show hidden folders: sometimes when you go down a directory that you found to contain a lot of data, you can’t find a directory in it that is very big. This means either the large sized item is a file (or more) in the directory you are in, or there are one or more hidden directories that are to blame.

So my du call looks like this:

sudo du -hd1 | sort -h

Some of that was picked up here. The main answer, containing sudo du -chs .[!.]* * | sort -h, and the suggestion sudo du -ahd1 | sort -h in the comments both work quite nicely. However, both yield folders AND files. From the output of du, it is often unclear which is which, and therefore I like to split the two, only checking folders with du.

Let’s brake that down:
sudo: you’ll need superuser privileges to see the content of all folders
du: print disk usage
-hd1: two arguments for du

• -h, –human-readable print sizes in human readable format
  
• -d1 –max-depth=N - maximum depth set to 1, so the call does not show sub-directories

|: pipe - connect multiple commands with the vertical bar
sort -h: sort the du output with the -h argument: -h, –human-numeric-sort compare human readable numbers (e.g., 2K 1G)

ls

In contrast to my standard ls approach (ls -lh), in this case, what goes for du, also goes for ls: you need to be able to see hidden files, and to sort your output.

ls -ahlsSr

Breakdown:
ls: list information about files
-ahls: six arguments for ls

• -a, –all do not ignore entries starting with .
  
• -h, –human-readable with -l and -s, print sizes like 1K 234M 2G etc.
  
• -l use a long listing format
  
• -s, –size print the allocated size of each file, in blocks
  
• -S sort by file size, largest first
  
• -r, –reverse reverse order while sorting

Procedure

So, I start in the root folder and work my way down.
Go to your root:

cd /

Then run the du command mentioned above:

sudo du -hd1 | sort -h

When in root folder, its output for instance looking like this:

0       ./dev
0       ./proc
0       ./sys
4,0K    ./cdrom
4,0K    ./srv
8,0K    ./mnt
16K     ./lost+found
136K    ./tmp
1,6M    ./run
8,3M    ./etc
12M     ./root
230M    ./boot
1,3G    ./opt
4,1G    ./var
4,6G    ./snap
6,8G    ./home
7,3G    ./usr
120G    ./media
144G    .

Then after looking at the output, and contemplating what should be in some of those directories, I go into one of the folders that take up a lot of disk space, run the du command again. If again most of the disk space is taken up by a lower level folder, I repeat this process. If most of the disk space seems to be taken up by the current directory, I run my ls command.

ls -ahlsSr

This command yields a long list of files, sorted by file sizes (largest last), and colour distinction between files, folders, archives, etc.
Output (bottom part) for /var/log looked like this:

Huge log files

By doing the above, I recently ended up in this specific folder (/var/log), because some of my log files blew up, taking up about 6GB each.
These were syslog and kern.log.

There may be a clear reason why these logs are increasing in size so much, so have a look before you try to shrink them.
Log files you can normally open in a text editor to browse through its content, but if they’re too big, this may fail.
In any case, you can run the following command to have a look at (the most recent) part of its content.

tail -n 100 /var/log/syslog
tail -n 100 /var/log/kern.log

If you see a clear reason, try to tackle that cause in order to stop it repeating.
For me, I didn’t immediately see a clear reason, so went on to clear their contents.

I first stumbled upon this Ask Ubuntu thread, that specifically asks about syslog, and the main answer is adequate enough to solve your issues. However, it does not explain what these commands do. This thread nicely adds to solving the log problem, but especially this thread explains things nicely.

First up: clear the log files, do NOT delete them.
When you delete them (rm), and then try to recreate an empty version of them (touch), it may be that in the meantime, the OS tries to write to the no longer existing log file. Besides that, you may get issues with permissions not set properly.

Instead, use the following command to clear a log - in this case syslog

sudo cat /dev/null > /var/log/syslog

Breakdown:
sudo: you’ll need superuser privileges to apply this command to syslog
cat: list the contents of the file coming after it
/dev/null: a simple device, that looks empty when reading from it
>: redirect the command on the left to the file on the right
/var/log/syslog: the log file that you want to clear

In short, you’re using cat to output the content of /dev/null (empty) to the log file, thereby clearing its original content. The explanations on Ask Ubuntu also show some shorthand commands to do the same.

Journal log

At the same time as I came across the above log issues, my journal log was also swamped. This needed a somewhat different approach, I found out.

Journal is the logging system for systemd, logging kernel and userland processes. And over time it can take up quite some of your disk space.
You can check its size by running the following command:

journalctl --disk-usage

Output should look something like this:

Archived and active journals take up 528.0M in the file system.  

You find the logs within /var/log/journal/<machineid>/ , where <machineid> is an alphanumeric string identifying your computer.
Content of the folder is a list of .journal logs.

It is possible to have a look at the content of any of these, but this mostly looks like gibberish, aka, not human readable.

tail -n 100 user-1000.journal

If the journal system takes up a lot of disk space, you can shrink the space it takes to a certain set size. The following command e.g. sets it to 500MB.

sudo journalctl --rotate --vacuum-size=500M

–rotate Request immediate rotation of the journal files
This argument marks active journal files as archived, making that they are affected by the second argument
–vacuum-size=BYTES Reduce disk usage below specified size

Additionally, you can completely empty it and it will start logging anew.

sudo journalctl --rotate --vacuum-time=1s

–vacuum-time=TIME Remove journal files older than specified time
In this case, with the settings I used, all journal files with data older than 1 second will be removed. A new active and empty journal file will be created for logging.

You can also automate this process, clearing journal logs when they exceed a certain size or age by editing the journald configuration file (/etc/systemd/journald.conf). I haven’t looked into this, yet. I might, if I need it more in the future.

Browser data

While I was at it, I decided to clear some more disk space, and one of the obvious places to look is my browsing history. If I don’t have this set to clearing automatically, you easily rack up a few GBs in data. In Firefox (FF), my preferred browser, you can simply go to Preferences > Privacy & Security > Cookies and Site Data. It looks like this, and tells you how much data you have stored in FF right now:

Just click ‘clear data’, and choose in the popup (see pic below) what you want to clear: then click ‘clear’

Additionally, I did a quick search for additional steps to clean up disk space. This It’s FOSS article has some nice additions, albeit some others seemed redundant.

Remove packages that are no longer needed

This removes packages that are no longer needed, and old Linux kernels.

sudo apt-get autoremove

Remove older versions of Snap applications

This addition is a nice one, because these can take up quite a bit of disk space. Snap packages are relatively big in size, and Snap keeps two older versions of each application. If you’re sure you don’t want to revert to an older version, you can simply remove these.

Check the size of the Snap

du -h /var/lib/snapd/snaps

Output looks like this:

4,0K    /var/lib/snapd/snaps/partial
1,2G    /var/lib/snapd/snaps

I already cleaned mine, so it’s not so big.

If you want to look at which Snap packages you have installed, run:

snap list --all

My output:

Name                    Version          Rev   Tracking       Publisher   Notes
chromium                81.0.4044.138    1143  latest/stable  canonical✓  -
core                    16-2.44.3        9066  latest/stable  canonical✓  core
core18                  20200427         1754  latest/stable  canonical✓  base
gtk-common-themes       0.1-36-gc75f853  1506  latest/stable  canonical✓  -
notepadqq               1.4.8            855   latest/stable  danieleds   -
slack                   4.4.2            23    latest/stable  slack✓      classic
snapd                   2.44.3           7264  latest/stable  canonical✓  snapd
wine-platform-3-stable  3.0.4            6     latest/stable  mmtrt       -
wine-platform-runtime   v1.0             123   latest/stable  mmtrt       disabled
wine-platform-runtime   v1.0             136   latest/stable  mmtrt       -

If you want to get rid of a Snap package, you can run the following command:

snap remove wine-platform-runtime --revision 123

Output:

wine-platform-runtime (revision 123) removed

By default all the snap revisions are removed. However, in this case, I only wanted to remove the disabled Snap package wine-platform-runtime (revision 123). Therefore I added the --revision argument to the command, followed by the revision number. Without this information both versions of wine-platform-runtime would have been removed.

If you want to get rid of all disabled Snap packages, you can create a shell script that does this, as described here

Go to your home folder:

cd ~

Create a file in your favourite text editor. I used Vim to do this, and called it remove_old_snaps:

vim remove_old_snaps

Copy-paste the following code into it and save the file.
So in Vim, press Ins
Paste the following code:

#!/bin/bash
# Removes old revisions of snaps
# CLOSE ALL SNAPS BEFORE RUNNING THIS
set -eu

LANG=en_US.UTF-8 snap list --all | awk '/disabled/{print $1, $3}' |
    while read snapname revision; do
        snap remove "$snapname" --revision="$revision"
    done

Press Esc
Type :exit! to save the file and quit Vim

To make the script executable, you have to assign the proper rights to it:

chmod +x remove-old-snaps

Now run the file:

sudo ./remove_old_snaps

This will give the same output as above, when I removed the single package wine-platform-runtime (revision 123), yielding a line like this for every package removed.

When you look at the script, what it does is the following: it calls snap list --all, looks for packages that are disabled, then takes the values of those packages that are in columns 1 (Name) and 3 (Revision), and uses these values, per package, in the snap remove command, in the exact same fashion as my manual removal above: snap remove <snap name> --revision <revision number>

Clean APT cache

The advanced package tool (APT) manages software on the system, and keeps a cache of downloaded and installed packages. Some of these may be redundant by now. This hardly takes up space on my system for now.

Check the size of the cache:

sudo du -sh /var/cache/apt 

Output:

74M     /var/cache/apt

Remove outdated packages only:

sudo apt-get autoclean

Clear entire cache:

sudo apt-get clean