I’m running Arch Linux, and use ext4 filesystems.
When I run ls in a directory that is actually small now, but used to be huge – it hangs for a while. But the next time I run it, it’s almost instantaneous.
I tried doing:
strace ls
but I honestly don’t know how to debug the output. I can post it if necessary, though it’s more than a 100 lines long.
And, no, I’m not using any aliases.
$ type ls ls is hashed (/usr/bin/ls) $ df . Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda9 209460908 60427980 138323220 31% /home
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
A directory that used to be huge may still have a lot of blocks allocated for directory entries (= names and inode numbers of files and sub-directories in that directory), although almost all of them are now marked as deleted.
When a new directory is created, only a minimum number of spaces are allocated for directory entries. As more and more files are added, new blocks are allocated to hold directory entries as needed. But when files are deleted, the ext4 filesystem does not consolidate the directory entries and release the now-unnecessary directory metadata blocks, as the assumption is that they might be needed again soon enough.
You might have to unmount the filesystem and run a e2fsck -C0 -f -D /dev/sda9 on it to optimize the directories, to get the extra directory metadata blocks deallocated and the existing directory entries consolidated to a smaller space.
Since it’s your /home filesystem, you might be able to do it by making sure all regular user accounts are logged out, then logging in locally as root (typically on the text console). If umount /home in that situation reports that the filesystem is busy, you can use fuser -m /dev/sda9 to identify the processes blocking you from unmounting /home. If they are remnants of old user sessions, you can probably just kill them; but if they belong to services, you might want to stop those services in a controlled manner.
The other classic way to do this sort of major maintenance to /home would be to boot the system into single-user/emergency mode. On distributions using systemd, the boot option systemd.unit=emergency.target should do it.
And as others have mentioned, there is an even simpler solution, if preserving the timestamps of the directory is not important, and the problem directory is not the root directory of the filesystem it’s in: create a new directory alongside the “bloated” one, move all files to the new directory, remove the old directory, and rename the new directory to have the same name as the old one did. For example, if /directory/A is the one with the problem:
mkdir /directory/B mv /directory/A/* /directory/B/ # regular files and sub-directories mv /directory/A/.??* /directory/B/ # hidden files/dirs too rmdir /directory/A mv /directory/B /directory/A
Of course, if the directory is being used by any services, it would be a good idea to stop those services first.
Method 2
Out of curiosity, let’s try to reproduce this:
$ mkdir test $ cd test $ time ls # Check initial speed of ls real 0m0,002s $ stat . # Check initial size of directory File: . Size: 4096 Blocks: 8 IO Block: 4096 directory ... $ seq 1 1000000 | xargs touch # Create lot of files $ echo 3 | sudo tee /proc/sys/vm/drop_caches # Clear cache $ time ls > /dev/null real 0m1.588s $ stat . # Check size of directory when files are there File: . Size: 22925312 Blocks: 44776 IO Block: 4096 directory
Ok, so now we have a large directory. Let’s remove the files and see what happens:
$ ls | xargs rm # To avoid too long argument list $ echo 3 | sudo tee /proc/sys/vm/drop_caches $ time ls > /dev/null real 0m1.242s $ stat . File: . Size: 22925312 Blocks: 44776 IO Block: 4096 directory
So yes, the allocated size for the directory does stay large and that does cause slow ls, like telcoM’s answer also indicated.
If it is just a single directory with the problem, there is a simpler solution that does not require unmounting or root access: Simply create a new directory, move remaining files to it and remove the bloated one.
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0