I untarred a corrupt tar file, and managed to end up with some directory
that I can not delete,
If I try to delete it, it seems like it can not be found, but ls shows it’s present, both with bash and with python I get similar behaviour, except right after I try to delete it with rm -rf, ls complains it can’t find it, then it lists it (see below after rm -rf). The find command shows the file is present,
but still I can’t think of a way to delete it.
Here are my attempts:
Here you see both ls and find agree we have a directory,
rl]$ ls mikeaâ??cnt rl]$ find -maxdepth 1 -type d -empty -print0 ./mikeaâcnt
But I can’t delete it:
rl]$ find -maxdepth 1 -type d -empty -print0 | xargs -0 rm -f -v rm: cannot remove `./mikeaâ302201302204cnt': Is a directory rl]$ ls mikeaâ??cnt
I can cd to it though and it’s empty:
rl]$ cd mikeaâ^Á^Äcnt/ mikeaâ^Á^Äcnt]$ ls mikeaâ^Á^Äcnt]$ pwd .../rl/mikeaâcnt mikeaâ^Á^Äcnt]$ cd ../ rl]$ ls mikeaâ??cnt
see below that is not a simple file but a directory, plus ls behaves funny after the rm -rf it says it can’t find the file then lists it straight after:
rl]$ rm mikeaâ^Á^Äcnt/ rm: cannot remove `mikeaâ302201302204cnt/': Is a directory rl]$ rm -rf mikeaâ^Á^Äcnt/ rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt rl]$
So this is the attempt with python, the file is found, but the name is not
usable as a name that can be deleted:
rl]$ python
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import shutil
>>> os.listdir('.')
['mikeaxc3xa2xc2x81xc2x84cnt']
>>> shutil.rmtree(os.listdir('.')[0] )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.6/shutil.py", line 204, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usr/lib64/python2.6/shutil.py", line 202, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'mikeaxc3xa2xc2x81xc2x84cnt'
even when I use tab completion the name it picks up is no usable:
rl]$ rm -rf mikeaâ^Á^Äcnt rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt
using the name that python shows with bash I get this:
rl]$ rm -rf "mikeaxc3xa2xc2x81xc2x84cnt" rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt
Is there anything I can do to get rid of this corrupt dir?
The underlying filesystem (NFS) seems functional and no other problems are reported, and I have had no such problems until the corrupt tar file.
EDIT:
Here is using find‘s own -exec option to call rm
rl]$ find -maxdepth 1 -type d -empty -exec rm -f {} ;
find: `./mikeaâ302201302204cnt': No such file or directory
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
rl]$
but the file is still there, (ls complains it can’t find it, but then shows it anyway)
2nd EDIT:
rl]$ find -maxdepth 1 -type d -empty -exec rm -rf {} ;
find: `./mikeaâ302201302204cnt': No such file or directory
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
The behaviour is still unchanged, the file still present
3rd EDIT:
rl]$ ls
mikeaâ??cnt
rl]$ find -maxdepth 1 -type d -empty -exec rm -rf {} +
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
There seems to be more to the name than mikeaâcnt from looking at the output of the python attempt mikeaxc3xa2xc2x81xc2x84cnt, and this screenshot:
4th EDIT:
This is the attempt with a wild card:
rl]$ echo * mikeaâcnt rl]$ echo mike* mikeaâcnt rl]$ rm -rf mike* rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt
and my locale:
rl]$ locale LANG=en_US.utf8 LC_CTYPE="en_US.utf8" LC_NUMERIC="en_US.utf8" LC_TIME="en_US.utf8" LC_COLLATE="en_US.utf8" LC_MONETARY="en_US.utf8" LC_MESSAGES="en_US.utf8" LC_PAPER="en_US.utf8" LC_NAME="en_US.utf8" LC_ADDRESS="en_US.utf8" LC_TELEPHONE="en_US.utf8" LC_MEASUREMENT="en_US.utf8" LC_IDENTIFICATION="en_US.utf8" LC_ALL=
5th Edit:
rl]$ ls -i ls: cannot access mikeaâcnt: No such file or directory ? mikeaâ??cnt
but also the behaviour has changed, now ls and cd do this:
rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt rl]$ cd mikeaâ^Á^Äcnt mikeaâcnt: No such file or directory.
This has happened after the attempts to delete, I’m thinking that it might be NFS issues as suggested in one of the answers here by vinc17.
6th EDIT:
This is the output of lsof and ls -a
rl]$ /usr/sbin/lsof mikeaâ^Á^Äcnt
lsof: status error on mikeaâxc2x81xc2x84cnt: No such file or directory
above is wrong, here is the correct lsof invocation:(rl is the parent directory)
rl]$ /usr/sbin/lsof | grep mike | grep rl tcsh 11926 mike cwd DIR 0,33 4096 19569249 /home/mike/mish/rl lsof 14733 mike cwd DIR 0,33 4096 19569249 /home/mike/mish/rl grep 14734 mike cwd DIR 0,33 4096 19569249 /home/mike/mish/rl grep 14735 mike cwd DIR 0,33 4096 19569249 /home/mike/mish/rl lsof 14736 mike cwd DIR 0,33 4096 19569249 /home/mike/mish/rl rl]$ rl]$ ls -a ls: cannot access mikeaâcnt: No such file or directory . .. mikeaâ??cnt
7th Edit:
move won’t work, (I tried it before all this, but I did not save the output), but it has the same problem as ls and rm with the file.
8th EDIT:
this is using the hex chars as suggested:
rl]$ ls --show-control-chars | xxd 0000000: 6d69 6b65 61c3 a2c2 81c2 8463 6e74 0a mikea......cnt. rl]$ rmdir $'mikea6d696b6561c3a2c281c284636e74acnt' rmdir: failed to remove `mikea06d6906b651c3a2c2\81c2\846306e74': No such file or directory rl]$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt rl]$
9th Edit:
for the stat command:
rl]$ stat mikeaâ^Á^Äcnt stat: cannot stat `mikeaâ302201302204cnt': No such file or directory rl]$
Its seems even more likely from all the output, there is a bug or other NFS misbehaviour as suggested in the comments.
Edit 10:
This is strace output in a gist since its so large,
its the output or these two commands:
strace -xx rmdir ./* | grep -e '-1 E'` strace -xx -e trace=file ls -li`
https://gist.github.com/mikeatm/e07fa600747a4285e460
Edit 11:
So before the above rmdir I noticed that I could cd into the directory,
but after the rmdir I could not cd again, similar to yesterday. The . and .. files were present:
rl]$ ls mikeaâ??cnt rl]$ cd mikeaâ^Á^Äcnt/ mikeaâ^Á^Äcnt]$ ls mikeaâ^Á^Äcnt]$ ls -a . .. mikeaâ^Á^Äcnt]$ cd ../
Final Edit:
I saw a local admin over this and it was dealt with by logging on to the server itself and deleting from there. The explanation from them is that it could be a problem with character sets in the name being inappropriate.
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
One way to delete files/direcories like this is by their inode-reference.
To find the inodes for elements in current dir:
ls -i 14813568 mikeaâcnt
To delete this:
find . -inum 14813568 -delete
Method 2
The following excerpt from this essay potentially explains why that directory refuses to be deleted:
NFSv4 requires that all filenames be exchanged using UTF-8 over the wire. The NFSv4 specification, RFC 3530, says that filenames should be UTF-8 encoded in section 1.4.3: “In a slight departure, file and directory names are encoded with UTF-8 to deal with the basics of internationalization.” The same text is also found in the newer NFS 4.1 RFC (RFC 5661) section 1.7.3. The current Linux NFS client simply passes filenames straight through, without any conversion from the current locale to and from UTF-8. Using non-UTF-8 filenames could be a real problem on a system using a remote NFSv4 system; any NFS server that follows the NFS specification is supposed to reject non-UTF-8 filenames. So if you want to ensure that your files can actually be stored from a Linux client to an NFS server, you must currently use UTF-8 filenames. In other words, although some people think that Linux doesn’t force a particular character encoding on filenames, in practice it already requires UTF-8 encoding for filenames in certain cases.
UTF-8 is a longer-term approach. Systems have to support UTF-8 as well as the many older encodings, giving people time to switch to UTF-8. To use “UTF-8 everywhere”, all tools need to be updated to support UTF-8. Years ago, this was a big problem, but as of 2011 this is essentially a solved problem, and I think the trajectory is very clear for those few trailing systems.
Not all byte sequences are legal UTF-8, and you don’t want to have to figure out how to display them. If the kernel enforces these restrictions, ensuring that only UTF-8 filenames are allowed, then there’s no problem… all the filenames will be legal UTF-8. Markus Kuhn’s utf8_check C function can quickly determine if a sequence is valid UTF-8.
The filesystem should be requiring that filenames meet some standard, not because of some evil need to control people, but simply so that the names can always be displayed correctly at a later time. The lack of standards makes things harder for users, not easier. Yet the filesystem doesn’t force filenames to be UTF-8, so it can easily have garbage.
Method 3
You should not use non-ASCII characters in the command line since as you could see, for some reason, they won’t necessarily correspond to the filename (Unicode has various ways for expressing accented letters). Something like:
rm -rf mike*
should work since the filename is directly generated by the shell. But make sure there’s only one match (do an echo mike* first to confirm).
Well, if cd works, then there’s no reason why rm or ls should say No such file or directory, so that the problem may be at the file system level.
Note: Do not use ls to find whether a directory is empty, but ls -a.
The directory may still be used by another process (including if it’s the cwd of some process). IMHO, that’s why it still “exists” but can yield errors, e.g. with ls; lsof may give you some information, but with NFS, you need to find which machine uses it. Especially with NFS, this can yield strange errors. ls -a in the parent directory could show you .nfs* files/directories in some cases.
When you get:
$ ls ls: cannot access mikeaâcnt: No such file or directory mikeaâ??cnt
I suspect that the file still exists in the directory table due to NFS caching and/or because it is used by another process, but without associated information. When ls tries to get information on the file itself, it gets an error as the file itself no longer exists (it is only in the directory table), hence the displayed error. Then ls outputs the filename because it is in the directory table. The fact you have question marks in one case but not in the other case is due to a display bug of ls IMHO (unrelated to your problem).
Method 4
I have personally tested using find‘s -exec directive:
$ mkdir -p mikeaâcnt
$ ls
mikeaâcnt
$ find -maxdepth 1 -type d -empty -exec rm -rf {} +
$ ls
$
The folder was correctly created and correctly removed.
As pointed out by @Igeorget, there’s an even simpler method if you have GNU find:
$ find -maxdepth 1 -type d -empty -delete
I also tested this command, and it functions correctly
Method 5
I have had the same problem, I believe. I have seen the problem earlier with a filename of ☃. ls in this case displayed the file as â??, but I was able to delete it with rm ☃.
This led me to the following way to convert the wrong name to the correct one:
First get the bytes of the filename:
$ ls --show-control-chars | xxd 0000000: 6d69 6b65 61c3 a2c2 81c2 8463 6e74 0a mikea......cnt.
Then decode these bytes as UTF-8, to get the unicode codepoints, using the hexadecimal input of this website for example: http://software.hixie.ch/utilities/cgi/unicode-decoder/utf8-decoder
U+006D LATIN SMALL LETTER M character U+0069 LATIN SMALL LETTER I character U+006B LATIN SMALL LETTER K character U+0065 LATIN SMALL LETTER E character U+0061 LATIN SMALL LETTER A character U+00E2 LATIN SMALL LETTER A WITH CIRCUMFLEX character (â) U+0081 <control> character () U+0084 <control> character () U+0063 LATIN SMALL LETTER C character U+006E LATIN SMALL LETTER N character U+0074 LATIN SMALL LETTER T character
Notice these are all below the byte boundary. We obtain the following bytes:
6D 69 6B 65 61 E2 81 84 63 6E 74
If we treat this sequence at UTF-8 we get:
U+006D LATIN SMALL LETTER M character U+0069 LATIN SMALL LETTER I character U+006B LATIN SMALL LETTER K character U+0065 LATIN SMALL LETTER E character U+0061 LATIN SMALL LETTER A character U+2044 FRACTION SLASH character (⁄) U+0063 LATIN SMALL LETTER C character U+006E LATIN SMALL LETTER N character U+0074 LATIN SMALL LETTER T character
And thus your filename is: mikea⁄cnt, with a fraction slash instead of a normal forward one. You may now pass this name to rmdir.
Method 6
For those who use a NetApp for the NFS3 volume, which is mounted on a linux machine:
cluster1::> set -privilege advanced cluster1::*> vserver nfs modify -vserver <SVM_NAME> -v3-search-unconverted-filename enabled cluster1::*> set -privilege admin
After I ran the command all non ASCII files were easily removed with the rm -rf <folder> command.
Method 7
After getting the correct hex code of file / folder name (using whatever method one sees fit, I may choose ls --show-control-chars | xxd), some special construct should be used to address such characters when running under bash:
rmdir $'mikeaxc3xa2xc2x81xc2x84cnt'
Otherwise backslashes are treated as vanilla backslash.
Method 8
Have you tried using rm -rf ./mikeaâcnt or rm -rf "./mikeaâcnt" or an absolute path? Also instead of rm, try rmdir ./mikeaâcnt.
Method 9
Have you tried to get the inode of that file with stat:
stat mike*
That should give you the inode number (and other data), and then you could try to delete it.
Method 10
I had similar issues. Do you have Gnome, KDE or some kind of Xwindow DM?. If you do open your file broser and remove the file from there.
It should work.
I would like to see a solution from the command line, but in my case and after losing a lot of time trying to figure out how to remove it from the command line I found that it was as simple as removing any other file from nautilus or any other file explorer (truth is I have only tried with nautilus).
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0