How to find all hardlinks in a folder?

Question

I have two external disks that has the same files. One is encrypted, the other is not. The encrypted one has a lot less space left than the non encrypted, I now assume that it is because of hardlinks on the non encrypted disks.

So I would like to check, if there are any hardlinked files that might be doubled on the encrypted disk. How can I identify a hardlink?

If you have any other ideas what the reason for the free space issue could be, I'am open to ideas. Is it possible that the files need more space because of the encryption?

Answer 1

$ find -type f -links +1

That will show all regular files that have more than one link (name) to them. It will not tell you which names are linked to the same file, for that you could use -samefile or -inum, e.g. find -samefile "$somefile"

In the technical sense, all files (file names) are (hard) links, it's just that files with more than one link pointing to them are interesting in this sense. But even in those cases, there's no way to say that one of them is the "proper" file, and the other a link, the links are equal.

As an example:

$ touch a b c
$ ln b b2 ; ln c c2
$ find -type f -links +1
./c2
./b
./b2
./c
$ find -samefile b
./b
./b2

Answer 2

Search for hard links

@ilkkachu's and @barrycarter's answers are good. This answer is an alternative, that describes some results with more details.

• If the linked {match is/matches are} in the same directory tree, you will find them directly.

• Otherwise you can search in the whole file system from the mount point, but only within the same file system using -xdev, which is important if you search the root partition / and there are other mounted partitions.

  $ sudo find / -xdev -type f -links +1 -ls | sort -n > hard-links-in-root.txt
  

The following is an example, where one hard linked pair is found in the current directory, and two hard linked matches are found in another directory by searching from the mount point /media/multimed-2 of the data partition.

$ sudo find . -xdev -type f -links +1 -ls | sort -n
  5242881    648 -rw-rw-r--   2 olle     nio        657936 jun 30  2015 ./like-this.png
  5242882    940 -rw-rw-r--   2 olle     nio        957688 jun 30  2015 ./from-here.png
 14843905   1620 -rw-r--r--   2 olle     nio       1652803 jun 30  2015 ./img_4810.jpg
 14843905   1620 -rw-r--r--   2 olle     nio       1652803 jun 30  2015 ./mid-sommer-night_4810.jpg

$ find /media/multimed-2/ -samefile ./like-this.png
/media/multimed-2/Photos/2015/06/30/like-this.png
/media/multimed-2/Bilder/kartor/like-this.png

$ find /media/multimed-2/ -samefile ./from-here.png
/media/multimed-2/Photos/2015/06/30/from-here.png
/media/multimed-2/Bilder/kartor/from-here.png

Other causes why different amount of drive space is used

• Different file systems (ext4, NTFS, FAT32 ...)

• Different partition size, which causes differences in the overhead (meta-data).

• Different sector size on the drive (maybe?)

Answer 3

In theory, hard links should be indistinguishable from regular files (that's sort of the point). If "x" is a hardlink to "y", then "y" is also a hardlink to "x". That being said, the second column of ls -l tells you how many links there are to a given file. If this number is bigger than 1, the file is or has a hardlink somewhere. This may not work for directories, but I'm not sure why. I initially said each file in a directory has a link to that directory, but I was wrong: I found a directory with 10 files whose "link count" was only 2.

Once you've found the hard link, you can do ls -i to see its inode, and then use find's inode option to find other file(s) with the same inode (thus making them hard links to each other). Be sure to restrict find to a specific device, otherwise you may get spurious results.

To find all hard links at once, have find spit out inodes for all files on a device, and then use things like sort and uniq to find duplicates.

Answer 4

You could do something like this:

find . -type t -ls | grep -v " 1 username"

This will list files in the current directory and perform a ls on it. As @barrycarter said, hard links are indistinguishable from real files, but in this listing they will show up as having more than one link. Using grep -v you weed out the files that have only one link. (The username in the grep command is to make grep look in the right place for the single 1. Replace by your own username.)