3

I was running diff --brief -r /home/mateusz/ /media/mateusz/Database/backup_test_tmp_folder/home/mateusz/ command that completed with error code 2. According to documentation "Exit status is 0 if inputs are the same, 1 if different, 2 if trouble."

Unfortunately command exited without giving any clue why it failed. I tried using straces but unfortunately I am unable to diagnose anything based on produced log (last 150 lines of 3.5GB strace file are posted to https://gist.github.com/matkoniecz/15ed855bd3f161ad6354c7d637234804 - and also for diff running with sudo that failed after producing 78KB of strace log).

I considered modifying diff command to output some explanation before dying and returning 2, but given that it is unlikely that my program is unique lack of support for such debugging seems to indicate that there are better solutions (hopefully it is not "learn how to interpret strace").

So how I am supposed to check why my diff command failed? Is there any better tool than strace or adding printf debugging to diff command?

reducing activity
  • 1,268
  • 4
    What did the last line of the stderr say? I usually see something like diff: /xyz: Permission denied when getting the exit status of 2. – choroba Apr 03 '16 at 14:19
  • @choroba No useful info, it is just difference between these two folders. Last line is "diff: /home/mateusz/.thumbnails/normal/ffffc3a5ff8a7679cb5a02f2qcec5153.png: No such file or directory". And it is not a reason for failure as I have entire screens of such messages. – reducing activity Apr 03 '16 at 17:23
  • Sounds like it couldn't find a matching file to compare... -q, --brief means "report only when files differ", remove that and try it again. And looks like you're comparing a bunch of files. If you ran diff once for each individual pair of files, it should be pretty obvious which files aren't the same. – Xen2050 Apr 03 '16 at 19:59
  • @Xen2050 "Sounds like it couldn't find a matching file to compare." - is it counted as "trouble" worth returning error code 2 (rather than 1 indicating differences) during recursive comparing of directories? "If you ran diff once for each individual pair of files, it should be pretty obvious which files aren't the same." well I want to achieve a different goal - to compare directories, each containing over 400k files - not just two versions of one file. "looks like you're comparing a bunch of files" - it is an intended effect. – reducing activity Apr 03 '16 at 22:25

1 Answers1

2

You can read the output. diff doesn't give different exit status codes other than:

  • 0 if inputs are the same,
  • 1 if different,
  • 2 if trouble.

(from man diff)

"Trouble" means it couldn't read a file, or pretty much anything else.

From a little testing using diff (GNU diffutils) 3.3 and comparing folders, diff outputs messages to stdout or stderr:

  • If one file is missing then the exit status is 1, and this appears on stdout:

      Only in folder1: file-a

  • If one file is unreadable then the exit status is 2, and this appears on stderr:

      diff: folder1/file-b: Permission denied

(If both errors occur, the exit status is 2.)

So, read / parse stdout and stderr to find out what the problems are.

Note that diff will continue processing further files after encountering "trouble", so the line reporting the reason for the exit status 2 may be at any part of the output.

See this GNU "diffutils manual" link for more info on comparing directories (and using diff in general , it's insanely more detailed than just man diff). Info like:

... if you use the --report-identical-files (-s) option, it reports pairs of identical files

If only one file exists, diff normally does not show its contents; it merely reports that one file exists but the other does not. You can make diff act as though the missing file is empty, so that it outputs the entire contents of the file that actually exists... use --new-file (-N)... If the older directory contains large files that are not in the newer directory, you can make the patch smaller by using the --unidirectional-new-file option instead of -N.

To ignore some files while comparing directories, use the --exclude=pattern (-x pattern) option

If you have been comparing two directories and stopped partway through, later you might want to continue where you left off. You can do this by using the --starting-file=file (-S file) option. This compares only the file file and all alphabetically later files in the topmost directory level.

Or these related Q's from other sites:

Community
  • 1
Xen2050
  • 8,953
  • Thanks for "diffutils manual"! It turned out that adding "--no-dereference" parameter fixed problem (for the first time I have problem with not enough parameter listed in man page). Though I am still not sure why as reading stderr, stdout provides no hints what was going wrong (only 'No such file or directory' entries). I am not accepting this answer yes, despite fact that it seems to be solving my problem as question remains not answered - but if nothing changes I will accept it within two weeks. For now I merely upvoted it. – reducing activity Apr 05 '16 at 12:29
  • I will rerun full backup check and compare differences for commands with --no-dereference and without it - maybe this will reveal something that I am missing for now. – reducing activity Apr 05 '16 at 12:32
  • And there is a reason found! diff: /media/mateusz/Database/backup_test_tmp_folder/home/mateusz/.kde/socket-Grisznak: Permission denied. My problem was that I assumed that on "trouble" diff will immediately stop, report problem and return 2. It rather continued checking other files and reporting "/home/mateusz/.thumbnails/normal/yet_another_thumbnail.png No such file or directory". – reducing activity Apr 05 '16 at 12:35
  • 1
    You're welcome. I actually prefer kdiff3 or similar like Kompare for comparing directories with a GUI, can quickly see files only in one dir, identical, different, etc, But I'm not sure about scripting it, – Xen2050 Apr 06 '16 at 02:24