173

What is the rationale for the "unix system resources", or /usr directory, as described here, which duplicates many of the directory names under the root directory /?

My purpose: I'm installing Oracle JDK for the umpteenth time and decided this time to just put it under /home/user and I'm just reading around a bit to see if it is a bad idea on a single user machine.

muru
  • 207,970
H2ONaCl
  • 10,079
  • 3
    Your home directory is a perfect place for installing third-party software as non-root. – Lekensteyn May 02 '12 at 17:06
  • 20
    ...the sad realization because you thought /usr meant a hidden "user" directory for so many years... – Govind Rai Nov 09 '16 at 16:46
  • 14
    I seriously thought it was "user" this whole time. Like user binaries – Tanner Babcock Dec 21 '17 at 22:51
  • @TannerBabcock I think it is. At least I think /usr/bin is meant to be where system wide binaries are installed that are not considered system essential. – Aslan French Oct 22 '19 at 18:15
  • Fwiw, "usr" originally stood for "user". And "unix system resources" is by no means a commonly agreed-on standard. You find all sorts of variations on this. Imho, it's totally fine to continue to pronounce it as is. Who tf says "u-s-r"? :) – maryisdead Jul 18 '24 at 09:54

1 Answers1

286

There are the short version and the long version of your answer...

History

Short version:

As your link already said, /usr is a place for system-wide, read-only files. So all your software installed via system repository packages goes there. It does not duplicate any names of / except /bin and /lib, but originally with a different purpose: /bin and /lib are only for binaries and libraries required for booting, while /usr/bin and /usr/lib are for all the other executables and libraries. (now please don't ask about /sbin, this is the Short Version after all)

Nowadays, the distinction between required and not required for booting has diminished, since most modern distros, including Ubuntu, cannot properly boot without several files from /usr. And that's why there was a strong movement towards merging /usr/bin and /bin. Starting from Ubuntu 19.04 /bin, /lib (and /sbin) are just symlinks to their /usr counterparts, and in some systems they might not even exist anymore.

But maybe you are confusing /usr and /usr/local? Because yes, there is (and should be) a lot of duplicated directory names. More on that later...

Long version:

Back in the 70's, in Unix (yes, Unix, way before Linux), disks had little space (no hard drives yet!), and at a given point the system binaries grew too much in number and size to a point they would not fit a single disk, and developers had to split them across several media and thus create new mount points for them. /bin filesystem was full, so they installed the new binaries at... /usr/bin. And /usr was, at that time, their... user directory!

After this split happened, which is almost embarrassing and often told as a joke/lore, they started to create artificial justifications and criteria to decide what would go to /bin and what would go to /usr/bin. The informal rule was: "essential" stuff go to /bin, "the rest" goes to /usr/bin. Same with /lib. It was not long before /usr became crowded with system-related directories mixed with actual users' homes. Thus /home was born, to hold all user-related directories and keep /usr clean for "system stuff" only.

This was long before the FHS exist. When it was created, it embraced and formalized that current tradition and kept the name /usr, although at that time it already had nothing to do with "user" anymore. So yes, fancy names like "UNIX Source Repository" or "UNIX System Resources" are all made-up names, and it's too late to rename it anyway. But, as we've seen it, not too late to toss /bin, /lib and /sbin into it. So /usr is now the main "system" directory.

"Ok, what about /usr/sbin?", you ask. Damn, I was hoping you had forgotten. Ok... /usr/sbin is for commands that can only be (or are only meaningful when) executed by the root user, like mount and fdisk.

"But isn't that almost the same as /bin?". Yes, sure, but...

"Wait, then why there is an /sbin too? That doesn't make any sense!". Well, that's because of... err... humm... hold on...

Look, a 3-headed monkey behind you!

Ok, hopefully, you got distracted enough. Moving on...

If you think I'm cheating, yes, I am indeed. But so is the "official" answer: "Essential commands that can only be executed by root and must be available before you even mount /." What? Come on! Truth is: the line is indeed blurry, and there are lots of legacy names and paths that just "stuck" and now it is quite hard to get rid of.

More on the Case for the /usr merge, from systemd docs:

The historical justification for a /bin, /sbin and /lib separate from /usr no longer applies today. They were split off to have selected tools on a faster hard disk (which was small, because it was more expensive) and to contain all the tools necessary to mount the slower /usr partition. Today, a separate /usr partition already must be mounted by the initramfs during early boot, thus making the justification for a split-off moot. In addition, a lot of tools in /bin and /sbin in the status quo already lost the ability to run without a pre-mounted /usr. There is no valid reason anymore to have the operating system spread over multiple hierarchies, it lost its purpose.

And an amazing reading about the /usr split and its rationale, by Rob Landley:

Understanding the bin, sbin, usr/bin , usr/sbin split

Nowadays

Currently, regarding install directories, your the best way to understand is to think this way:

  • /usr - all system-wide, read-only files installed by (or provided by) the OS

  • /usr/local - system-wide, read-only files installed by the local administrator (usually, you). And that's why most directory names from /usr are duplicated here.

  • /opt - an atrocity meant for system-wide, read-only and self-contained software. That is, software that does not split their files over bin, lib, share, include like well-behaved software should.

  • ~/.local - the per-user counterpart of /usr/local, that is: software installed by (and for) each user. Like /usr, it has its own ~/.local/share, ~/.local/bin, ~/.local/lib.

  • ~/.local/opt - the per-user counterpart of /opt

So where to install software?

The above list is already half the answer of you Oracle JDK question, at least it gives several clues. The checklist to "Where should I install software X?" goes by:

  • Is it a completely self-contained, single directory software, like Eclipse IDE and other downloaded java apps, and you want it to be available to all users? Then install in /opt

  • Same as above, but you don't care about other users and I want to install for your user alone? Then install in ~/.local/opt

  • Its files split over multiple dirs, like bin and share, like traditional software compiled and installed with ./configure && make && sudo make install, and should be available to all users? Then install in /usr/local

  • Same as above, but for your user only? Then install in ~/.local

  • Software installed by the OS, or via package managers (like Software Center), and, most importantly, which any local modification might be overwritten when update manager upgrades it to a new version? It goes to /usr

Notes:

  • This explains why default install prefix for compiled software is /usr/local, and why you should change it to ./configure --prefix=$HOME/.local when installing software for your own user only

  • You may have noticed that all above directories are read-only (except, of course, when you install/remove software). The writable files (like config files) usually go to /etc (for system-wide software) and ~/.config (for per-user settings). Although many legacy software (and, unfortunately, some modern too) still use ~/.<software-name>, cluttering your home folder root with dozens of hidden dirs and files.

  • ~/.local and ~/.config are not part of the FHS specification. FHS does not deal with user's home folder. They are an attempt of Freedesktop/XDG, another standard organization geared towards Desktop Environments (like Gnome, KDE, and Unity), to try to set some conventions regarding the structure of the user's home. Not all software adheres to it (for example, distros have only recently included ~/.local/bin in user's default $PATH, and only if it exists), and no user is forced to follow it, but both gain many interoperability benefits if they do.

I hope this helps to clarify things a bit. Feel free to ask anything so I can improve the answer!

(and I also hope purists don't kill me for such an extremely informal language and explanation. It was intentional, and it surely has many inaccuracies, but I do believe it's a good way to make a newcomer have a brief overview understanding about install directories rationales)

MestreLion
  • 20,785
  • 2
    You summed it up very concisely and yes, it's true- the complete answer is a much longer story than the above! – papashou May 11 '12 at 22:17
  • Why not? The same logic applies: an attacker may create an executable under the home or a subdirectory of its, named after a system command. – ignis Jan 31 '13 at 16:03
  • 3
    @ignis: if an attacker has access to create and modify files under a user's home, then that user is already completely compromised, and $PATH is irrelevant. The attacker can even change that via ~/.profile, so your point is moot. ~/.local/bin is as secure (or insecure, if you want) as ~/bin, which is common practice in most distros. The idea that a user should not have any dir to keep and execute personal scripts in $PATH is absurd. – MestreLion Feb 01 '13 at 14:25
  • Thus, ~/bin is as secure as ~/.profile, and $PATH does not protect me from malware run by me (which has the permission to write in my own home). I did not know this file, I'm sorry. Thank you for the clarification, sorry for my previous comment. – ignis Feb 01 '13 at 19:25
  • The longer history it has, the more improvement/mess there is. – smwikipedia Mar 21 '16 at 02:45
  • 1
    Thanks for the detailed answer! Any updates to the current version? 17.04? – danfromisrael Aug 27 '17 at 05:38
  • 3
    This answer is like a journey! This man rules :) – Jacek Dziurdzikowski Sep 07 '21 at 12:17
  • @danfromisrael: no big news for 17.04 (or 2022): XDG standard is now more widely adopted and has more dirs in the spec, such as runtime and state, /opt is also more common, and there are the self-contained snaps, but the Nowadays section still remain true. – MestreLion Oct 05 '22 at 17:23
  • I'm just learning. Why should well-behaved software split their files over bin, lib, share, and include? – dotancohen Sep 15 '24 at 14:52
  • "So all your installed software goes there" That's not correct / needs clarification. For example Steam installs everything under ~ unless you manually change this. – Ash Jan 10 '25 at 02:52
  • @Ash: Steam has a very non-traditional installer that does not follow FHS or XDG or any Linux convention. Still, Ubuntu's steam-installer package only installs files in /usr, so the claim still holds. The launcher in /usr downloads the client to ~ only at run time, when launched by the user, and I believe it would not be appropriate to clarify such a corner case in the preamble of the short version (where the quote was taken from). – MestreLion Jan 10 '25 at 06:47
  • @Ash: That said, I think a clarification was needed regarding "installed software". In a world of Snaps, Flatpak, AppImage and such, "installed" means "via deb packages and apt repositories" – MestreLion Jan 10 '25 at 07:01
  • @dotancohen: as this layout is used for decades, many tools and features were built with this in mind to improve security and interoperability of software. Stuff in bin are executables, so it needs that permission, and it may be architecture-dependent, so it cannot be shared with other OS/Hosts. While share is just "data", needs no executable permissions, is architecture-independent, so it can be shared (hence the name), and even be in a network mount. Configuration in /etc may have sensitive data, so it requires more strict and granular access control... – MestreLion Jan 10 '25 at 07:22
  • @dotancohen: and "files split over multiple places" depends on the point of view. Yes, if you think about one app, it might seem "split all over". But from your system point of view, that deals with several apps and packages, all executables are in a single place, all desktop launchers are in a single place, all documentation is neatly organized, same as icons, and all unique state (/var) and config are too, so they're easy to backup (and those are not available via re-download) – MestreLion Jan 10 '25 at 07:35
  • It's not true that since Ubuntu 20.04 /bin, /lib and /sbin "don't exist anymore". They definitely exist in Ubuntu 20.04, as symlinks to respective directories under /usr, as said earlier. – raj Jan 10 '25 at 08:34
  • @raj, true for 24.04 too, I'll fix it – MestreLion Jan 16 '25 at 01:29
  • @MestreLion So when I unpack a compressed application, say Kafka, it has various sub-directories like bin/, libs/... Should I unpack this application directly to /usr/local/ or should I put the files from kafka/libs in usr/local/libs and the files from kafka/bin/ in /usr/local/bin?

    If the answer is the latter, then where would I put the other directories that come with that application besides bin and lib?

     – rocksNwaves May 12 '25 at 20:17
  • @rocksNwaves: The way you described it looks like Kafka is packaged as a self-contained software, so it most likely requires that directory structure as-is. Just unpack it at /opt/kafka. As a convenience, you might symlink its main executable (or a launcher script) to /usr/local/bin or ~/.local/bin. If this is a Desktop app, instead create a .desktop launcher at /usr/local/share/applications (or ~/.local/share/applications) pointing directly to its main executable and icon. – MestreLion Jun 04 '25 at 00:58
  • Great explanation! However, the real atrocity isn't /opt, it's using directory paths to encode information. Sometimes, you want to access all of a program's files at once, in which case /opt/PROGRAM is a much cleaner way to store the data (everything is contained in a single folder). Other times, e.g. when specifying your $PATH, you want to access all executables at once so /bin is more convenient. These notations should be interchangeable: instead of paths, we should be using (commutative) attributes for file selection, like get(prog=NAME, filetype=executable, user=USER). – Closed Limelike Curves Jun 30 '25 at 18:29
  • This, of course, is also why many people ignore the XDG desktop spec: Nobody wants their program's files to be shattered over a half-dozen directories. – Closed Limelike Curves Jun 30 '25 at 20:59
  • @ClosedLimelikeCurves: most software already have files over many directories, regardless of XDG: /etc for configuration, /usr/bin for executables, /usr/share for data, /var, /tmp, and so on. XDG only mimics the FHS rationale for the user's home. And if a dev wants to keep user files in a single dir, ~/.local/share/<appname> is a better path than ~/.<appname>, if only for uncluttering ~ and making backup scripts a lot easier. – MestreLion Jul 10 '25 at 16:51
  • @ClosedLimelikeCurves: and I agree that any hierarchical path will favor either purpose-first (the Unix/FHS/XDG approach) or app-first (the Windows way), and neither are inherently better than the other: some tasks will find one notation to be more convenient than the other, while other tasks will benefit more the other approach. Only a "pathless selector" like your hypothetical get() would be a win-win. Still, since the Unix ecosystem is mostly "purpose-first" paths, mixing this as /opt does lead to many interoperability issues – MestreLion Jul 10 '25 at 17:20
  • Yep, agree with your comments. Consistency is best, and most packages don't use /opt, so that should be a last resort. – Closed Limelike Curves Jul 11 '25 at 23:48