Follow the /opt standard for added software

What features/apps/bugfixes needed in a future Puppy
Message
Author
User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

Follow the /opt standard for added software

#1 Post by Q5sys »

It seems that for the most part we the Puppy Linux Community have not been following the /opt standard for additional non-default included software.

Iguleder seems to have taken the lead here with the packages he's built for Slacko, I made a comment about it HERE, and Aitch agreed as well. Hence the creation of this thread. That way we can keep the slacko thread on topic.

Comments? Ideas? Opinions? Post away. :)

Unsure about /opt? Here is a little blurb about it:


Purpose:
/opt is reserved for the installation of add-on application software packages.

A package to be installed in /opt must locate its static files in a separate /opt/<package> directory tree, where <package> is a name that describes the software package.

Requirements:
"/opt"
<package> "Add-on application software packages"
Static package objects


Tree:
The directories /opt/bin, /opt/doc, /opt/include, /opt/info, /opt/lib, and /opt/man are reserved for local system administrator use. Packages may provide "front-end" files intended to be placed in (by linking or copying) these reserved directories by the local system administrator, but must function normally in the absence of these reserved directories.

Programs to be invoked by users must be located in the directory /opt/<package>/bin. If the package includes UNIX manual pages, they must be located in /opt/<package>/man and the same substructure as /usr/share/man must be used.

Package files that are variable (change in normal operation) must be installed in /var/opt. See the section on /var/opt for more information.

Host-specific configuration files must be installed in /etc/opt. See the section on /etc for more information.

No other package files may exist outside the /opt, /var/opt, and /etc/opt hierarchies except for those package files that must reside in specific locations within the filesystem tree in order to function properly. For example, device lock files must be placed in /var/lock and devices must be located in /dev.

Distributions may install software in /opt, but must not modify or delete software installed by the local system administrator without the assent of the local system administrator.
BEGIN RATIONALE

The use of /opt for add-on software is a well-established practice in the UNIX community. The System V Application Binary Interface [AT&T 1990], based on the System V Interface Definition (Third Edition), provides for an /opt structure very similar to the one defined here.

The Intel Binary Compatibility Standard v. 2 (iBCS2) also provides a similar structure for /opt.

Generally, all data required to support a package on a system must be present within /opt/<package>, including files intended to be copied into /etc/opt/<package> and /var/opt/<package> as well as reserved directories in /opt.

The minor restrictions on distributions using /opt are necessary because conflicts are possible between distribution-installed and locally-installed software, especially in the case of fixed pathnames found in some binary software.
END RATIONALE

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#2 Post by 01micko »

Puppy Linux Blog - contact me for access

aragon
Posts: 1698
Joined: Mon 15 Oct 2007, 12:18
Location: Germany

#3 Post by aragon »

as aitch asked, here are my thoughts about that:
- yes, i would personally follow that route, if bk will include the settings in std. woof2 for all upcoming puppies. remember that we often had problems with /usr/local/lib.
- the exact structure has to be clear before usage. For example:
-- will there be an /opt/share
-- will there be an /opt/local/...
- it should be clear, how to handle updates to preinstalled packages (bins and libs)
- what 'rank' should the new paths have in paths access-sequence.

So before simple doing it, it might be better to first have a little concept and discuss that with bk. It would not be that good, if some packagers use the new techniques, and users have the problem that they will need to edit system variables to get the packages working...

Personally i will not use opt (actually), as i'm compiling on 4.3.1 and i wan't to keep compatibility as long as i use this version.

aragon

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#4 Post by technosaurus »

Q5sys wrote:It seems that for the most part we the Puppy Linux Community have not been following the /opt standard for additional non-default included software.
/usr is dumb
/usr/local is stupid
/opt is just retarded
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#5 Post by Flash »

I'm all for following standards, but I have to ask, instead of the totally unhelpful name of /opt, why not say "/Additional_software" or something that is self-explanatory? That way, maybe developers would just naturally do the right thing. :lol:

User avatar
Makoto
Posts: 1665
Joined: Fri 04 Sep 2009, 01:30
Location: Out wandering... maybe.

#6 Post by Makoto »

I've been wondering - what does opt stand for, anyway? Optional? :D

I was half-expecting someone to suggest something like /usr/(username)/bin, etc., too. /opt came out of left field, for me (the first time I'd heard of it, I'd encountered someone insisting that it was the right thing to do.)
[ Puppy 4.3.1 JP, Frugal install ] * [ XenialPup 7.5, Frugal install ] * [XenialPup 64 7.5, Frugal install] * [ 4GB RAM | 512MB swap ]
In memory of our beloved American Eskimo puppy (1995-2010) and black Lab puppy (1997-2011).

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#7 Post by technosaurus »

Rob Landley wrote:Filesystem Layout

Firmware Linux's directory hierarchy is a bit idiosyncratic: some redundant directories have been merged, with symlinks from the standard positions pointing to their new positions. On the bright side, this makes it easy to make the root partition read-only.
Simplifying the $PATH.

The set "bin->usr/bin, sbin->usr/sbin, lib->usr/lib" all serve to consolidate all the executables under /usr. This has a bunch of nice effects: making a a read-only run-from-CD filesystem easier to do, allowing du /usr to show the whole system size, allowing everything outside of there to be mounted noexec, and of course having just one place to look for everything. (Normal executables are in /usr/bin. Root only executables are in /usr/sbin. Libraries are in /usr/lib.)

For those of you wondering why /bin and /usr/sbin were split in the first place, the answer is it's because Ken Thompson and Dennis Ritchie ran out of space on the original 2.5 megabyte RK-05 disk pack their root partition lived on in 1971, and leaked the OS into their second RK-05 disk pack where the user home directories lived. When they got more disk space, they created a new direct (/home) and moved all the user home directories there.

The real reason we kept it is tradition. The excuse is that the root partition contains early boot stuff and /usr may get mounted later, but these days we use initial ramdisks (initrd and initramfs) to handle that sort of thing. The version skew issues of actually trying to mix and match different versions of /lib/libc.so.* living on a local hard drive with a /usr/bin/* from the network mount are not pretty.

I.E. The separation is just a historical relic, and I've consolidated it in the name of simplicity.

On a related note, there's no reason for "/opt". After the original Unix leaked into /usr, Unix shipped out into the world in semi-standardized forms (Version 7, System III, the Berkeley Software Distribution...) and sites that installed these wanted places to add their own packages to the system without mixing their additions in with the base system. So they created "/usr/local" and created a third instance of bin/sbin/lib and so on under there. Then Linux distributors wanted a place to install optional packages, and they had /bin, /usr/bin, and /usr/local/bin to choose from, but the problem with each of those is that they were already in use and thus might be cluttered by who knows what. So a new directory was created, /opt, for "optional" packages like firefox or open office.

It's only a matter of time before somebody suggests /opt/local, and I'm not humoring this. Executables for everybody go in /usr/bin, ones usable only by root go in /usr/sbin. There's no /usr/local or /opt. /bin and /sbin are symlinks to the corresponding /usr directories, but there's no reason to put them in the $PATH.
Consolidating writeable directories.

All the editable stuff has been moved under "var", starting with symlinking tmp->var/tmp. Although /tmp is much less useful these days than it used to be, some things (like X) still love to stick things like named pipes in there. Long ago in the days of little hard drive space and even less ram, people made extensive use of temporary files and they threw them in /tmp because ~home had an ironclad quota. These days, putting anything in /tmp with a predictable filename is a security issue (symlink attacks, you can be made to overwrite any arbitrary file you have access to). Most temporary files for things like the printer or email migrated to /var/spool (where there are persistent subdirectories with known ownership and permissions) or in the user's home directory under something like "~/.kde".

The theoretical difference between /tmp and /var/tmp is that the contents of /tmp should be deleted by the system init scripts on every reboot, but the contents of /var/tmp may be preserved across reboots. Except there's no guarantee that the contents of any temp directory won't be deleted. So any program that actually depends on the contents of /var/tmp being preserved across a reboot is obviously broken, and there's no reason not to just symlink them together.

(I case it hasn't become apparent yet, there's 30 years of accumulated cruft in the standards, covering a lot of cases that don't apply outside of supercomputing centers where 500 people share accounts on a mainframe that has a dedicated support staff. They serve no purpose on a laptop, let alone an embedded system.)

The corner case is /etc, which can be writeable (we symlink it to var/etc) or a read-only part of the / partition. It's really a question of whether you want to update configuration information and user accounts in a running system, or whether that stuff should be fixed before deploying. We're doing some cleanup, but leaving /etc writeable (as a symlink to /var/etc). Firmware Linux symlinks /etc/mtab->/proc/mounts, which is required by modern stuff like shared subtrees. If you want a read-only /etc, use "find /etc -type f | xargs ls -lt" to see what gets updated on the live system. Some specific cases are that /etc/adjtime was moved to /var by LSB and /etc/resolv.conf should be a symlink somewhere writeable.
The resulting mount points

The result of all this is that a running system can have / be mounted read only (with /usr living under that), /var can be ramfs or tmpfs with a tarball extracted to initialize it on boot, /dev can be ramfs/tmpfs managed by udev or mdev (with /dev/pts as devpts under that: note that /dev/shm naturally inherits /dev's tmpfs and some things like User Mode Linux get upset if /dev/shm is mounted noexec), /proc can be procfs, /sys can bs sysfs. Optionally, /home can be be an actual writeable filesystem on a hard drive or the network.

Remember to put root's home directory somewhere writeable (I.E. /root should move to either /var/root or /home/root, change the passwd entry to do this), and life is good.

Firmware Linux is an embedded Linux distribution builder, which creates a bootable single file Linux system based on uClibc and BusyBox/toybox. It's basically a shell script that builds a complete Linux system from source code for an arbitrary target hardware platform.

The FWL script starts by building a cross-compiler for the appropriate target. Then it cross-compiles a small Linux system for the target, which is capable of acting as a native development environment when run on the appropriate hardware (or under an emulator such as QEMU). Finally the build script creates an ext2 root filesystem image, and packages it with a kernel configured to boot under QEMU and shell scripts to invoke qemu appropriately.

The FWL boot script for qemu (/tools/bin/qemu-setup.sh) populates /dev from sysfs, sets up an emulated (masquerading) network (so you can wget source packages or talk to distcc), and creates a few symlinks needed to test build normal software packages (such as making /lib point to /tools/lib). It also mounts /dev/hdb (or /dev/sdb) on /home if a second emulated drive is present.

For most platforms, exiting the command shell will exit the emulator. (Some, such as powerpc, don't support this yet. For those you have to kill qemu from another window, or exit the xterm. I'm working on it.)

To use this emulated system as a native build environment, see native compiling.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Makoto
Posts: 1665
Joined: Fri 04 Sep 2009, 01:30
Location: Out wandering... maybe.

#8 Post by Makoto »

So, it's another example of, "it's not broke, but we're fixing it, anyway"?

Sheesh. I was already confused about where to put some packages I was compiling or repackaging, a while back (some of those maintainers wanted all their files left intact, so I'd been debating creating a subdirectory for one or more of the packages... fun.)
[ Puppy 4.3.1 JP, Frugal install ] * [ XenialPup 7.5, Frugal install ] * [XenialPup 64 7.5, Frugal install] * [ 4GB RAM | 512MB swap ]
In memory of our beloved American Eskimo puppy (1995-2010) and black Lab puppy (1997-2011).

User avatar
puppyluvr
Posts: 3470
Joined: Sun 06 Jan 2008, 23:14
Location: Chickasha Oklahoma
Contact:

#9 Post by puppyluvr »

:D Hello,
Isnt that why /root/my-applications/bin is in the path???
Close the Windows, and open your eyes, to a whole new world
I am Lead Dog of the
Puppy Linux Users Group on Facebook
Join us!

Puppy since 2.15CE...

User avatar
Makoto
Posts: 1665
Joined: Fri 04 Sep 2009, 01:30
Location: Out wandering... maybe.

#10 Post by Makoto »

Sure, but isn't that cheating? :P

I wasn't sure at the time whether or not anything I put in a subdirectory of any of those directories (including /my-applications/bin) would still be executable as if they were on the path, without having to create symlinks. I think I asked about it, but never really had much of an answer.

Yeah, I know, silly question. But I thought I'd make sure before I started creating links. :/
[ Puppy 4.3.1 JP, Frugal install ] * [ XenialPup 7.5, Frugal install ] * [XenialPup 64 7.5, Frugal install] * [ 4GB RAM | 512MB swap ]
In memory of our beloved American Eskimo puppy (1995-2010) and black Lab puppy (1997-2011).

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#11 Post by Q5sys »

Makoto wrote:So, it's another example of, "it's not broke, but we're fixing it, anyway"?

Sheesh. I was already confused about where to put some packages I was compiling or repackaging, a while back (some of those maintainers wanted all their files left intact, so I'd been debating creating a subdirectory for one or more of the packages... fun.)
if you ask me, putting executables in a half a dozen directories is a broken idea. root binaries can go to /sbin, system required binaries like gnu tools work well in /bin distro binaries can go to /usr/bin, add on stuff can go to /opt or some other dir.

Currently most linux distros like puppy, put things all over the freaking place. You've got:
/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
hell my opensuse box even had binaries in
/usr/share and /usr/local/share
and in directories like /usr/bin there is no organization... just a long list of binaries, that a non skilled person will have no clue about.



for a person coming from a windows background who is used to everything in its own nice and tidy c:/program files/DIRECTORY/ the linux file system seems like a complete and utter chaotic mess. yes windows also has c:/user/USERNAME/appdata/ but thats like linux's home directory for each user.

the /opt standard was to help keep things seperate. Which actually works out well for helping newbies. Instead of them installing software and then wondering where on earth to find it... its somewhat simplier.
If you install Metasploit for instance, it'll install to /opt/msf3/ (msf3 = meta sploit framework 3)
If you install another gui like KDE and wonder where the files are, well they'd be under /opt/kde.

yes /opt is an old standard, but as things continue to get widely out of control with where people put executable things, there does need to be some order or simplicity brought to the table.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#12 Post by Iguleder »

Wow, so much ... anger :lol:

Let me explain why I like putting packages under /opt. There are three reasons for this.

First, these packages I built may conflict with the built-in ones and they're "unofficial", or, as some of you may describe them, optional. Packages such as libav (the FFmpeg fork) have the same file names and I don't want to ruin users' systems.

Second, /opt is extremely useful for whole suites of packages that need to be separated from the core system. A good example for this is Trinity (the KDE 3.x fork), which depends on Qt3. Since we don't want conflicts between Trinity and KDE (or between Qt3 and Qt4), putting it under /opt/trinity brings two major advantages: packages do not even attempt to link against Trinity or Qt3 (since this is a "non-standard" path) and the system is much more organized and elegant, since Trinity has dozens of executables and libraries, which get installed in their own directory instead of getting merged with /usr, which is messy anyway.

Third, /opt is the standard directory for these things. /usr/local/apps is an "underground" hack and I think it's extremely ugly. Also, "/opt" is shorter :lol:

Having support for /opt is a very nice feature for 5.3.1, since SFSs and PETs can be installed there from now on, which means Puppy can be much more elegant and well-structured.

EDIT: forgot to mention, the paths are under /opt (/opt/bin, /opt/sbin, /opt/include, /opt/lib, /opt/share, etc'), but configuration is stored in /etc/opt and variable data in /var/opt (/var/opt/mail, etc').

EDIT 2: you got me excited! Thanks guys, you inspired me to do a nice experiment: put each package in something like /apps/category, so multimedia applications go to /opt/Multimedia, office applications go to /apps/Office, etc', but the core system and the libraries needed by many packages from various categories go to / and /usr, in the traditional split. Since the chance two packages from different categories share a dependency (I'm not talking about glibc or zlib, something like libsigc++), it's a good approach.

I'm currently running the first build (a total rebuild of my repository, fully automated and easy) to see if it's any good :)
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
8-bit
Posts: 3406
Joined: Wed 04 Apr 2007, 03:37
Location: Oregon

#13 Post by 8-bit »

I do not care what the directory is as long as it is in the path and any supporting binaries an installed program needs are also in the path or made in directories that it can find.
If you have a program that installs and expects some supporting file to be in usr/bin and that supporting file is instead in usr/local/bin and the program hard codes that path, you have problems.
That is why so little hard coding of files is done and for those that use hard coding, a link has to be made to the actual location of the support file.

When PET files are made, I have seen them put their executables everywhere. So I agree that some standardization is needed.
If it was just the Puppy branch of linux, that would be one thing. But it seems to be in most linux releases.

Can someone give me an example of at least one version of linux that had got it right as to the directory tree and executable placement?

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#14 Post by technosaurus »

8-bit wrote: Can someone give me an example of at least one version of linux that had got it right as to the directory tree and executable placement?
There are 2 drastically different approaches that work equally well:
Aboriginal (formerly firmware) just 2 binary dirs ... The rest are symlinks
Gobolinux ... Each app in its own dir (similar to a rox app) with symlinks into the path iirc

It really doesn't matter as long as everyone is on the same page.

Most successful versions that aren't for full blown environments involve symlinking of some sort. Tinycore and Choicepup, for instance, mounts its sfs files and symlinks each file.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#15 Post by Aitch »

I'm running Slacko
I have FF Aurora, Chromium, and Opera, yet only Opera installed itself into /opt

I have learned in various puppies when using rox, to hit the top left up arrow, until it won't go any further, and start there, as files/folders with the same label, in different locations/paths, confused the heck out of me....I tried Commander, looking for an explorer type treeview, but found rox was actually easier to use, and don't use Commander anymore
It was confusing enough to learn that / was both root and slash, and /root was slashroot, when I thought they were both the root people were talking about.....slash-et-cee, /etc, is another....seeing things written and hearing them is different, too

http://community.spiceworks.com/how_to/show/2287
17 on the list is a linux filesystem plan

I guess Barry really needs to chip in on this, though I'm glad it's being discussed....thanks for starting a new thread, Q5sys :D

Imagine a possible future headline:
Download Puppy Ver.xxx - with the first ORGANISED linux file system
Oh no, it's that dreaded word again.....where's everyone gone....? :wink: :lol:

Aitch :)

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#16 Post by Q5sys »

Aitch wrote: I guess Barry really needs to chip in on this, though I'm glad it's being discussed....thanks for starting a new thread, Q5sys :D

Imagine a possible future headline:
Download Puppy Ver.xxx - with the first ORGANISED linux file system
Oh no, it's that dreaded word again.....where's everyone gone....? :wink: :lol:

Aitch :)
cough cough... you mean 'organized'. Silly Brit's putting a 's' where a 'z' goes. ;)

Yea it would be nice if Barry would give us his thoughts. I personally like /opt because it keeps things neat and tidy. I am not a zealot for the idea though. Us talking it out and getting everyones ideas and opinions on the matter is really the best benefit of this thread.
When the idea came in the slacko thread, i realized (Aitch, notice the z there, ;) haha)... wow, I have no clue what anyone elses opinion on this issue is.


(and if anyone cant tell, my s vs z banter is just sarcasm from a colonial to a royalist. ;) heh)

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#17 Post by jemimah »

Iguleder wrote:Wow, so much ... anger :lol:

Let me explain why I like putting packages under /opt. There are three reasons for this.

First, these packages I built may conflict with the built-in ones and they're "unofficial", or, as some of you may describe them, optional. Packages such as libav (the FFmpeg fork) have the same file names and I don't want to ruin users' systems.

Second, /opt is extremely useful for whole suites of packages that need to be separated from the core system. A good example for this is Trinity (the KDE 3.x fork), which depends on Qt3. Since we don't want conflicts between Trinity and KDE (or between Qt3 and Qt4), putting it under /opt/trinity brings two major advantages: packages do not even attempt to link against Trinity or Qt3 (since this is a "non-standard" path) and the system is much more organized and elegant, since Trinity has dozens of executables and libraries, which get installed in their own directory instead of getting merged with /usr, which is messy anyway.
Iguleder does a nice job explaining the situations in which opt should be used. Opt is for portable apps or those that don't play nice with the rest of the os.

Compiling everything as a portable app is convenient for the user, but not small.

Generally it's good to keep things out of /usr/local - reserve that for the user's local programs.

Personally, I don't think moving a bunch of stuff to /opt makes things any less confusing.

What I do is make a rox-app (done using a script added to the end of fixmenus) for each menu item and put it in /root/my-roxapps. This makes it easy to find your apps and put them on the desktop and you don't have to care where they are in the filesystem.

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#18 Post by Aitch »

Ah, thanks Jemimah
I was just coming back to say I'd posted a link here in the Saluki thread, and Barry's Blog
cough cough... you mean 'organized'. Silly Brit's putting a 's' where a 'z' goes :wink:
Now, now, no 'cross the pond wars' please....if the English had their way, you'd all fail your English exams.... :lol:

Thankfully, the world's full of foreigners....

Aitch :)

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#19 Post by Q5sys »

Aitch wrote:
cough cough... you mean 'organized'. Silly Brit's putting a 's' where a 'z' goes :wink:
Now, now, no 'cross the pond wars' please....if the English had their way, you'd all fail your English exams.... :lol:

Thankfully, the world's full of foreigners....

Aitch :)
the funny thing is I still catch myself spelling colour with a 'u'. One of my elementary teachers was British and she taught us the 'proper' spelling of those words. However the rest of my teachers in my educational career did not appreciate that trend. haha
I got nothing but love for the Brits, still hoping to marry a sweet British girl one of these days. haha

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#20 Post by mavrothal »

/opt make sense only for cross-distribution apps (Browsers, OOo) trying to navigate the plethora of linux variations and file system trees out there. For any given one distribution, not only it does not make sense but is a call for trouble. Apps put their files in /opt so will not conflict with distro files. If you start putting distro files in /opt, you are just inviting these conflicts!
Frankly, I doubt if a more rational and consistent file system tree was commonplace, the need for /opt would emerged.

The (arguably) most bleeding edge distribution, the one backed up by the (certainly) most successful Linux enterprise, Fedora, is moving this way with their next release, Fedora17. Here is why.
So consolidation is the name of the game. Not further dispersion.

For local/experimental stuff there is $HOME or /root/my-applications (in puppy) or anything the user wants. But not for "distro" stuff.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

Post Reply