Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 22 Oct 2019, 02:59
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Misc
How is PUPPY filesystem structured in LIVE & Frugal layouts?
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [7 Posts]  
Author Message
gcmartin

Joined: 14 Oct 2005
Posts: 6730
Location: Earth

PostPosted: Sat 15 Aug 2015, 15:54    Post subject:  How is PUPPY filesystem structured in LIVE & Frugal layouts?
Subject description: The older documentation of this is scattered. This corral's the kennel.
 

Over the years, developers has introduced various ways to structure Puppy for booting. As well, over the years, meaningful documentation on the ways and means make it difficult for the new and experience members to find and correlate these structures in an easily understood instance.

This thread is a requested attempt to gather useful understanding so that users can identify the means and the layout of the PUPs they use, and what is contained in those files that users see when viewing,either, the ISO, the DVD drive if booted Live or the Frugal folder if booted frugally.

I ask community to assist with any knowledge they choose to share on this thread so that users can understand the various means of Puppy boot system layouts.

These are a few that I ask community to express their knowledge of layout (pictures sometimes help too).

I am aware of several in primary use today: The following are examples taken from Puppy ISOs:
  • layouts with vmlinux, initrd, and distro's-SFS, for example:


  • layouts with humonguous, for example:


  • layouts with vmlinux, initrd, zdrv-SFS, and distro's-SFS, for example:


  • there are others, for example:
This thread is inspired by members trying to assist on this question, here, as I can see why the OP started the thread in absence of finding a good clear explanation of the differing packaging developers would do in any PUP distro.

No matter how simplistic you offer answers, all answers can be "a useful help" to members who review this thread for better Puppy understanding. I will update this opening post from time to time as you offer any responses to help in user understanding.
For example, But, it has been made know via other threads, that in some PUPs, following the Linux standard is not always taken. And, it is also well-known that the filesystems in other Linux distros may, too, interpret the standard different and unique to their desire and need.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile

Last edited by gcmartin on Sun 30 Aug 2015, 20:50; edited 4 times in total
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 3415
Location: 500 seconds from Sol

PostPosted: Sat 15 Aug 2015, 18:20    Post subject: Full Install Layout of TahrPup --/ and /usr/lib views
Subject description: Examination of SaveFolder would be similar
 

Hi gcmartin,

I realize that you only asked about Live and Frugal layouts, but thought this was as good a place as any to demonstrate what a Full install looks like. [Especially since I've had this screenshot sitting around for awhile and want to get rid of it].

The attached is a composite of two screenshots:

The top-right corner shows /, and the various folders within it, including /lib. It was superimposed on a screenshot of /lib, which shows the folders and files within that folder.

If the user is operating with a SaveFolder --rather than a SaveFile-- an examination of such SaveFolder will reveal a similar structure: folder, folders within folders, and files within a folder.

Depending on what applications the user has installed into a Frugal install using a SaveFolder, examination of / of the SaveFolder will reveal folders having some or all of the same names as those shown on the top-right corner of the screenshot; and browsing into any folder will reveal folders having some or all of the same names which would appear in a Full Install.

The same condition would be true of a SaveFile, except that you will not be able to see the contents of a SaveFile unless it is mounted or opened using a program which can edit SaveFiles.

mikesLr
Full-Install.png
 Description   File Structure of Full Install
 Filesize   141.95 KB
 Viewed   890 Time(s)

Full-Install.png

Back to top
View user's profile Send private message 
gcmartin

Joined: 14 Oct 2005
Posts: 6730
Location: Earth

PostPosted: Sun 16 Aug 2015, 13:24    Post subject:  

The vmlinux component (kernel)
An excellent description of VMLinux is found here.
Several PUPPY Linux members have produced tools for creating the Linux kernel. Currently, the most active member is @StemSee's Kernel Builder utility found here.

The initrd component (initial ramdisk boot subsystem)

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 3415
Location: 500 seconds from Sol

PostPosted: Sun 30 Aug 2015, 13:14    Post subject: Files "in" RAM with a Frugal Install - SFS processing  

Hi All,

Hopefully gcmartin or someone who actually knows will correct any misinformation the following contains.

I'd like to be able to tell you that when you're running a Frugally Installed Pup, what you'll see in RAM (random access memory) is similar to what you see when you look at a Full install, or look into a SaveFolder. But I can't. To a large extent what you see “in RAM” is the result of “smoke and mirrors”: well, sort of mirrors, anyway.

To really understand what's going on would take the explanation of a technical expert. I can only provide the layman's view I use to visualize the relationship between what's “in RAM” to whatever storage media you are using. For a more technical –and undoubtedly more accurate-- explanation, see http://murga-linux.com/puppy/viewtopic.php?p=827458&sid=180a2614ecae6cf2578f37eb14775300#827458 and read at least the first couple of pages of this thread, http://murga-linux.com/puppy/viewtopic.php?p=281842#281842

On storage (CD/DVD, USB-stick, hard-drive) a frugal Puppy consists of compressed/squashed files-systems which usually bear the acronym “sfs” --without the quotes, but on occasion may have an ending of 2fs or 3fs. The images gcmartin provided in the first post of this thread show the basic SFSes contained in the ISOs of several Pups. As I'll discuss its significance later, I'll mention here that if you were viewing the files of a Carolina ISO you would also see an adrv_lina_1.x.sfs.

A frugally installed Pup utilizes what I refer to as “layered-file-system.” You can see part of this layering taking place if you read the text as a Frugal Pup is booting to desktop. The first* thing you'll see is that drivers needed to read the storage media are copied, although the text reads “loaded”. [If your Pup came with a zdrv_xxx.sfs, I think that's where drivers were found]. Then vmlinuz –the kernel which handles memory and process management-- is copied from vmlinuz to RAM. Then one SFS is “loaded”, meaning copied. Than another. You'll also note a sentence which begins with the use the term “loading” but which ends with the phrase “copying to ram.” Then the system is stabilized. Kind of like layering one image on top of another in a graphic application and then selecting “merge-down” except: (a) the merge takes place in RAM; (b) Pup has greater flexibility as to which layer will have priority; and (c) most importantly, not every file in an SFS may, in fact, have been copied to RAM.

* You won't see it, but I think, the first thing which happens is vmlinux is copied to RAM, then the initial ramdisk is either copied or created using the instructions found in initrd. [I'm not sure which]. Those instructions are then followed until the instruction “switch root" is reached. It is into this ramdisk, this space in RAM, that the a Pup's files are copied to a particular pattern creating the Pup's file structure. When the instruction “switch root” is reached and exercised, root is switched from the Ramdisk to the root and system which was built. At least, that's my guess.

How many of the files in SFSes are copied to RAM will depend on which Pup you're using and how much RAM your system has (and perhaps whether your system uses a Swap File/Partition and its size). [Hint: if you're running a frugal install from removable media and receive notification that it is safe to remove it, it's probable that all files have been copied] Smile . The minimum that a Pup will copy into RAM are those files sufficient for you to open the applications which have somehow become a part of your system. What Pup also did during the boot process was to create in RAM a list pointing to where the rest of the files somehow on your system can be found.

I've chosen abiword for illustration purposes because that application is found on almost every Pup. If you pfind abiword [Menu>Filesystem>pfind, select System files] you'll get a window similar to the attached screen shot. It lists 15 files. While all those files may, in fact, have been copied to RAM, it is possible** –and for purpose of these illustrations will be assumed-- that when you first bootup a Pup only three abiword files may have been copied into RAM:

/usr/bin/abiword –abiword's executable,
/usr/share/icons/hicolor/48x48/apps/abiword.png –the icon which appears next to the listing of abiword on Menu>Document, and
/usr/share/applications/abiword.desktop – the file Pup uses to generate abiword's listing on the Menu.

** In actuality, I think it probable that abiword –which would only need about 15 mbs of RAM to be fully copied into RAM-- is, in fact, fully copied. I know from experience that a LibreOffice.sfs occupies about 175 Mb of storage, and if fully decompressed and copied into RAM would require about 500 mb +/-. However, only about 35 Mb of RAM are needed if a LibreOffice.sfs is loaded –available to the system-- but not opened.

Again assuming files are copied to RAM only as needed, when you open abiword, Pup examines the list in RAM and follows the instructions pertaining to abiword to copy into RAM those of abiword's files which are necessary to provide a window of its Graphical User Interface. Then, as and when made necessary by your selection of one of abiword's Tools or Menu entries, Pup again follows those instructions to copy into RAM the files necessary to perform the actions you've commanded.

When you boot into Slacko64_5.8.8 the first time –assuming that Slacko has abiword as a builtin application-- Puppy's list will include instructions to look for abiword files within the puppy_slacko64_5.8.8.sfs. The puppy_xxx.sfs file included in an ISO is the usual location for a dev to includes the files of builtin applications. The Carolinas, however, are constructed differently: Its devs have, with two exceptions, only included in puppy_lina_1.x.sfs those files necessary for Carolina to run, configure settings, download files and perform other basic file manipulation functions. “User applications” – those required to perform “real world tasks” such as word-processing, graphic creation and editing, display a Webpage and so on-- are included in the adrv_lina_1.x.sfs. So –assuming Carolina came with abiword builtin-- the first time you boot into Carolina Puppy's list would include instructions to look for abiword files within adrv_lina_1.x.sfs.

SFSes can be either READ-ONLY or READ/WRITE; but as far as I know, Puppies are built so that only a SAVEFILE is READ/WRITE and, regardless of how many layers a Pup has been instructed to copy into RAM, a SaveFile and its contents will always have priority: that is, the list will always point to the files in the SaveFile. Included in that SaveFile will be a list pointing to those applications and files [such as the lib you needed to add in order to run xyz application] you've installed to your SaveFile, but also respecting those applications you've decided to remove using Menu>Setup>Remove builtins. [Remove builins does not remove the application and its files from the SFS where it is located. It merely writes to the SaveFile instructions “Whiting out” the affected files].

Well, now, a new version of abiword has become available and you've installed the pet of it. Puppy's list regarding abiword now points to your SaveFile rather than the Pup_xxx.sfs or adrv_xxx.sfs. Or you've converted the new abiword pet to an SFS, or downloaded the new abiword in SFS form and Menu>Setup>SFS-load loaded the abiword.sfs. Then Puppy's list (in your SaveFile) will point to the abiword.SFS. These relationships may become permanent (until you Menu>Setup>SFS-load, unload abiword or PPM Uninstall abiword) if you manually Save (to your SaveFile) or, if you haven't turned off automatic Save such Save is automatically performed. But if you did turn-off automatic Save and don't manually Save, the next time you boot into your Pup, its list still contain the old instructions prior to your installing the pet or loading the SFS. Because, in fact, you didn't. When automatic-save is turned off, “installing” a pet only copies its files to RAM and the instructions regarding that new version only exist in RAM. Unless you instruct Pup to Save, when you shut down whatever was in RAM is cleared. [For instructions regarding how to turnoff Automatic Save, see http://murga-linux.com/puppy/viewtopic.php?p=662326#662326].

When you “pfind abiword –system” what Pup did was not only show you what files had actually been copied to RAM, but also --by following the pointers on the list-- showed you what files exist somewhere on your entire system.

File-managers work the same way as pfind. When you browse to a folder you can see not only the files actually in RAM, but also the files somewhere on your system which are being pointed to. When a file you see is selected to be copied or moved if it is not already in RAM it will be copied into RAM in order for the action to take place. When you create a symlink to a file what you are actually doing is editing the list to add a pointer to the actual file from the location of the symlink. And, when you “delete” a file it is only the pointer to the actual file which is effected.

At least, that the way I conceptualize what browsing to files –other than those in folders on storage media-- is showing me.

mikesLr
abiword.png
 Description   Files displayed by pFinding Abiword
 Filesize   143.38 KB
 Viewed   809 Time(s)

abiword.png


Last edited by mikeslr on Tue 09 Jul 2019, 19:33; edited 6 times in total
Back to top
View user's profile Send private message 
anikin

Joined: 10 May 2012
Posts: 1020

PostPosted: Sun 30 Aug 2015, 14:55    Post subject:  

First things first.
Before this thread goes too far, here is some *required* reading:
Filesystem Hierarchy Standard
Quote:
The Filesystem Hierarchy Standard (FHS) defines the directory structure and directory contents in Unix and Unix-like operating systems.
https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
https://wiki.linuxfoundation.org/en/FHS
For example, what's /opt for
http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
or what's the purpose of /usr/bin
http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s04.html
If the above refs and specs sound too dull and dry for you, have a look at this lovely piece:
http://tuxradar.com/content/take-linux-filesystem-tour/#null
and some nice visuals:
https://www.google.com/search?q=linux+file+system+hierarchy&biw=1024&bih=454&tbm=isch&imgil=yY0Chw_l87M2XM%253A%253BOxRcdMcPErez2M%253Bhttp%25253A%25252F%25252Fmyshortnotes.info%25252Flinux-filesystem-hierarchy%25252F&source=iu&pf=m&fir=yY0Chw_l87M2XM%253A%252COxRcdMcPErez2M%252C_&usg=__LYMEVYNo9qcP3Zy8vlEjz3QQJc8%3D#imgrc=yY0Chw_l87M2XM%3A&usg=__LYMEVYNo9qcP3Zy8vlEjz3QQJc8%3D
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2647

PostPosted: Sun 30 Aug 2015, 15:09    Post subject:  

Yeah, there's no way to fathom what puppy is doing without comparing it with any 'standard' distro. And really, one must understand the distro as it is installed -and understand the installer (because every installer is a mini-LiveCD.

It is, quite correctly, all done with smoke and mirrors, now known as links, union-mounts, loop devices, layers & Co. These days, distros like fedora actually use device-mapper to create COW systems which combine read-only and writable devices, file-systems, mounts, images and archives.
Back to top
View user's profile Send private message 
mikeslr


Joined: 16 Jun 2008
Posts: 3415
Location: 500 seconds from Sol

PostPosted: Sun 30 Aug 2015, 18:03    Post subject: The map is not the territory  

Thanks anikin for stepping in. I especially think your link to illustrations of the Linux File system will be helpful to newbies –both new and, like myself, perpetual-- trying to understand what's going on.

Hopefully, amigo, I made it clear that I was not trying to set out how Puppy actually works; but only how I visualize it working on those occasions when I have an interest in somehow changing an already existing Pup.

When I first started using Puppy, it was pretty much “a black box”, not unlike a battery. Attach one wire here, another there and voila! Then there was light. Very Happy

As a newbie, all I might want to do is install a pet or load an SFS, the equivalent of attaching the wires from a battery to some other instrument, a radio, a fan. It didn't really matter to me what was in the black box. It could just as well have been a genie which was aroused by fiddling with wires as a combination of materials and chemicals generating an electric current when, by attaching an appliance, a circuit is closed creating a bridge between the anode and the cathode.

If I ever want to create a functional black box of my own, it would be important that I look inside the black box. I suspect I'd have a different outcome whether I read documents about electronics, gathered schematics and go to an electronics store, or read documents about magic, gathered amulets and wandered in the desert.

I'll probably never build my own Pup, at least not from scratch/woof. But from time to time I take a run at cobbling together pets and/or debs to build an SFS. Having an idea of “what's in the box” and “what makes it tick” and especially, “what could have gone wrong” seems prudent. For example, my current working hypothesis is that the reason the Openshot pet installable into Tahrpup using its quickpet fails to run if converted to an SFS is that Openshot depends on the python version included in the pet. When installed the python files will be in the SaveFile which has priority. When Openshot is converted to an SFS, “it's” python files, outside the SaveFile, may not have priority. Well, it's just a theory.

As is what's really in the black box. For all I know, there really is just a genie – one that responds to such magic words as anode and cathode.

The map is not the territory.

mikesLr
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [7 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Misc
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1239s ][ Queries: 13 (0.0088s) ][ GZIP on ]