Ram Full on bootup

Discuss anything specific to using Puppy on a multi-session disk
Post Reply
Message
Author
Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

Ram Full on bootup

#1 Post by Perkins »

I'm running a multisession live DVD on an older machine, and have run into an irritating issue. I started with 4.00, and went to 4.1 to see if this problem was fixed, but no luck.

This machine has 512MB of memory. This turns into about 200MB on the free memory indicator. This amount stays pretty well constant since most of what the machine does is web browsing. It fluctuates about four meg depending on the state of the browser cache.

However, as the saved sessions stack up, I am starting to get "RAM FULL" messages as it is loading folders from the disk on startup. If I save any large files, the message appears earlier in the process. If I delete them, it moves back down the list. Regardless, when the boot completes, I have 200MB of memory on the indicator. My guess would be that as it's loading files it's still loading old copies for some reason, even though it subsequently crunches it all down to a smaller size.

Plugging in a USB drive formatted as linux-swap increases the free space on the indicator, but has no effect whatsoever on how far it gets through the loading process before the ram fills up.

This is turning into a rather ugly problem since as the disk fills up it's failing to load configuration settings and such. Any tips on how to fix it would be appreciated.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#2 Post by mikeb »

Hmm your assessment sounds about right which results in more memory space is needed during boot than once fully running.

Suggestions

1) only save when you need to...the browser for example creates a large amount of fresh files everytime it runs which subsequently get saved and loaded again.

2)If you have decent internet speed set browser cache to 0...plus do you use email?..can you use imap in that case?

3) there is/was an /archive folder...the purpose was that any files put in here would be saved to the cd but not loaded into ram...ie still accessible through the mounted cd....good place for large data files.

regards

mike

User avatar
HairyWill
Posts: 2928
Joined: Fri 26 May 2006, 23:29
Location: Southampton, UK

#3 Post by HairyWill »

It may be that there is a bug in init that doesn't include the swap space in its ram calculations.

I suppose I ought to look at the code before making this suggestion, but I don't have time right now and want to get it written down.
Will
contribute: [url=http://www.puppylinux.org]community website[/url], [url=http://tinyurl.com/6c3nm6]screenshots[/url], [url=http://tinyurl.com/6j2gbz]puplets[/url], [url=http://tinyurl.com/57gykn]wiki[/url], [url=http://tinyurl.com/5dgr83]rss[/url]

User avatar
trapster
Posts: 2117
Joined: Mon 28 Nov 2005, 23:14
Location: Maine, USA
Contact:

#4 Post by trapster »

there is/was an /archive folder...the purpose was that any files put in here would be saved to the cd but not loaded into ram...ie still accessible through the mounted cd....good place for large data files.


Whooooa......is this still true???
Interesting, I will play with when I get home.
trapster
Maine, USA

Asus eeepc 1005HA PU1X-BK
Frugal install: Slacko
Currently using full install: DebianDog

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#5 Post by mikeb »

I forgot to ask which puppy version.....was something i noticed on 2.02...been a while since I multisessioned.

mike

User avatar
alienjeff
Posts: 2265
Joined: Sat 08 Jul 2006, 20:19
Location: Winsted, CT - USA

#6 Post by alienjeff »

Do you have a swap partition set up on a integral (not USB) hard drive? If so, what size?

I'll be curious to see what Flash has to say about this one. He's our resident Multisession Advocate.
[size=84][i]hangout:[/i] ##b0rked on irc.freenode.net
[i]diversion:[/i] [url]http://alienjeff.net[/url] - visit The Fringe
[i]quote:[/i] "The foundation of authority is based upon the consent of the people." - Thomas Hooker[/size]

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#7 Post by Perkins »

Nope, no swap on an internal disk. It was a temporary solution for my mother's XP machine when it died. I have since rebuilt it. (It's still XP since she uses programs that defy my attempts to make run or find substitutes for on Linux.)


I tried both 3.something and 4.something, both exhibited the same problems.

I suppose the thing to do would be to make it check the size of the data on the disk and size the initial ramdisk appropriately. I'm not facile with Puppy's internal workings, so I'm not sure if that's possible. True, it might slow it down, but booting from a multisession disk isn't exactly fast anyway...

One could put the initial size into a file, and then only recalculate if it gets filled up... Not sure how difficult that would be.

User avatar
alienjeff
Posts: 2265
Joined: Sat 08 Jul 2006, 20:19
Location: Winsted, CT - USA

#8 Post by alienjeff »

Put a swap partition on the HD and you should enjoy a more pleasant experience.
[size=84][i]hangout:[/i] ##b0rked on irc.freenode.net
[i]diversion:[/i] [url]http://alienjeff.net[/url] - visit The Fringe
[i]quote:[/i] "The foundation of authority is based upon the consent of the people." - Thomas Hooker[/size]

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#9 Post by Perkins »

Well, I'll keep that in mind. I usually end up using the multisession option though when I don't want to mess with the harddrive.

User avatar
Béèm
Posts: 11763
Joined: Wed 22 Nov 2006, 00:47
Location: Brussels IBM Thinkpad R40, 256MB, 20GB, WiFi ipw2100. Frugal Lin'N'Win

#10 Post by Béèm »

Perkins, I suppose you have knowledge of this document?
Time savers:
Find packages in a snap and install using Puppy Package Manager (Menu).
[url=http://puppylinux.org/wikka/HomePage]Consult Wikka[/url]
Use peppyy's [url=http://wellminded.com/puppy/pupsearch.html]puppysearch[/url]

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#11 Post by Perkins »

Yes, I've read it before. I just went through it again, and it kind of confirms my guess about what was happening. I'm thinking the pup_ro1 ramdisk is too small to hold all the saved files, so it skips some, and then when the pup_rw is created and layered on top of it it gives the same 200MB of free space. Meaning that I have 200MB of working space once the computer is up and running, but not necessarily that amount of permanent storage.

Does that seem to be a correct analysis? Or is there some other reason why I'd have a fairly constant amount of free space, but fail to load files at startup? I'm not terribly good at bash, so reading the startup script myself to see what's happening kind of makes my brain try to melt.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#12 Post by Flash »

Here is a way to "defragment" a multisession DVD that might help.

Assuming you can get your multisession Puppy DVD booted up, use Burniso2cd to burn a new DVD with the same iso (or a newer one, if you want to upgrade) then shut down leaving the newly burned DVD in the drive you booted from. Choosing to save when you shut down will transfer everything from the old DVD to the first session of the new DVD. Walla, you have combined all those sessions into one. The next time the new DVD boots, it will not have to load all the intermediate sessions that caused you trouble on the old DVD. This trick works particularly well with a DVD+RW; Burniso2cd will overwrite whatever is on a DVD+RW without having to blank it first. Of course, if you are using multisession Puppy for forensic reasons, you would start a new DVD+R instead of using a rewritable DVD.

Incidentally, have you noticed that when multisession Puppy boots up, the sessions are loaded starting with the last one saved on the DVD and working to the first? I don't understand how that works, but it obviously does.
[url=http://www.murga-linux.com/puppy/viewtopic.php?t=69321][color=blue]Puppy Help 101 - an interactive tutorial for Lupu 5.25[/color][/url]

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#13 Post by Perkins »

The time I tried moving to a new disc, it actually made the problem worse. I suppose I should clarify a bit with the results of some of the experimenting I did once I no longer needed the system to work.

The amount of data on the disc seems to actually be fine. It all fits into memory on bootup. However, if I add more than one or two MB to it, it bumps the last couple of sessions. If I then delete the extra data it will go back to working properly. So somehow I'm right on the edge of how much space it has on bootup. If I make a new disc with it over the size limit, then it has one big set of files it fails to load instead of a couple of little ones. So it's ok for the first session, but after that the whole thing craps out and dies.

The sessions, Flash, are loaded in reverse order because it's more efficient that way. You load the newest copy of any file first, and can then skip all the older copies. If you did it the other way, you'd have to overwrite all the old copies with newer ones until you get to the newest set, and then delete any files that were deleted. More complicated, and would take much more time since you would essentially have to read *all* the data on the disc instead of skipping anything which has already been superseded.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#14 Post by Flash »

Perkins wrote:The time I tried moving to a new disc, it actually made the problem worse. I suppose I should clarify a bit with the results of some of the experimenting I did once I no longer needed the system to work.

The amount of data on the disc seems to actually be fine. It all fits into memory on bootup. However, if I add more than one or two MB to it, it bumps the last couple of sessions. If I then delete the extra data it will go back to working properly. So somehow I'm right on the edge of how much space it has on bootup. If I make a new disc with it over the size limit, then it has one big set of files it fails to load instead of a couple of little ones. So it's ok for the first session, but after that the whole thing craps out and dies.
Hmm, I don't remember anything that I would describe like that.
The sessions, Flash, are loaded in reverse order because it's more efficient that way. You load the newest copy of any file first, and can then skip all the older copies. If you did it the other way, you'd have to overwrite all the old copies with newer ones until you get to the newest set, and then delete any files that were deleted. More complicated, and would take much more time since you would essentially have to read *all* the data on the disc instead of skipping anything which has already been superseded.
Well, the details still elude me, but that's nothing new. :)
It appears that every session is read from the DVD even if everything in one session was deleted in a later session. If you're going to read every session anyway, surely it takes much longer to read the sessions last-to-first than it would to read them from the DVD from the beginning. My DVD drive is capable of reading a DVD at over 10x, but I'm sure that is only if the read is continuous. Loading Puppy must take longer if the read head has to reposition itself to search for each session, than if the data were read from the DVD in one long continuous stream, and reconstructed in RAM on the fly. Maybe I'm wrong, but that's how it seems to me. :?
[url=http://www.murga-linux.com/puppy/viewtopic.php?t=69321][color=blue]Puppy Help 101 - an interactive tutorial for Lupu 5.25[/color][/url]

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#15 Post by Perkins »

Except that you don't have to read all the data. Just the file names. If it doesn't exist already, and there's no deletion marker, then you copy it. Otherwise you don't. Basically it's a "cp -u" (copy only if destination is older than source or does not exist), and since you're going last to first, whatever you have at the end is the latest version. First to last you can't skip any data at all, unless you read all the filenames and then decide which ones to copy, which would be needlessly complicated.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#16 Post by Flash »

Perhaps we're talking past each other. I agree that it would be a waste of time to load a large file from an early session, only to find that file was deleted in a later session. However, I've found that most sessions are pretty small (the sessions appear as directories on a mounted multisession Puppy DVD) so it seems to me that for most people a multisession DVD would boot faster if the sessions were read from the DVD in the natural order of first-to-last.

I don't know much about the details of how a DVD works. I know that the data on a DVD is recorded on a single track which begins at the innermost edge of the data area and extends in a continuous spiral to the outer edge of the data area, similar to the track on an old LP record. By design, a DVD drive can only read from or write data to that track in one direction. There is no physical reason I can think of why it couldn't read the track backward. The DVD drive designers probably just couldn't imagine why anyone would want to, if it occurred to them to wonder about it at all.

It takes the read head a while to synchronize (or whatever the technical term is) with the track before it can read the data. Given that a drive is only designed to read data from the spiral track in one direction, the read head must read the last session, then double back to a previous location on the track and re-synchronize before it can read the previous session, and so on until all the sessions are read. This offends my sense of order somehow.. I imagine that the seek-and-synchronize time takes such a long time that it might be faster to simply zip through the sessions on the track starting with the first one, and let the operating system, which is already loaded and operating, deal with straightening everything out in RAM. The drive starts out loading the main Puppy files from the beginning of the track anyway. Might as well just keep going while the drive is on a roll. That's all I'm trying to say.

Perkins
Posts: 62
Joined: Sun 25 Sep 2005, 05:45
Contact:

#17 Post by Perkins »

I have never disassembled one enough that I could watch the inner workings while it is reading, but listening to an optical drive, it sounds to me a lot like it seeks back and forth between the track's table of contents and the actual files while it is reading. Leastwise, I can hear the read-head skipping back and forth, even when doing a straight copy. So the fact that starting at the end would let you read *just* the table for the majority of the files probably adds more speed than reading in reverse order costs.

Now, if you were to tarball and/or compress the data in each folder so that it could be read as a single file, then it might make a difference... Might get you some extra storage space on your disk too... Except that then you *would* have to read all the data in every session just to pick out the bits you wanted, so I bet it would be slower.

But anyway, we're drifting a little off topic. I'm still curious if anyone has an explanation for why my multisession disk is behaving in this odd manner.

Post Reply