Release cached memory manually (with user input)

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

Release cached memory manually (with user input)

#1 Post by nic007 »

This may be useful to those running fully in RAM (and especially those not using swap). I have 2GB of RAM and run without swapfile. I've observed that watching HD videos for an extended period on my machine results in an inevitable system crash and was wondering why because VLC was mostly the only application in use. Somehow this must have something to do with system memory control. I noticed that the crash generally occurs (or is likely to occur) when all the "free" memory is cached. The biggest chunk of cached memory is actually still supposed to be available to the system and should not compromise the stability of the system. However, this does not seem to be the case in my experience using Puppy. Since clearing the cached memory in regular intervals, my system has been more stable (haven't had a crash since, touch wood). There are two ways to flush the cached memory manually:
1) Using this command:

Code: Select all

echo 3 > /proc/sys/vm/drop_caches
I run this in intervals of 30 minutes (using Pschedule) when watching very high bitrate HD videos.
2) the second method is to simply unmount the drive you were reading/writing from, memory will be flushed. This method can obviously only be used if the drive is not in use.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#2 Post by musher0 »

Hi nic007.

Good reminder, this. Thanks.

I use the < sync > command before so that any data still in transit is sent to the
destination before I start "vacuuming" the RAM, like this:

Code: Select all

sync;echo 3 > /proc/sys/vm/drop_caches;wait
Also with < wait > at the end, to make sure my "vacuuming" is over before bash
processes another command.

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#3 Post by nic007 »

Hi, yes aware of the sync command but as far as I know the system runs that regularly anyway. I mean, every second script has sync in it. Also, the cached memory is not flushed completely so wonder if sync really is necessary. I have previously tried sync && ...... to run as "one command" but it doesn't work that way when using Pschedule. I'll try the way you suggested, thanks.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#4 Post by greengeek »

Hi Nic - two questions:

- how can I work out the current state of cache? Is there some way to know if it is choked?

- What form of system crash were you experiencing? Did you experience the personal storage icon turning red?

B.K. Johnson
Posts: 807
Joined: Mon 12 Oct 2009, 17:11

#5 Post by B.K. Johnson »

Hello nic007

Code: Select all

echo 3 > /proc/sys/vm/drop_caches
I want to use this, but...
  • I don't understand what it does [I have my newbie dunce cap on :lol: ]
    Despite not getting an error when it's run, how do I know it is working: cache used before and after
    My Firefox cache wasn't emptied when run
[color=blue]B.K. Johnson
tahrpup-6.0.5 PAE (upgraded from 6.0 =>6.0.2=>6.0.3=>6.0.5 via quickpet/PPM=Not installed); slacko-5.7 occasionally. Frugal install, pupsave file, multi OS flashdrive, FAT32 , SYSLINUX boot, CPU-Dual E2140, 4GB RAM[/color]

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#6 Post by nic007 »

greengeek wrote:Hi Nic - two questions:

- how can I work out the current state of cache? Is there some way to know if it is choked?

- What form of system crash were you experiencing? Did you experience the personal storage icon turning red?
Hi, greengeek. This explains it well: https://www.linuxatemyram.com/ Use the free command in terminal to check the status. The important thing is that the total free memory as reported, could actually be less than the actual free memory available. Evenso, in my experience, the indication of very low total free memory alone could be an indication of an imminent crash.
Crash as in freeze up, computer needs to be rebooted. Instability of software and drivers can of course also play a role. My conclusion is that under normal use the system seems to respond adequately and it should not be necessary to flush the memory cache at any stage. Under heavy load for a prolonged period, flushing the cache every now and then could be useful and prevent an imminent crash (it does for me).
I don't have a savefile. The personal storage metre almost always report 1GB free.

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#7 Post by nic007 »

B.K. Johnson wrote:Hello nic007

Code: Select all

echo 3 > /proc/sys/vm/drop_caches
I want to use this, but...
  • I don't understand what it does [I have my newbie dunce cap on :lol: ]
    Despite not getting an error when it's run, how do I know it is working: cache used before and after
    My Firefox cache wasn't emptied when run
Hi, B.K. Johnson. This relates to the system memory cache (not your browser cache). Run the

Code: Select all

free
command in terminal before and after you have run the command to flush the cache and you will see the amount of free memory increase. Also - read my answer to greengeek.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#8 Post by greengeek »

nic007 wrote:I don't have a savefile. The personal storage metre almost always report 1GB free.
Same here, no savefile. My storage meter also never used to suggest any problem either - except I have noticed lately it starts to run low (red) under two specific conditions:

1) When I am using mksquashfs to build large puppy sfs files

2) When i am using mhwaveedit to examine/cut/encode large audio files.

In both cases my system grinds to a very slow state and programs seem to fail with odd errors.

(I have 6GB swap on hard disk and this seems to match the total shown in my storage icon)

I don't know why this has started happening - except that it seems to be noticeable when I have been using Google Chrome - and I am starting to wonder if Chrome mucks up my cache and/or RAM somehow.

Chrome has also started mucking up my LibreOffice suite - causing "bus" errors - whatever they are.

I will try your cachedrop when it next happens and see if anything improves.

cheers!

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#9 Post by nic007 »

greengeek, correct. My storage space metre drops only when one do operations relating to writing to / which will involve using RAM if you work without a savefile. That metre has nothing to do with the system process of caching memory, I think. PS. I don't have a swapfile, could that large swapfile of yours be part of your problem?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#10 Post by greengeek »

nic007 wrote: That metre has nothing to do with the system process of caching memory I think
This is an area I do not understand well enough. I always thought that if we ran without savefile then the "personal storage" meter became irrelevant - but that is not the case. It is certainly reporting that something is getting clagged up under heavy resource usage.

Clearly that icon is also reflecting something to do with RAM usage and the ability of the system to move data (and I guess code) around the place effectively.

Cache handling must be part of the "code moving" process I would think?

However, after reading some of the internet threads concerning cache dropping it sounds as if:
- The Linux cache controller "looks ahead" to decide what code to move from HDD (or RAM?) to the faster cache and to store it there so it is readily available again later
- RAM control is more dynamic and one of its functions is to move code out of RAM back to disk - especially but not exclusively - to swap partition or swapfile. (so yes - the presence of a large swap partition could potentially be part of my particular problem)

Once the system gets busy I imagine both processes get a real workout. One of the threads I read seemed to suggest that Linux layering processes can trigger bugs with caching (on some Linux kernels) - so I wonder whether Puppy's layering might cause oddball results with both RAM and cache handling.

How does Puppy calculate "free storage" when it has multiple layers in play, and has those layers moving into and out of both cache and swap?? How does it decide when to change the storage icon from green to orange to red? If cache is full of unneeded code does that mean that the rest of the system resources get a harder workout than necessary?

Unless I have a developing hardware problem, then something in my software has been changing lately - which seems odd as my system boots to exactly the same state every time (No savefile and never any changes to the system sfs files). I really do suspect that Google Chrome is misallocating my system resources and somehow cluttering up the way my machine runs.

Can a software program have an impact on what gets cached or prioritised for storage in RAM or swap? I think so.

Maybe dumping cache will help me track down what is really going on

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#11 Post by nic007 »

Run that cache command in intervals of say 1 hour and see if you get more overall stability. As mentioned, my personal storage indicator seems to drop only when large data writing operations to / (memory being used if no savefile is in use) takes place... but will go up if I delete data that has been written to / . This follows that all operations involving data being written to / will stop if the indicator reaches zero...and software requiring data to be written to / continuously will probably fail at that point. Are you sure it is not the browser cache consuming all your memory?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#12 Post by greengeek »

nic007 wrote: Are you sure it is not the browser cache consuming all your memory?
No, I can't say I am sure of that. I don't really understand all the different caches. However, I don't think the browser can be using all my memory as I never see memory fill up when just browsing - even long sessions on youtube or downloading vids etc. That does not seem to reduce the "personal storage" figure seen by hovering over the icon (and there would have to be enormous browser cache to fill 6GB).

Unless it's something weird like the browser cache has first dibs on free RAM and any other applications are forced out to disk swapping.

I think I need to find out more about that personal storage icon and work out what it really is trying to tell me. I just can't understand how 6GB of swap can fill up just because I am doing a little browsing then manipulating audio files with mhwaveedit. 2GB Ram plus 6GB swap can't be disappearing.

I also need to know more about kernel cache as one of the messages I get sometimes is "unable to fork - cannot allocate memory" which according to a bit of googling seems to be trying to tell me about kernel memory running out. Don't know if that is to do with cache (of some kind) or main RAM, or Puppy's handling of layers.

TyroBGinner
Posts: 236
Joined: Wed 30 Mar 2016, 20:18

#13 Post by TyroBGinner »

Try a crontab directly and see if it works...it is easier as the "pschedule" program is confusing. Enter

Code: Select all

crontab -u root
Then enter

Code: Select all

 */30 * * * * sync && echo 3 > /proc/sys/vm/drop_caches && wait
in the text editor once it opens.
Win10 and lovin' it!

User avatar
bigpup
Posts: 13886
Joined: Sun 11 Oct 2009, 18:15
Location: S.C. USA

#14 Post by bigpup »

nic007 wrote:
That metre has nothing to do with the system process of caching memory I think
This is an area I do not understand well enough. I always thought that if we ran without savefile then the "personal storage" meter became irrelevant - but that is not the case. It is certainly reporting that something is getting clagged up under heavy resource usage.

Clearly that icon is also reflecting something to do with RAM usage and the ability of the system to move data (and I guess code) around the place effectively.
If you are running with no save at all.
The personal storage icon in the tray is reading what is being used as the storage while operating Puppy.
Usually that is about half the system RAM as a ram disk.
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected :shock:
YaPI(any iso installer)

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#15 Post by greengeek »

musher0 wrote:I use the < sync > command before so that any data still in transit is sent to the destination before I start "vacuuming" the RAM, like this:

Code: Select all

sync;echo 3 > /proc/sys/vm/drop_caches;wait
Also with < wait > at the end, to make sure my "vacuuming" is over before bash
processes another command.

BFN.
Hi Musher - is there any functional difference bwteen your syntax - using semicolon delimiter ";" and Tyrobeginners syntax using "&&" ?

User avatar
nic007
Posts: 3408
Joined: Sun 13 Nov 2011, 12:31
Location: Cradle of Humankind

#16 Post by nic007 »

Revisiting this thread. After thorough testing for months I do NOT recommend this command to free memory, it leads to unwanted machine freezes which can be avoided.

My scenario. I run in RAM without swapfile. I have 2GB of RAM and often watch videos for an extended period of time (on most occasions HD video). Generally everything goes well for a few hours before a freeze/lock up may occur. Using the drop caches command to free memory definitely increases the occurance of machine freezes. I have found that the most effective way to avoid any freezes is to occasionally unmount and re-mount the drive which you are playing videos from. Unmounting the drive automatically clears the memory caches but does not present the same problems afterwards as with using the drop caches command.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#17 Post by musher0 »

greengeek wrote:
musher0 wrote:I use the < sync > command before so that any data still in transit is sent to the destination before I start "vacuuming" the RAM, like this:

Code: Select all

sync;echo 3 > /proc/sys/vm/drop_caches;wait
Also with < wait > at the end, to make sure my "vacuuming" is over before bash
processes another command.

BFN.
Hi Musher - is there any functional difference bwteen your syntax - using semicolon delimiter ";" and Tyrobeginners syntax using "&&" ?
Double Ampersand
A double ampersand && in Bash means AND and can be used to separate
a list of commands to be run sequentially.

Commands separated by a double ampersand && are to be run
synchronously, with each one running only if the last did not fail (a fail is
interpreted as returning a non-zero return status).

This can be especially handy for when you have a couple of commands to
run, but you don’t want the second to be run if the first fails.
Source

Whereas the semi-colon is a typograpic convention replacing the line feed.
This convention enables the user to write several commands on the same
line. It is handy to write "one-liners", but it does not check the status of
the previous command.

Use Tyro's suggestion if you prefer. It can't hurt. He's probably using the
double ampersands because he is suggesting a cron job context.

As for me, I do not see the need for checking if you flush the caches
occasionally. So I use the semi-colon as separator.

Same for "Médor", the forum member who taught me this sequence of
commands for this purpose.

The form with semi-colons for this trick can also be seen in this tecmint
article and elsewhere.

IHTH.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Post Reply