Can 32-bit Puppy use >3 GB of RAM? (Yes, with PAE)

What features/apps/bugfixes needed in a future Puppy
Message
Author
gcmartin

RAM performance impact

#91 Post by gcmartin »

jamesbond wrote:Only performance impact is still dangling. ... BTW, I tested PAE Wary with shutdown - and it created a good savefile for me, hmm :?
I agree with JamesBond on this need. The other items are trivial (mostly non-existent on 4GB PCs).

2 Questions:
Can someone help me. Does anyone have a PAE ISO available for download that I can access for test? If so, post or send me a PM. I'm interested in testing the exact "Live" configurations you are using so that my measurements and reports can be consistent as much as possible with what's already been reported.
Edited: JamesBond's Wary. is here

Is there a hardinfo item which tells if PAE support is active in the running 32bit PUPs?.

Performance Expectations
On the performance, I want to share an experience: On a 2010 benchmark at a system center, we were not able to exercise the newer system for 2 reasons: it was so much faster that it was completing work faster than we could generate work for it? To overcome this, we had to create a timed-loop application to build enough of a load to make an attempt at comparison.
Then, on memory saturation, it was impossible due to the nature of the internet. 2 DS3 links inbound to the system failed saturate memory in such a way as to get a true measure.

Advice: We should not be too disappointed to find that the hardware PAE is just as effective at delivering pages as old frame management is. After all, the manufacturers have been doing this for over 10 years. (If fact I wouldn't be surprise if a manufacturer had only one model and doesn't tell us about it....since, in theory, a circuit design could be indistinguishable in real time.)

This community has...
JamesBond has produced a positive shockwave in the PUPs community. Thanks, enormously, for your work in helping all of us.

Everyone here has contributed as we have gone from problem identification, to measurements, to identifying the missing element(s), to producing a testbed, to testing, and hopefully (and I do mean hopefully) to 32bit ISO productions that take advantage of the finding. There are going to be many more users configuration that meet this memory model and thus, they will have a PUP solution available without thought...no matter what size they have from 512MB to 16GB.
Last edited by gcmartin on Sun 17 Apr 2011, 22:40, edited 3 times in total.

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#92 Post by p310don »

Can someone help me. Does anyone have a PAE ISO available for download that I can access for test? If so, post or send me a PM. I'm interested in testing the exact "Live" configurations you are using so that my measurements and reports can be consistent as much as possible with what's already been reported.
jamesbond made hugemem wary5.11, see here http://murga-linux.com/puppy/viewtopic. ... 7&start=60
Is there a hardinfo item which tells if PAE support is active in the running 32bit PUPs?.


I didn't see it, but it may be there under the kernel section. The obvious part is if you have higher than 3100ish meg of ram, you've got the PAE kernel running.

Hope to see your results soon gcmartin. I will be posting more soon also.

gcmartin

There IS a way for 32 bit PUPs to use >3.1 GB of RAM

#93 Post by gcmartin »

Running the original 32bit WARY is immaterial based upon the work already done by this community. And, I dont have a PC that has less than 512MB of memory. So, running the non-PAE mode WARY provides no benfit to this discussion.

The resolution by JamesBond is working as expected. I see no degradation. Here's my report of running Humungous WARY o a 4GB PC!

Code: Select all

-Computer-
Processor		: 2x AMD Athlon(tm) 64 X2 Dual Core Processor 4600+
Memory		: 4150MB (159MB used)
Operating System		: Puppy Linux 0.51
User Name		: root (root)
Date/Time		: Wed 13 Apr 2011 03:17:38 AM UTC
-Display-
Resolution		: 1280x1024 pixels
OpenGL Renderer		: Mesa GLX Indirect
X11 Vendor		: The X.Org Foundation
-Multimedia-
Audio Adapter		: HDA-Intel - HDA NVidia
Audio Adapter		: USB-Audio - C-Media USB Headphone Set  
-Input Devices-
 MosArt Optical Mouse
   USB Keyboard
   USB Keyboard
 C-Media USB Headphone Set
 Power Button
 Power Button
 PC Speaker
-Printers (CUPS)-
CUPS-PDF		: <i>Default</i>
-SCSI Disks-
ATA TOSHIBA MK5055GS
TSSTcorp CDDVDW SH-S223B
Optiarc DVD RW AD-7240S
Maxtor 6 L200P0
Here's my report of running Humungous WARY on a 1.5GB laptop.

Code: Select all

-Computer-
Processor		: 2x Genuine Intel(R) CPU           T2080  @ 1.73GHz
Memory		: 1545MB (176MB used)
Operating System		: Puppy Linux 0.51
User Name		: root (root)
Date/Time		: Wed 13 Apr 2011 03:11:57 AM UTC
-Display-
Resolution		: 1440x900 pixels
OpenGL Renderer		: Mesa GLX Indirect
X11 Vendor		: The X.Org Foundation
-Multimedia-
Audio Adapter		: HDA-Intel - HDA Intel
-Input Devices-
 AT Translated Set 2 keyboard
 SynPS/2 Synaptics TouchPad
 CHICONY HP Basic USB Keyboard
 Power Button
 Sleep Button
 Lid Switch
 Power Button
 PC Speaker
-Printers (CUPS)-
CUPS-PDF		: <i>Default</i>
-SCSI Disks-
Optiarc DVD RW AD-7560A
USB 2.0 USB Flash Drive
I am unable to generate any saturation workload on iethr of these PCs.

the HardInfo benchmark tools are about the only way to measurement data.

Let me know if here is other information you would like provvided. I am not seeing any performance solwing. In fact, it seems faster.. At the very worst, its about the same.

Hope this helps

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

Re: RAM performance impact

#94 Post by James C »

gcmartin wrote: Can someone help me. Does anyone have a PAE ISO available for download that I can access for test? If so, post or send me a PM. I'm interested in testing the exact "Live" configurations you are using so that my measurements and reports can be consistent as much as possible with what's already been reported..
Good luck on this one. As far as I know,almost all distros use a non-PAE kernel as default. The iso everyone downloads will have a non-PAE kernel, usually during an install the PAE kernel can be downloaded and installed if sufficient memory is detected. Naturally the kernel can be installed later as well, if the user adds ram for an example.
Anyway, I can't think of a distro that downloads a PAE kernel as default, except maybe Red Hat.

gcmartin

Re: RAM performance impact

#95 Post by gcmartin »

James C wrote:Good luck on this one. As far as I know,almost all distros use a non-PAE kernel as default. The iso everyone downloads will have a non-PAE kernel, usually during an install the PAE kernel can be downloaded and installed if sufficient memory is detected. Naturally the kernel can be installed later as well, if the user adds ram for an example.
Anyway, I can't think of a distro that downloads a PAE kernel as default, except maybe Red Hat.
JamesC by saying this, gives support to what we are doing.
  1. We are demonstrating that running a single kernel on PC built since PentiumPro does NOT produce measurable downside in Linux operations (especially in PCs with 512MB+ RAM)
  2. If Puppy distros, who understand this demonstration, adopts the PAE kernel as their standard, then there is less work of their part to have to ever address a RAM limitation as they go forward, no matter what CPU is used....as long as its PentiumPRO forward.
Puppy, truly, then becomes the RAM centric system it advertises to be no matter how much RAM in present.

Thanks for pointing that out. You are right to have helped us with that understanding.

Hope this helps.
P.S. In my eyes, this is historical in the Puppy world. Almost like discovering the "world is not flat"

gcmartin

#96 Post by gcmartin »

I think most will understand why., but, for me, this may be a more meaningful dispay in how the OS is seeing RAM management.

Below on 4GB PC running "JamesBond's WARY distro"

Code: Select all

# free
              total         used         free       shared      buffers
  Mem:      4150616       477208      3673408            0        61676
 Swap:      4233116            0      4233116
Total:      8383732       477208      7906524
Below on 1.5GB laptop running "JamesBond's WARY distro"

Code: Select all

# free
              total         used         free       shared      buffers
  Mem:      1545372       449524      1095848            0        59008
 Swap:      4216988            0      4216988
Total:      5762360       449524      5312836
Hope this helps

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#97 Post by Flash »

Give us a link to where we can download JamesBond's Wary. :)

gcmartin

A working real world distro for ALL PCs with 384MB+

#98 Post by gcmartin »

Flash wrote:Give us a link to where we can download JamesBond's Wary. :)
JamesBond's Wary. is here And I display his instructions.
jamesbond wrote: ... PAE-enabled Wary 511 is available at http://mydrive.ch, under puppy32 folder, the filename is wary-511-hugemem.iso. Username is dotpet@puppyshare, password is puppylinux. It only contains the basic drivers - I didn't compile the extra modem drivers etc. Enjoy.
EDIT: md5sum is 350990e629fdb0f58c0dd65bbf2c2f49
EDIT: For reference, everything inside that iso is Wary511. Kernel is 2.6.32.28 based on Barry's kernel-2.6.32.28.sfs. DOTconfig is based on Barry's DOTconfig-6FEB2011, I only change the CPU architecture to 586 and enable HIGHMEM64GB.
Hope this helps

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#99 Post by Flash »

Thanks! :)

gcmartin

Barry positions WARY platform support

#100 Post by gcmartin »

Saw this post from BarryK. Its a response that should help all PUP developers with an understanding.

My intention in this post, by me, is NOT to position BarryK, at all; BUT it is to help anyone here to understand what current 32bit PUPs are designed for: 2 year old machines WITHOUT PAE.

His blog statement should help everyone with some key points to consider:
Point: Until last year's drops in cost of memory, most PUP machine users could NOT afford to upgrade. Now, with cost to upgrade being so low, most of us are doing memory upgrades, foregoing new purchases. And, we are seeing a much larger number of PC users with RAM exceeding 3/4GB.
Point: There is a lack of understanding of what PAE is and how Linux is designed to take advantage of it.
Point: Further, (and for good reason), there is a reluctance to take advantage of this "old" hardware built-in FEATURE because there is no predictor of what will occur with applications running in a "PAE aware" PUP.
Point: Barry's blog on this is the first "marker" for any PUP PC system postition statement I have seen. And, it is important, to me, as it targets its audience with a specific support position for mainline PUPs. (I have been asking for some sign like this since joining the forum...what PC configuration is PUP, WARY, etc. designed for?). So if PUPs don't run on or take advantage of newer hardware, its because of this design point.
BTW, please, everyone understand that 15 year old PAE is NOT A NEW PC FEATURE. Things like SATA III or USB3 or 6-Core are NEW features.

Together, all of this should help us to understand these key points, until a new PC system position statement for PUPs is upgraded or emerges.

In the meantime, all of us, here, should understand the following:
  • FATDOG 64bit PUP is "an expanded, out of mainline PUP design".
  • Any new 64bit OS using PUP desktops are "an expanded, out of mainline PUP design".
  • JamesBond "Proof of Concept WARY" is "an expanded, out of mainline PUP design".
  • Any 32bit PUP forthcoming with PAE operational is "an expanded, out of mainline PUP design".
Even though PAE has been around since 1995 PentiumPro days, for both Intel and AMD, This, though, is NOT a new feature to the Linux kernel. But, the kernel must turn it on for Linux to use it. Any PAE aware kernel PUP is to be considered "an expanded, out of mainline PUP design" until such time in the future that it may become available via PPM or Woof builds.

Most importantly: BarryK did NOT say to "don't build a PUP that does not contain PAE". In fact, it appears that he encourages PAE builds in his blog. And, now that this thread has shown the PUP community that LARGE RAM PCs could be fully utilized with the proper PUP kernel built, we should expect and see, sometime in the future, this not being overlooked anymore. It should (and I hope will) become standard.

This has been one of the more exciting projects that I have worked on. Its has shown how this diverse community comes together to announce, understand, research, discuss, build a model and test. Lets not drop the ball and help Barry should he make a WARY kernel available to use. The Puppy community should be VERY PROUD of its ability to be responsive as has been demonstrated here.

Maybe some PUP member could start a thread, where the community can contribute to a PUP distro (WARY-4GB or WARY-for-all-PCs) with this thread's findings. Or maybe someone will present a guideline thread for 3.5GB+users who need to run a 32bit PUP. Or maybe a new "PUP525 for all PCs, new and old". Or .... (I don't possess the talents to achieve this, but I will help wherever possible....testing perhaps, etc.)

Thanks again, everyone!
Last edited by gcmartin on Wed 08 Jun 2011, 14:27, edited 1 time in total.

gcmartin

Everyone here will want to see this

#101 Post by gcmartin »

Wow ..... Look!
James C wrote:
pemasu wrote:I have now highmem and PAE compiled in. It means that 4 Gb ram is shown as that and Ice Puppy also supports ram more than 4 Gb. Side effects. It will use ram about 20 mb more than without it.
This probably will not be the last one. It is just for testing and for those who want Puppy to recognize 4 gb or more ram as it is.
Guess I'll need to stick more ram in the old P4 box. :)
Its found here, "32bit for ALL PCs since PentiumPro; no matter what RAM size"

Edited: @JamesBond has produced a Wary which works on all PC over last 10 years.
@Pemasu has produced a Luci (like Puppy 5.2) which works on all PCs over the last 10 years
Now its up to the others in the 32bit PUP community to take advantage of this technology "break-thru".

Let's help by testing this most current Puppy offering.
Hope this helps
Last edited by gcmartin on Thu 28 Apr 2011, 16:30, edited 1 time in total.

User avatar
duke93535
Posts: 194
Joined: Thu 05 May 2005, 16:45
Location: California , High Desert

#102 Post by duke93535 »

For us on newer hardware James Bond’s Wary makes the cut. We see no problems with it on an older backup computer, and that is 2GHz single core and lower memory (768 MB). Wary see our 6 core processor and the 4GB of memory now. We are using DSL and have no use for the older modems in the kernel.

Fatdog doesn’t make the cut because of the 64bit flashplayer (a work in progress) and no ability to use 32bit flashplayer. Fatdog also needed the xdg menus fixed by us. Kubuntu LTS 64 bit does have that ability to use 32bit flashplayer. Kirk, it seems must have not had any luck doing this
as he said he’d try a couple of times. We tried doing this, but it was over our heads. Lupu’s kernel doesn’t even see 6 cores. Notable difference in testing: The 32bit machine is using less that half the memory of the 64bit Kubuntus with about the same things install.

Computer
Processor 4x AMD Phenom(tm) II X4 945 Processor
Memory 4058MB (517MB used)
Operating System Ubuntu 10.04.2 LTS

Computer
Processor 4x AMD Phenom(tm) II X4 945 Processor
Memory 4151MB (93MB used)
Operating System Puppy Linux 0.51

Here is our 6 core.

Computer
Processor 6x AMD Phenom(tm) II X6 1055T Processor
Memory 4056MB (507MB used)
Operating System Ubuntu 10.04.2 LTS

Computer
Processor 6x AMD Phenom(tm) II X6 1055T Processor
Memory 4148MB (104MB used)
Operating System Puppy Linux 0.51

Here is backup 32bit computer.

Computer
Processor AMD Athlon(tm) XP 2400+
Memory 767MB (212MB used)
Operating System Ubuntu 10.04.2 LTS

Computer
Processor AMD Athlon(tm) XP 2400+
Memory 774MB (85MB used)
Operating System Puppy Linux 0.51

Thanks, jamesbond for the modified Wary.

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#103 Post by p310don »

duke93535, thanks for your input. Its comments like yours that are the reason for my creating this thread.

Thanks for testing jamesbond's excellent wary modification. If you have any issues with it, please let us know.

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#104 Post by p310don »

Just came across this testing from Phoronix.

Ubuntu 11.04: i686 vs. i686 PAE vs. x86_64

http://www.phoronix.com/scan.php?page=a ... ae64&num=1

With Puppies now coming with PAE and 64 bit as well as standard kernels there is much discussion of benefits / drawbacks.

Just some more info for knowledge on the topic. :)

gcmartin

Great Performance reports

#105 Post by gcmartin »

@P310Don, Thanks for this.

Its what this community has already reported in a little difference way.

It should provide a guiding light to all of the 32bit distro developers.

This report adds validity in the testing done here and with @Pemasu's forward thinking approach for Puppy kernels-ISOs.

The overall community has done a lot of work to show that there is little to no drawback or impact in PAE. So, in essence, for modern day needs as memory becomes more abundant in individual platforms, there is NO NEED TO CHANGE DISTRO when anyone exceeds 3.5GB RAM in 32bit systems....None! (Assuming you started with a PAE aware distro).

64bit platforms that are shown in that report only servers to show what was proven by Intel 8 years ago (AMD as well).

Hope this helps
Last edited by gcmartin on Wed 20 Jul 2011, 21:31, edited 1 time in total.

gcmartin

#106 Post by gcmartin »

In the prior post, I alluded to the reports that many have shared who have tested the various 32bit PAEs available to the Puppy community.

There is one things that is "overbearing" by the majority of EVERYONE who has tested. The general consensus is that there appears to be an improved desktop performance.

When running the benchmarks as I have, I have wrestled with a reasonable way to demonstrate desktop performance comparison. I have not found a reasonable way to do this that I feel many would agree is accurate. I don't trust my senses as being an accurate measure.

Further, in my tests using the Hardinfo reporting tool and its imbedded utilities, I have found that some of the reporting tools do NOT spread across the processor pool (Cores) as I would have expected, Thus, even those measurements I, personally, don't feel is a "truly" accurate measurement of the system's performance or potential..

Further, again, in my test, those utilities do not seem to address memory saturation in a way I would like to have seen it so that I would have an even better gauge on perfomance when the system is under a high RAM load with SWAP active.

But, if we can accept the assumption that in the 32bit comparison, it uses the processor pools very similarly(ie PAE vs non-PAE), then, it can be viewed as an accurate measure.

Hope this helps

P.S. The vendors (Intel/AMD) have the measurement tools and have done this measurements already. Understanding this, in OS systems development, these Processor hardware vendors make the processor-chipset advancements available to the OS vendors with the information necessary for exploitation to the user benefits.

Post Reply