64bit PCs and PAE in Puppy World - Facts versus Myths

What works, and doesn't, for you. Be specific, and please include Puppy version.
Message
Author
User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

Re: Fact

#21 Post by Q5sys »

Jim1911 wrote:My experience with Slacko 5.6.
Thanks for sharing!
If you get time and are bored one day, could you run a full hardinfo test and post the reports here? It'd be nice to get more of a baseline for pae/non-pae systems.
Sadly only a few members like yourself have published test results.

If I get time this weekend Ill test Slacko 5.7 pae/non-pae across 4 different machines and report the results.

gcmartin

#22 Post by gcmartin »

Q5sys, your hard-on for PAE is not supported by the industry evidence. You keep purporting that this is some sort of MYth! Why do you continue to belabor this. Are you so blinded in your belief that you'll go to any extreme.

Sorry. whether you or I like it or not, PAE as a hardware built-in feature of the CPU has been in the build of over 99% of all 32bit systems built since 1995.

What's your beef! You upset about something else? You want the community to stop? Or what is it you are trying to assist the community with, that provides useful community guidance?

The preponderance of evidence is published by several authorities all over the internet. And, internal reports (that I sure no one lets you see) have shown that PAE provides comparable performances without degradation harm to the OSes.

It works, OK. GET IT! It works. Get it.

Unless you can provide good guidance on its use, stop flapping. (But, I sure you don't get.)

Maybe you can go up against the Industry Giants to show that the whole industry has lied to you.

But, until you do show the industry this, you are just flapping at me ... again.

Edit: One additional thing. It appears your REALLY dont know how the OSes work on use of accessed data as you "sound off" as an authority on system performance. WOW! Shame on you, my friend. I am inclined to believe Intel and AMD who build and test this, versus your emotional perceptions.

If you dont want to use it, that your choice.

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#23 Post by mavrothal »

gcmartin wrote: It works, OK.
No one can argue that PAE does not work (but in some Pentium M) or that is not a hardware feature.
However, no one can argue that does not tax the system either.
It is also true that 32bit systems with more that 3MB memory are really rare.
So in real life PAE is useful for people with 64bit machines that for some mysterious reason want to use 32bit OSs. Is their right, but taxing older 32bit machines (a major "market" for puppy) by default, to satisfy them would be unreasonable.
If a pupplet developer wants to built a PAE version, fine, but a 32bit puppy with PAE kernel only, is really out of place.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#24 Post by Q5sys »

gcmartin wrote:Sorry. whether you or I like it or not, PAE as a hardware built-in feature of the CPU has been in the build of over 99% of all 32bit systems built since 1995.
You know what else has been around since 1995? i386 technology. Doesn't mean we should be using it. Just like i386, PAE needs to be left it the wastebin of history.
gcmartin wrote:The preponderance of evidence is published by several authorities all over the internet. And, internal reports (that I sure no one lets you see) have shown that PAE provides comparable performances without degradation harm to the OSes.
You have failed at ANY point to provide evidence for your claims. Others have provided verifiable sources. You continue to make claims with no supporting evidence.
And whats this... internal reports no one is letting me see. What this community has secret reports that only a select few people get to see. LMAO. That's got to be the most outragous claim I've ever heard on here.

gcmartin wrote:Maybe you can go up against the Industry Giants to show that the whole industry has lied to you.
Well lets see what the Industry giants say, shall we?

Linus Torvalds wrote:And dammit, in this age and date when almost everybody has a gigabyte of RAM in any new machine, anybody who still thinks that "not that many people need 64-bits" is simply not aware of what he's speaking of.
Linus Torvalds wrote:anybody who still doesn't get why 64 bits is a requirement should just shut up rather than make a total fool of himself.
Source = http://www.realworldtech.com/forum/?thr ... stid=76973


Linus Torvalds wrote:Once you start having to access physical memory through some kind of HIGHMEM.SYS window (which is what PAE is), and cannot map it all, you can no longer keep normal pointers to such memory around. Instead, you are basically using a really ugly and strange segmented architecture, where you keep some kind of indirect pointer, and every time before you use it, you have to map it into the virtual address space, access it, and then unmap it again.

So performance plummets, and the code actually gets really nasty too, so you don't actually use the high memory for any random data, you only use it for special stuff. As an example, you'd use it for disk caching (that's what 90% of all HIGHMEM.SYS usage was too - a lot of people just set it all - or at least a big chunk of it - aside as a harddisk cache, because so few programs could use it very well for anything else).
Linus Torvalds wrote:And yes, there were serious problems. In theory, you can have 64GB of RAM with Linux on a PAE x86 box. In practice, it seldom worked very well past the 4GB mark, so PAE itself was almost totally useless. The reason was that once you had more than 4GB of memory, you usually had filled out a large chunk of the easily accessible memory with just all the data structures to keep track of the rest of memory (that's exaggerated, but it's not entirely off).
Source = http://www.realworldtech.com/forum/?thr ... stid=76980

gcmartin wrote:Edit: One additional thing. It appears your REALLY dont know how the OSes work on use of accessed data as you "sound off" as an authority on system performance. WOW! Shame on you, my friend. I am inclined to believe Intel and AMD who build and test this, versus your emotional perceptions.
Well lets see what AMD has to say about this, shall we?
AMD wrote:Benchmarks

First we picked some real world benchmarks for our 32-bit vs. 64-bit comparisons. Oggenc, Mencoder and Povray as well as some compilation tests. Furthermore micro benchmarks were used to show specific performance differences for syscalls and 64-bit arithmetics.

We set up three system configurations – a 32-bit installation, a 64-bit installation and a combination of 32-bit installation with 64-bit kernel to challenge the compat layer. All tests were performed on a dual-core AMD-K8™ processor with 1 GB RAM.

The tests showed that the penalty of using the compat layer instead of running your 32-bit application on a native 32-bit kernel is about 1-2 percent. So it is almost negligible.

64-bit took the lead in the media encoding tests. Our Povray and Mencoder benchmarks took about 5% less time in the 64-bit case, Oggenc even 25%. Just C-compilation tests showed a performance advantage of 5% to 8% for 32-bit versus 64-bit.

Native arithmetic performance (64-bit data types used in 64-bit software vs. 32-bit data types used in 32-bit) showed a gain of 10% for the 64-bit case. Using 64-bit data types on 32-bit and 64-bit in the arithmetic performance test showed that 64-bit is more than twice as fast as 32-bit.
Source = http://developer.amd.com/community/blog ... bit-linux/



Dont know what 'experts' you are talking about and what 'industry' you are keep talking about. The IT industry and experts have been VERY clear about the performance benefits of X64 over PAE even with limits below 4GB ram.

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#25 Post by Q5sys »

mavrothal wrote:
gcmartin wrote: It works, OK.
No one can argue that PAE does not work (but in some Pentium M) or that is not a hardware feature.
However, no one can argue that does not tax the system either.
It is also true that 32bit systems with more that 3MB memory are really rare.
So in real life PAE is useful for people with 64bit machines that for some mysterious reason want to use 32bit OSs. Is their right, but taxing older 32bit machines (a major "market" for puppy) by default, to satisfy them would be unreasonable.
If a pupplet developer wants to built a PAE version, fine, but a 32bit puppy with PAE kernel only, is really out of place.
But that seems to be the very issue at hand, the performance hit. But some people refuse to believe this in face of overwhelming evidence.

3rd party tests:
phoronix wrote:It should be to no surprise, for most computational workloads the 64-bit version of Ubuntu 12.04 LTS is much faster than the 32-bit and 32-bit kernel with PAE support versions. The performance advantage of 64-bit over 32-bit Ubuntu is clear. If you are still running the 32-bit version on 64-bit capable hardware you should really consider switching with the "Precise Pangolin" now that the 64-bit Flash is on par with the 32-bit version, OpenJDK works well on 64-bit, etc.
Source = http://www.phoronix.com/scan.php?page=a ... 3264&num=1
phoronix wrote:It is not new or surprising that the x86_64 Ubuntu is much faster than i686 Ubuntu on supported hardware, but it is somewhat surprising that Canonical continues to push the 32-bit version as the "recommended" version of Ubuntu from their web-site, etc.
Source = http://www.phoronix.com/scan.php?page=a ... ae64&num=1
phoronix wrote:To no surprise compared to our 32-bit vs. 64-bit benchmarking over nearly the past decade on Phoronix, the 64-bit version of Ubuntu generally delivers superior performance over 32-bit Ubuntu. This strong 64-bit performance is even when using an old Intel Core 2 Duo system with just 1GB of memory. The reasons for avoiding 64-bit Linux are generally moot these days with Flash, Java, and other applications now running just fine on x86_64 distributions. The state of multi-lib on Ubuntu and other Linux distributions is also in good standing.
Source = http://www.phoronix.com/scan.php?page=a ... 1304&num=1


But lets focus JUST on the memory issue. How much of a hit is there? Well I decided to test that out tonight on my machine. Using a vanilla install of LHP using an old 3.8.7 kernel, and the fresh new Slacko 5.6 with the 3.10.5 kernel. Now the 3.10.5 kernel has some improvements over the 3.8.7, but its the closest test I could do. Both systems are made from Slackware 14.0; so difference in binaries can be discounted.

The test I ran was rather simple. Boot the system. It has 32gb, so I created 14 2GB tmpfs drives. I then wrote to each one of those drives and then did file comparisons between them to check the memory access (read/write) bandwidth.

64Bit bandwidth was a consistant 2.4 to 2.5 GB/s.
PAE bandwidth was a consistant 1.9 to 2.0 GB/s

That's a 20% memory bandwidth reduction due to PAE.

When memory is fully utilized, reading a 2gb file from memory - PAE was 33% slower which resulted in a full second penaty


Here are the links. (complete with my typos). People can use the same commands to run tests on their systems.

64-Bit:
http://q5sys.info/pae/memtests/64-2.png
http://q5sys.info/pae/memtests/64-3.png
http://q5sys.info/pae/memtests/64-4.png
http://q5sys.info/pae/memtests/64-5.png
http://q5sys.info/pae/memtests/64-6.png
http://q5sys.info/pae/memtests/64-7.png

PAE:
http://q5sys.info/pae/memtests/pae-1.png
http://q5sys.info/pae/memtests/pae-2.png
http://q5sys.info/pae/memtests/pae-3.png
http://q5sys.info/pae/memtests/pae-4.png
http://q5sys.info/pae/memtests/pae-5.png
http://q5sys.info/pae/memtests/pae-6.png
http://q5sys.info/pae/memtests/pae-7.png

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#26 Post by James C »

http://askubuntu.com/questions/60155/do ... 0183#60183
PAE does not improve speed but gives you the chance to use more than 4GB of memory. The performance impact is very little actually and you will only "feel it" if you start using the memory above the 4GB limit. Basically if you have a 32Bit PAE Enabled system with 8GB RAM the performance impact will only come AFTER an application starts using the memory above de 4GB (This is the one from 4GB to 8GB). But the impact is very little. You will not notice it. What you will notice is 4GB of more ram.
Also for a system with 4GB of ram is not needed or recommended to use PAE since you are not going over the limit which is the 4GB of ram (which is a 2 by the power of 32 in bytes)

For videos like flash games, flash youtube videos, etc.. you won't notice anything different from a 32Bit with PAE or a 32bit without PAE. It will be the same performance.
Now for the "Should I buy or not buy the laptop with 8GB or ram". Well if you are like me on my home that the things i use it are to listen to music, see movies in HD quality, download stuff, open like 10 Libreoffice documents to work on, open Firefox and Chrome to work, etc.. then you do not need 8GB of ram. With 4GB of Ram you should be not just fine but happy (since you did not overspend on 4 more GB of ram to not use them).

Now if you are like me on my work place and you like to compile stuff, configure servers, render images and videos, etc.. Those are some heavy usage cases. So you need more ram for them to operate faster. Rendering something with 4GB is not the same as 8GB. Same for compiling (with the appropiate flags) and for the rest of the intensive cpu/memory apps.

But for the normal work, video watching, typical downloader and common user 4GB is enough.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#27 Post by James C »

http://people.redhat.com/nmurray/RHEL-2 ... epaper.pdf
The performance impact is highly workload dependent, but on a fairly typical kernel
compile, the PAE penalty works out to be around a 1% performance hit on Red
Hat’s test boxes. Testing with various other workload mixes has given performance
hits ranging from 0% to 10%.

User avatar
Q5sys
Posts: 1105
Joined: Thu 11 Dec 2008, 19:49
Contact:

#28 Post by Q5sys »

James C wrote:http://people.redhat.com/nmurray/RHEL-2 ... epaper.pdf
The performance impact is highly workload dependent, but on a fairly typical kernel
compile, the PAE penalty works out to be around a 1% performance hit on Red
Hat’s test boxes. Testing with various other workload mixes has given performance
hits ranging from 0% to 10%.
Ive been looking for that report for a while now and could never find it. Thanks!

Really, this issue is two fold.

First you have the PAE penalty, which only affects memory bandwidth. This only affects a system when you are doing memory read/write intensive tasks
Second you have the additional Instructions that come with a x86_64 CPU.

In most real world scenarios when you are browsing the web, watching a movie, etc, you wont notice much from a PAE kernel. Once you start getting into more active memory access from photo editing, compiling, or other more intensive workloads you will.

But honestly there's more of a performance penalty between the i686 vs x86_64 than there is with PAE/NON-PAE on a modern system.
As I said before, without the extra instructions that come along with x86_64, you're effectively running your Intel I5, or Core Duo like its an old P3 with a much higher clock cycle.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#29 Post by James C »

http://linux-memhotadd.sourceforge.net/hotadd.html
Addressing physical memory above 4 GB on 32-bit Intel Pentium processor needs a Linux kernel running with PAE mode enabled. To enable the PAE mode, the kernel needs to be recompiled with the CONFIG_HIGHMEM and CONFIG_HIGHMEM64G flags enabled. A PAE-enabled kernel uses three level page tables for VM address translation instead of two level page tables used by a non-PAE-enabled kernel. This can cause some performance impact that can be observed by running a benchmark suite that stresses the VM subsystem.
The benchmark suite chosen to highlight this performance impact is UnixBench available at http://www.tux.org/pub/tux/benchmarks/System/unixbench/ .

The table illustrates the difference in performance between a non-PAE kernel and a PAE-enabled kernel. The specifications of the test system are as follows:

Processor: 2-way SMP Pentium-III 1Gz
Memory: 1 GB RAM
Model: Compaq ProLiant DL380
Distribution: RedHat 7.1
Kernel: 2.4.13


Test Name

Code: Select all

Non-PAE Kernel PAE-Kernel

Dhrystone 2	180.6	180.5
Double-precision Whetstone	97.7	97.6
Execl Throughput	274.8	250.1
File Copy (1024 bufsize)	328.3	318.9
File Copy (256 bufsize)	373.4	366.8
File Copy (4096 bufsize)	295.2	292.1
Pipe Throughput	397.8	387.4
Process Creation	391.5	300.7
Shell Scripts (8 concurrent)	610.2	576.8
System Call Overhead	311.5	292.6
FINAL SCORE	296.2	280.0
From the final score, there is a performance degradation of 4.1% (the usual range is 3-6%). It can also be observed that the main difference stems from "Process Creation", which is quite worse with PAE, because the 'density' of the page-tables is half of that of non-PAE page tables (i.e, twice as much has to be copied).
Last edited by James C on Fri 23 Aug 2013, 05:49, edited 1 time in total.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#30 Post by James C »

http://wiki.novell.com/index.php/Mem...#How_PAE_works
Dynamically changing the page tables to map memory into the 4GB address space
In this case, the application reserves a certain address range within its 4GB address space, and then as needed maps different memory pages to that address space. This method has an important performance penalty because not only does it add overhead to change the paging tables each time, but also the TLB (translation lookaside buffer, a special fast cache for paging tables) needs to be flushed each time you change the page table. This is the method similar to what we had in the old DOS days with EMS memory.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#31 Post by James C »

http://www.linux-magazine.com/Issues/20 ... ernel-News

Linus Torvalds wrote:
PAE is 'make it barely work'. The whole concept is fundamentally flawed, and anybody who runs a 32-bit kernel with 16GB of RAM doesn't even understand *how* flawed and stupid that is.

Don't do it. Upgrade to 64-bit, or live with the fact that IO performance will suck. The fact that it happened to work better under your particular load with one particular IO size is entirely just 'random noise'.

Yeah, the difference between 'we can cache it' and 'we have to do IO' is huge. With a 32-bit kernel, we do IO much earlier now, just to avoid some really nasty situations. That makes you go from the 'can sit in the cache' to the 'do lots of IO' situation. Tough.

Seriously, you can compile yourself a 64-bit kernel and continue to use your 32-bit user-land. And you can complain to whatever distro you used that it didn't do that in the first place. But we're not going to bother with trying to tune PAE for some particular load. It's just not worth it to .anybody

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#32 Post by p310don »

I'm glad to see some actual facts being fleshed out in this thread. Reading through the past 4 or 5 posts, the conclusion is,32bit PAE vs 32bit Non PAE is barely noticeable. 64bit is better.

The problem to me seems to be not whether Puppy uses PAE or not for modern hardware. It is that puppy isn't developing a mainstream 64bit version.

The reason I would still recommend PAE on modern hardware when using puppy is that almost all the apps / pets / programs are compiled for 32bit, and thus they will work, and your system will use all the memory. Even if it is horribly.

PAE is a perfect solution for the meantime to use on modern hardware, and use all of the puppy stuff we have.

64bit is what Puppy should be aiming to deliver, build on Fatdog and LHP and really make Puppy a modern distro for use on modern hardware.

I'm not a builder / compiler / anything good, just a user so I can't help build anything. But, my opinion, Puppy 6.0 should be 32 and 64bit.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#33 Post by jamesbond »

James C wrote: Linus Torvalds wrote:
...
Seriously, you can compile yourself a 64-bit kernel and continue to use your 32-bit user-land. ....
Ah yes, time for shameless plug for my wiki: http://jamesbond3142.no-ip.org/wiki/wik ... koInFatdog.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

p310don
Posts: 1492
Joined: Tue 19 May 2009, 23:11
Location: Brisbane, Australia

#34 Post by p310don »

jamesbond - nice shameless plug. Most of it goes beyond my skill set.

This kernel solution might be similar to yours? I don't really know, but I thought it was promising when announcer announced it...

http://www.murga-linux.com/puppy/viewtopic.php?t=74884

Would be good to see a puppy built with a kernel like that

gcmartin

#35 Post by gcmartin »

Thanks @JamesBond for your post.

Most everyone knows that I have been a user of 64bit FATDOG since forever. And that I have also been a user of LightHouse64 since its beginning. I run both of these ONLY on my 64bit PCs. (I don't bother with running 32bit OS on them except on very rare occasions for testing something someone has observed.)

That being said, JamesBond shows us how to use FATDOG to run a 32bit OS (Slacko is an example), Simultaneously. You dont reboot ... you just start the 32bit OS and you have both systems on your fingertips.

Also, JamesBond and Kirk have also assisted with adding a subsystem (available thru the FATDOG repos) either, VBox and KVM (which I have tested extensively), which also allow one to run multiple simultaneous OSes (32bit or 64bit) within a single FATDOG host system.

The functionality is enormous, and extends a base system's operations for so many additional things we can imagine.

The OSes for 64bit systems are designed to fully take advantage of your hardware and exploit that hardware for user advantage without any existing physical RAM limitations.

This the the primary reason I have and will continue to use plus recommend 64bit OSes on 64bit PCs.

If you have a 64bit PC you will find lots of comfort booting FATDOG or its fuller software complement, LIghtHouse64.

And, if you are a current user of either, the link provided by @Jamesbond is a good one to allow you to experience both the 64bit OS; and a 32bit OS you start as well, simultaneously.

Here to help

Post Reply