Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 24 Nov 2014, 10:19
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Suggestions
Can 32-bit Puppy use >3 GB of RAM? (Yes, with PAE)
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 4 of 8 [106 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8 Next
Author Message
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Tue 22 Mar 2011, 07:04    Post subject:  

Hehehe ... I compiled Wary kernel with HIGHMEM64G option also, and yes available memory went from 2.9GB to 3.9GB (this is on a 4GB machine).

Bad news though, enabling this option is enough to change the module layout signature, so existing modules won't load. You will also need to re-compile all the modules also (and then install that modules to pup.sfs and initrd.gz). I didn't do that, so I only got as far as the ramdisk shell Very Happy

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3972
Location: Kiel,Germany

PostPosted: Tue 22 Mar 2011, 12:24    Post subject:  

Quote:
Hehehe ... I compiled Wary kernel with HIGHMEM64G option also, and yes available memory went from 2.9GB to 3.9GB (this is on a 4GB machine).

You always have to compile all modules I think everytime you even make small changes to the kernel .config

May I ask about the architecture you did choose ?

Common seems to be the CONFIG_M586
or CONFIG_M686

but 4GB+ Machines might be worth _MCORE2 or _MATOM

I have no machine above 1GB and my testbox is a P4 with 448MB RAM and until now I can say that it is no performance difference in _M386, _M486, _M586, _M586TSC, _M686
and _PREEMPT or _PREEMPT_VOLUNTARY
Back to top
View user's profile Send private message Visit poster's website 
technosaurus


Joined: 18 May 2008
Posts: 4378

PostPosted: Tue 22 Mar 2011, 13:37    Post subject:  

the biggest difference in the architectures is the stack size. medium sized apps can fit in later architectures whereas only really small apps can fit on older ones. it makes a big difference if for instance you can tweak the kernel and compiler enough to fit your high load web server.
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Tue 22 Mar 2011, 16:56    Post subject: Barry produces the base 32bit kernel(s)  

For mainstream 32bit Pups, Barry produces the kernels. Have we taken the time to present this issue to Barry? Anyone? He may have consciously done this for a reason he feels passionate about. We won't understand unless he responds.

Hope this helps

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 732
Location: Brisbane, Australia

PostPosted: Tue 22 Mar 2011, 19:41    Post subject:  

I did pm Barry to reference him to this thread, but as yet have not heard back.

Gcmartin is absolutely right, this is something that Barry, or whoever is creating the puppy needs to do at the beginning, rather than as a patch, as shown by Jamesbond.

Perhaps this is something that should be considered when building puppy 6? Something for the future?
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Tue 22 Mar 2011, 21:48    Post subject:  

Barry won't take this lightly. The thing is, enabling PAE has performance impacts - and it will make the kernel actually consume more memory to hold additional page tables. So while this is good for systems with large memory pool, this isn't so great idea for 512MB systems (not to say it won't work with i486 - Barry's usual CPU target).

That being said, Barry is flexible enough to provide ISO with different kernels (retro, ide, etc) ... so perhaps he will consider releasing one with 64GB-enabled kernel.

@Karl, I compiled a few times, one with M586 and the other with MCORE2. I didn't get far enough to be able to test any performance difference, because I didn't recompile the modules. I'd take technosaurus' say on it. Oh, one more thing - if you're lucky, sometimes you can get away with modifying .config and reusing existing modules ... but apparently for this particular change, it won't work.

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Tue 22 Mar 2011, 23:42    Post subject: Reporting the findings should help every system builder  

I figured that Barry is wondering how to handle this dilemma.

There are several Puppy 32bit "litters" as I have seen. Each litter uses a base that is markedly different from other litters. (Excuse my Puppy vocabulary)

Some litters are "official" litters. Then, there are the unofficial ones. The official ones are the only ones I have tested (excepting LightHouse). And all official litters (and LightHouse) have this issue.

But, it seems that several "unofficial" litters addressed this issue (either by accident or on purpose). What may work to this discussion's advantage is to get all of the Pups where the problem does NOT exist on this thread so that all who would want to use memory above 3GB could select that unofficial PUP.

I am not sure how this thread could, in one place, track the haves from the have nots. But, so that this discussion is never revisited, there probably should be a location somewhere to make public this very important information for 32bit distro users.

There is one more item that should accompany that information for users. That is which kernel and what kernel parms are used with that distro. For example, Barry and Shinobar have released 4 versions of WARY 5.1.1. Each of these, BTW, have this problem that we are discussing here. If one of them decided to address this and maintain the functionality currently in 5.1.1, then we really would have even more confusion than already exist. Instead of 4 WARY's 5.1.1, we wold have 6 or 8 versions of the same distro...some with the problems others without the problem.

Finally, how can JameBond, Technosaurus, Karl Godt, ecube and all of the distro developers report in such a way as there is some consistency in what is being reported. What I mean is, "there does NOT appear to be any standard measure or measurement tool for what everyone is/has done. And how does what they've done impact capacity or performance"? So who is to say whether there are performance/capacity impacts without some way to validate whether there is truth to what is being reported? Don't get me wrong, I am not making any claim that there is distorted information being passed around. I do, though, understand that there is a need to have a "measure"!

There are some kernel approaches that some of the 32bit Puppy distro builders have employed. We should, though, know what they did where they resolve this specific issue....whether or not there is some perceived negative impact.

I am on record for having asked several times on several different threads why no-one is reported what PC platform(s) their distros are targeted for. An example of why this information would be needed is this very thread with the 4 WARY distros among others.

By us thinking about this, here, and contributing, we open the doors for the intelligent communications that we are witnessing on this thread. Let's make this as beneficial as we can for old and new Puppy users.

Hope this helps

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 732
Location: Brisbane, Australia

PostPosted: Wed 23 Mar 2011, 01:52    Post subject:  

Gcmartin, perhaps what you are alluding to, is a horses for courses situation, puppy for all, as we have now, puppy for old, as the waries are and puppy hi-tech for brand new PCs, and PCs in the future.

I know Barry's idea is to make puppy work on all the old stuff, but there are a lot of people who are starting to use linux as a viable alternative to M$ products on their new machines. As I said at the beginning of this thread, I suggested that a new user, with a new machine might be disappointed to learn that this distro does not appear to be taking full advantage of every drop of power in the machine. Then they might go to another distro that does appear to take advantage of all the hardware available.

I know its been suggested before to have a heavier puppy, perhaps one with all the prettiness and eyecandy thrown in. This would of course need to be made clear that it is suited to machines of a certain spec. Just like Window$ says on the box, to run win7, you need 1ghz cpu, 1 gig ram, 16gig hdd space etc (which with win7 would run like a dog, not a frisky puppy!!)

Jamesbond, if there is a performance hit by enabling the PAE kernel, do you know what it is? Say its 20%, but it allows you to fully utilise a machine that is 100% faster, you're still in front. This is exactly what gcmartin is talking about with measurement and benchmarking. Knowing what I have seen in some linux info, the performance hit may only be theoretical, or miniscule. I did read someone complaining that a kernel mod increased the load time by 125%, which equated to nearly half a second. Percentage wise, that's terrible, real world wise, would you notice?
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Wed 23 Mar 2011, 04:21    Post subject:  

p310don wrote:
Jamesbond, if there is a performance hit by enabling the PAE kernel, do you know what it is? Say its 20%, but it allows you to fully utilise a machine that is 100% faster, you're still in front. This is exactly what gcmartin is talking about with measurement and benchmarking. Knowing what I have seen in some linux info, the performance hit may only be theoretical, or miniscule. I did read someone complaining that a kernel mod increased the load time by 125%, which equated to nearly half a second. Percentage wise, that's terrible, real world wise, would you notice?

I wouldn't be able to quantify the performance hit, because, as gcmartin said above, I didn't do any kind of testing. And there is so many CPU models, the hit may be different from different models. But one thing is sure - there will be a hit - because if you look at the CPU specsheet of how the PAE is implemented, you'll see that more memory is required to to the memory management (this sounds stupid, but yes, you need memory to do memory management), and there is a increase of complexity in managing that memory under PAE.

That being said, on qualitative level, I agree with your assesment - if you have a relatively powerful machine with loads of RAM, the hit is probably be more than offset by the advantage you gain by being able to use that extra memory. But this isn't true for less powerful machine.

Now that being said - I read elsewhere that some mainline distros have started to ship with PAE-enabled kernel by default. Not because of they want to enable 64GB access (that's just a happy side benefit), but because they want to make use of the NX bit (for malware protection), and use of NX bit necessitates PAE kernel. Even though it causes worse performance on less-capable machines.

So yeah ... it's a trade-off Smile

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
myke


Joined: 15 Mar 2011
Posts: 102
Location: Québec

PostPosted: Wed 23 Mar 2011, 11:11    Post subject: Kernel versions
Subject description: Should be reported
 

Ideally, each official puppy and each puppy derivative should report the kernel version used and at least for the older versions, what extra drivers have been compiled. This would simplify the search for people like me with netbooks and such.

Frankly, it depends on the skill and interest levels of the developer for this to happen.

The only Puppy derivative that I noticed doing this is Fluppy. I knew before I downloaded Fluppy that it would work.

All it takes to obtain the kernel version is to do the command: "uname -r" (without the quotes) in a terminal.

myke

_________________
AA1 D255E-keucr slacko 5.3;luci;mijnpup; tw-os; with:Emacs,gawk,noteboxmismanager,treesheets, freeplane, libreoffice, tkoutline, Sigil, calibre, calendar. magic&Noteliner(wine), kamas (DOS)
Back to top
View user's profile Send private message 
Karl Godt


Joined: 20 Jun 2010
Posts: 3972
Location: Kiel,Germany

PostPosted: Wed 23 Mar 2011, 17:16    Post subject:  

First impression for CONFIG_MATOM 64GB on my testbox is very normal
no problems so far execpt the usual it87module<>acpi , enabling the bootlogo stopped showing debug output at
<7>[ 0.382560] PM: Adding info for No Bus:fbcon
<7>[ 0.382815] PM: Adding info for platform:vesafb.0
<6>[ 0.384172] vesafb: framebuffer at 0xe4000000, mapped to 0xdca80000, using 5120k, total 65536k
<6>[ 0.384201] vesafb: mode is 1280x1024x16, linelength=2560, pages=1
<6>[ 0.384211] vesafb: protected mode interface info at c000:ed00
<6>[ 0.384220] vesafb: pmi: set display start = c00ced36, set palette = c00ceda0
<6>[ 0.384229] vesafb: pmi: ports = 3b4 3b5 3ba 3c0 3c1 3c4 3c5 3c6 3c7 3c8 3c9 3cc 3ce 3cf 3d0 3d1 3d2 3d3 3d4 3d5 3da
<6>[ 0.384263] vesafb: scrolling: redraw
<6>[ 0.384272] vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
<7>[ 0.384474] PM: Adding info for No Bus:fb0
<7>[ 0.384717] PM: Adding info for No Bus:vtcon1
<4>[ 0.439052] Console: switching to colour frame buffer device 160x64
<6>[ 0.489738] fb0: VESA VGA frame buffer device
<7>[ 0.490265] PM: Adding info for platform:vga16fb.0
<7>[ 0.490620] vga16fb: initializing
<6>[ 0.490842] vga16fb: mapped to 0xc00a0000
<7>[ 0.491159] checking generic (e4000000 4000000) vs hw (a0000 10000)
<7>[ 0.491730] PM: Adding info for No Bus:fb1
<6>[ 0.492142] fbcon: VGA16 VGA (fb1) is primary device
<6>[ 0.492463] fbcon: Remapping primary device, fb1, to tty 1-63
and took 1 minute until desktop
agpgart did not show @
Code:
elspci -l

nor any `modprobe -l | grep fb` and nouveau like the _M686 4GB config yesterday
# cat /proc/cmdline
panic=56 root=/dev/hdb8 ro vga=ask debug nosmp acpi=force pci=noacpi thermal.off=1 log_buf_len=1M
tty1-4 are messed for the moment so I have to boot the other kernels again to compare and diff the configs
Quote:
All it takes to obtain the kernel version is to do the command: "uname -r" (without the quotes) in a terminal.

2.6.37.4-KRG-iatom-1
xosview_on_iatom.png
 Description   
 Filesize   86.41 KB
 Viewed   992 Time(s)

xosview_on_iatom.png

Back to top
View user's profile Send private message Visit poster's website 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Wed 23 Mar 2011, 21:52    Post subject: PAE or 64bit seems to be the answer  

Just a note about performance and PAE.

PAE - This is not an item that we should be guessing at.
PAE is a hardware feature. It is not something that adds overhead, per se. It gives 32bit machines a method of addressing memory by handling addressing in hardware vs having to have the OS do it in its software!

We should be discussing how to decide if Puppy 32bit developers want to have their user base use any-all memory no matter what user use the distro. This is in total disregard of whether he has 1GB or 16GB running a 32bit Puppy.

This is hardware everyone. There is no guessing about whether there is some magical impact. Either the OS has the feature which enables it to use the hardware that is present (kinda like a missing driver). So when the kernel is compiled it can see it and use it, or it can't so it wont use it. (I have personally witness uninformed people arguing that "if an unused driver is present that it has a negative impact on an OS". Wow, talking about uninformed.)

In some of the internet threads I've read, some have found that the magical negative impact they initially reported were just that....magical. PAE; they reported, "that negative impact seen was not due to PAE. That it was there long before PAE was implemented." From the one lab report I obtained, they report that it has 0 impact. "Its hardware" according to Intel. So the OS has been designed to look at his hardware or to look at the address-space when running its code. PAE is hardware translation! NOT a lot of monkeying around in the OS. The OS, if enabled, will use it, if present.

Also, there was some other discussion about something called SLUB which has NO PLACE in this thread's topic, but was mentioned in several threads as having a positive impact on system behavior.

Everyone, now, based upon ALL contributions by everyone here, should have a very good idea of
  1. what the problem is,
  2. what PUP(s) have the problem
  3. which PUP(s) have gotten around it (even though they may not have told us how they did it), and
  4. that PAE is a significant oversight for newer, larger, modern machines who may want to us 32bit PUPs.
This does NOT detract from current operational use of PUP(s), it merely means that PUP(s) canNOT take advantage of your PC's hardware to its fullest.

And, in the future, PUP distro developers will either compile the OS so that it can handle PAE or other Intel hardware or they will continue to ignore it. (In case some have missed it, PAE has been around since Pentium Pro days...think about it.)

Hope this helps.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile

Last edited by gcmartin on Wed 23 Mar 2011, 22:42; edited 1 time in total
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 732
Location: Brisbane, Australia

PostPosted: Wed 23 Mar 2011, 22:33    Post subject:  

1. what the problem is,

Puppy, along with other 32bit os's doesn't see all of the memory in a high memory machine. This, in puppy's case, is largely superficial, as normally puppy doesn't need that much ram.

2. what PUP(s) have the problem

Most of them. As puppy is a 32bit platform. Obvious exemption would be Fatdog64.

3. which PUP(s) have gotten around it (even though they may not have told us how they did it), and

Not sure. More info needed here.

4. that PAE is a significant oversight for newer, larger, modern machines who may want to us 32bit PUPs.

This is correct, but, possibly in contradiction to Barry's ideas about legacy support. It would be a significant oversight IF enabling PAE turned out to NOT have an (significant) affect on real world performance and ISO size. If it did have an affect, then dual releases, one for old hardware, one for new, clearly labelled would be a winner.
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Thu 24 Mar 2011, 05:22    Post subject:  

gcmartin, sorry I need to correct you here.
Yes PAE is a hardware feature. But hardware feature requires implementation in software to be used (that's why the kernel has a configuration option to enable managing that feature). It doesn't just magically used just because it's there. That extra implementation code takes additional CPU time - for every memory-allocation-related operations. In addition, it takes additional overhead to manage RAM in PAE-mode (compared to non PAE-mode) - simply because there is more level of indirection in page tables http://en.wikipedia.org/wiki/Physical_Address_Extension.

What exactly is the impact is debatable, but as I said, for newer and faster CPUs, I think the impact is minimal or non-existent. For legacy CPUs, however, it may be a different story.

p310don, from ISO size point of view, I think the difference will be minimal. The PAE-enabled kernel I compiled only have tens of KB of differences, and I think the modules' sizes shouldn't be too different as well.

Now, apparently Karl Godt has compiled a PAE-enabled kernel, and got it to the point of working (based on the screenshot) - which is a step further than what I did. Karl, would you be willing to share the ISO with the rest of us? If you can compile the kernel with M586, it would be great I think (not everyone owns Core2 or Atom CPU ...) p310don is waiting and eager to test Smile

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 732
Location: Brisbane, Australia

PostPosted: Thu 24 Mar 2011, 08:43    Post subject:  

I have a bunch of old PCs laying around here, as well as newer ones, getting a shiny new one next week too Very Happy

I think the important consideration here, if a PAE kernel is compiled, and a working puppy made, is that it is tested not only on a good pc to see that it gets the highmem working, but, also tested versus standard kernel on an old pc to see if it makes a genuine negative difference.

As gcmartin said earlier, a system of measurement is needed. What do you guys think, would hardinfo's benchmarking tools be enough, or are there better tests available?
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 4 of 8 [106 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, 8 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Suggestions
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1185s ][ Queries: 12 (0.0056s) ][ GZIP on ]