Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 11 Dec 2019, 06:57
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Kernel hyperthreading is now forced on
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [18 Posts]   Goto page: 1, 2 Next
Author Message
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Sun 16 Dec 2018, 06:49    Post subject:  Kernel hyperthreading is now forced on
Subject description: CONFIG_SCHED_SMT now on whether you want it or not
 

Firstly, a quick explanation:

The Linux kernel configuration has CONFIG_SMP, to enable support for multiple CPU cores. There is another option, CONFIG_SCHED_SMT, known as "hyperthreading", that simulates more than one core for each physical core.

In all of my releases of Puppy Linux, and the distros I now work on (Quirky and EasyOS), I left CONFIG_SCHED_SMT turned off, as I distrusted it -- caused file corrupt back in the early days, recently it is looking like a security risk.

Anyway, a kernel developer submitted a patch to force CONFIG_SCHED_SMT as always enabled, and removed the choice from the kernel configuration. It was signed off by the big guns, even Linus, and is now in the kernel, backported to earlier versions.

In the 4.14.x series, it arrived with 4.14.86.

What I object to is this choice being taken away from us, with very little reason, it seems, just for the developers coding convenience.

I posted about this to my blog, along with an email I sent to some kernel developers:

http://bkhome.org/news/201812/kernel-41487-compiled.html

Sent the email this morning, and have received a reply from Greg KH, asking for further clarification, and this is my reply:

Quote:
On 12/16/18, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Sun, Dec 16, 2018 at 02:21:55PM +0800, Barry Kauler wrote:

>
> Is having this option enabled causing a problem for you? Does it cause
> slowdowns or runtime issues?
>
> thanks,
>
> greg k-h
>

Greg,
I have retained a distrust of hyperthreading, since the early days, when I first tried it, got file corruption, so turned it off and left it off, for the last ten years, well longer than that.

Then recently, I was feeling smug that I was avoiding some security issues by having it off. From light reading on the topic anyway, so it seemed.

Apart from my own distros, I am sure that there are other non-mainstream distros and thousands of embedded applications of the kernel out there, that have SCHED_SMT turned off.

Anyway, how many distros have or have not got it enabled, is not the main issue. SMT is a distinct feature, separate from SMP, and I think that it is important to keep that distinction. Even if "just in case".

Also, from the description in the submitted patch, I can't see any real justification for forcing SCHED_SMT on, just a small amount of coding convenience for the security hacks they are putting into the kernel. That's the impression I got anyway, from reading the changelog.

Another comment I would make, is that although there appears to be some speed gain from hyperthreading, on paper, in practice it is mostly "corner cases". Nobody has ever said Puppy Linux is slow, just the opposite -- other mainstream distros are often reported as sluggish in comparison.

Regards,
Barry kauler


I don't know whether the latest woof-CE built pups still have SCHED_SMT turned off. But it is the principle of thing, our choice being arbitrarily taken away.

My comment about negligible speed loss if SCHED_SMT is off, was thrown in there to stir things up. I know that there are some tests that show speed improvement with hyperthreading on certain CPUs.

But, would you notice any difference, practically, running a pup with it on or off?

Thought that I would post this here, in case anyone would like to comment, and maybe further clarify some of my comments. Even if you want to refute some of my comments, that is fine, I want to see this from all angles.

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
Smithy


Joined: 12 Dec 2011
Posts: 1076

PostPosted: Sun 16 Dec 2018, 08:51    Post subject:  

Hi Barry, what about disabling hyperthreading in the bios?

It does seem a bit retrograde to not offer the kernel option for people tweaking and compiling kernels.
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Sun 16 Dec 2018, 17:35    Post subject:  

I have posted a question to the kernel developer guys:

Quote:
I am an outsider, looking in, have only followed the RETPOLINE etc.
stuff fairly superficially. Would this statement be correct or
incorrect?:

With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
security reasons.
Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
CONFIG_RETPOLINE slows it down again.
End result, not much speed gain, far more complicated kernel.

Regards,
Barry Kauler


The changes that they have made are because of the new security threats, one of which is known as STIPB, and has resulted in a new kernel configuration option, CONFIG_RETPOLINE, to tackle this threat.

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Sun 16 Dec 2018, 18:02    Post subject:  

Had a response, so I have responded:

Quote:
On 12/17/18, Jiri Kosina <jikos@kernel.org> wrote:
> On Mon, 17 Dec 2018, Barry Kauler wrote:
>
>> With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
>
> This is not true. Retpoline protects also against a scenario where process
> A posions BTB for process B that gets scheduled afterwards on the same
> core, irrespective of SMT.
>
>> With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
>> security reasons.
>
> This is answered by the above as well. Retpoline is needed irrespective of
> SMT.
>
>> Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
>> CONFIG_RETPOLINE slows it down again.
>
> There is no 'again', as those two things are not really dependent in any
> way.
>
>> End result, not much speed gain, far more complicated kernel.
>
> And therefore this is bogus Smile
>

Thanks for the response.
So I guess it comes down to basic philosophy:

"SMT is a distinct feature, separate from SMP, and I think that it is
important to keep that distinction."

You guys have chosen to remove that distinction, and to remove the
choice for many specialised, such as embedded, situations, where they
would maybe want that choice.

I guess this will have to be my last post on the topic, as I can't
offer any other justification for my stance, other than the above.

Regards,
Barry Kauler



I like this, having a big fight with the kernel developers! Twisted Evil

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
rufwoof


Joined: 24 Feb 2014
Posts: 3672

PostPosted: Sun 16 Dec 2018, 18:55    Post subject:  

Some (Google staff) background reading ...

Retpoline: a software construct for preventing branch-target-injection
Author: Paul Turner, Senior Staff Engineer, Technical Infrastructure

https://support.google.com/faqs/answer/7625886

Also (maybe) of interest https://www.openbsd.org/papers/rop.pdf

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Sun 16 Dec 2018, 20:41    Post subject:  

I had another feedback, one of the kernel developers has informed me that the retpoline security threat is not just when CONFIG_SCHED_SMT is enabled, it may also be a threat when disabled.

That changes things. It seems that my understanding was too simplistic.

So, might just give in, and go along with what the kernel developers have decided. Don't have a choice anyway.

It is disconcerting though, when a kernel configure option suddenly disappears, and the explanation doesn't definitively justify it's removal.

I still have suspicions about enabling SCHED_SMT, from way back in the "early days" when I experienced file corruption, but it seems whatever that problem was, it has long ago been fixed.

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
mikeb


Joined: 23 Nov 2006
Posts: 11281

PostPosted: Mon 17 Dec 2018, 06:12    Post subject:  

If hyperthreading is turned off in the bios does this change override it?
I never felt it was worth the extra complexity.

I also wonder how much development is spent on 'hypothetical' threats as apposed to real improvements....or have there been any actual infringements in the real world?

mike
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Mon 17 Dec 2018, 07:13    Post subject:  

mikeb wrote:
If hyperthreading is turned off in the bios does this change override it?


I don't know what the difference is, turning it off in the kernel, or in the BIOS.

This site:

https://serverfault.com/questions/806770/turning-off-hyperthread-by-bios-or-kernel

Quoting:

Quote:
There should be no difference in performance between disabling hyperthreading in the BIOS or disabling it in the operating system.


If they both effectively do the same thing, then according to the feedback I got from a kernel developer, turning off hyperthreading in the BIOS still won't protect from the STIPB threat -- the retpoline patch is still going to be required. Bummer.

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 9093
Location: Perth, Western Australia

PostPosted: Mon 17 Dec 2018, 07:18    Post subject:  

Interesting reference:

https://wikitech.wikimedia.org/wiki/Hyper-threading

_________________
https://bkhome.org/news/
Back to top
View user's profile Send private message Visit poster's website 
nosystemdthanks


Joined: 03 May 2018
Posts: 708

PostPosted: Tue 18 Dec 2018, 01:56    Post subject: Re: Kernel hyperthreading is now forced on
Subject description: CONFIG_SCHED_SMT now on whether you want it or not
 

BarryK wrote:
I don't know whether the latest woof-CE built pups still have SCHED_SMT turned off. But it is the principle of thing, our choice being arbitrarily taken away.


unfortunately, this is more like what things are going to be like when linus steps down and gkh takes over.

"does it work? are you using a version that is still on the support cycle?"

if the first question is yes and the second is no, it will be considered irrelevant.

linux has transitioned from project, to product. its license has not changed-- the way it is managed is changing. systemd is another manifestation of this.

choice is irrelevant in this new era of the linux kernel. linux is not about choice-- it is about support, until we move onto whats new.

your kernel is supported. if you want choice, youll have to fork the kernel itself.

i realise some people will think this is unfair, unwarranted or unfounded-- but this cultural shift has happened for the past 4 years, and also existed in gnome 3. red hat is now ibm and that will contribute to this further. i said msft was going to "buy redhat next" when they bought github.

this isnt your linux, it isnt our linux-- this wont be posix. this is what i call redix. and this is going to keep happening.

some people say hurd-- i doubt it, ive tried it. its certainly an option, but not a likely one.

most likely? a fork of the linux kernel. sometime during the tenure of gkh as linuss replacement.

because the future of linux kernel development isnt about choice. not anymore. its about the linux kernel developers. just like debian for the past 4 years, is about dds. take it or leave it.

_________________
The freedom to NOT run the software, to be free to avoid vendor lock-in through appropriate modularization/encapsulation and minimized dependencies; meaning any free software can be replaced with a user’s preferred alternatives.
Back to top
View user's profile Send private message Visit poster's website 
8Geee


Joined: 12 May 2008
Posts: 2095
Location: N.E. USA

PostPosted: Tue 18 Dec 2018, 10:47    Post subject:  

Unfortuneately Barry, 99% of CPU's have Out-of-Order Execution, and Branch Prediction/Speculation. I do believe this is CPU mitigation at the Kernel level.

The early Intel Atom CPU's are the only exception, and do not need a vast majority of these recent patches. Nonetheless, I do see your point, and it was wise to shut off SMT Just in case. Today's CPU security flaws have 'disabled' that concern, as the CPU manufaturer will not/cannot cure the CPU patient. That really sucks, period.

Regards and thanks
8Geee

_________________
Linux user #498913 "Some people need to reimagine their thinking."
"Zuckerberg: a large city inhabited by mentally challenged people."
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 11281

PostPosted: Tue 18 Dec 2018, 15:23    Post subject:  

Quote:
The early Intel Atom CPU's are the only exception,

well a comfort for someone who just bought a spare atom ITX board as a spare.
They also will run anything you like.
mike
Back to top
View user's profile Send private message 
8Geee


Joined: 12 May 2008
Posts: 2095
Location: N.E. USA

PostPosted: Tue 18 Dec 2018, 18:44    Post subject:  

Yah, I found a D2700 Intel "Kit". Just add memory and a box... Good for Pups.

Regards
8Geee

_________________
Linux user #498913 "Some people need to reimagine their thinking."
"Zuckerberg: a large city inhabited by mentally challenged people."
Back to top
View user's profile Send private message 
Marv


Joined: 04 May 2005
Posts: 1216
Location: SW Wisconsin

PostPosted: Mon 07 Jan 2019, 11:06    Post subject:  

mikeb wrote:
If hyperthreading is turned off in the bios does this change override it?

Just a note:

I ran into an interesting quirk/bug on a second generation i5 laptop. It took some time, memory testing, multiple machines, and inter-kernel comparing to sort it. With the 4.19 series 64b kernel (earliest tested 4.19.11), when I disabled hyperthreading in the BIOS I got occasional program lockups. This is in the current LxPupSc and LxPupSc64 and showed up first when I was using galculator a lot and then confirmed in other programs, mostly those using gtk I believe. After 5 to 10 sucessive calculations the calculator would would lock up. The programs could always be killed OK and restarted but on continued use would lock up again (no keyboard input or response) and the pup kept running fine. It did not occur using the 4.9.134 32b kernel on that same hardware. Re-enabled hyperthreading in the BIOS and no recurrence of the lockups at all for several days of identical use of those same installs now. I've been using the calculator a lot so I would definitely have seen it. Odd but quite reproducible. Perhaps the kernel uses the CPU info and doesn't read/respect the BIOS settings in this case?

Checked the DOTconfig on those kernels and SMT is explicitly enabled there fwiw...
Code:
CONFIG_SCHED_SMT=y

_________________
Pups currently in kennel Very Happy Older LxPupSc and X-slacko-4.4 for my users; LxPupSc, LxPupSc64 and upupee for me. All good pups indeed, and all running savefiles for look'n'feel only. Browsers, etc. solely from SFS.
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 12986
Location: S.C. USA

PostPosted: Mon 07 Jan 2019, 12:13    Post subject:  

This is way above my knowledge level.

But I will make this observation.
Basing any decisions on how stuff worked 10 or more years ago, is very wrong to me.

Puppy Linux is very much improved and better at doing stuff, than it was 10 years ago.

Computer processors are designed to now work using multiple cores, to do anything. The cores are there, so use them.
When this multiple core construction was first developed. No one really had a good understanding of how to use it.
No software really made full use of it.
Now it is normal operation of a processor.

Puppy Linux is really not a good OS to judge speed or operation on.
A lot of Puppy programs are very simple small code.
Not much processor demand being made.
Try to have 2 or more programs running at the same time and doing high demand processing or a program that is properly coded to use multiple processor cores..
Those multiple cores really show what they provide.

Some of this reminds me of all the talk about was 8 bit going to 16 bit, going to 32 bit, going to 64 bit really needed.
I remember the early days of 64 bit. Almost no software used it and it took several years for full support to develop.
That is even talked about in Puppy development, going from 32 bit to 64 bit Puppy.
Does Puppy really need 64 bit?
Some say yes. Others say no.

_________________
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
YaPI(any iso installer)
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [18 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0695s ][ Queries: 11 (0.0096s) ][ GZIP on ]