Kernel hyperthreading is now forced on

[BarryK]

Firstly, a quick explanation:

The Linux kernel configuration has CONFIG_SMP, to enable support for multiple CPU cores. There is another option, CONFIG_SCHED_SMT, known as "hyperthreading", that simulates more than one core for each physical core.

In all of my releases of Puppy Linux, and the distros I now work on (Quirky and EasyOS), I left CONFIG_SCHED_SMT turned off, as I distrusted it -- caused file corrupt back in the early days, recently it is looking like a security risk.

Anyway, a kernel developer submitted a patch to force CONFIG_SCHED_SMT as always enabled, and removed the choice from the kernel configuration. It was signed off by the big guns, even Linus, and is now in the kernel, backported to earlier versions.

In the 4.14.x series, it arrived with 4.14.86.

What I object to is this choice being taken away from us, with very little reason, it seems, just for the developers coding convenience.

I posted about this to my blog, along with an email I sent to some kernel developers:

http://bkhome.org/news/201812/kernel-41 ... piled.html

Sent the email this morning, and have received a reply from Greg KH, asking for further clarification, and this is my reply:

On 12/16/18, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Sun, Dec 16, 2018 at 02:21:55PM +0800, Barry Kauler wrote:

>
> Is having this option enabled causing a problem for you? Does it cause
> slowdowns or runtime issues?
>
> thanks,
>
> greg k-h
>

Greg,
I have retained a distrust of hyperthreading, since the early days, when I first tried it, got file corruption, so turned it off and left it off, for the last ten years, well longer than that.

Then recently, I was feeling smug that I was avoiding some security issues by having it off. From light reading on the topic anyway, so it seemed.

Apart from my own distros, I am sure that there are other non-mainstream distros and thousands of embedded applications of the kernel out there, that have SCHED_SMT turned off.

Anyway, how many distros have or have not got it enabled, is not the main issue. SMT is a distinct feature, separate from SMP, and I think that it is important to keep that distinction. Even if "just in case".

Also, from the description in the submitted patch, I can't see any real justification for forcing SCHED_SMT on, just a small amount of coding convenience for the security hacks they are putting into the kernel. That's the impression I got anyway, from reading the changelog.

Another comment I would make, is that although there appears to be some speed gain from hyperthreading, on paper, in practice it is mostly "corner cases". Nobody has ever said Puppy Linux is slow, just the opposite -- other mainstream distros are often reported as sluggish in comparison.

Regards,
Barry kauler

I don't know whether the latest woof-CE built pups still have SCHED_SMT turned off. But it is the principle of thing, our choice being arbitrarily taken away.

My comment about negligible speed loss if SCHED_SMT is off, was thrown in there to stir things up. I know that there are some tests that show speed improvement with hyperthreading on certain CPUs.

But, would you notice any difference, practically, running a pup with it on or off?

Thought that I would post this here, in case anyone would like to comment, and maybe further clarify some of my comments. Even if you want to refute some of my comments, that is fine, I want to see this from all angles.

[Smithy]

Hi Barry, what about disabling hyperthreading in the bios?

It does seem a bit retrograde to not offer the kernel option for people tweaking and compiling kernels.

[BarryK]

I have posted a question to the kernel developer guys:

I am an outsider, looking in, have only followed the RETPOLINE etc.
stuff fairly superficially. Would this statement be correct or
incorrect?:

With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
security reasons.
Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
CONFIG_RETPOLINE slows it down again.
End result, not much speed gain, far more complicated kernel.

Regards,
Barry Kauler

The changes that they have made are because of the new security threats, one of which is known as STIPB, and has resulted in a new kernel configuration option, CONFIG_RETPOLINE, to tackle this threat.

[BarryK]

Had a response, so I have responded:

On 12/17/18, Jiri Kosina <jikos@kernel.org> wrote:
> On Mon, 17 Dec 2018, Barry Kauler wrote:
>
>> With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
>
> This is not true. Retpoline protects also against a scenario where process
> A posions BTB for process B that gets scheduled afterwards on the same
> core, irrespective of SMT.
>
>> With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
>> security reasons.
>
> This is answered by the above as well. Retpoline is needed irrespective of
> SMT.
>
>> Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
>> CONFIG_RETPOLINE slows it down again.
>
> There is no 'again', as those two things are not really dependent in any
> way.
>
>> End result, not much speed gain, far more complicated kernel.
>
> And therefore this is bogus
>

Thanks for the response.
So I guess it comes down to basic philosophy:

"SMT is a distinct feature, separate from SMP, and I think that it is
important to keep that distinction."

You guys have chosen to remove that distinction, and to remove the
choice for many specialised, such as embedded, situations, where they
would maybe want that choice.

I guess this will have to be my last post on the topic, as I can't
offer any other justification for my stance, other than the above.

Regards,
Barry Kauler

I like this, having a big fight with the kernel developers!

[rufwoof]

Some (Google staff) background reading ...

Retpoline: a software construct for preventing branch-target-injection
Author: Paul Turner, Senior Staff Engineer, Technical Infrastructure

https://support.google.com/faqs/answer/7625886

Also (maybe) of interest https://www.openbsd.org/papers/rop.pdf

[BarryK]

I had another feedback, one of the kernel developers has informed me that the retpoline security threat is not just when CONFIG_SCHED_SMT is enabled, it may also be a threat when disabled.

That changes things. It seems that my understanding was too simplistic.

So, might just give in, and go along with what the kernel developers have decided. Don't have a choice anyway.

It is disconcerting though, when a kernel configure option suddenly disappears, and the explanation doesn't definitively justify it's removal.

I still have suspicions about enabling SCHED_SMT, from way back in the "early days" when I experienced file corruption, but it seems whatever that problem was, it has long ago been fixed.

[mikeb]

If hyperthreading is turned off in the bios does this change override it?
I never felt it was worth the extra complexity.

I also wonder how much development is spent on 'hypothetical' threats as apposed to real improvements....or have there been any actual infringements in the real world?

mike

[BarryK]

If hyperthreading is turned off in the bios does this change override it?

I don't know what the difference is, turning it off in the kernel, or in the BIOS.

This site:

https://serverfault.com/questions/80677 ... -or-kernel

Quoting:

There should be no difference in performance between disabling hyperthreading in the BIOS or disabling it in the operating system.

If they both effectively do the same thing, then according to the feedback I got from a kernel developer, turning off hyperthreading in the BIOS still won't protect from the STIPB threat -- the retpoline patch is still going to be required. Bummer.

[BarryK]

Interesting reference:

https://wikitech.wikimedia.org/wiki/Hyper-threading

[nosystemdthanks]

I don't know whether the latest woof-CE built pups still have SCHED_SMT turned off. But it is the principle of thing, our choice being arbitrarily taken away.

unfortunately, this is more like what things are going to be like when linus steps down and gkh takes over.

"does it work? are you using a version that is still on the support cycle?"

if the first question is yes and the second is no, it will be considered irrelevant.

linux has transitioned from project, to product. its license has not changed-- the way it is managed is changing. systemd is another manifestation of this.

choice is irrelevant in this new era of the linux kernel. linux is not about choice-- it is about support, until we move onto whats new.

your kernel is supported. if you want choice, youll have to fork the kernel itself.

i realise some people will think this is unfair, unwarranted or unfounded-- but this cultural shift has happened for the past 4 years, and also existed in gnome 3. red hat is now ibm and that will contribute to this further. i said msft was going to "buy redhat next" when they bought github.

this isnt your linux, it isnt our linux-- this wont be posix. this is what i call redix. and this is going to keep happening.

some people say hurd-- i doubt it, ive tried it. its certainly an option, but not a likely one.

most likely? a fork of the linux kernel. sometime during the tenure of gkh as linuss replacement.

because the future of linux kernel development isnt about choice. not anymore. its about the linux kernel developers. just like debian for the past 4 years, is about dds. take it or leave it.

[8Geee]

Unfortuneately Barry, 99% of CPU's have Out-of-Order Execution, and Branch Prediction/Speculation. I do believe this is CPU mitigation at the Kernel level.

The early Intel Atom CPU's are the only exception, and do not need a vast majority of these recent patches. Nonetheless, I do see your point, and it was wise to shut off SMT Just in case. Today's CPU security flaws have 'disabled' that concern, as the CPU manufaturer will not/cannot cure the CPU patient. That really sucks, period.

Regards and thanks
8Geee

[mikeb]

The early Intel Atom CPU's are the only exception,

well a comfort for someone who just bought a spare atom ITX board as a spare.
They also will run anything you like.
mike

[8Geee]

Yah, I found a D2700 Intel "Kit". Just add memory and a box... Good for Pups.

Regards
8Geee

[Marv]

If hyperthreading is turned off in the bios does this change override it?

Just a note:

I ran into an interesting quirk/bug on a second generation i5 laptop. It took some time, memory testing, multiple machines, and inter-kernel comparing to sort it. With the 4.19 series 64b kernel (earliest tested 4.19.11), when I disabled hyperthreading in the BIOS I got occasional program lockups. This is in the current LxPupSc and LxPupSc64 and showed up first when I was using galculator a lot and then confirmed in other programs, mostly those using gtk I believe. After 5 to 10 sucessive calculations the calculator would would lock up. The programs could always be killed OK and restarted but on continued use would lock up again (no keyboard input or response) and the pup kept running fine. It did not occur using the 4.9.134 32b kernel on that same hardware. Re-enabled hyperthreading in the BIOS and no recurrence of the lockups at all for several days of identical use of those same installs now. I've been using the calculator a lot so I would definitely have seen it. Odd but quite reproducible. Perhaps the kernel uses the CPU info and doesn't read/respect the BIOS settings in this case?

Checked the DOTconfig on those kernels and SMT is explicitly enabled there fwiw...

Code: Select all

CONFIG_SCHED_SMT=y

[bigpup]

This is way above my knowledge level.

But I will make this observation.
Basing any decisions on how stuff worked 10 or more years ago, is very wrong to me.

Puppy Linux is very much improved and better at doing stuff, than it was 10 years ago.

Computer processors are designed to now work using multiple cores, to do anything. The cores are there, so use them.
When this multiple core construction was first developed. No one really had a good understanding of how to use it.
No software really made full use of it.
Now it is normal operation of a processor.

Puppy Linux is really not a good OS to judge speed or operation on.
A lot of Puppy programs are very simple small code.
Not much processor demand being made.
Try to have 2 or more programs running at the same time and doing high demand processing or a program that is properly coded to use multiple processor cores..
Those multiple cores really show what they provide.

Some of this reminds me of all the talk about was 8 bit going to 16 bit, going to 32 bit, going to 64 bit really needed.
I remember the early days of 64 bit. Almost no software used it and it took several years for full support to develop.
That is even talked about in Puppy development, going from 32 bit to 64 bit Puppy.
Does Puppy really need 64 bit?
Some say yes. Others say no.

[8Geee]

Frankly, the 32/64 bit Pup was/is not a real choice... the CPU manufacturers made that change for us. Puppy had no other choice but to make 64b OS's so that newer CPU/MPU could take advantage of a 1% speed increase. Naturally 64b OS's exclude 32 bit CPU/MPU... and that was the true aim of the other OS developers.

FWIW
8Geee

[ozsouth]

This may explain some of the issues I've had on new laptops on which I can't disable hyperthreading.
I've made a 64bit kernel with CONFIG_SCHED_SMT=y. Will test over next week or so to see.

[peterw]

Interesting discussion. Made me think, "How do you know that hyperthreading is in use or not?" You can easily find out if the CPU is capable of it but you are not certain if it is actually using it.
Anyhow after reading about various commands and Python scripts, I came across this http://www.murga-linux.com/puppy/viewtopic.php?t=59771 Puppy followers had the easiest answer which is to use htop. I wanted to see if it was in use on a Slacko that was on a single core Atom CPU Acer Aspire One and found out that it was.