Get the top HN stories in your inbox every day.
topspin
MrGilbert
> [...] a bunch of dramatic YouTube content [...]
That framing doesn't do him and the team justice. There is (or better, was) a 3.5h long story about NVIDIA GPUs finding their ways illegaly from the US to China, which got taken down by a malicious DMCA claim from Bloomberg. It is quite interesting to watch (Can be found archive.org).
GN is one of the last pro-consumer outlets, that keep on digging and shaking the tree big companys are sitting on.
themafia
> which got taken down
Not everywhere:
https://archive.org/details/the-nvidia-ai-gpu-black-market-i...
mrheosuper
When something is uploaded to the internet, it won't be easy to take it down.
Ask Beyonce.
adrian_b
The small coolers used by them are not recommended by Noctua for 9950X. Noctua recommends only bigger coolers for 9950X, which dissipates 200 W permanently on a workload like theirs (which is much less than the over 250 W dissipated in similar conditions by the competing Intel CPUs).
Despite this, the overtemperature protection of the CPUs should have protected the CPUs and prevent any kind of damage like this.
Besides the system that varies continuously the clock frequency to keep the CPU within the current and power consumption limits, there is a second protection that stops temporarily the clock when a temperature threshold is exceeded. However, the internal temperature sensors of the CPUs are not accurate, so the over-temperature protection may begin to act only at a temperature that is already too high.
So these failures appear to have been caused by a combination of not using the appropriate coolers for a 200 W CPU, combined with the fact that AMD advertises a 200-W CPU as an 170-W CPU, fooling naive customers into believing that smaller coolers are acceptable, and with either some kind of malfunction of the over-temperature protection in these CPUs or with a degradation problem that happens even within the nominal temperature range, but at its upper end.
ollien
> The small coolers used by them are not recommended by Noctua for 9950X
Noctua's CPU compatibility page lists the NH-U9s as "medium turbo/overclocking headroom" for the 9950X [0]. I don't think it's fair to suggest their cooler choice is the problem here.
adrian_b
That means pretty much "not recommended".
On the same page linked by you, Noctua explains that the green check mark means that with that cooler the CPU can run all-core intensive tasks, exactly like those used by the gmplib developers, only at the base clock, which is 4.3 GHz for 9950X, with turbo disabled in BIOS.
Only then the CPU might dissipate its nominal TDP of 170 W, instead of the 200 W that it dissipates with turbo enabled.
With "best turbo headroom", you can be certain that the CPU can run all-core intensive tasks with turbo enabled. Even if you do no overclocking, but you run all-core intensive tasks with turbo enabled, this is the kind of cooler that you need.
Noctua does not define what "medium headroom" means, but presumably it means that you can run with turbo enabled all-core tasks that have medium intensity, not maximum intensity.
There is no doubt that it is a mistake to choose such a cooler when you intend to run intensive multi-threaded computations. A better cooler, but not much bigger, like NH-U12A, has an almost double cooling capacity.
That said, there is also no doubt that AMD is guilty of at least having some bugs in their firmware or in failing to provide adequate documentation for the motherboard manufacturers that adapt the AMD firmware for their MBs.
nerdsniper
Wendell at Level1Techs often goes more in-depth on the software testing and datacenter use-case analysis through partnerships with friends who run lots of machines in datacenters.
GN is unique in paying for silicon-level analysis of failures.
der8auer also contributes a lot to these stories.
I tend to wait for all 3 of their analyses, because each adds a different "hard-won" perspective.
fxtentacle
He's a bit sensationalist, yes, but I am thankful that he saved us from buying affected Intel CPUs.
bayindirh
He's a "student" and friend of late Gordon Mah Ung. He's carrying his torch forward.
This was Gordon's style, and Steve is continuing it. He has the courage to hit Bloomberg offices with a cameraman, so I don't think his words ring hollow.
We need that kind of in your face, no punches held back type of reporting when compared to "measured professionals".
mft_
Absolutely - this is the sort of direct citizen journalism I expect (sort of hope?) we'll see more and of as traditional investigative journalism dies its slow death.
tpurves
Yes. When he's right, he's right. However the main issue I have with GN is how Steve tends to go full Leeroy Jenkins pitchforks and torches for 9 out of every 5 actual scandals in the tech industry.
jchw
When it comes to interpersonal drama, the "Shoot first, ask questions later" style of reporting is terrible. However, for consumer advocacy it's basically the opposite, especially because in most cases it's easy for companies to turn the narrative around by simply handling the issue well. It's almost more about how they handle it than the actual issue in many cases.
CaptainBanger
I felt the same way, but over time I have come to respect those with the Crusader personality archetype, we need these people to do their thing and they need us to balance them out.
spookie
Not sure of sensationalist or just doing great reporting. I take him as one of the last good tech journalists on the platform.
hnuser123456
GN wasn't the first to break the story the 13/14th gen was defective. The thousands and thousands of users experiencing the issues collectively noticed pretty quick. If anything, there was a period where he was saying "We've talked to Intel but we won't say anything yet until they do."
BoorishBears
[flagged]
RachelF
AMD has failed to be reliable with its Zen 4 and Zen 5 consumer CPUs, just at the same time Intel did the same with their 13k and 14k higher end CPUs.
AMD is somewhat worse than Intel as their DDR5 memory bus is very "twitchy" making it hard to get the highest DDR5 timings, especially with multiple DIMMs per channel.
akimbostrawman
I don't think it's reasonable to call memory timing tweaking stability issues worse than a cpu dying from heat under normal usage.
AuryGlenz
I had to put together an AM5 computer pretty quickly after I accidentally fried some components in my last computer, so I got a Microcenter bundle.
I got 2x32GB sticks of RAM with the plan to throw in another two sticks later. I had no idea that was now a bad plan. I wish manufacturers would have just put 2 DIMM slots on motherboards as a “warning.”
rangestransform
I think that's just a result of being at the limit of what a right-angle memory slot can handle, it's about time that desktop move to CAMM or soldered memory
a-french-anon
What do you mean? Is your second sentence the only reason for the first?
trebligdivad
They don't say what temperature the CPU was reporting which seems like an odd omission. Whatever the specs of your cooler etc check the temperature it's actually running at. Go by what the CPU is saying! I've got the older 3950x, and the first one died after a few months (still in warranty) with a cooler in spec, but it would go into the 90s at full load just doing big builds. I replaced the heatsink with a basic watercooler when the replacement chip arrived and it's running at least 20c cooler at full load.
Symmetry
A modern CPU should be able to detect temperature excursions and bring itself to a safe halt even if you power it up without any cooler attached. It's normal and expected that people making mistakes around the cooling systems of their CPUs will accidentally give themselves terrible performance. It is not normal that the CPUs will break.
account42
Zen 2 is supposed to be able to work up to 95 C so that shouldn't have caused your CPU to fail. And it should clock down before it fails anyway, way below the specified "minimum" frequency if needed - got to experience that with a failing AIO. A better cooler should only be required to make full use of your CPU not to protect it.
trebligdivad
I kind of agree with you and Symmetry; but having had a fried CPU I'm more careful. No electronics like running very hot - so even if you're just inside spec on something for the heat it's likely to live a shorter life than if you kept it more comfortable - and it'll let it clock faster if you keep it cool! And really my points are: * the standard spec coolers just don't manage that on these hot CPUs, even if they claim to. * If you're building a machine and you know you're pushing it hard, just check the temperatures to check that cooling you bought is working.
wkat4242
Maybe they didn't have anything logging the temperature. They didn't expect it to die after all.
spoaceman7777
All you really need to see is the picture of the CPU with thermal paste only on one half. Thermal throttling is tuned to work when there is 1. a sufficient heatsink (theirs was significantly below requirements) and 2. it is installed correctly so that its triggers for downclocking happen with the correct timing. This is just another instance of ridiculous PEBCAK error
ndiddy
This is per design. On AM5 processors, there's a hotspot on the lower half of the processor where the dies that contain the CPU cores are located. Noctua recommends that AM5 users mount their coolers shifted towards the lower side of the processor for optimal cooling performance, see https://noctua.at/en/offset-am5-mounting-technical-backgroun... . You may have missed the paragraph in the article that explicitly points this out:
> We use a Noctua cooling solution for both systems. For the 1st system, we mounted the heat sink centred. For the 2nd system, we followed Noctua's advice of mounting things offset towards what they claim to be the hotter side of the CPU. Below is a picture of the 2nd system without the heat sink which shows that offset. Note the brackets and their pins, those pins are where the heat sink's pressure gets centred. Also note how the thermal paste has been squeezed away from that part, but is quite thick towards the left.
nolok
While it is noctua advice, I don't think AMD supports that view, so it would seem correct to at least test the cpu the way the vendor recommends before making conclusions
dukezzz
Noctua recommends mounting their cooler so that the center is shifted toward the lower part of the CPU. From your picture with the thermal paste, it’s clear that your cooler is only making contact with about two-thirds of the CPU, meaning you mounted it incorrectly. The cooler’s contact area must always cover the entire CPU; otherwise, you reduce heat transfer capacity. On top of that, you’re already using an undersized cooler for this CPU. I think you don’t understand the basics of thermodynamics.
marshray
Clearly paste was squeezed out from the entire perimeter of the CPU. Offset mounting is used intentionally for this CPU.
Probably there's less paste remaining on the south end of the CPU because that's where the mounting force is greatest.
If anything, there's too much paste remaining on the center/north end of the CPU. Paste exists simply to bridge the roughness of the two metal surfaces, too much paste is a bad sign.
My guess is that the MB was oriented vertically and that big heavy heat sink with the large lever arm pulled it away from the center and north side of the CPU.
IMO, the CPU is still responsible for managing its power usage to live a long life. The only effect of an imperfect thermal solution ought to be proportionally reduced performance.
mrheosuper
Many reviewers have tested that too much paste is not an issue, except being messy to clean.
userbinator
I'm not as sure about AMD CPUs (and they were known for having far worse overheat behaviour back in the early 2000s) but there are plenty of stories of Intel CPUs working for many years, sitting at the thermal limits, with the (stock) heatsink not even in contact, thanks to their cheap push-pin retention mechanism.
munchlax
Those dreadful plastic knobs never want to sit right. Simple lever over that shit any time of day, pls.
db48x
> The so-called TDP of the Ryzen 9950X is 170W. The used heat sinks are specified to dissipate 165W, so that seems tight.
TDP numbers are completely made up. They don’t correspond to watts of heat, or of anything at all! They’re just a marketing number. You can't use them to choose the right cooling system at all.
https://gamersnexus.net/guides/3525-amd-ryzen-tdp-explained-...
bayindirh
When I see the term TDP, I remember what I have read in the "Thermal Design Document" of Intel Core2Quad Q6600 and the family it belongs:
> The thermal solution bundled with the CPUs is not designed to handle the thermal output when all the cores are utilized 100%. For that kind of load, a different thermal solution is strongly recommended (paraphrased).
I never used the stock cooler bundled with the processor, but what kind of dark joke is this?
johncolanduoni
Most states of “100% utilization” as you’d see in `top` are not 100% thermal output or even close. Cores waiting for memory accesses count as utilized in the former sense but will not produce as much heat as one that is actually using the ALU etc. That’s why special make-work like Prime95 is used for stress testing overclocking/thermals: it will saturate the cores with enough unblocked arithmetic work to generate more heat than having 1000 browser tabs open does.
account42
You're not going to get anywhere near full thermal load with just integer arithmetic either - you need to saturate the floating point units for that.
kokada
This is more how I think too: using a cooler that supports your CPU TDP is generally fine because most people will not run a CPU 100% for an extended amount of time. But in this case they seem to be running the CPU 100% for an extended amount of time AND are using an under-spec'ed cooler (even if it is just by 5W).
You don't even need to change the actual cooler since for AMD CPUs you can pretty much customize the TDP whatever way you want, and by default they run well above their efficiency curve. For example, my 7600X has a default TDP of 105W but I run it in Eco Mode (65W) with undervolt and I barely lose any performance. Even if I did no undervolt, running the CPU in Eco Mode is generally preferable since the performance loss is still negligible (~5%).
bayindirh
For a general purpose system, this line of thinking makes sense. However, the desktop system in question was built to be daily driven and support some high performance code research, so it had to endure some serious loads for a desktop computer.
I went the other way and overspecced the CPU cooler and added some silent but high CFM capable fans on the system. The motherboard I got was able to adjust all fans depending on the system temps, so it scaled from a very silent desktop to a low-key space heater automatically under load.
Instead of undervolting the processor, I was using a tweaked on-demand governor on the system which stuck to lower power levels more than usual, so unless I was doing software development and testing things, it stayed cool and silent.
BTW, by 100%, I'm talking about completely saturating the CPU pipeline. Not pseudo 100% where CPU reports saturation but most of the load is iowait.
cogman10
Man that was a beast of a CPU back in the day.
The Conroe Intel era was amazing for the time.
keanebean86
That was such a fun time to be into hardware. For years Intel had the money and relationships to keep the Pentium 4 everywhere even though AMD had the better product. The P4 might edge ahead in video rendering but the Athlon would win overall and use less power.
Then Conroe launched and the balance shifted. Even the cheapest Core2Duo chips were competitive against the best P4s and the high-end C2Ds rivaled or beat AMD. https://web.archive.org/web/20100909205130/http://www.anandt...
AND those chips overclocked to the moon. I got my E6420 to 3.2ghz (from 2.133ghz) just by upping the multiplier. A quick search makes me think my chip wasn't even that great.
bayindirh
Buying parts for that particular desktop was quite fun:
- Me: Can I get a Q6600?
- Seller: But, that's... Quad core?
- Me: Yes, I'll have it.
- Seller: OK. RAM?
- Me: I'll get OCZ Flex-XLC Hybrids. 1GB.
- Seller: *Gives one*
- Me: I'll get four.
- Seller: ?
- Me: Yes, four please.
undefined
lofaszvanitt
I always used the stock cooler, because it's quiet and nothing uses the cpu to its fullest :).
mrb
You are correct. In fact these guys measured a maximum socket power consumption of 240 watt using a 9950X at stock settings, running prime95. So far above the "170 watt" TDP:
https://hwbusters.com/cpu/amd-ryzen-9-9950x-cpu-review-perfo...
dcrazy
I don’t understand this argument. If the CPU dissipated an equal number of watts of heat energy as it consumed from the wall, there wouldn’t be any energy left to do actual useful work. Isn’t the extra 100W accounted for by things like changing the state of flip-flops? In other words, mustn’t one consider the entropy reduction of the system as an energy sink?
marshray
Clocking and changing register states requires charging and discharging the gate capacitance of a bunch of MOSFET transistors. The current that results from moving all that charge around encounters resistance, which converts it to heat. Silicon is only a "semi" conductor after all.
You are correct that there is energy bound in the information stored in the chip. But last I checked, our most efficient chips (e.g., using reversible computing to avoid wasting that energy) are still orders of magnitude less efficient than those theoretical limits.
MadnessASAP
I think the numbers are more like <1W used in actual information processing, >239W lost to heat. Information and the transformation of it does have some inherent energy cost. But it is very, very small. And you end up getting that back as heat somewhere else down the line anyways.
db48x
Nope. Remember that you cannot destroy energy. The energy you use to flip the flip flop still exists, only now it’s just disordered waste heat instead of electricity.
arcade79
What happens to the energy that did the useful work?
aidenn0
I have a 65W TDP CPU, and the difference in power draw (measured at the outlet) from idle to full CPU load is over 100W; it seems to just raise the clock until it hist 95C, so if I limit the CPU fan's top speed, the power draw goes down.
db48x
Yep. Modern CPUs continually adjust their clock multiplier based on what their temperature is doing, plus a few timers. If you have a better cooler then you’ll get more performance out of the same CPU, but at the cost of drawing more power and producing more heat.
einpoklum
Wow, I can't believe how BS this TDP is! I feel like a total idiot! I've always assumed it's sorta-kinda a tight upper bound on power consumption, perhaps with some allowance for "imperfections" in the dissipation properties of the CPU, and that I shouldn't sweat the details.
Couldn't this count as false/misleading advertizing though?
gruez
It's thermal design power, ie. it's the power that it's designed for, not absolute max.
db48x
No, they don’t design the chip with these numbers in mind. The marketing department picks the number they want based on how they want customers to think about the chip, and which competitors they want you to compare it against. They just plug in whatever numbers are needed into the formula so that the number comes out how they want it.
taneq
Huh, I always thought it was “total dissipated power”. Like you’d use to spec a power supply.
vel0city
Its pretty insane to see someone say something like: “TDP is about thermal watts, not electrical watts. These are not the same.” Watts are watts.
But yeah, TDP means nothing. If you stick plenty of cooling and run the right motherboard board revision your "TDP" can be whatever you want it to be until the thing melts.
o11c
"TDP is about average watts, not peak watts" would be an honest way of saying it.
kllrnohj
> Couldn't this count as false/misleading advertizing though?
For what, exactly? TDP stands for "thermal design power" - nothing in that means peak power or most power. It stopped being meaningful when CPUs learned to vary clock speeds and turbo boost - what is the thermal design target at that point, exactly? Sustained power virus load?
einpoklum
> For what, exactly? TDP stands for "thermal design power"
The chip is not designed for this rate of power dissipation; and it is not the rate of power dissipation that you can expect to get from the chip.
OhMeadhbh
When I worked at Linden Lab we had a deal going with IBM. Either as part of that deal or in an attempt to impress our larger partner, many of us got Thinkpads. I actually kind of like them, since the cost wasn't coming out of my budget.
Inside Linden, about 90% of meetings were held in-world, so we constantly had the Second Life viewer up. About three months later our Thinkpads started failing. Apparently they thought people who would a) buy a thinkpad and b) use it to play video games wouldn't be playing video games 12 hours per day (though as many have pointed out, does one "play" Second Life? especially if you're using it for work.)
After 3 months of use, the Second Life client had caused sufficient heating cycles so as to delaminate the PCB under the GPU.
I'm sort of proud of this. Our software was dangerous.
Gracana
Delaminating the PCB is absolutely wild. I'm really fascinated by the idea of doing meetings in Second Life, though. What did people use for avatars at work?
OhMeadhbh
We used whatever avatars we put together.
Here's a link to Philip's avatar, which he intentionally kept basic for quite some time: https://community.secondlife.com/forums/topic/517881-help-me...
And here's a shot of mine: https://www.flickr.com/photos/opensourceobscure/2476204733/i...
And about in-world meetings. One thing Second Life did VERY WELL was it was always clear who was speaking. If someone was talking, there were green arrows sort of exploding out of their avatar's head. You couldn't miss it. Web-Ex at the time was HORRIBLE in this regard. Teams, Google Meet and Zoom are a little better than Web-Ex, but when meeting in Second Life, you could adjust your camera to get a good view of everyone in the meeting which also helped out.
tux3
The room temperature or precise way the paste was applied should not matter. Modern CPUs have very advanced dynamic voltage and frequency scaling (DVFS), which accounts for several sensors, including temperature.
These big x86 CPUs in stock configuration can throttle down to speeds where they can function with entirely passive cooling, so even if the cooler was improperly mounted, they'd only throttle.
All that to say, if GMP is causing the CPU to fry itself, something went very wrong, and it is not user error or the room being too hot.
mk_stjames
This was my first question as well- I thought it had been a long, long time since you could fry a CPU by taking away the heatsink.
As in... what, AMD K6 / early Pentium 4 days was the last time I remember hearing about cpu cooler failing and frying a cpu?
dwood_dev
Athlon era when AMD had no IHS but Intel had one. Intel also had thermal controls that AMD lacked.
I once worked on a piece of equipment that was running awful slow. The CPU was just not budging from its base clock of 700Mhz. As I was removing the stock Intel cooler, I noticed it wasn't seated fully. Once I removed it and looked I saw a perfectly clean CPU with no residue. I looked at the HSF, the original thermal paste was in pristine condition.
I remounted the HSF and it worked great. It ran 100% throttled for seven years before I touched it.
userbinator
This infamous video: https://www.youtube.com/watch?v=06MYYB9bl70
p_l
K6 depended on motherboard having thermal sensors - and which had to properly attach to the CPU in the first place.
Built-in thermal sensing came later.
RachelF
Yes, this is the point - software should never be able to physically damage the hardware it is on.
If it can, then the hardware is to blame.
mrheosuper
As a FW engineer, my software has released the magic smoke a lot.
account42
That's why firmware is often considered a separate category from software even though technically it's the same thing. Software is code that expects the hardware to works as specified, firmware is what achieves that.
themafia
If the throttling is not stable it could increase stress on the part by creating a bunch of transient but large thermal cycles through the chip. It would need to have some kind of exponential backoff on throttle so it doesn't immediately try to raise the frequencies when the temperature slightly dips.
secabeen
I would be interested to see if they had the same result with PTM7950 thermal material instead of paste. I've seen significantly better temps with these modern phase-change compounds, and they essentially eliminate application errors.
FuriouslyAdrift
Most likely it's the motherboard. ASRock is getting nailed right now for unstable XMP and CPU voltages (it's recommended to undervolt a little just in case).
The Asus Prime B650M motherboards they are using aren't exactly high end.
wmf
Yikes, this is the cheapest motherboard and failed Hardware Unboxed VRM tests. https://youtu.be/DTFUa60ozKY?t=744
caycep
conversely the asrocks actually did pretty good in that test...
J_Shelby_J
My friend just had an ASRock board cook his AMD CPU. Apparently a very common problem.
aidenn0
Can you link to a reputable source for what settings I should use on my asrock motherboard? I'd like to avoid this.
FuriouslyAdrift
No more than 1.2 volts on vsoc... but YMMV.
"According to new details from Tech Yes City, the problem stems from the amperage (current) supplied to the processor under AMD's PBO technology. Precision Boost Overdrive employs an algorithm that dynamically adjusts clock speeds for peak performance, based on factors like temperature, power, current, and workload. The issue is reportedly confined to ASRock's high-end and mid-range boards, as they were tuned far too aggressively for Ryzen 9000 CPUs."
https://www.tomshardware.com/pc-components/cpus/asrock-attri...
kvemkon
And the close-up photos of the socket with pins are missing.
T-A
A quick search on the NH-U9S shows it's a compact cooler for small systems, rated for up to 140 W (see e.g. [1]).
The 9950X's TDP (Thermal Design Power) is 170 W, its default socket power is 200 W [2], and with PBO (Precision Boost Overdrive) enabled it's been reported to hit 235 W [3].
[1] https://www.overclockersclub.com/reviews/noctua_nh_u9s_cpu_c...
[2] https://hwbusters.com/cpu/amd-ryzen-9-9950x-cpu-review-perfo...
[3] https://www.tomshardware.com/pc-components/cpus/amd-ryzen-9-...
BugsJustFindMe
Noctua does not use TDP for their heatsinks and instead have CPU compatibility charts. They say it's fine, with "medium turbo/overclocking headroom". https://ncc.noctua.at/cpus/model/AMD-Ryzen-9-9950X-1831
T-A
> Noctua does not use TDP for their heatsinks and instead have CPU compatibility charts.
Reviewers and sellers do, though. Here are a few more: [1][2][3][4]
The highest rating is from [1], which says
You cannot access the TDP guide from here, but we will tell you that it displays 140W TDP; however, it also says you can overclock that to closer to 160W or 180W TDP overall.
AVADirect advertises it as good for 115W [4].
It also beggars belief that a single 92mm fan would suffice to cool a 9950X, when the best 120 and 140 mm air coolers just barely reach 240W [5]. The only Noctua in that review, the 140mm NH-D15S, gets to 233W.
> They say it's fine, with "medium turbo/overclocking headroom"
Hopefully just an innocent mistake...
[1] https://www.tweaktown.com/reviews/7038/noctua-nh-u9s-cpu-coo...
[2] https://www.hardwareslave.com/reviews/cooling/noctua-nh-u9s-...
[3] https://www.frostytech.com/articles/2781/index.html
[4] https://www.avadirect.com/NH-U9S-chromax-black-125mm-Height-...
[5] https://www.tomshardware.com/pc-components/air-cooling/therm...
stouset
That’s a good catch, but don’t modern CPUs thermally throttle, rather than risk damage? Not that you should rely on this with an underpowered cooling solution but I would expect worse performance, not a fried chip.
spoaceman7777
Not really a lot it can do rapidly enough if there's only thermal paste on half the CPU.
It sounds like the user likely did the opposite of the "offset seating" of the heatsink that Noctua recommended.
account42
There thermal paste on the whole CPU in TFA, it's just thinner on one side because there was more pressure there. Or are you looking at the pic of the heat sink, which is larger than the CPU heat spreader and thus only partially covered by paste?
craftkiller
Looking at the AM5 pinout[0], it looks like those pins are VDDCR and VSS. There might be a little bit of PCIe sprinkled in towards the outer edges, but I'm not 100% on the orientation of this pinout vs the orientation of the CPU. I don't know anything about electricity so I've got nothing else to add.
[0] https://upload.wikimedia.org/wikipedia/commons/2/2d/Socket_A...
raverbashing
This is a nice guess but the likelihood that actual silicon area is closely connected to the pins in that area is not so obvious
nsteel
Isn't almost every other pin going to be power/ground on a high-power chip like this? On both the package and the die.
fxtentacle
"We suspect that GMP's extremely tight loops around MULX make the Zen 5 cores use much more power than specified, making cooling solutions inadequate."
I feel like if this was heat related, the overall CPU temperature should still somewhat slowly creep up, thereby giving everything enough time for thermal throttling. But their discoloration sure looks like a thermal issue, so I wonder why the safety features of the CPU didn't catch this...
touisteur
I'm guessing the temperature could increase quite fast (milliseconds or less) in heavy duty areas, especially when going scalar-to-dense-vector operations.
My best understanding of the avx-512 'power license' debacle on Intel CPUs was that the processor was actually watching the instruction stream and computing heuristics to lower core frequency before reaching avx512 or dense-avx2 instructions. I guessed they knew or worried that even a short large-vector stint would fry stuff...
Apparently voltage and thermal sensor have vastly improved and looking at the crazy swings on NVIDIA GPU's clocks seem to agree with this :-)
jeffbee
Are we talking "slowly" in a relative sense? A silicon die of this size has a thermal mass (guessing) around 10⁻³ J/K but a power dissipation rate over 200W, so it can rise from room temperature to junction temperature limits almost instantly.
topspin
People without a background in electronics don't appreciate what modern CPUs and GPUs are doing: the amount of current flowing through these devices is just mind blowing. With adequate cooling, a Ryzen 9 9950X is handling somewhere in the neighborhood of 150-200 amps under high load.
nisegami
I initially scoffed at the 150-200 amps. But I know core voltage is usually in the neighbourhood of 1V so to draw 200W, you really would have to basically be moving 200A of current. That's wild.
BearOso
They said it took months for each CPU to fail. Both systems used the same inadequate heatsink/fan. Then there's also the lower-end motherboards (they are not "top-quality", the brand means nothing) and the miniscule 450W power supply used in the initial configuration, which are confusingly paired with a 16-core CPU and 64/96GB of RAM.
It doesn't strike me as odd that running an extremely power-heavy load for months continuously on such configurations eventually failed.
edgineer
"We don't overclock or overvolt or play other teen games with our hardware."
You overclock as a teen so that as an adult you know to verify your CPU's voltage, clock speeds, and temperature at a minimum when you build your own system.
They made no mention of monitoring of CPU temperature, ECC corrected/detected errors, or throttling. They then ran CPU benchmark loads for several months on the system.
"The so-called TDP of the Ryzen 9950X is 170W. The used heat sinks are specified to dissipate 165W, so that seems tight."
Yikes. You need a heatsink rated much higher. These CPUs were overheated for months.
Gracana
CPUs with stock cooling solutions will turbo boost up to max temp and stay there, that's completely normal and shouldn't cause a CPU to physically burn up, even if you do it for months.
edgineer
"boost up to max temp and stay there"
At stock settings CPUs will boost depending on many factors. Once one of several different limits is hit the CPU will not boost as high, trying to find a steady state where it stays below Tjmax.
Note the following: the 9950x does not come with a stock cooler. AMD recommends water cooling for the 9950x. Transistor lifetime decreases exponentially with temperature.
I'd expect that a 9950x under sustained load paired with a "165W" cooler would not only not boost, but would throttle to below base clocks.
In the case of CPU cooling, I don't agree that relying on the CPU's thermal safety nets to continuously regulate the system to avoid damage is good practice. With additional cooling to ensure it never reaches Tjmax, this also will result in better CPU performance, a tangible benefit.
Had the author monitored his systems, he would have observed high temperature and throttling. Yes, in 2025 it's arguable that a CPU's safety net should be reliable wrt temperature, even if you run with no heatsink at all. I also agree that TDP specifications are unclear.
But the bottom line is that you should pay the extra $100 or so to cool your CPU properly. It will be faster and more reliable.
Please take care of your equipment; do not take it for granted.
thway15269037
I don't know about GMP, but I recently built a PC with 9950X3D. As part of initial testing, I ran Prime95 for 48 hours. Everything ran stable, but I noticed that part of the tests, I think it was FFT or something like that, caused incredibly sharp increase in temp. We are talking 60C average in the rest of the test vs immediate (less than a 5 seconds) 95+ degrees when that FFT thingie started. It was very weird.
That's when I discovered actually ancient term "power virus". Anyway, after talking to different people I dismissed this weird behavior and moved on.
Reading this makes me worry I actually burned mobo in that testing.
jmb99
Different use patterns will result in different temperatures. Very tight math loops (no memory/IO wait) will lead to higher temperatures than something that that relies on L2/3 cache or main memory, even though they’ll both report “100% CPU use” and probably use similar amounts of power. And, different operations will produce heat in different areas of the die; depending on physical layout, some operations might generate heat in a tiny cluster, whereas some others might generate heat in larger spread out areas. Even though both of those cases might use the same amount of power and generate the same amount of heat, the temperatures will be drastically different due to the heat concentration.
Iirc the FFT step uses AVX, and on Zen 5 that’ll be AVX-512. It should keep 100% of the required data in L1 caches, so you’re keeping the AVX units busy literally 100% of the time if things are working right. The rest of the core will be cold/inactive, so if you’re dumping an entire core’s worth of power into a teeny tiny ALU, which is gonna result in high temps. Most (all?) processors downclock under heavy AVX load, sometimes by as much a 1GHz (compared to max boost), because a) the crazy high temperatures results in more instability at higher frequencies, and b) if the clocks were kept high, temperatures would get even higher.
userbinator
Try LINPACK, it's even more stressful than Prime95.
bob1029
Could be the power supply and load profile?
I've heard some really wild noises coming out of my zen4 machine when I've had all cores loaded up with what is best described as "choppy" workloads where we are repeatedly doing something like a parallel.foreach into a single threaded hot path of equal or less duration as fast as possible. I've never had the machine survive this kind of workload for more than 48 hours without some kind of BSOD. I've not actually killed a cpu yet though.
userbinator
I've never had the machine survive this kind of workload for more than 48 hours without some kind of BSOD.
Then you shouldn't trust the results of your work either, as that's indicative of a CPU that's producing incorrect results. I suggest lowering the frequency or even undervolting if necessary until you get a stable system.
...and yes, wildly fluctuating power consumption is even more challenging than steady-state high power, since the VRMs have to react precisely and not overshoot or undershoot, or even worse, hit a resonance point. LINPACK, one of the most demanding stress tests and benchmarks, is known for causing crashes on unstable systems not when it starts each round, but when it stops.
bob1029
The results might be invalid for one generation but the model is resilient to these kinds of events overall. Far more resilient than my operating system is.
Randomly flipped genome bits could even be beneficial for escaping local minima and broken RNG in evolutionary algorithms. One bad evaluation won't throw the whole thing off. It's gotta be bad constantly.
fc417fc802
I experienced that with a GPU years ago. A workload I wrote caused a pronounced high frequency noise from the card that I've never encountered the like of before or since. I'd describe it as a very high frequency chirping. I refactored the program rather than seeing what would come of it.
bee_rider
Is that, like, an intentional stress-test for the hardware that you’ve come up with?
bob1029
No. It is just how the algorithms play out:
1. Evaluate population of candidates in parallel
2. Perform ranking, mutation, crossover, and objective selection in serial
3. Go to 1.
I can very accurately control the frequency of the audible PWM noise by adjusting the population size.
nromiun
How is that possible? Even if the chip did not get enough cooling it should have been just throttled heavily.
jsheard
Modern silicon is so dense and heats up so fast that throttling is easier said than done. I think they have to model and predict the thermals ahead of time nowadays, because by the time they could react to a temp sensor alone, the chip might already be toast.
tliltocatl
Maybe the throttling circuitry/firmware simply doesn't have enough time to react.
Get the top HN stories in your inbox every day.
As there is ongoing drama with Zen 5 and power issues, there are people with the instruments and the motivation to investigate this. You should consider contacting Gamers Nexus, and help them to get your test suite running. They can measure power draw and do a thermal analysis of this CPU, and they'd likely be eager to do it, given the possibility of making a bunch of dramatic YouTube content about design flaws in widely used hardware. That's pretty much their whole schtick in recent years.
> Modern CPUs measure their temperature and clock down if they get too hot, don't they?
Yes. It's rather complex now and it involves the motherboard vendor's firmware. When (not if) they get that wrong CPUs burn up. You're going to need some expertise to analyze this.