Duet mini 5+ freezes and shuts off heaters

droftarts

@michael-hathaway are Nylon and CF more prone to static buildup? Perhaps grounding the nozzle would help. Make sure the extruder motor is grounded, too. I still have no idea how it’s triggering a ‘manual reset’, unless a sudden zap is enough to drag the reset signal to ground.

Ian

Michael Hathaway

The printer is grounded and there is no carpet, only concrete in workshop. If static were the culprit, it would not print at lower temperatures.

It was 45-46°C in my workshop last week and I think the higher printing temperatures over loaded the board. While it was triggering a reset, one time it also triggered an emergency stop on its own.

It is now 26°C in the shop and it is printing my lower temp Nylon fine. I will load the HTNCF next and see if fails or continues to print.

droftarts

@michael-hathaway I asked @dc42 if he thought high ambient temperature causing the Duet MCU to overheat would trigger a reset that looks like a manual reset. He said "I think that's unlikely, but not impossible." Without knowing how hot the Duet board is getting (map the on-board MCU temp sensor to see a graph) that's difficult to know. An MCU reset or shutdown would be triggered somewhere above 85°C.

I know you said "the printer is grounded", but specifically are the metal parts of the hotend grounded? I still think a static shock is more likely, and grounding the hot end would eliminate this. You said you swapped out the PT1000 for a new one and got a different failure, so possibly static charge in the the hot end has arced through that and caused a reset. High ambient temperature + high temperature nozzle (this would lower humidity, making static worse) + possibly CF and Nylon that produce more static charge than PETG might be enough to reset the MCU.

However, @chrishamm thinks that a static shock to the temperature inputs may cause a MemoryProtectionFault reset (it does on the 6HC, not sure about the Mini 5+), which would show up in the M122 report.

One other question: How did the old Mini 5+ fail? Could it be related? Maybe the MCU got zapped?

Ian

Michael Hathaway

@droftarts I ordered a ring terminal to fit the hot end for grounding, will be here tomorrow. It was extremely humid here and hot.

All the boards show the same failure. I have had trouble free performance for 1.5 years. I have had the mini board shut down before when the chamber temperature went over 50C. This is an unusual heat wave in California. If this keeps up, I think I will install air conditioning in the workshop.

The other Matterhacker guys prefer their Azteeg boards. But for me, I only use Duet.

Michael Hathaway

Added electrical ground wire from the hotend to the frame and to electrical earth ground. Printer still failed prints.

Pulled printer from workshop into air conditioned building. Printer has produced several 10+ hour prints.

For fun, I added a copper heat sink to the MCU. Temperature now reading a -6.7°C drop. Probably not worth doing, but if a few degrees means the difference between another stall and not, then I am ok with it. I can now be part of the "stuck a heat sink on my MCU" fan club. lol

I think that I will make a new case cover that brings ambient air from outside the heat chamber to the Duet and then draw it back out again. Shouldn't be too difficult, I think two 60mm fans would work well.

infiniteloop

@michael-hathaway

I think that I will make a new case cover that brings ambient air from outside the heat chamber to the Duet and then draw it back out again. Shouldn't be too difficult, I think two 60mm fans would work well.

I often see housings with the Duet mounted near the bottom, cooling them from above. In your case, I’m not sure, but it is really essential to follow the explicit recommendations: cool the bottom of the board! Then, even a mediocre blower at low speed is sufficient.

Michael Hathaway

@infiniteloop I am not sure I follow you here. Why would I want to cool the bottom of the board? Are you talking about the back side of the board?

Cooling the back side of the board is for the stepper drivers as well as a couple of other items. I had a rather large fan cooling the back side of the board at the start of this. Didn't help what so ever. No, what ever is tripping the reset of the board is either on the front side or the board in it's entirety.

My board enclosure is located inside the heated print chamber. Blowing hot air over an overheated board is not going to be the answer for me. Bringing in cool air to cool the board and then exhausting it so that it doesn't cool the print chamber is what is needed.

My warped sense of humor here

infiniteloop

@michael-hathaway said in Duet mini 5+ freezes and shuts off heaters:

Are you talking about the back side of the board?

Sorry for my English: it's the back side what I mean.

Michael Hathaway

My original case had a 92mm fan on the front side and it was designed to blow air on the front and force the air around to the back side to cool the steppers. Still its hot air circulating and I need to do this a little differently this time. I didn't really have problems until we had a few really hot days and I was printing high temperature plastics, which made the chamber even hotter, surpassing the limits of the system.

droftarts

@michael-hathaway when you initially asked about this problem, there were a number of useful pieces of information that you weren’t forthcoming with: that the Duet (and PSU, I think) are mounted inside the heated chamber, the high ambient temperature at the time, printing high temperature materials. This lead to a focus on other factors that might cause the reset, rather than the obvious one; the board temperature.

What sort of temperature does the chamber get to? You are stressing the electronics on the board running it inside the enclosure. The MCU itself may be rated to 85C (operating temperature, not ambient temperature), but other components on the board may not be, and could be causing voltage fluctuations and the ‘phantom’ resets. I don’t think we publish a temperature range of operation for Duet boards (I’ll ask @dc42), but subjecting it to heat intentionally seems less than ideal. I think you have got away with it in past, but a combination of factors has pushed the board beyond its temperature limits. Is it possible to mount it outside the enclosure?

Ian

Michael Hathaway

@droftarts I agree with you completely.

The boards are not faulty, I am just pushing them to their limits. It's not really intentional though, I have just gotten away with it until now.

I don't think it is the MCU causing the shutdown, but something else on the board. I have printed no less than 20,000 prints on my original Duet mini 5. IMO it's way better than the American or any other boards on the market.

It's just a Prusa aluminum clad enclosure, many people use these. I have never pushed the chamber over 60°C and I know it was over 50 when I was having problems with the MCU in the 67-70C range. It was 48°C here in the shade and my workshop was close to that temperature. Both the combined high ambient temperature and the increase current through the board to bring the hotend up to 330C is what I think caused the resets/emergency shutdown.

Michael Hathaway

Will the Duet 6HC handle higher temperatures better than the mini 5+? Are the components on the board higher grade?

TLAS

@michael-hathaway said in Duet mini 5+ freezes and shuts off heaters:

The printer went 5 layers and stopped. I have no idea why it is freezing.

We’ll maybe you shouldn’t (should) leave it out in the cold? Sorry, couldn’t resist…

I’m fully certified to use dad jokes…
https://youtu.be/4CeWYM08s7A

Michael Hathaway

@tlas I don't wish to wire for external board. I am not the first person to print inside an enclosure. I just didn't know the thermal limits of this board until now. I will end up switching over to the 6HC as it seems to be more robust.

TLAS

@michael-hathaway
Mostly a joke there (board freezing… leaving it out in the cold…).

I can understand why folks would want to keep the boards in heated chambers.

FYI, bringing in a line of compressed air at ~10 psi is helpful to provide local cooling without impacting the overall chamber too much.

Michael Hathaway

@tlas I joked with a friend this morning when I told him I was going to water cool my MCU.

droftarts

@michael-hathaway Unfortunately, I don't think it's just the MCU, and this issue isn't just limited to the Mini 5+. Other components aren't rated higher than 85C, and running at high temperatures will shorten the life of all the components. We don't give recommended operating temperature limits (though we are talking about it), because the board heating itself up due to FETs/stepper drivers/power supplies on the board is very application specific.

With the 6HC, the higher rating on the stepper drivers mean they run cooler, so don't add as much self-heating to the board, so perhaps that's what you mean when you say they are more robust.

We really don't advise running Duets in heated chambers; it will shorten their life.

Ian

Michael Hathaway

@droftarts Yes, this is what I said in my previous statement to you. It is all the components on the board.

The chamber is heated by the bed and hotend. The reason why it prints petg and not the high temp nylon is because the chamber temp rises much higher with the higher temp plastics. Especially on very hot days. I realize now that I cannot print my higher temp plastics until I bring cold air in to cool the electronics.

Michael Hathaway

A small update: The issue has nothing to do with heat of the chamber or the Duet 3 mini. After a ton of testing, I have narrowed the issue down around the extruder.

Exhausting all other tests, I decided to run a print cycle with no filament running through the extruder and it finished the print. I am able to print at high temperature with flexible Nylon, but the stiff carbon fiber nylon throws a reset.

I installed a new stepper and Bontech extruder. I will post results later.

Michael Hathaway

10/25/2022, 1:15:28 AM 	m122
=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.4.1 (2022-06-01 21:06:56) running on Duet 3 Mini5plus Ethernet (standalone mode)
Board ID: 6MFQS-V096U-D65J0-40KMJ-JS03Z-R72JU
Used output buffers: 1 of 40 (22 max)
=== RTOS ===
Static ram: 103684
Dynamic ram: 105204 of which 0 recycled
Never used RAM 32752, free system stack 168 words
Tasks: NETWORK(ready,25.9%,260) ETHERNET(notifyWait,0.3%,568) HEAT(notifyWait,0.0%,358) Move(notifyWait,0.0%,290) CanReceiv(notifyWait,0.0%,942) CanSender(notifyWait,0.0%,372) CanClock(delaying,0.0%,339) TMC(notifyWait,0.7%,81) MAIN(running,72.1%,422) IDLE(ready,0.2%,29) AIN(delaying,0.8%,264), total 100.0%
Owned mutexes:
=== Platform ===
Last software reset at 2022-10-25 00:59, reason: User, GCodes spinning, available RAM 32752, slot 0
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00489000 BFAR 0xe000ed38 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
MCU revision 3, ADC conversions started 101415, completed 101415, timed out 0, errs 0
Step timer max interval 1486
MCU temperature: min 41.6, current 42.2, max 43.3
Supply voltage: min 23.8, current 23.9, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 5301, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 5301, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 12, reads 5298, writes 12, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 5301, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 9, reads 5301, writes 9, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2022-10-25 01:15:24
Cache data hit count 172505060
Slowest loop: 12.16ms; fastest: 0.13ms
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 1.0ms, write time 9.1ms, max retries 0
=== Move ===
DMs created 83, segments created 3, maxWait 42811ms, bed compensation in use: mesh, comp offset 0.000
=== MainDDARing ===
Scheduled moves 2, completed 2, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 2], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.6
Heater 1 is on, I-accum = 0.5
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== Filament sensors ===
Extruder 0 sensor: ok
=== CAN ===
Messages queued 909, received 0, lost 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 18 (min 18), ts 506/0/0
Tx timeouts 0,0,505,0,0,402 last cancelled message type 30 dest 127
=== Network ===
Slowest loop: 9.29ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- Ethernet -
State: active
Error counts: 0 0 0 0 0
Socket states: 2 2 2 2 2 0 0 0