New heater tuning algorithm

dc42

@gloomyandy said in New heater tuning algorithm:

I originally posted by mistake in 3.2-beta3.2 thread, but I think it is better here...

I've been testing the new algorithm on my printer and have seen some strange results when tuning the bed heater. My bed is a 310x310x6mm aluminium plate with a magnetically attached steel sheet on top. The heater is a mains powered 250x250mm 500W Silicon pad (with the thermistor embedded in it). When tuning my target temperature is 60 degrees. Ambient temperature is around 21 degrees. This combination is probably not ideal as there is a lot of lag between the pad thermistor and the actual bed plate temperature.

If I run the tuning from cold then it always seems to run for the full 30 cycles. However if I run it when it has cooled (from 60) to say 30 degrees (using the A parameter to specify the ambient temperature) then it will complete after 5 cycles. To try and understand what is going on I have added some additional debug to the tuning process. Here is the output from the first case:
I suspect that the relatively large thermal mass means that as the full bed heats up it is impacting the heating/cooling times of the test cycle. But there also seems to be something a little odd going on with the "dLow" values as these are sometimes zero.
...
It may well be that I need to add an additional thermistor (perhaps embedded in the edge of the bed) to get more accurate readings of the actual bed temperature. However I don't think the setup I have is that unusual so I thought the data might be of interest to you.

I think I know what is going on. My theory is that systems such as yours behave somewhat like a FOPDT (first order process plus dead time) plus a thermal reservoir that is somewhat weakly coupled to it. When heating from cold, the temperature of the reservoir lags the temperature of the heating element and temperature sensor. It is only when it has heated up to their average temperatures that the behaviour during the tuning cycles becomes stable.

This phenomenon also occurs with hot ends to some extent, which is why the tuning algorithm does two heating/cooling cycles before is starts collecting data. It's clear that with bed heaters like yours, more of these dummy cycles are needed.

I too have a bed heater that behaves in this way, however in my case the thermistor readings have become noisy, so it is not surprising that it always goes the full 30 cycles.

Any solution must not only solve the problem you encountered but also handle noisy thermistors reasonably well, by continuing to average over a reasonably large number of cycles. One option would be to check the consistency after 5 cycles, and if it is poor then throw the data away and start again. So instead of doing 2 dummy cycles and up to 30 real cycles, it would do 7 dummy cycles and up to 25 real cycles. Do you think this would work for your heater?

gloomyandy

@dc42 said in New heater tuning algorithm:

Any solution must not only solve the problem you encountered but also handle noisy thermistors reasonably well, by continuing to average over a reasonably large number of cycles. One option would be to check the consistency after 5 cycles, and if it is poor then throw the data away and start again. So instead of doing 2 dummy cycles and up to 30 real cycles, it would do 7 dummy cycles and up to 25 real cycles. Do you think this would work for your heater?

On my test build I cleared the data every 5 cycles and continued on up to a max of 30 cycles in total. So basically I tuned in groups of 5 cycles. In my case this resulted in the tuning completing after 10 cycles total. So yes I'm pretty sure that your proposal would work in my case (though I'm not sure if it would in the case of even bigger/thicker beds as they might take even longer to heat up). I wonder if there is any way to differentiate between a noisy reading (which will benefit from more readings) and a lagging reading (which probably needs a restart)? But yes I'm pretty sure that being able to extend the dummy cycles would help.

dc42

Some other solutions come to mind:

We could store the last 5 readings instead of just the mean and standard deviation; then we could check whether the last 5 readings are good enough. I am reluctant to do this because of the additional RAM that would be needed. One of the goals of the new algorithm was to use minimal RAM so that it can be implemented on the Duet 3 tool board.
We could monitor the ratio of heating time to cycle time, and continue doing dummy cycles until this stabilises or has the appearance of being noise rather than a trend. For example, wait for the ratio of heating time to cycle time to be at least 95% of the value in the previous cycle.

Your thoughts?

gloomyandy

I agree that it would be a pity not to be able to use the same mechanism for the tool boards. On my system the way that the cooling time gets longer and heating time shorter is very noticeable so yes perhaps that could be used to indicate that things have not stabilised. I'm not show how much noise impacts those readings though?

Oh and I think there may be a bug in the code that determines the dLow time. In the data I posted above this is sometimes 0 which I don't think is correct. I added some extra code to my experimental version which I think fixes it. I'm not at home at the moment so will post more details later. But I think the problem was that sometimes we fail to detect a "low point" correctly after the heater has been turned on and it is using time values from the previous cycle.

dc42

@gloomyandy said in New heater tuning algorithm:

I agree that it would be a pity not to be able to use the same mechanism for the tool boards. On my system the way that the cooling time gets longer and heating time shorter is very noticeable so yes perhaps that could be used to indicate that things have not stabilised. I'm not show how much noise impacts those readings though?

If the temperature readings are noisy then sooner or later the heating time will be at least 95% of the heating time in the last cycle, which would trigger the start of data collection for tuning. So the effect of noise may be to suggest that stabilisation has occurred sooner than it actually has. I am not concerned by this, because we can't expect good results from noisy temperature readings anyway.

Oh and I think there may be a bug in the code that determines the dLow time. In the data I posted above this is sometimes 0 which I don't think is correct. I added some extra code to my experimental version which I think fixes it. I'm not at home at the moment so will post more details later. But I think the problem was that sometimes we fail to detect a "low point" correctly after the heater has been turned on and it is using time values from the previous cycle.

Thanks, I'll wait for the detail from you.

gloomyandy

Hi @dc42 , sorry that took a little longer then I expected!

So the problem I was seeing was that when the bed was hot, occasionally when the tuning code switched from heater off to heater on the temperature would not drop below the temperature recorded in peakTemp, or rather it had already heated to be above that temperature by the next time that the tuning code runs. This resulted in new values for peakTemp, afterPeakTemp, peakTime and afterPeakTime not being captured which in turn resulted in the later if statement:

		else if (afterPeakTime == peakTime && temperature - tuningTargetTemp >= TuningPeakTempDrop - TuningHysteresis)
		{
			afterPeakTime = now;
			afterPeakTemp = temperature;
		}

never being executed. This then gives an odd value for heatingRate as the calculation used the times and values from the previous cooling cycle for afterPeakTime and afterPeakTemp.

I've made some changes to my test branch which seem to help with this (but this may not be the best fix). You can see the changes here:
https://github.com/gloomyandy/RepRapFirmware/commit/1726cdf3647ecf812400cded0b04d295a30420fc

Let me know if any of that makes sense!

Oh and one final thought. With a hot bed we can end up with some very short times in dLow (I've seen values as low as 250mS or 500ms and occasionally 0). With the sample time being 250mS any "jitter" from say 1000 to 750 or 1250 can result in a pretty big impact on the standard deviation and so on the stability test.

0

gloomyandy committed to gloomyandy/RepRapFirmware

Experimental fix for when temperature drop is not detected with heaters
on

dc42

Thanks Andy, I agree with your fix and I have committed it.

We may need to reduce the heat sample time to handle heaters with very short dead times. It's on my work list to make it variable again.

jay_s_uk

@chrishamm
The new heater tuning output doesn't get saved using M500 when a duet is connected to an SBC.
They save fine when in standalone mode

chrishamm

@jay_s_uk This is known and it will be addressed in 3.2-b4. DSF 3.2-b3 doesn't have the new object model properties either, so it cannot save them yet.

jay_s_uk

@chrishamm good. Just wanted to make sure it was known

jay_s_uk

@dc42

Here is a snapshot of the bed tuning being carried out on a my duet 3 in standalone mode.
As you can see, the cool down period takes longer and longer as the bed heats up.
I have a 500x500x10 aluminium bed with a 2kW 240v silicone heater with a thermocouple embedded in it.
These were my results.

Warning: heater behaviour was not consistent during tuning
Auto tuning heater 0 completed after 30 cycles in 1047 seconds. This heater needs the following M307 command:
 M307 H0 R1.489 C145.3 D2.44 S1.00 V27.0
Send M500 to save this command in config-override.g

gloomyandy

@dc42 Hi David,
I've been taking another look at this and have implemented your suggestion:
"One option would be to check the consistency after 5 cycles, and if it is poor then throw the data away and start again. So instead of doing 2 dummy cycles and up to 30 real cycles, it would do 7 dummy cycles and up to 25 real cycles."

Though in my case it will actually still run up to 30 real cycles after the 7 dummy ones. I basically added a new phase to insert the extra dummy cycles. I've been testing it and it works fine for me and the results it produces are pretty consistent. You can see the changes I made here: https://github.com/gloomyandy/RepRapFirmware/commit/868993b6e534e985bf5fe689a76ed33ce4ddb9a1

I've also been running a number of tests using the results from the new tuning code on both my bed and hot end heaters and so far it looks pretty good. I see very stable temperatures both with and without the cooling fan. The initial overshoot is pretty low (approx 2 degrees on the hotend and less than 1 degree on the bed).

0

gloomyandy committed to gloomyandy/RepRapFirmware

Run extra 5 tuning cycles if readings are not stable

pixelpieper

Hi, @dc42 - I think I found a bug when using multiple thermistors together with M143 monitors. I use a duet Maestro with an AC powered silicone mat heater. I have the heater configured such that a) we regulate for a thermistor placed on the edge of the aluminum bed (Bedplate) but I want the thermistor integrated inside the silicone mat (Bedmat) not to reach temperatures higher than 130°C - hence I have configured two monitors with M143, disabling the heater temporarily once the mat reaches 130°C:

;; thermal -------------------------------------------------
; Sensors --------------------------------------------------
M308 S0 P"bedtemp" Y"thermistor" T100000 B3950 A"Bedmat"; configure sensor 0 as thermistor on pin temp0
M308 S1 P"e0temp" Y"pt1000" A"Hotend"  ; configure sensor 1 as thermistor
M308 S2 P"ctemp" Y"thermistor" T100000 B3950 A"Chamber"	;Chamber fan
M308 S3 Y"mcu-temp" A"Board"
M308 S4 P"e1temp" Y"thermistor" T100000 B3950 A"Bedplate"
 
; Heaters --------------------------------------------------
;Bed
M950 H0 C"bedheat" T4                           ; create bed heater output on out0 and map it to sensor 4
M950 H1 C"e0heat" T1                           ; create nozzle heater output on e0heat and map it to sensor 1
M140 H0   

;PID Settings
M307 H0 A301.0 C845.3 D1.4 S1.00 V23.6 B0	
M307 H1 A482.6 C291.7 D5.5 S1.00 V23.6 B0 ;V6

; Monitors & Limits
M143 H0 P1 T0 A2 S130 C0                     ; Regulate (A2) bed heater (H0) to have pad sensor (T0) below 130°C. Use Heater monitor 1 for it
M143 H0 P2 T0 A0 S135 C0                     ; Fault (A0) bed heater (H0) if pad sensor (T0) exceeds 135°C. Use Heater monitor 2 for it
M143 H0 P0 S120                            ; Set bed heater max temperature to 120°C, use implict monitor 0 which is implicitly configured for heater fault
M143 H1 S400                                  ; set temperature limit for heater 1 to 275C

I would have assumed these monitors would be also considered when tuning, but that does not seem to be the case, I peak 40°C over my limit of 130°C:

After tuning M143 works again as expected:

dc42

@gloomyandy said in New heater tuning algorithm:

I've been taking another look at this and have implemented your suggestion:
"One option would be to check the consistency after 5 cycles, and if it is poor then throw the data away and start again. So instead of doing 2 dummy cycles and up to 30 real cycles, it would do 7 dummy cycles and up to 25 real cycles."

Thanks; but I'd rather try my other suggestion, i.e. do idle cycles until the on-time settles (or a limit is reached). That way it should usually do less than 7 idle cycles.

dc42

@pixelpieper said in New heater tuning algorithm:

Hi, @dc42 - I think I found a bug when using multiple thermistors together with M143 monitors. I use a duet Maestro with an AC powered silicone mat heater. I have the heater configured such that a) we regulate for a thermistor placed on the edge of the aluminum bed (Bedplate) but I want the thermistor integrated inside the silicone mat (Bedmat) not to reach temperatures higher than 130°C - hence I have configured two monitors with M143, disabling the heater temporarily once the mat reaches 130°C:

As with the old heater tuning algorithm, the new one will always overshoot the target temperature. You need to be aware of this when choosing the target temperature. The heater itself is switched off when the target temperature is reached, so the overshoot is purely due to excess thermal mass in the heater and poor coupling to the thermistor.

gloomyandy

@dc42 said in New heater tuning algorithm:

Thanks; but I'd rather try my other suggestion, i.e. do idle cycles until the on-time settles (or a limit is reached). That way it should usually do less than 7 idle cycles.

Hi I'll be interested to see how well that works. I did play around with something along those lines but could not find a good stability test that only needed a small number of cycles. I should also have said that in the my test version if the overall results are stable after 5 cycles then it will use those results (as it does now), the extra cycles are only used if things are not stable.

Let me know if you want me to give anything a try.

jaymcd0626

Tried this and the cycle never seems to stop.

https://forum.duet3d.com/topic/20046/attempting-to-pid-tune-my-heat-bed?_=1606325917554

jay_s_uk

@jaymcd0626 it'll get to 30 cycles and then fail but will give you an output. This can be used for the time being.
Dc42 is going to be tweaking the heater algorithm

jbarros

Just to add some data into this, I just performed pid tuning on a new printer I'm finishing, new heater model so I don't have a baseline for it.

Heater: Keenovo 240V 750W 300x300mm
Bed: 310x310x4mm

RepRapFirmware for Duet 3 MB6HC version 3.2-beta4 running on Duet 3 MB6HC v1.01 or later (standalone mode)

M308 S0 P"temp0" Y"thermistor" T100000 B3950       ; configure sensor 0 as thermistor on pin temp0 - 100k beta 3950 1% thermistor (Used in Keenovo AC silicone mats)

28/11/2020, 22:47:58 M307 H0
Heater 0 model: heating rate 0.911, cooling time constant 393.9, dead time 1.81, max PWM 1.00, calibration voltage 24.1, mode PIDComputed PID parameters: setpoint change: P108.5, I4.069, D137.1, load change: P108.5, I13.719, D137.1
28/11/2020, 22:47:53 M303
Heater 0 tuning succeeded, use M307 H0 to see result
28/11/2020, 22:44:05 Warning: heater behaviour was not consistent during tuningAuto tuning heater 0 completed after 30 cycles in 1711 seconds. This heater needs the following M307 command: M307 H0 R0.911 C393.9 D1.81 S1.00 V24.1
Edit the M307 H0 command in config.g to match this.
28/11/2020, 22:42:06 M303
Heater 0 is being tuned, phase 4 of 4
28/11/2020, 22:18:24 Auto tune starting phase 3, fan off
28/11/2020, 22:17:03 Auto tune starting phase 2, heater settling
28/11/2020, 22:16:29 M303
Heater 0 is being tuned, phase 2 of 4
28/11/2020, 22:15:39 Auto tune starting phase 1, heater on
28/11/2020, 22:15:33 M303 H0 S80
Auto tuning heater 0 using target temperature 80.0°C and PWM 1.00 - do not leave printer unattended

hmortensen

Hi

I've tried tuning my hotend heater after updating to 3.2b4, but this step "Auto tune starting phase 3, fan on" ran for 45 min without finishing, and then I turned it off.

Is something wrong, or should it take this long?

Attached snip of temp chart and console log.

Screenshot 2020-11-29 200054.png

11/29/2020, 7:47:45 PM	M303
Heater 1 is being tuned, phase 5 of 5
11/29/2020, 7:37:59 PM	M303
Heater 1 is being tuned, phase 5 of 5
11/29/2020, 7:11:50 PM	M303
Heater 1 is being tuned, phase 5 of 5
11/29/2020, 7:07:56 PM	M303
Heater 1 is being tuned, phase 5 of 5
11/29/2020, 7:05:29 PM	Auto tune starting phase 3, fan on
11/29/2020, 7:04:57 PM	M303
Heater 1 is being tuned, phase 4 of 5
11/29/2020, 7:02:30 PM	M303
Auto tune starting phase 3, fan off
Heater 1 is being tuned, phase 4 of 5
11/29/2020, 7:01:05 PM	Auto tune starting phase 2, heater settling
11/29/2020, 7:01:03 PM	M303
Heater 1 is being tuned, phase 2 of 5
11/29/2020, 6:58:30 PM	M303
Heater 1 is being tuned, phase 2 of 5
11/29/2020, 6:58:05 PM	Auto tune starting phase 1, heater on
11/29/2020, 6:58:00 PM	M303 T0 S250
Auto tuning heater 1 using target temperature 250.0°C and PWM 1.00 - do not leave printer unattended
11/29/2020, 6:57:47 PM	Connection established
11/29/2020, 6:57:46 PM	Connection interrupted, attempting to reconnect...
HTTP request timed out
11/29/2020, 6:57:36 PM	Upload of config.g successful after 0s
11/29/2020, 6:56:36 PM	M303 T0 S250
Error: M303: heater 1 target temperature must be below the temperature limit for this heater (250.0C)
11/29/2020, 6:44:03 PM	M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.2-beta4 running on Duet WiFi 1.02 or later
Board ID: 08DGM-917NK-F2MS4-7JTDJ-3SS6J-KWVNH
Used output buffers: 3 of 24 (15 max)
=== RTOS ===
Static ram: 24108
Dynamic ram: 104312 of which 40 recycled
Never used RAM 1588, free system stack 188 words
Tasks: NETWORK(ready,148) HEAT(blocked,294) MAIN(running,459) IDLE(ready,20)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:46:27 ago, cause: software
Last software reset details not available
Error status: 0x00
MCU temperature: min 28.1, current 28.9, max 29.4
Supply voltage: min 23.9, current 24.0, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: position 0, standstill, SG min/max not available
Driver 1: position 0, standstill, SG min/max not available
Driver 2: position 0, standstill, SG min/max not available
Driver 3: position 0, standstill, SG min/max not available
Driver 4: position 0, standstill, SG min/max not available
Driver 5: position 0
Driver 6: position 0
Driver 7: position 0
Driver 8: position 0
Driver 9: position 0
Driver 10: position 0
Driver 11: position 0
Date/time: 2020-11-29 18:43:55
Cache data hit count 4294967295
Slowest loop: 259.02ms; fastest: 0.19ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 4.6ms, write time 39.8ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 169, MinFreeDm: 169, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 1754.37ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.24
WiFi MAC address 60:01:94:2e:a5:fa
WiFi Vcc 3.40, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 23840
WiFi IP address 192.168.1.7
WiFi signal strength -64dBm, mode none, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0
11/29/2020, 6:43:25 PM	M115
FIRMWARE_NAME: RepRapFirmware for Duet 2 WiFi/Ethernet FIRMWARE_VERSION: 3.2-beta4 ELECTRONICS: Duet WiFi 1.02 or later FIRMWARE_DATE: 2020-11-26
11/29/2020, 6:41:14 PM	Connected to 192.168.1.7