Solved Duet 3 +SBC : Print randomely freeze when runin my script on RPI

[chrishamm]

What DSF version are you using, 3.1.1? I noticed you are mixing async with non-async code in your C# app, which isn't recommended. Here a slightly improved version:

using DuetAPI.Commands;
using DuetAPI.Connection;
using DuetAPI.Machine;
using DuetAPIClient;
using System;
using System.IO;
using System.Net.Sockets;
using System.Text.Json;
using System.Threading.Tasks;

namespace dws_tst1
{
    class Program
    {
        static AnalogSensor oDuetSensor;

        static SubscribeConnection oSubscription;
        static CommandConnection oCommand;

        static MachineModel oModel;

        static double fFanLastValue;

        static async Task Main(string[] args)
        {
            Console.WriteLine("Start");
            fFanLastValue = 0;

            Console.WriteLine("Connections");
            oSubscription = new SubscribeConnection();
            oCommand = new CommandConnection();

            Console.Write("Connecting");
            await oSubscription.Connect(SubscriptionMode.Patch);
            await oCommand.Connect();
            Console.WriteLine("Connection OK");

            Console.WriteLine("Reading model");
            oModel = await oCommand.GetMachineModel();

            Console.WriteLine("Resetting MCU/CPU Fan");
            await oCommand.PerformSimpleCode("M106 P3 S0");

            Console.WriteLine("Selecting Duet sensor");
            oDuetSensor = oModel.Sensors.Analog[3];

            Console.WriteLine("Checking");
            while (true)
            {
                try
                {
                    double fDuetPwm = await checkDuetTemp();
                    double fPiPwm = await checkPiTemp();
                    await setFan(Math.Max(fDuetPwm, fPiPwm));
                }
                catch (SocketException)
                {
                    Console.WriteLine("Lost connection to DCS");
                    try
                    {
                        await oSubscription.Connect(SubscriptionMode.Patch);
                        await oCommand.Connect();
                    }
                    catch (Exception e)
                    {
                        Console.WriteLine(" -- Connection error: {0}", e.Message);
                        await Task.Delay(2000);
                    }
                }
            }
        }

        public static async Task<double> checkDuetTemp()
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 10);

            using JsonDocument oPatch = await oSubscription.GetMachineModelPatch();
            oModel.UpdateFromJson(oPatch.RootElement);
            if (oDuetSensor.LastReading == null) return 0;

            double fMin = 40;
            double fMax = 70;

            double currentOverrun = oDuetSensor.LastReading.Value - fMin;
            if (currentOverrun <= 0)
            {
                Console.WriteLine("Duet   " + Math.Round((decimal)oDuetSensor.LastReading, 1).ToString() + " C°                     ");
                Console.SetCursorPosition(iLeft, iTop);
                return 0;
            }

            double fPwm = Math.Round(currentOverrun / fMax, 2);
            Console.WriteLine("Duet " + Math.Round((double)oDuetSensor.LastReading, 1).ToString() + " C° : Fan at " + (fPwm * 100).ToString() + " %");
            Console.SetCursorPosition(iLeft, iTop);
            return fPwm;
        }

        public static async Task<double> checkPiTemp()
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 11);

            string sTemp = await File.ReadAllTextAsync("/sys/class/thermal/thermal_zone0/temp");
            double fTemp = double.Parse(sTemp) / 1000;

            double fMin = 55;
            double fMax = 70;

            double currentOverrun = fTemp - fMin;
            if (currentOverrun <= 0)
            {
                Console.WriteLine("Pi   " + Math.Round(fTemp, 1).ToString() + " C°                       ");
                Console.SetCursorPosition(iLeft, iTop);
                return 0;
            }

            double fPwm = Math.Round(currentOverrun / fMax, 2);
            Console.WriteLine("Pi   " + Math.Round((decimal)fTemp, 1).ToString() + " C° : Fan " + (fPwm * 100).ToString() + " %");
            Console.SetCursorPosition(iLeft, iTop);
            return fPwm;

        }
        private static async Task setFan(double _fValue)
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 12);

            if (fFanLastValue == 0 && _fValue == 0)
            {
                fFanLastValue = _fValue;
                Console.SetCursorPosition(iLeft, iTop);
                return;
            }

            if (fFanLastValue < 1 && _fValue >= 1)
            {
                Console.WriteLine("Fan On Max");
            }
            else if (fFanLastValue > 0 && _fValue == 0)
            {
                Console.WriteLine("Fan Off          ");
            }
            else
            {
                Console.WriteLine("Fan On          ");
            }
            await oCommand.PerformSimpleCode("M106 P3 S" + _fValue.ToString());

            fFanLastValue = _fValue;
            Console.SetCursorPosition(iLeft, iTop);
        }
    }
}

If it crashes again at some point, please share the output of M122.

[Donpi]

@chrishamm thanks for the rewrite, I'll check that later.

I think It will crash the same as the probleme occurs even with juste as bash script.

here is my M122 results :

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.1.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TN-6JTDG-3SD6J-TABLT
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 154604
Dynamic ram: 163424 of which 44 recycled
Exception stack ram used: 544
Never used ram: 74600
Tasks: NETWORK(ready,1968) HEAT(blocked,1188) CanReceiv(suspended,3820) CanSender(suspended,1384) CanClock(blocked,1436) TMC(blocked,204) MAIN(running,2672) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:03:57 ago, cause: software
Last software reset at 2020-12-07 11:16, reason: User, spinning module LinuxInterface, available RAM 74600 bytes (slot 0)
Software reset code 0x0010 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0444a000 BFAR 0x00000000 SP 0xffffffff Task MAIN
Error status: 0
MCU temperature: min 31.4, current 37.0, max 37.2
Supply voltage: min 24.8, current 24.8, max 25.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.1, max 12.1, under voltage events: 0
Driver 0: standstill, reads 41519, writes 14 timeouts 0, SG min/max 0/0
Driver 1: ok, reads 41517, writes 17 timeouts 0, SG min/max 0/99
Driver 2: standstill, reads 41512, writes 22 timeouts 0, SG min/max 0/969
Driver 3: standstill, reads 41512, writes 22 timeouts 0, SG min/max 0/1023
Driver 4: ok, reads 41518, writes 17 timeouts 0, SG min/max 0/142
Driver 5: standstill, reads 41525, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-12-07 11:20:27
Slowest loop: 4.70ms; fastest: 0.14ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 374, MinFreeDm: 373, MaxWait: 23941ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 65, completed moves: 64, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: 3
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by Aux
HTTP* is ready with "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux* is idle in state(s) 35 0, running macro
Trigger* is idle in state(s) 0
Queue* is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 1.03ms; fastest: 0.01ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 0 of 8
- Ethernet -
State: disabled
Error counts: 0 0 0 0 0
Socket states: 0 0 0 0 0 0 0 0
=== CAN ===
Messages sent 940, longest wait 0ms for type 0
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 18ms ago
RX/TX seq numbers: 7547/7549
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.1.1
Aux:
Finishing macro deployprobe.g, started by G29
> Next stack level
Executing macro bed.g, started by system
Number of flush requests: 1
> Next stack level
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 32.60

[A Former User]

If you have a unused input pin that is capable of analouge input you can probably add a thermistor move the control to the Duet making it kina simpler and more reliable.

[Donpi]

@bearer of course. Or I can set my fan at 20% all the time.

How to do it, is not my question.
I think that no matter how I do it, le DWS schould'nt crash my print.

And that's precisely why I'm asking.

[chrishamm]

@Donpi I have just tried the updated test app and it's working on my test setup with 3.2.0-b4.1:

   Duet   21.4 C°
Reading model                                               Pi   47.7 C°

If you still have problems with 3.1.1, please consider upgrading to the latest unstable version. We're hoping to publish a first release candidate for 3.2 quite soon.

[Donpi]

@chrishamm it crash the same with your rewrite.
It stoped printing after about 30 minutes.

How can I switch to 3.2.0-b4.1 ?

[Phaedrux]

@Donpi said in Duet 3 +SBC : Print randomely freeze when runin my script on RPI:

How can I switch to 3.2.0-b4.1 ?

You'd have to switch to the unstable branch and then update.

https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Software_Installation

[Donpi]

It crash the print even with the 3.2.0-b4.1

here is my M122

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta4.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TN-6JTDG-3SD6J-TABLT
Used output buffers: 4 of 40 (15 max)
=== RTOS ===
Static ram: 123292
Dynamic ram: 137988 of which 36 recycled
Never used RAM 130876, free system stack 188 words
Tasks: Linux(ready,77) HEAT(blocked,297) CanReceiv(blocked,947) CanSender(blocked,371) CanClock(blocked,352) TMC(blocked,49) MAIN(running,1161) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:25:28 ago, cause: software
Last software reset at 2020-12-09 15:50, reason: HeatTaskStuck, GCodes spinning, available RAM 130188, slot 0
Software reset code 0x4143 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0040080f BFAR 0x00000000 SP 0x20410144 Task MAIN
Stack: 2042328c 00429512 81000000 00000000 3e958000 00000000 4079a000 401fc5e4 3f9ca0f0 4530c6d9 4530c6d9 44b0c6d9 49371b00 37533333 43d697a0 00000000 42117d81 3f800000 00000000 60000011 20426c58 2040e0b4 20426c58 00000000 20423af8 00000080 a5a5a5a5
Error status: 0x00
MCU temperature: min 35.1, current 37.4, max 37.6
Supply voltage: min 24.8, current 25.0, max 25.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 12980, writes 14 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 12981, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 12980, writes 15 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 12981, writes 15 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 12983, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 12987, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-12-09 16:16:44
Slowest loop: 4.12ms; fastest: 0.12ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
FreeDm 375 (min 375), maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 6111, send timeouts 13750, received 0, lost 0, longest wait 0ms for reply type 0, free buffers 47
=== SBC interface ===
State: 0, failed transfers: 0
Last transfer: 19ms ago
RX/TX seq numbers: 15632/53872
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x209d8
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.0-beta4
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 1.96

[Donpi]

This post is deleted!

[Donpi]

The duet stop printing but the script continue

[chrishamm]

Many thanks for reporting this, I've just reproduced and fixed this problem. It will be fixed in the next version.

[Donpi]

Are there come log that I can check to see what is appening in the Duet ?
Or what gcode is effetively executed .

[Phaedrux]

There is this: https://duet3d.dozuki.com/Wiki/Gcode?revisionid=HEAD#Section_M929_Start_stop_event_logging_to_SD_card

And this: https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

[chrishamm]

TBH I had some problems with your plugin but I've got a fix ready - at least I haven't been able to reproduce it with the latest dev version. This problem only showed up when two G/M/T-codes from two different G-code channels were frequently sent.

To work-around this problem try to send your M106 code to the File channel (that's the same channel used for file prints) and consider adding a delay (e.g. await Task.Delay(1000);) after it. The second parameter of PerformSimpleCode lets you choose the target channel.

[Donpi]

@chrishamm Great, I'll test that today.

I thought I was abandoned alone with my problem

[Donpi]

@chrishamm said in Duet 3 +SBC : Print randomely freeze when runin my script on RPI:

await Task.Delay(1000);

The workaround seems to work but it gives me many "Error parsing response" in the PanelDuet console

[chrishamm]

@Donpi I tested your plugin with 3.2-RC1 and I no longer get unexpected crashes or other problems. In any case I recommend adding a short delay once object model updates have been received to reduce the CPU usage a bit.

[Donpi]

Good to hear that
Thank you for the support