Duet 3 +SBC : Print randomely freeze when runin my script on RPI

chrishamm

What DSF version are you using, 3.1.1? I noticed you are mixing async with non-async code in your C# app, which isn't recommended. Here a slightly improved version:

using DuetAPI.Commands;
using DuetAPI.Connection;
using DuetAPI.Machine;
using DuetAPIClient;
using System;
using System.IO;
using System.Net.Sockets;
using System.Text.Json;
using System.Threading.Tasks;

namespace dws_tst1
{
    class Program
    {
        static AnalogSensor oDuetSensor;

        static SubscribeConnection oSubscription;
        static CommandConnection oCommand;

        static MachineModel oModel;

        static double fFanLastValue;

        static async Task Main(string[] args)
        {
            Console.WriteLine("Start");
            fFanLastValue = 0;

            Console.WriteLine("Connections");
            oSubscription = new SubscribeConnection();
            oCommand = new CommandConnection();

            Console.Write("Connecting");
            await oSubscription.Connect(SubscriptionMode.Patch);
            await oCommand.Connect();
            Console.WriteLine("Connection OK");

            Console.WriteLine("Reading model");
            oModel = await oCommand.GetMachineModel();

            Console.WriteLine("Resetting MCU/CPU Fan");
            await oCommand.PerformSimpleCode("M106 P3 S0");

            Console.WriteLine("Selecting Duet sensor");
            oDuetSensor = oModel.Sensors.Analog[3];

            Console.WriteLine("Checking");
            while (true)
            {
                try
                {
                    double fDuetPwm = await checkDuetTemp();
                    double fPiPwm = await checkPiTemp();
                    await setFan(Math.Max(fDuetPwm, fPiPwm));
                }
                catch (SocketException)
                {
                    Console.WriteLine("Lost connection to DCS");
                    try
                    {
                        await oSubscription.Connect(SubscriptionMode.Patch);
                        await oCommand.Connect();
                    }
                    catch (Exception e)
                    {
                        Console.WriteLine(" -- Connection error: {0}", e.Message);
                        await Task.Delay(2000);
                    }
                }
            }
        }

        public static async Task<double> checkDuetTemp()
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 10);

            using JsonDocument oPatch = await oSubscription.GetMachineModelPatch();
            oModel.UpdateFromJson(oPatch.RootElement);
            if (oDuetSensor.LastReading == null) return 0;

            double fMin = 40;
            double fMax = 70;

            double currentOverrun = oDuetSensor.LastReading.Value - fMin;
            if (currentOverrun <= 0)
            {
                Console.WriteLine("Duet   " + Math.Round((decimal)oDuetSensor.LastReading, 1).ToString() + " C°                     ");
                Console.SetCursorPosition(iLeft, iTop);
                return 0;
            }

            double fPwm = Math.Round(currentOverrun / fMax, 2);
            Console.WriteLine("Duet " + Math.Round((double)oDuetSensor.LastReading, 1).ToString() + " C° : Fan at " + (fPwm * 100).ToString() + " %");
            Console.SetCursorPosition(iLeft, iTop);
            return fPwm;
        }

        public static async Task<double> checkPiTemp()
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 11);

            string sTemp = await File.ReadAllTextAsync("/sys/class/thermal/thermal_zone0/temp");
            double fTemp = double.Parse(sTemp) / 1000;

            double fMin = 55;
            double fMax = 70;

            double currentOverrun = fTemp - fMin;
            if (currentOverrun <= 0)
            {
                Console.WriteLine("Pi   " + Math.Round(fTemp, 1).ToString() + " C°                       ");
                Console.SetCursorPosition(iLeft, iTop);
                return 0;
            }

            double fPwm = Math.Round(currentOverrun / fMax, 2);
            Console.WriteLine("Pi   " + Math.Round((decimal)fTemp, 1).ToString() + " C° : Fan " + (fPwm * 100).ToString() + " %");
            Console.SetCursorPosition(iLeft, iTop);
            return fPwm;

        }
        private static async Task setFan(double _fValue)
        {
            int iLeft = Console.CursorLeft;
            int iTop = Console.CursorTop;
            Console.SetCursorPosition(60, 12);

            if (fFanLastValue == 0 && _fValue == 0)
            {
                fFanLastValue = _fValue;
                Console.SetCursorPosition(iLeft, iTop);
                return;
            }

            if (fFanLastValue < 1 && _fValue >= 1)
            {
                Console.WriteLine("Fan On Max");
            }
            else if (fFanLastValue > 0 && _fValue == 0)
            {
                Console.WriteLine("Fan Off          ");
            }
            else
            {
                Console.WriteLine("Fan On          ");
            }
            await oCommand.PerformSimpleCode("M106 P3 S" + _fValue.ToString());

            fFanLastValue = _fValue;
            Console.SetCursorPosition(iLeft, iTop);
        }
    }
}

If it crashes again at some point, please share the output of M122.

Donpi

@chrishamm thanks for the rewrite, I'll check that later.

I think It will crash the same as the probleme occurs even with juste as bash script.

here is my M122 results :

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.1.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TN-6JTDG-3SD6J-TABLT
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 154604
Dynamic ram: 163424 of which 44 recycled
Exception stack ram used: 544
Never used ram: 74600
Tasks: NETWORK(ready,1968) HEAT(blocked,1188) CanReceiv(suspended,3820) CanSender(suspended,1384) CanClock(blocked,1436) TMC(blocked,204) MAIN(running,2672) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:03:57 ago, cause: software
Last software reset at 2020-12-07 11:16, reason: User, spinning module LinuxInterface, available RAM 74600 bytes (slot 0)
Software reset code 0x0010 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0444a000 BFAR 0x00000000 SP 0xffffffff Task MAIN
Error status: 0
MCU temperature: min 31.4, current 37.0, max 37.2
Supply voltage: min 24.8, current 24.8, max 25.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.1, max 12.1, under voltage events: 0
Driver 0: standstill, reads 41519, writes 14 timeouts 0, SG min/max 0/0
Driver 1: ok, reads 41517, writes 17 timeouts 0, SG min/max 0/99
Driver 2: standstill, reads 41512, writes 22 timeouts 0, SG min/max 0/969
Driver 3: standstill, reads 41512, writes 22 timeouts 0, SG min/max 0/1023
Driver 4: ok, reads 41518, writes 17 timeouts 0, SG min/max 0/142
Driver 5: standstill, reads 41525, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-12-07 11:20:27
Slowest loop: 4.70ms; fastest: 0.14ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 374, MinFreeDm: 373, MaxWait: 23941ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 65, completed moves: 64, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: 3
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by Aux
HTTP* is ready with "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux* is idle in state(s) 35 0, running macro
Trigger* is idle in state(s) 0
Queue* is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 1.03ms; fastest: 0.01ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 0 of 8
- Ethernet -
State: disabled
Error counts: 0 0 0 0 0
Socket states: 0 0 0 0 0 0 0 0
=== CAN ===
Messages sent 940, longest wait 0ms for type 0
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 18ms ago
RX/TX seq numbers: 7547/7549
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.1.1
Aux:
Finishing macro deployprobe.g, started by G29
> Next stack level
Executing macro bed.g, started by system
Number of flush requests: 1
> Next stack level
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 32.60

A Former User

If you have a unused input pin that is capable of analouge input you can probably add a thermistor move the control to the Duet making it kina simpler and more reliable.

Donpi

@bearer of course. Or I can set my fan at 20% all the time.

How to do it, is not my question.
I think that no matter how I do it, le DWS schould'nt crash my print.

And that's precisely why I'm asking.

chrishamm

@Donpi I have just tried the updated test app and it's working on my test setup with 3.2.0-b4.1:

   Duet   21.4 C°
Reading model                                               Pi   47.7 C°

If you still have problems with 3.1.1, please consider upgrading to the latest unstable version. We're hoping to publish a first release candidate for 3.2 quite soon.

Donpi

@chrishamm it crash the same with your rewrite.
It stoped printing after about 30 minutes.

How can I switch to 3.2.0-b4.1 ?

Phaedrux

@Donpi said in Duet 3 +SBC : Print randomely freeze when runin my script on RPI:

How can I switch to 3.2.0-b4.1 ?

You'd have to switch to the unstable branch and then update.

https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Software_Installation

Donpi

It crash the print even with the 3.2.0-b4.1

here is my M122

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta4.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TN-6JTDG-3SD6J-TABLT
Used output buffers: 4 of 40 (15 max)
=== RTOS ===
Static ram: 123292
Dynamic ram: 137988 of which 36 recycled
Never used RAM 130876, free system stack 188 words
Tasks: Linux(ready,77) HEAT(blocked,297) CanReceiv(blocked,947) CanSender(blocked,371) CanClock(blocked,352) TMC(blocked,49) MAIN(running,1161) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:25:28 ago, cause: software
Last software reset at 2020-12-09 15:50, reason: HeatTaskStuck, GCodes spinning, available RAM 130188, slot 0
Software reset code 0x4143 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0040080f BFAR 0x00000000 SP 0x20410144 Task MAIN
Stack: 2042328c 00429512 81000000 00000000 3e958000 00000000 4079a000 401fc5e4 3f9ca0f0 4530c6d9 4530c6d9 44b0c6d9 49371b00 37533333 43d697a0 00000000 42117d81 3f800000 00000000 60000011 20426c58 2040e0b4 20426c58 00000000 20423af8 00000080 a5a5a5a5
Error status: 0x00
MCU temperature: min 35.1, current 37.4, max 37.6
Supply voltage: min 24.8, current 25.0, max 25.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 12980, writes 14 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 12981, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 12980, writes 15 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 12981, writes 15 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 12983, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 12987, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-12-09 16:16:44
Slowest loop: 4.12ms; fastest: 0.12ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
FreeDm 375 (min 375), maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 6111, send timeouts 13750, received 0, lost 0, longest wait 0ms for reply type 0, free buffers 47
=== SBC interface ===
State: 0, failed transfers: 0
Last transfer: 19ms ago
RX/TX seq numbers: 15632/53872
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x209d8
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.0-beta4
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 1.96

Donpi

This post is deleted!

Donpi

The duet stop printing but the script continue

chrishamm

Many thanks for reporting this, I've just reproduced and fixed this problem. It will be fixed in the next version.

Donpi

Are there come log that I can check to see what is appening in the Duet ?
Or what gcode is effetively executed .

Phaedrux

There is this: https://duet3d.dozuki.com/Wiki/Gcode?revisionid=HEAD#Section_M929_Start_stop_event_logging_to_SD_card

And this: https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

chrishamm

TBH I had some problems with your plugin but I've got a fix ready - at least I haven't been able to reproduce it with the latest dev version. This problem only showed up when two G/M/T-codes from two different G-code channels were frequently sent.

To work-around this problem try to send your M106 code to the File channel (that's the same channel used for file prints) and consider adding a delay (e.g. await Task.Delay(1000);) after it. The second parameter of PerformSimpleCode lets you choose the target channel.

Donpi

@chrishamm Great, I'll test that today.

I thought I was abandoned alone with my problem

Donpi

@chrishamm said in Duet 3 +SBC : Print randomely freeze when runin my script on RPI:

await Task.Delay(1000);

The workaround seems to work but it gives me many "Error parsing response" in the PanelDuet console

chrishamm

@Donpi I tested your plugin with 3.2-RC1 and I no longer get unexpected crashes or other problems. In any case I recommend adding a short delay once object model updates have been received to reduce the CPU usage a bit.

Donpi

Good to hear that
Thank you for the support