Intercepting Messages from DSF/DCS with dsf-python

rero

@davidjryan there is a perfect example in the dsf python-bindings:
examples -> custom_m_codes.py

(Sry, don't have the reputation (!) to post links in this forum)

droftarts

@rero You should have enough reputation to post links now. It's an anti-spam measure. I think the link you mean is https://github.com/Duet3D/dsf-python/blob/v3.6-dev/examples/custom_m_codes.py

Ian

davidjryan

@droftarts @rero

Son of a #$@#!

Thanks gents! I'll start playing around with that example.

davidjryan

@droftarts

I have the example running and I'm watching the stream. I have set it to intercept "everything" (removed filters from the InterceptConnection call).

With the example program NOT running, if I execute an G90 G1 X-10 F6000 while X is at 0mm from the DWC Dashboard tab, I get "Error G1: target position outside machine limits" banner message and it goes to the Console. All correct as my X limit is -1mm.

With the example program running, if I execute an G90 G1 X-10 F6000 while X is at 0mm from the DWC Dashboard tab, The "Error G1: target position outside machine limits" banner message does NOT display in DWC nor in the Console, and the example app receives the following:

pi@A1000-0:~/A1000 $ python intercept.py
send: {"mode":"Intercept","version":12,"InterceptionMode":"Pre","Channels":["HTTP","Telnet","File","USB","Aux","Trigger","Queue","LCD","SBC","Daemon","Aux2","Autopause","File2","Queue2","Unknown"],"AutoFlush":true,"AutoEvaluateExpressions":true,"Filters":null,"PriorityCodes":false}
recv: {"success":true}
recv: {"sourceConnection":0,"result":null,"type":"M","channel":"Trigger","lineNumber":null,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":409,"minorNumber":null,"flags":4097,"comment":null,"filePosition":null,"length":null,"parameters":[{"letter":"K","value":"network","isString":true},{"letter":"I","value":"1","isString":false}],"command":"Code"}
1 CDE:
M409 K"network" I1
1 CDE Type:
M
send: {"command":"Flush","Channel":"Trigger","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":43,"result":null,"type":"M","channel":"HTTP","lineNumber":1,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":120,"minorNumber":null,"flags":2049,"comment":null,"filePosition":0,"length":5,"parameters":[],"command":"Code"}
2 CDE:
M120
2 CDE Type:
M
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":43,"result":null,"type":"G","channel":"HTTP","lineNumber":2,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":91,"minorNumber":null,"flags":2049,"comment":null,"filePosition":5,"length":4,"parameters":[],"command":"Code"}
3 CDE:
G91
3 CDE Type:
G
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":43,"result":null,"type":"G","channel":"HTTP","lineNumber":3,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":1,"minorNumber":null,"flags":2049,"comment":null,"filePosition":9,"length":14,"parameters":[{"letter":"X","value":"-10","isString":false},{"letter":"F","value":"6000","isString":false}],"command":"Code"}
4 CDE:
G1 X-10 F6000
4 CDE Type:
G
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":43,"result":null,"type":"M","channel":"HTTP","lineNumber":4,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":121,"minorNumber":null,"flags":2049,"comment":null,"filePosition":23,"length":5,"parameters":[],"command":"Code"}
5 CDE:
M121
5 CDE Type:
M
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":0,"result":null,"type":"M","channel":"Trigger","lineNumber":null,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":409,"minorNumber":null,"flags":4097,"comment":null,"filePosition":null,"length":null,"parameters":[{"letter":"K","value":"network","isString":true},{"letter":"I","value":"1","isString":false}],"command":"Code"}

Switching to InterceptConnection(InterceptionMode.POST gives this:

pi@A1000-0:~/A1000 $ python intercept.py
send: {"mode":"Intercept","version":12,"InterceptionMode":"Post","Channels":["HTTP","Telnet","File","USB","Aux","Trigger","Queue","LCD","SBC","Daemon","Aux2","Autopause","File2","Queue2","Unknown"],"AutoFlush":true,"AutoEvaluateExpressions":true,"Filters":null,"PriorityCodes":false}
recv: {"success":true}
recv: {"sourceConnection":0,"result":null,"type":"M","channel":"Trigger","lineNumber":null,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":409,"minorNumber":null,"flags":4099,"comment":null,"filePosition":null,"length":null,"parameters":[{"letter":"K","value":"network","isString":true},{"letter":"I","value":"1","isString":false}],"command":"Code"}
1 CDE:
M409 K"network" I1
1 CDE Type:
M
send: {"command":"Flush","Channel":"Trigger","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":0,"result":null,"type":"M","channel":"Trigger","lineNumber":null,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":409,"minorNumber":null,"flags":4099,"comment":null,"filePosition":null,"length":null,"parameters":[{"letter":"K","value":"network","isString":true},{"letter":"I","value":"1","isString":false}],"command":"Code"}
2 CDE:
M409 K"network" I1
2 CDE Type:
M
send: {"command":"Flush","Channel":"Trigger","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":0,"result":null,"type":"M","channel":"Trigger","lineNumber":null,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":409,"minorNumber":null,"flags":4099,"comment":null,"filePosition":null,"length":null,"parameters":[{"letter":"K","value":"network","isString":true},{"letter":"I","value":"1","isString":false}],"command":"Code"}
3 CDE:
M409 K"network" I1
3 CDE Type:
M
send: {"command":"Flush","Channel":"Trigger","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":52,"result":null,"type":"M","channel":"HTTP","lineNumber":1,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":120,"minorNumber":null,"flags":2051,"comment":null,"filePosition":0,"length":5,"parameters":[],"command":"Code"}
4 CDE:
M120
4 CDE Type:
M
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":52,"result":null,"type":"G","channel":"HTTP","lineNumber":2,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":91,"minorNumber":null,"flags":2051,"comment":null,"filePosition":5,"length":4,"parameters":[],"command":"Code"}
5 CDE:
G91
5 CDE Type:
G
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":52,"result":null,"type":"G","channel":"HTTP","lineNumber":3,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":1,"minorNumber":null,"flags":2051,"comment":null,"filePosition":9,"length":14,"parameters":[{"letter":"X","value":"-10","isString":false},{"letter":"F","value":"6000","isString":false}],"command":"Code"}
6 CDE:
G1 X-10 F6000
6 CDE Type:
G
send: {"command":"Flush","Channel":"HTTP","SyncFileStreams":false,"IfExecuting":true}
recv: {"result":true,"success":true}
send: {"command":"Resolve","Type":0,"Content":null}
recv: {"sourceConnection":52,"result":null,"type":"M","channel":"HTTP","lineNumber":4,"indent":0,"keyword":0,"keywordArgument":null,"majorNumber":121,"minorNumber":null,"flags":2051,"comment":null,"filePosition":23,"length":5,"parameters":[],"command":"Code"}

So the error response is being redirected away from DWC, but where is it going?

Looking at the responses, I can't decipher if one of those is the error message. They all just look like G and M code intercepts that are confirmation that the DCS received the command. Is there a specific flag or function for errors/faults?

Is the error trapped in the M121 response? Is it the "majorNumber":409 or "flags":4900 keys? If so, is there a lookup table for these values? I didn't see anything on Github.

droftarts

@davidjryan one for @chrishamm , I think!

Ian

rero

@davidjryan sorry I was mislead by your thread-subject and did not carefully read your full post.

I assume you want to check the messages-member of the ObjectModel (see https://github.com/Duet3D/dsf-python/blob/v3.6-dev/src/dsf/object_model/object_model.py and https://github.com/Duet3D/dsf-python/blob/v3.6-dev/src/dsf/object_model/messages/messages.py for details).

I did a quick test with my c#-application and M118- and Plugin-Output arrived successfully ...

Check also https://github.com/Duet3D/dsf-python/blob/v3.6-dev/examples/subscribe_object_model.py - this example shows how to access the ObjectModel.

HTH

PS: Link-reputation achieved!

davidjryan

@rero unfortunately, your assumption is incorrect. I am asking about those messages that are not a part of the object model. I have been capturing the object model for 2 years now with my custom app and I am looking to be able to disable the DWC once I can get the error messages read by my app. I am creating my own "DWC" that is custom to my application (which is not a printer).

The object model messages do not include the "error (RED)" or "warning (YELLOW)" type of messages when there is an overall problem with the system, i.e, any yellow or red message that pops up on the DWC as a banner on the bottom of the DWC webpage or itemized in the Console page/tab as they occur.

From what I can tell, when an error occurs in DSF, the message is "pushed" out there. I need to know where "there" is so I can capture it. I believe the intercept example you and @droftarts sent has me on the right track but I'm still missing a piece of the puzzle.

If I send a non-supported G code from my app, the DWC reports with a yellow warning as an non-supported G command. If I sent a poorly formatted G command, the DWC reports with a red error message with the issue with the command. Neither were sent by DWC yet both were captured as a warning and error by DWC. So something else is at play than just the reading the OM.

Running the intercept example as .PRE or .POST will stop the DWC from reporting those messages. But I can't see within the intercept responses the error message itself unless it is what I mentioned before and there is a lookup table in DWC to textualize the error from the reported code.

rero

@davidjryan if your assumption is correct, two instances/connections/channels of DWC via different browsers should have equal logs.
So if you run a command through the "Send"-function of DWC, the (error-)message will/should be shown everywhere ...

For me that is not the case: every command performed/executed directly reports only to the source-channel which send the command.

For your control-app: what is the answer from the DCS if you perform your test-command?
Like in https://github.com/Duet3D/dsf-python/blob/v3.6-dev/examples/send_simple_code.py ?

If you app is the only controlling-instance, this functionality should be suitable for you ...

chrishamm

@davidjryan That isn't quite right. DWC receives many code replies via the messages array of the object model, at least if they originate from system macros or job files. If you need results of all G-codes (e.g. also from HTTP, USB, or other channels), you should intercept executed G-codes using the IPC API. When you receive them, they should have the result property set which is what you seem to be interested in. Running /opt/dsf/bin/CodeLogger -t executed displays all executed codes, too.

davidjryan

@chrishamm Do you have a code snippet for this? Either C or python?

How is it that DWC gets the messages when it's my app that sends the G or M code that causes the warning or error?

I send some commands via HTTP Request and some commands via Command Request. My app uses PyQt for the GUI and their threading classes so I can run in somewhat of a multitasking environment.

My HTTP request:

class HTTPPostRequestThread(QRunnable):
    """
    Send commands to Duet
    via HTTP request
    """
    def __init__(self, *args):
        super(HTTPPostRequestThread, self).__init__()
        self.url = args[0]
        self.command = args[1]
        self.log = False
        try:
            self.logger = args[2]
            self.log = True
        except:
            pass

    @pyqtSlot()
    def run(self):
        try:
            if self.log:
                self.logger.info(f'Sending {self.command} to DCS via HTTP')        
            result=requests.post(self.url, self.command)
            print(f'HTTP Result = {result}')
        except:
            if self.log:
                self.logger.info(f'Failed to send {self.command} to DCS via HTTP')        
            pass

My CommandConnection request:

class CommandRequestThread(QRunnable):
    """
    Send commands to Duet
    via TCPIP socket
    """
    def __init__(self, *args):
        super(CommandRequestThread, self).__init__()
        self.command = args[0]
        self.log = False
        try:
            self.logger = args[1]
            self.log = True
        except:
            pass
        
    @pyqtSlot()
    def run(self):
        if self.command!='':
            try:
                if self.log:
                    self.logger.info(f'Sending {self.command} to DCS via socket')        
                self.connect = CommandConnection(debug=False)
                self.connect.connect()
                result = self.connect.perform_simple_code(self.command, async_exec=False)
                print(f'Result = {result}')
                self.connect.close()
            except:
                if self.log:
                    self.logger.info(f'Failed to send {self.command} to DCS via socket')        
                pass
        else:
            self.logger.info(f'Command for DCS socket is blank')

Result values:

2025-02-25 08:56:50.964 - INFO - Sending M98 P"configuration/om_faults_reset.g" to DCS via HTTP
HTTP Result = <Response [200]>
2025-02-25 08:56:51.253 - INFO - Tab button (5) Manual pressed
2025-02-25 08:56:52.967 - INFO - Sending set global.bFaultPresent = true to DCS via HTTP
2025-02-25 08:56:52.986 - ERROR - At Least One Axis Is Not Homed
2025-02-25 08:56:52.991 - INFO - Sending set global.bCycleAbort = true to DCS via HTTP
HTTP Result = <Response [200]>
HTTP Result = <Response [200]>
2025-02-25 08:56:56.188 - INFO - okPushButton button pressed
2025-02-25 08:56:58.885 - INFO - Tab button (2) Stock pressed
2025-02-25 08:57:02.301 - INFO - Tab button (7) Recovery pressed
2025-02-25 08:57:06.287 - INFO - pbNegDir button pressed
2025-02-25 08:57:06.299 - INFO - Sending M98 P"functions/jog_axis.g" X"X" Y-1 Z"Neg" to DCS via socket
Result =

Here's the same result but with debug turned on for Command Request:

2025-02-25 09:07:52.514 - INFO - pbNegDir button pressed
2025-02-25 09:07:52.547 - INFO - Sending M98 P"functions/jog_axis.g" X"X" Y-1 Z"Neg" to DCS via socket
send: {"mode":"Command","version":12}
recv: {"success":true}
send: {"command":"SimpleCode","Code":"M98 P\"functions/jog_axis.g\" X\"X\" Y-1 Z\"Neg\"","Channel":"SBC","ExecuteAsynchronously":false}
recv: {"result":"","success":true}
Result = 
2025-02-25 09:07:55.234 - INFO - Sending M98 P"configuration/om_faults_reset.g" to DCS via HTTP
HTTP Result = <Response [200]>

The HTTP request in the above example is just setting globals so the [200] is expected as there are no errors.
The Command request shows no result returned (or success in debug mode). The command for this instance was a G90 G1 X-1 when the axis was not homed, so there was an insufficient axes homed error, which the DWC caught, somehow.

DWC:

chrishamm

@davidjryan I'm sorry but I can't really help with Python scripts. Here the corresponding example: https://github.com/Duet3D/dsf-python/blob/v3.6-dev/examples/custom_m_codes.py Instead of PRE you need to set the interception mode to Executed, then you should be able to inspect the results of executed G/M/T-codes. Of course that plugin needs to run on the SBC itself, you cannot intercept codes remotely without additional infrastructure. Also make sure that your user is part of the dsf group or let it run as root to avoid permission issues while developing.

You can find the source code of the CodeLogger utility here: https://github.com/Duet3D/DuetSoftwareFramework/blob/v3.5-dev/src/CodeLogger/Program.cs See the corresponding wiki -> Third-Party plugins for further details if you want to write a .NET console app.