Re: Deadlock reading the object model from a plugin
Hello! Sorry to bother you again with this issue. But I still get occasional deadlocks in my plugin, even though I applied your suggestions from the mentionned post. The debugging information still shows DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel) and the machine is stuck.
Do you have any updates on that problem with the newest version of the firmware ?
With the v flat, the output is the following:
{"key":"state","flags":"d99vn","result":{"atxPower":null,"atxPowerPort":null,"beep":null,"currentTool":0,"deferredPowerDown":null,"displayMessage":"","gpOut":[],"laserPwm":null,"logFile":"","logLevel":"off","machineMode":"FFF","macroRestarted":false,"messageBox":null,"msUpTime":275,"nextTool":0,"powerFailScript":"M913 X0 Y0 G91 M83 G1 Z3 E-5 F1000","previousTool":1,"restorePoints":[{"coords":[0,0,0,0,0],"extruderPos":0,"fanPwm":0,"feedRate":50.0,"ioBits":0,"laserPwm":null,"toolNumber":-1},{"coords":[140.418,136.962,248.086,2.000,5.000],"extruderPos":8.9,"fanPwm":1.00,"feedRate":30.0,"ioBits":0,"laserPwm":null,"toolNumber":0},{"coords":[385.000,122.605,249.936,2.000,5.000],"extruderPos":-10.0,"fanPwm":1.00,"feedRate":108.3,"ioBits":0,"laserPwm":null,"toolNumber":1},{"coords":[0,0,0,0,0],"extruderPos":0,"fanPwm":0,"feedRate":50.0,"ioBits":0,"laserPwm":null,"toolNumber":-1},{"coords":[0,0,0,0,0],"extruderPos":0,"fanPwm":0,"feedRate":50.0,"ioBits":0,"laserPwm":null,"toolNumber":-1},{"coords":[0,0,0,0,0],"extruderPos":0,"fanPwm":0,"feedRate":50.0,"ioBits":0,"laserPwm":null,"toolNumber":-1}],"startupError":null,"status":"idle","thisActive":true,"thisInput":8,"time":"2024-12-05T14:41:38","upTime":21767}}
Still no messageBox key inside it...
@chrishamm Okay thanks for all the pointers ! I switched all my machines to 3.5.3 and made the changes with the connections that you suggested, and it solves most of my problems.
For the problem that I mentionned due to the machine being shifter along the Z axis after executing a macro from the plugin, I found something that might have caused the issue:
When intercepting a command inside the plugin, I was always starting by sending G90 and M83 to the machine to make sure that whatever mode it is in, I can always perform moves and extrusions in absolute and relative mode, respectively. I suspect that those two commands might have caused the machine to shift position along Z due to synchronization issues because I removed them and the issue never happened again...
@chrishamm
Yes, that's what I meant, currently my InterceptConnection is shared between all the modules of my code, so that all the commands get executed before the interception finishes.
What is weird though is that we used to have a separate CommandConnection in a previous version of the plugin and that bug never happened back then...
@chrishamm Thanks for your issue on GitHub! Is there any way I could help tracking it down ?
Regarding the command connection, I am passing the intercept connection to the different parts of my plugin so that all the commands are sent through it, so this should be working, right ?
The weird thing is that this issue happens randomly. That is, if I launch the same GCode several times in a row, it might succeed, or fail, but at different places. The only common denominator is that it fails to move at the right height after it resumes execution from my plugin.
@Ant1 Oh and by the way, we also observed a new, more concerning issue with the plugin. When resuming the GCode execution after a custom command has been intercepted by my plugin, the printer shifts everything vertically, which basically makes the print fail.
I have checked the GCode, and there is always an instruction G1 Z... after the call to the macro that gets intercepted, so the machine should move to a specific height. But instead, it moves 1 or 2mm higher than that and it prints in the air.
We have checked, and this issue only happens when the plugin is activated. Could it also be some kind of synchronization issue between the plugin and the duet ? I am kinda lost on this one...
@chrishamm
Yes, it was the same issue with Resending package #0.
Okay I will check if disabling the PanelDue works.
And thank you for the tip, I will add that line of code to the plugin.
@Ant1 Hey! Unfortunately the issue just happened again today 
Here is information with regard to my configuration:
config.g
And here are some information about the plugin that I am developping. I only included the interception part, but if you want to see the other files I can share.
plugin.json
intercept.py
@chrishamm Okay, good to know. I've made the update to 3.5.3 on one of my machines this week, and it seems to be working for now. I will make an update if I still observe issues.
Thanks for your help!
@chrishamm Sorry for the later reply on this, I forgot about it and got no notifications!
Here is a minimal setup that reproduces my issue. In this setup, I use a separate connection to intercept and to send commands. If I use a single intercept connection, the issue still happens, but less often.
Also, note that the message box shows up on the PanelDue in about 50% of the cases when sending M2000. But in the the other cases, the message box only pops up in the web interface. I didn't manage to see if it was correlated to something else, as it seemed pretty random to me.
#!/usr/bin/python3
from dsf.connections import InterceptConnection, InterceptionMode, CommandConnection
from dsf.object_model.messages import MessageType
if __name__ == "__main__":
try:
intercept = InterceptConnection(InterceptionMode.PRE, filters=['M2000'])
command = CommandConnection()
command.connect()
intercept.connect()
while True:
code = intercept.receive_code()
try:
if code.short_str() == 'M2000':
command.perform_simple_code('M291 S2 P"Test" X1 Y1 Z1')
print(command.perform_simple_code('M409 K"state" F"d99fn"'))
intercept.resolve_code(MessageType.Success)
except Exception as e:
intercept.resolve_code(MessageType.Error, str(e))
finally:
intercept.close()
command.close()
The output of M409 K"state" F"d99fn" is the following:
{"key":"state","flags":"d99fn","result":{"currentTool":2,"gpOut":[],"laserPwm":null,"msUpTime":646,"status":"idle","time":"2024-10-14T11:27:13","upTime":2238}}
I don't see any messageBox value inside it.
@chrishamm Okay I will update my machines and see if it solves the issue. Thanks!
Otherwise, is there a way to only get a specific key of the object model that prevents full updates to be propagated to the raspberry PI ? Maybe this would help reducing too long responses.
Hello,
I am developping a interception plugin on my duet, and I am experiencing occasional deadlocks when reading from the object model, which cause some prints to hang forever.
More details:
CommandConnection.get_object_model() from the dsf-python package.sudo journalctl -u duetcontrolserver -r, and I got the following suspicious lines:Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
Oct 04 09:02:35 alpha DuetControlServer[29146]: [warn] Resending packet #0 (request GetObjectModel)
I suspect that the command to retrieve the object model is stuck in a deadlock, which sounds like a synchronization issue.
I have tried to call CommandConnection.sync_object_model() before retrieving the object model, but the deadlock also occured once in that case.
Do you have any ideas on how to solve this issue ?
Do you think that I am doing something wrong, or could it be an issue in the firmware ?
In that case, should I update it to the latest version (3.5.3) ?
Thank you very much for your help!
Antoine
@Ant1 As a note, this issue happens with DSF 3.5.2. I tried to reproduce it with DSF 3.5.1, but the issue doesn't happen and the message gets shown properly on the PanelDue, so I think that the issue comes from the update.
@chrishamm I experience a similar issue, but when M291 is called from plugins... Here is the issue that I have opened to describe my problem: https://forum.duet3d.com/topic/36365/m291-issued-from-a-plugin-doesn-t-show-anything-on-the-paneldue
I am creating a plugin which intercepts a certain M-code (M1000 in my case), and then performs a bunch of things with the machine. At some point, it needs to open a message box using M291. It does so using CommandConnection.perform_simple_code('M291 ...').
Depending on how the plugin is called, the message box doesn't get showed on the PanelDue. Here is what I have observed:
Expected:
M291 should open the message box on the PanelDue in all cases, otherwise it is annoying because I need to open a web interface to run this macro.
@chrishamm Thanks a lot, that was the problem ! I was looking at an old documentation page that didn't show that this permission was possible... Thank you !
Hello !
I am writing a plugin that uses a raspberry PI camera to calibrate print heads. I encounter an issue where the camera won't start when constructing the camera object:
from picamera2 import PiCamera2
cam = Picamera2()
Returns the following error:
[0:32:17.478523088] [4325] INFO Camera camera.cpp:990 Pipeline handler in use by another process
Camera __init__ sequence did not complete.
I am pretty sure that this issue is due to the apparmor configuration because if I run the exact same script from another folder on the raspberry PI, I don't get any errors...
Do you have any experience using a Raspberry PI camera from the plugins ?
Thanks for your help !
@achrn
Thank you a lot ! This was indeed the problem, I didn't have launchProcesses. I removed it at some point because I didn't think it was necessary to run the python script, but since it depends on an external library, that might cause the issue 
Thank you !
@chrishamm
Yes I have already done that. My plugin.json contains the following:
"sbcPackageDependencies": ["libzbar0", "python3-picamera2"],
"sbcPythonDependencies": ["opencv-python", "pyzbar", "Pillow", "numpy", "dsf-python>=3.5.0.2rc2"],
I have tried to change the line ending with dos2unix but it still gives me the same error...
If I change the first line to #!/usr/bin/python3, as you suggested, I get the following error:
from pyzbar.pyzbar import decode, ZBarSymbol
...
ImportError: Unable to find zbar shared library
Which is weird, because I have installed this library and all its dependencies, and I can get the import to work in a python3 interpreter, with no errors:
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyzbar
>>>
So it seems to me that something is not quite right with the DSF environment or PATH... Any ideas ?