New M-code to send a network request

frank26080115

@danal said in New M-code to send a network request:

Obviously, that connection should start when the first "M xxx URL payload" (for a given URL) is executed. When should it end? What action should be taken if it won't open? Should other G-Code commands 'wait' until the connection either suceeds or times out? How should duet manage memory if many, many, many M commands are executed all with different URLs? etc, etc.

It should be one GET request and no keepalive. Reply optional. Blocking with a short timeout. Single instance at a time.

It should never be used for mission critical actions like a filament swap. Fail safe and such.

lostapathy

@danal said in New M-code to send a network request:

How about UDP? Keep it simple. "M xxx (resolvable name|ip|broadcast) payload" where the payload is limited to an ASCII string that fits normal Gcode syntax rules (including future Duet/RepRap only extensions for symbolics, etc. ).

I offered the option of MQTT as a lighter alternative to HTTP. It's a super light protocol, comparatively, and since it's common among IoT devices, we wouldn't be plowing new ground with it. I think it's important in matters like this to adopt existing protocols where we can, rather than devising our own UDP protocol. If I wanted to run another bespoke service to collect events, I'd just connect over telnet and watch the stream, after all.

Already, there's a lot of esp8266-based devices that send out MQTTT telemetry, so it's obviously well within reach of the device horsepower, and example code should exist.

lostapathy

@frank26080115 said in New M-code to send a network request:

It should be one GET request and no keepalive. Reply optional. Blocking with a short timeout. Single instance at a time.

I agree, the duet should do nothing to process a response. I would envision we might want a parameter for the m-code to specify a timeout and/or pause (depending on whether this was HTTP or MQTT), though, rather than trying to pick a universal default for everyone.

I should know when I generate g-code that I need to stay put for up to a couple seconds for a picture, or if I'm just sending something "fire and forget"

Danal

MQTT: Where would the broker be in a typical 3D Printer scenario?

The various discussions of "One HTTP request" and a parameter to wait for a response or not... that all sounds right.

lostapathy

@danal said in New M-code to send a network request:

MQTT: Where would the broker be in a typical 3D Printer scenario?

The various discussions of "One HTTP request" and a parameter to wait for a response or not... that all sounds right.

Mine's on my NAS, but you can run that anywhere, or use a public one in the cloud.

OddKode

I'm writing a "DuetTimelapseClient" in C# as a WinForms app for my own purposes. The laptop I use for slicing is in the same room (and same network) as the printer and it's a Windows 10 machine. The client is using a HttpWebRequest / WebRequest to communicate /w the Duet and takes care of polling the Duet. It stores the current layer #. When the current layer # on the app doesn't match the current layer coming from the Duet, that's a layer change, and it will hit my IP cam's static image URL and download that to a folder (which is the name of the timelapse + current date / time). At the end, it executes FFMPEG to stitch the images together into a timelapse. I'm also planning at having a timelapse by interval instead of just z-change. This will take me a while as I burst some discs in my back and my Hypercube build with my new Duet 2 Ethernet board was put on hold till I'm more mobile again. Plus my day job is a software dev so it's hard to put in a full day doing that and coming home and doing more of that - gotta keep that healthy work / life balance!

Another idea I've been toying with. I've downloaded DWC to try and see if it's possible to raise a JS event upon layer change. I was hoping I could tap into the field that stores the IP / web camera URL (I'm using a Wyzecam v2), and upon layer change it raises a JS event, which uses that URL to call another "API" URL hosted on an old linux (Ubuntu) PC I'm using as a utility box (PiHole, Plex, file shares) - which is basically just a simple PHP script that takes in that URL, accesses it and downloads the resulting image to a file share.

All the while I'm also checking printing status. When a job completes, I'd have another event raised to call that same PHP script, writing out a file called "TimeLapseName-END.jpg". I'd have a cron job wake up on the machine every 5 minutes which looks at the root folder ("Timelapses"), digs into each one looking at all *.jpg files and finding those that end with "-END". That tells the script to run FFMPEG to stitch all jpg images inside each folder into a timelapse. After stitching, it wipes the images, then the containing folder and copies the stitched timelapse to a file share and sends me an email.

I think having two extra fields "API Image URL" (for taking the snaps) and "API Job Complete URL" (for rendering the timelapse) would make this possible - but this whole process only works if you're using an IP cam that has a static image capture function - direct web cam wouldn't work using this I don't think. For me, if I experiment with this, I'll probably just hardcode all of the values - I don't want to mess /w the general layout of DWC, just add some additional logic to it.

Sorry to ramble! I'm sure there's much better solutions out there, but I love to tinker so

Danal

@lostapathy said in New M-code to send a network request:

Mine's on my NAS, but you can run that anywhere, or use a public one in the cloud.

No thanks. With all due respect, the concept of having a NAS, or server, or cloud be REQUIRED for my printer to trigger the camera that is bolted to it (or similar)...

I totally get having such a service/broker for tens, hundreds, thousands, ... of IoT devices. For what should be totally local interaction between two or three endpoints, MQTT feels (to me) even "heavier" than some of the things we've already rejected.

Danal

HTTP with option for ultra-quick timeout seems to be a pretty good hammer.

Still like to see an UDP option, but I seem to be in the miniority for that, and the existing internals of Duet/RepRap/WiFi-module also don't seem amenable.

So... HTTP with timeout options. Sounds pretty good.

Mnnn [resolvable name|IP] [timeout in ms] "payload"

Should there be an option to express a payload buffer entirely in Hexadecimal?

Danal

@oddkode said in New M-code to send a network request:

Another idea I've been toying with. I've downloaded DWC to try and see if it's possible to raise a JS event upon layer change.

Yes, but... DWC just polls. Nothing is ever "pushed" to DWC from the printer. If you are already polling, no need to 'hook' DWC.

I was hoping I could tap into the field that stores the IP / web camera URL

You can certainly get that from DWC internally (JS read the HTML field, among other things), but there is a potentially easier way: DWC stores its persistent settings in a file on the printer itself, and pulls them via an AJAX call. You can either:

HTTP download the file

http://192.168.1.100/rr_download?name=DWCconfig.txt

Or make an AJAX call (still really an HTTP GET) for a JSON object (containing the same info)

http://192.168.1.100/dwc.json

Note: REALLY old DWC kept config in a cookie; then DWC kept it in local browser storage... but for some time now, the above has worked. So you should be able to ignore those old ways.

Nightreaver

I think having MQTT for this scenario makes sense to me.

I already like the Idea of having MQTT as part of the DuetFirmware as described here: https://github.com/dc42/RepRapFirmware/issues/284

Having a more detailed status will allow you to use MQTT to do lots of things with the report.
Even taking a picture would be possible that way if the status is properly prepared.

So lets say the MQTT connection is setup like wifi with some M-code, and you have another "M-code" (like M118 Px) to send a message to a MQTT topic, which is checked form another service/agent that will take care of anything else.

Also like mentioned in my ticket, you can use it to tigger power supplies like Tasmota-Compatible via Homeassistant/OpenHAB.

Setting up the Broker itself is part of another system, you usually dont use MQTT if there itsn't anything else in place. For these scenarios you might already have a broker running. Even if not, that broker can simply run on a RaspberryPI that you might use for the Webcam as well.

I really support that MQTT idea. MQTT with a proper topic-layout can be used to do many many more things...

Nightreaver created this issue in dc42/RepRapFirmware

closed MQTT Status Update / MQTT power control #284

fma

Why not just use a little RPi Zero W, and connect it to the Duet via I²C? RRF can send I²C commands, so once the RPi is set up, you can do whatever you want (up to cmplex tasks). An ESP8266/32 could also do the job.

Jacotheron

Regarding the RPi Zero W on I2C, we should just remember that at present a RaspberryPi can't be an I2C slave (it can communicate with I2C devices, but other devices can't establish a connection to them). I have spent a few days trying to set it up (in my case it was trying to safely shutdown the Pi, before the printer shuts itself down).

fma

I didn't know that. Too bad So one should use Arduino/ESP...

Nightreaver

yes you might use it, but then I'm having another ESP32/8622 device beside the esp of the duet, to get a feature via I2C that could be "easily" integrated via MQTT on the current duet?

I know there is many workaround to solve my issue, but MQTT is a neat solution.

Also think of this
https://forum.duet3d.com/topic/9668/telnet-printing-using-a-filament-monitor

There is already solution that work with MQTT and telegram.

So i.e. the Duet is updating a "Status" mqtt topic, when this is changing to "out of filament" send a telegram message. no need of utilizing and coding arduino or micropython. it can be done with much simpler and higher level language.

And as other already said, why do i have need to have another device for much simpler things when i already have a 179$ duet in place.

Feels like fixing my porsche with cheap volvo parts...

fma

@nightreaver said in New M-code to send a network request:

And as other already said, why do i have need to have another device for much simpler things when i already have a 179$ duet in place.

Well, I like the idea to have multiple simple systems doing specific tasks, and doing them well, rather than heavy things doing badly a lot of stuffs... Maybe RRF needs a good hooks system, with a simple and powerfull com design, to allow us to expand its capabilities without the need to modify the firmware, introducing complexity and potential unstability...

Feels like fixing my porsche with cheap volvo parts...

What a waste of Volvo parts

dc42

I have it on the list for RRF3 for the Duet to be able to send network messages. Whether we support sending via HTTP or MQTT or something else has not been decided yet.

dc42

@fma said in New M-code to send a network request:

... Maybe RRF needs a good hooks system, with a simple and powerfull com design, to allow us to expand its capabilities without the need to modify the firmware, introducing complexity and potential unstability...

That's hard to do on Duet 2, but it will be provided on Duet 3 right from the start.

Nightreaver

@dc42 said in New M-code to send a network request:

I have it on the list for RRF3 for the Duet to be able to send network messages. Whether we support sending via HTTP or MQTT or something else has not been decided yet.

IMHO is MQTT THE IoT protocol.

Other opinions ( "mqtt vs http" ) :

https://medium.com/mqtt-buddy/mqtt-vs-http-which-one-is-the-best-for-iot-c868169b3105
https://www.linkedin.com/pulse/internet-things-http-vs-websockets-mqtt-ronak-singh-cspo
https://cloud.google.com/blog/products/iot-devices/http-vs-mqtt-a-tale-of-two-iot-protocols

I'd probably not go, an reinvent the wheel with "something else". But im biased

Nightreaver

@fma said in New M-code to send a network request:

Why not just use a little RPi Zero W, and connect it to the Duet via I²C? RRF can send I²C commands, so once the RPi is set up, you can do whatever you want (up to cmplex tasks). An ESP8266/32 could also do the job.

So how would I utilize I²C on duet? I mean.. in need something while no solution is there.

I'd like to get:

power on/off before/after print
print status / filament status

I think if you can point me to any of these, I can figure out everything else

fma

M260 can send data over I²C bus:

https://duet3d.dozuki.com/Wiki/Gcode#Section_M260_i2c_Send_and_or_request_Data

On the I²C slave, just mimic some registers you will write from Duet to take the actions you need.

I recently developped a project using an ATtiny841 as I²C slave (this chip as a hardware I²C slave implementation), working this way. See:

https://framagit.org/fma38/Servo_Node

This is for an hexapod (I will use 6 of them, to control the 4 servos of each leg). But I will certainly use this chip for other projects, like here.

Code in Arduino/ dir is a master example; the real master will be a RPi, running my framework Py4bot: https://framagit.org/fma38/Py4bot