AI Print Monitoring / Printer Health monitoring
-
@Peter-Lebiedzinski thanks for posting here! I am looking forward to trying this out!
It could help to specify the minimum requirements for your implementation that people need to try it out.
-
Of course, thank you for reminding me.
The minimum requirements are as follows:
Visual Failure Monitoring:
-
Firmware Requirements:
- RepRapFirmware 3.4.5b3+ and above, this firmware contains support for the application sending commands to the printer when a failure has occurred. The software will still work and be able to notify you of failures if not using this firmware, however no actions will be able to be taken on behalf of the printer.
-
Supported formats:
- HTTP/S or MJPEG stream (preferred)
- RTSP
- USB
- PiCamera
-
Resolution:
- 480 x 480 pixels @ 100% quality (no compression) --> 768 x 768 @ 100% quality or greater recommended
-
Hardware:
- Able to process 1 image every 10 seconds
- Internet connected (Ethernet or WiFi)
- Can communicate with the Duet3D board
- Examples:
- Raspberry Pi 3
- Raspberry Pi 4
- Raspberry Pi zero W
- ESP32 - does not have ethernet option to communicate with Duet3D board
- Edge option:
- Embedded AI is an option and the hardware will fit inside the electronics cabinet of the printer. For this you'd have to contact us.
Anomaly Detection:
- Firmware requirements:
- Most recent firmware with support for datalogging (TBD)
- Hardware:
- Although this should be available for everyone in some capacity, hardware that provides more feedback information is better, this includes, but is not limited to:
- Closed-loop steppers/servos
- Accelerometer
- Lidar
- Temperature sensors
- Flow sensors
- Voltage/Amperage sensors
- Although this should be available for everyone in some capacity, hardware that provides more feedback information is better, this includes, but is not limited to:
-
-
To all who upvoted (and those who didn't but are still interested): we have the working beta version of the AI monitoring plugin available for testing. Currently, the plugin will only work on Duet printers running in SBC mode.
The ZIP install file for the plugin may be found on GitHub: https://github.com/printpal-io/DSF_PrintWatchAI_Plugin
Please reach out to me personally (lebiedzinskip@printpal.io) to receive a free API key for testing without limits. The free version without the API key will only allow you up to 12 hours of AI monitoring.
This is what the plugin currently looks like, any feedback is taken seriously, and very much appreciated:
-
@Peter-Lebiedzinski
Hi Peter, I received your email.
Any recommendations as to where the camera should be placed?
I ask because while I was researching other versions of monitoring software, they stated that the camera should be looking at the table at no more than a specified angle above the build surface.Thank you for keeping me in the loop.
DNvM -
@DNvM84 I would recommend placing the camera off of any corner of the print bed, with the center of the frame pointed at where the majority of the print will take.
I would also recommend placing it above the print bed such that it is 15 - 45 degrees from the vector of the plane that defines the print bed:
The angle can be determined by what you are printing and by how it works with the toolhead on your machine. Play around and find out what works best. Also, if printing an item that has a highly detailed infill that may confuse the AI, you would want to have the camera at a lower/higher angle so it does not perceive the infill as highly dense details.
-
Hi Peter,
I am very interested in your work and would like to try it.
Unfortunately I can't install your plugin on my system.
I have downloaded the file "PrintWatch-3.4.2.zip" ver.1.0 but there are compatibility problems, with the latest firmware version 3.5.0.RC1.
Are there any suggestions for a successful installation, or will other versions be released?
-
@Gianluca I will create a ZIP installer for your version of DSF. I will get back to you within the hour.
-
@Peter-Lebiedzinski said in AI Print Monitoring / Printer Health monitoring:
@Gianluca I will create a ZIP installer for your version of DSF. I will get back to you within the hour.
Thank You!!!
-
Sounds like a really interesting service.
I think it'd be a good investment in terms of the subscription when I'm in production mode, and the free license would be possibly enough for now and then small hobby printers.
https://printpal.io/printwatch/
Definitely going to keep up with developments and give it a try on my printers once they're back together again after the current round of upgrades.
-
In terms of AI capabilities I guess the ultimate goal would be to recognise any deviation from the STL or gcode file associated with the print.
Perhaps the plugin would build a representative visual model from the gcode and then alarm if the print head deviated...
Interested stuff...
-
@CNCModeller I cannot say too much, but we have been working with 3D Convolutions and Pointclouds of various CAD models, and there are very exciting things coming in the future.
-
I guess another way would be to put tracking markers on the bed and print head then visually track those to confirm the printer is following the gcode...
-
@Peter-Lebiedzinski I have a few questions regarding "anomaly detection". Assuming a print is showing signs of under extrusion, would your system detect and report that? If so, is it "intelligent" enough to diagnose the reason which could be say a partially blocked nozzle, print speed too high and exceeding the maximum volumetric flow rate of the hot end, temperature too low for the filament in use, motor current too low for the extruder in use, or any of the other reasons that can cause under extrusion? Also, with things such as bulging corners or other surface defects, is it intelligent enough to discriminate between genuine defects and features of the model itself?
-
@deckingman This would depend on what sensor values are available. If we have feedback data from the extruder motor (current, position, command position, etc), then yes. That is why we like the way Duet is moving with adding support for closed-loop steppers and low-level variable sampling at 4 Hz.
As for diagnosing the root cause of the issue, maybe, but not with high certainty, at least not as of now with our testing. We plan on trying various simple methods first before developing an end-to-end model for this, some of which would be clustering the data from our AD ("Anomaly Detection") model and seeing if we can segment out any distinct clusters that we can label as "pertaining to an issue with X, Y, or Z".
Regarding your last question, if it is a computer vision model then it is very unlikely to be able to discriminate genuine defects vs model features. It takes everything at face value as it sees it on the sensor. This of course could be addressed with a CAD model informed ML model, but we do not have plans on developing that as of now.
-
@CNCModeller This would be helpful, but not necessary. At the end of the day we want to make it as modular and easy to use for the user.
With nowadays computer vision operations, we can easily pointcloud the current print from a single camera angle.
We could perhaps wrap this into a feature rather quickly and release it for beta testing. The end feature would be a detector for clog, something conventional object detectors struggle to detect as it is trying to detect the absence of something in the frame.
-
@Gianluca
We have created a ZIP installer for the 3.5.0-rc.1 versions of RRF/DSF/DWC, it can be found here: https://github.com/printpal-io/DSF_PrintWatchAI_Plugin/releases/tag/v3.5.0-rc.1Please note that the 3.5.0-rc.1 currently has an issue with the HTTP endpoints being registered. This can be easily worked around by doing the following:
- Make sure the plugin is installed and started
- SSH into the SBC
- Enter the command:
sudo systemctl restart duetwebserver
This will restart the webserver and register the endpoints correctly. This must be done every time the SBC is started up. This should be fixed in the next release.
-
@Peter-Lebiedzinski said in AI Print Monitoring / Printer Health monitoring:
@Gianluca
We have created a ZIP installer for the 3.5.0-rc.1 versions of RRF/DSF/DWC, it can be found here: https://github.com/printpal-io/DSF_PrintWatchAI_Plugin/releases/tag/v3.5.0-rc.1Please note that the 3.5.0-rc.1 currently has an issue with the HTTP endpoints being registered. This can be easily worked around by doing the following:
- Make sure the plugin is installed and started
- SSH into the SBC
- Enter the command:
sudo systemctl restart duetwebserver
This will restart the webserver and register the endpoints correctly. This must be done every time the SBC is started up. This should be fixed in the next release.
Thank you so much for your precious help. This evening, after work, I will do all the tests and update you. I'm sure others who have installed the latest firmware version will find your work very useful!
-
@DNvM84 @jay_s_uk
I've created a Plugin for DWC-only configurations to be able to run the AI monitoring. It will work for DWC 3.4, 3.4.6, and 3.5.0-rc as of now. It can be found here: https://github.com/printpal-io/DWC_PrintWatchAI_PluginIn order to use the plugin, you still need a device (Raspberry Pi, or any SBC) running the AI backend component, and it needs to be on the same network (Ethernet or WiFi) as the Duet mainboard. The backend component as well as how to set it up can be found here: https://github.com/printpal-io/PrintWatchAI_Backend
I would recommend using a Raspberry Pi for running the backend since they are simple and easy to use.
I have also tested the Plugin's Backend component with great success on an Orange Pi Zero3 board and it works just as well. The Orange Pi Zero3 is a little more economical ($22.99 on amazon: https://www.amazon.com/Orange-Pi-Allwinner-Quad-Core-Development/dp/B0CB1BYTT8/ref=sr_1_5?crid=BYJ6JUDXT64O&keywords=orange+pi+zero+3&qid=1694217352&sprefix=orange+pi+zero+3%2Caps%2C118&sr=8-5)When testing, please run with
Test Mode
enabled, since the code that checks if the printer is 'Printing' has not been fully tested. -
@Peter-Lebiedzinski
I first tried to install PrintWatchAI_Backend on my RPI 4 from the command prompt, and in point 5 of the list I get an error that prevents the installation. I attach a list.pi@VORON-24-SBC:~ $ cd /home/pi pi@VORON-24-SBC:~ $ wget https://github.com/printpal-io/PrintWatchAI_Backend/archive/refs/tags/raspberrypi.zip --2023-09-21 11:40:21-- https://github.com/printpal-io/PrintWatchAI_Backend/archive/refs/tags/raspberrypi.zip Risoluzione di github.com (github.com)... 140.82.121.4 Connessione a github.com (github.com)|140.82.121.4|:443... connesso. Richiesta HTTP inviata, in attesa di risposta... 302 Found Posizione: https://codeload.github.com/printpal-io/PrintWatchAI_Backend/zip/refs/tags/raspberrypi [segue] --2023-09-21 11:40:21-- https://codeload.github.com/printpal-io/PrintWatchAI_Backend/zip/refs/tags/raspberrypi Risoluzione di codeload.github.com (codeload.github.com)... 140.82.121.10 Connessione a codeload.github.com (codeload.github.com)|140.82.121.10|:443... connesso. Richiesta HTTP inviata, in attesa di risposta... 200 OK Lunghezza: non specificato [application/zip] Salvataggio in: "raspberrypi.zip" raspberrypi.zip [ <=> ] 19,14K --.-KB/s in 0,03s 2023-09-21 11:40:22 (550 KB/s) - "raspberrypi.zip" salvato [19599] pi@VORON-24-SBC:~ $ unzip raspberrypi.zip && rm raspberrypi.zip Archive: raspberrypi.zip ff8985d86b20f8dac85b9908a5057b61b5a3df26 replace PrintWatchAI_Backend-raspberrypi/LICENSE? [y]es, [n]o, [A]ll, [N]one, [r]ename: y inflating: PrintWatchAI_Backend-raspberrypi/LICENSE inflating: PrintWatchAI_Backend-raspberrypi/README.md replace PrintWatchAI_Backend-raspberrypi/main.py? [y]es, [n]o, [A]ll, [N]one, [r]ename: a error: invalid response [a] replace PrintWatchAI_Backend-raspberrypi/main.py? [y]es, [n]o, [A]ll, [N]one, [r]ename: A inflating: PrintWatchAI_Backend-raspberrypi/main.py inflating: PrintWatchAI_Backend-raspberrypi/printwatch.service extracting: PrintWatchAI_Backend-raspberrypi/printwatch/__init__.py inflating: PrintWatchAI_Backend-raspberrypi/printwatch/client.py inflating: PrintWatchAI_Backend-raspberrypi/printwatch/core.py inflating: PrintWatchAI_Backend-raspberrypi/printwatch/interface.py inflating: PrintWatchAI_Backend-raspberrypi/printwatch/utils.py inflating: PrintWatchAI_Backend-raspberrypi/requirements.txt inflating: PrintWatchAI_Backend-raspberrypi/ustreamer.service pi@VORON-24-SBC:~ $ cd PrintWatchAI_Backend-raspberrypi pi@VORON-24-SBC:~/PrintWatchAI_Backend-raspberrypi $ pip3 install -r requirements.txt Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from -r requirements.txt (line 1)) (40.8.0) Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (from -r requirements.txt (line 2)) (0.32.3) Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (3.8.5) Requirement already satisfied: ujson in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 4)) (5.7.0) Collecting typing (from -r requirements.txt (line 5)) Using cached https://www.piwheels.org/simple/typing/typing-3.7.4.3-py3-none-any.whl Collecting uvicorn (from -r requirements.txt (line 6)) Using cached https://files.pythonhosted.org/packages/ad/bd/d47ee02312640fcf26c7e1c807402d5c5eab468571153a94ec8f7ada0e46/uvicorn-0.22.0-py3-none-any.whl Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from -r requirements.txt (line 7)) (5.4.1) Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (2.21.0) Collecting fastapi (from -r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/76/e5/ca411b260caa4e72f9ac5482f331fe74fd4eb5b97aa74d1d2806ccf07e2c/fastapi-0.103.1-py3-none-any.whl Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (23.1.0) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.3.3) Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (4.7.1) Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (3.2.0) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.9.2) Requirement already satisfied: asynctest==0.13.0; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (0.13.0) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (4.0.3) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (6.0.4) Requirement already satisfied: click>=7.0 in /usr/lib/python3/dist-packages (from uvicorn->-r requirements.txt (line 6)) (7.0) Collecting h11>=0.8 (from uvicorn->-r requirements.txt (line 6)) Using cached https://files.pythonhosted.org/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl Collecting anyio<4.0.0,>=3.7.1 (from fastapi->-r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/19/24/44299477fe7dcc9cb58d0a57d5a7588d6af2ff403fdd2d47a246c91a3246/anyio-3.7.1-py3-none-any.whl Collecting pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 (from fastapi->-r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/82/06/fafdc75e48b248eff364b4249af4bcc6952225e8f20e8205820afc66e88e/pydantic-2.3.0-py3-none-any.whl Collecting starlette<0.28.0,>=0.27.0 (from fastapi->-r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/58/f8/e2cca22387965584a409795913b774235752be4176d276714e15e1a58884/starlette-0.27.0-py3-none-any.whl Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from attrs>=17.3.0->aiohttp->-r requirements.txt (line 3)) (6.7.0) Requirement already satisfied: idna>=2.0 in /usr/lib/python3/dist-packages (from yarl<2.0,>=1.0->aiohttp->-r requirements.txt (line 3)) (2.6) Collecting exceptiongroup; python_version < "3.11" (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 9)) Downloading https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl Collecting sniffio>=1.1 (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 9)) Downloading https://files.pythonhosted.org/packages/c3/a0/5dba8ed157b0136607c7f2151db695885606968d1fae123dc3391e0cfdbf/sniffio-1.3.0-py3-none-any.whl Collecting pydantic-core==2.6.3 (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi->-r requirements.txt (line 9)) Using cached https://files.pythonhosted.org/packages/cb/fe/8c9363389f8f303fb151895af83ac30e06c0406779fe188b4281a64e4c50/pydantic_core-2.6.3.tar.gz Installing build dependencies ... done Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.7/tokenize.py", line 447, in open buffer = _builtin_open(filename, 'rb') FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-install-r73yhlap/pydantic-core/setup.py' ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-r73yhlap/pydantic-core/
Even though the installation of PrintWatchAI_Backend was not successful, I installed the plugin on DWC, but once I enter the data in "settings" when I regenerate the page, they are not saved and you have to enter them again.
In the "preview" screen the webcam does not work.
For now I don't have any other tests I can do...
-
@Gianluca This looks like it might be an issue with the version of pip installed. Can you try updating it with:
pip3 install --upgrade pip
or
pip install --upgrade pip
and then re-running the installation commands that failed for the backend?
None of the DWC plugin settings or UI items will work if the Backend is not running so both of your issues should be fixed by doing this. Let me know if this fixes your issue.