AI Print Monitoring / Printer Health monitoring

CNCModeller

In terms of AI capabilities I guess the ultimate goal would be to recognise any deviation from the STL or gcode file associated with the print.

Perhaps the plugin would build a representative visual model from the gcode and then alarm if the print head deviated...

Interested stuff...

Peter Lebiedzinski

@CNCModeller I cannot say too much, but we have been working with 3D Convolutions and Pointclouds of various CAD models, and there are very exciting things coming in the future.

CNCModeller

@Peter-Lebiedzinski

I guess another way would be to put tracking markers on the bed and print head then visually track those to confirm the printer is following the gcode...

deckingman

@Peter-Lebiedzinski I have a few questions regarding "anomaly detection". Assuming a print is showing signs of under extrusion, would your system detect and report that? If so, is it "intelligent" enough to diagnose the reason which could be say a partially blocked nozzle, print speed too high and exceeding the maximum volumetric flow rate of the hot end, temperature too low for the filament in use, motor current too low for the extruder in use, or any of the other reasons that can cause under extrusion? Also, with things such as bulging corners or other surface defects, is it intelligent enough to discriminate between genuine defects and features of the model itself?

Peter Lebiedzinski

@deckingman This would depend on what sensor values are available. If we have feedback data from the extruder motor (current, position, command position, etc), then yes. That is why we like the way Duet is moving with adding support for closed-loop steppers and low-level variable sampling at 4 Hz.

As for diagnosing the root cause of the issue, maybe, but not with high certainty, at least not as of now with our testing. We plan on trying various simple methods first before developing an end-to-end model for this, some of which would be clustering the data from our AD ("Anomaly Detection") model and seeing if we can segment out any distinct clusters that we can label as "pertaining to an issue with X, Y, or Z".

Regarding your last question, if it is a computer vision model then it is very unlikely to be able to discriminate genuine defects vs model features. It takes everything at face value as it sees it on the sensor. This of course could be addressed with a CAD model informed ML model, but we do not have plans on developing that as of now.

Peter Lebiedzinski

@CNCModeller This would be helpful, but not necessary. At the end of the day we want to make it as modular and easy to use for the user.

With nowadays computer vision operations, we can easily pointcloud the current print from a single camera angle.

We could perhaps wrap this into a feature rather quickly and release it for beta testing. The end feature would be a detector for clog, something conventional object detectors struggle to detect as it is trying to detect the absence of something in the frame.

Peter Lebiedzinski

@Gianluca
We have created a ZIP installer for the 3.5.0-rc.1 versions of RRF/DSF/DWC, it can be found here: https://github.com/printpal-io/DSF_PrintWatchAI_Plugin/releases/tag/v3.5.0-rc.1

Please note that the 3.5.0-rc.1 currently has an issue with the HTTP endpoints being registered. This can be easily worked around by doing the following:

Make sure the plugin is installed and started
SSH into the SBC
Enter the command: sudo systemctl restart duetwebserver

This will restart the webserver and register the endpoints correctly. This must be done every time the SBC is started up. This should be fixed in the next release.

Gianluca

@Peter-Lebiedzinski said in AI Print Monitoring / Printer Health monitoring:

@Gianluca
We have created a ZIP installer for the 3.5.0-rc.1 versions of RRF/DSF/DWC, it can be found here: https://github.com/printpal-io/DSF_PrintWatchAI_Plugin/releases/tag/v3.5.0-rc.1

Please note that the 3.5.0-rc.1 currently has an issue with the HTTP endpoints being registered. This can be easily worked around by doing the following:

Make sure the plugin is installed and started

SSH into the SBC

Enter the command: sudo systemctl restart duetwebserver

This will restart the webserver and register the endpoints correctly. This must be done every time the SBC is started up. This should be fixed in the next release.

Thank you so much for your precious help. This evening, after work, I will do all the tests and update you. I'm sure others who have installed the latest firmware version will find your work very useful!

Peter Lebiedzinski

@DNvM84 @jay_s_uk
I've created a Plugin for DWC-only configurations to be able to run the AI monitoring. It will work for DWC 3.4, 3.4.6, and 3.5.0-rc as of now. It can be found here: https://github.com/printpal-io/DWC_PrintWatchAI_Plugin

In order to use the plugin, you still need a device (Raspberry Pi, or any SBC) running the AI backend component, and it needs to be on the same network (Ethernet or WiFi) as the Duet mainboard. The backend component as well as how to set it up can be found here: https://github.com/printpal-io/PrintWatchAI_Backend

I would recommend using a Raspberry Pi for running the backend since they are simple and easy to use.
I have also tested the Plugin's Backend component with great success on an Orange Pi Zero3 board and it works just as well. The Orange Pi Zero3 is a little more economical ($22.99 on amazon: https://www.amazon.com/Orange-Pi-Allwinner-Quad-Core-Development/dp/B0CB1BYTT8/ref=sr_1_5?crid=BYJ6JUDXT64O&keywords=orange+pi+zero+3&qid=1694217352&sprefix=orange+pi+zero+3%2Caps%2C118&sr=8-5)

When testing, please run with Test Mode enabled, since the code that checks if the printer is 'Printing' has not been fully tested.

Gianluca

@Peter-Lebiedzinski
I first tried to install PrintWatchAI_Backend on my RPI 4 from the command prompt, and in point 5 of the list I get an error that prevents the installation. I attach a list.

pi@VORON-24-SBC:~ $ cd /home/pi
pi@VORON-24-SBC:~ $ wget https://github.com/printpal-io/PrintWatchAI_Backend/archive/refs/tags/raspberrypi.zip
--2023-09-21 11:40:21--  https://github.com/printpal-io/PrintWatchAI_Backend/archive/refs/tags/raspberrypi.zip
Risoluzione di github.com (github.com)... 140.82.121.4
Connessione a github.com (github.com)|140.82.121.4|:443... connesso.
Richiesta HTTP inviata, in attesa di risposta... 302 Found
Posizione: https://codeload.github.com/printpal-io/PrintWatchAI_Backend/zip/refs/tags/raspberrypi [segue]
--2023-09-21 11:40:21--  https://codeload.github.com/printpal-io/PrintWatchAI_Backend/zip/refs/tags/raspberrypi
Risoluzione di codeload.github.com (codeload.github.com)... 140.82.121.10
Connessione a codeload.github.com (codeload.github.com)|140.82.121.10|:443... connesso.
Richiesta HTTP inviata, in attesa di risposta... 200 OK
Lunghezza: non specificato [application/zip]
Salvataggio in: "raspberrypi.zip"

raspberrypi.zip         [ <=>                ]  19,14K  --.-KB/s    in 0,03s   

2023-09-21 11:40:22 (550 KB/s) - "raspberrypi.zip" salvato [19599]

pi@VORON-24-SBC:~ $ unzip raspberrypi.zip && rm raspberrypi.zip
Archive:  raspberrypi.zip
ff8985d86b20f8dac85b9908a5057b61b5a3df26
replace PrintWatchAI_Backend-raspberrypi/LICENSE? [y]es, [n]o, [A]ll, [N]one, [r]ename: y
  inflating: PrintWatchAI_Backend-raspberrypi/LICENSE  
  inflating: PrintWatchAI_Backend-raspberrypi/README.md  
replace PrintWatchAI_Backend-raspberrypi/main.py? [y]es, [n]o, [A]ll, [N]one, [r]ename: a
error:  invalid response [a]
replace PrintWatchAI_Backend-raspberrypi/main.py? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
  inflating: PrintWatchAI_Backend-raspberrypi/main.py  
  inflating: PrintWatchAI_Backend-raspberrypi/printwatch.service  
 extracting: PrintWatchAI_Backend-raspberrypi/printwatch/__init__.py  
  inflating: PrintWatchAI_Backend-raspberrypi/printwatch/client.py  
  inflating: PrintWatchAI_Backend-raspberrypi/printwatch/core.py  
  inflating: PrintWatchAI_Backend-raspberrypi/printwatch/interface.py  
  inflating: PrintWatchAI_Backend-raspberrypi/printwatch/utils.py  
  inflating: PrintWatchAI_Backend-raspberrypi/requirements.txt  
  inflating: PrintWatchAI_Backend-raspberrypi/ustreamer.service  
pi@VORON-24-SBC:~ $ cd PrintWatchAI_Backend-raspberrypi
pi@VORON-24-SBC:~/PrintWatchAI_Backend-raspberrypi $ pip3 install -r requirements.txt
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from -r requirements.txt (line 1)) (40.8.0)
Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (from -r requirements.txt (line 2)) (0.32.3)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (3.8.5)
Requirement already satisfied: ujson in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 4)) (5.7.0)
Collecting typing (from -r requirements.txt (line 5))
  Using cached https://www.piwheels.org/simple/typing/typing-3.7.4.3-py3-none-any.whl
Collecting uvicorn (from -r requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/ad/bd/d47ee02312640fcf26c7e1c807402d5c5eab468571153a94ec8f7ada0e46/uvicorn-0.22.0-py3-none-any.whl
Requirement already satisfied: Pillow in /usr/lib/python3/dist-packages (from -r requirements.txt (line 7)) (5.4.1)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from -r requirements.txt (line 8)) (2.21.0)
Collecting fastapi (from -r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/76/e5/ca411b260caa4e72f9ac5482f331fe74fd4eb5b97aa74d1d2806ccf07e2c/fastapi-0.103.1-py3-none-any.whl
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (23.1.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.3.3)
Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (4.7.1)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (3.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (1.9.2)
Requirement already satisfied: asynctest==0.13.0; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (0.13.0)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (4.0.3)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->-r requirements.txt (line 3)) (6.0.4)
Requirement already satisfied: click>=7.0 in /usr/lib/python3/dist-packages (from uvicorn->-r requirements.txt (line 6)) (7.0)
Collecting h11>=0.8 (from uvicorn->-r requirements.txt (line 6))
  Using cached https://files.pythonhosted.org/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl
Collecting anyio<4.0.0,>=3.7.1 (from fastapi->-r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/19/24/44299477fe7dcc9cb58d0a57d5a7588d6af2ff403fdd2d47a246c91a3246/anyio-3.7.1-py3-none-any.whl
Collecting pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 (from fastapi->-r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/82/06/fafdc75e48b248eff364b4249af4bcc6952225e8f20e8205820afc66e88e/pydantic-2.3.0-py3-none-any.whl
Collecting starlette<0.28.0,>=0.27.0 (from fastapi->-r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/58/f8/e2cca22387965584a409795913b774235752be4176d276714e15e1a58884/starlette-0.27.0-py3-none-any.whl
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from attrs>=17.3.0->aiohttp->-r requirements.txt (line 3)) (6.7.0)
Requirement already satisfied: idna>=2.0 in /usr/lib/python3/dist-packages (from yarl<2.0,>=1.0->aiohttp->-r requirements.txt (line 3)) (2.6)
Collecting exceptiongroup; python_version < "3.11" (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 9))
  Downloading https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl
Collecting sniffio>=1.1 (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 9))
  Downloading https://files.pythonhosted.org/packages/c3/a0/5dba8ed157b0136607c7f2151db695885606968d1fae123dc3391e0cfdbf/sniffio-1.3.0-py3-none-any.whl
Collecting pydantic-core==2.6.3 (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi->-r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/cb/fe/8c9363389f8f303fb151895af83ac30e06c0406779fe188b4281a64e4c50/pydantic_core-2.6.3.tar.gz
  Installing build dependencies ... done
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib/python3.7/tokenize.py", line 447, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-install-r73yhlap/pydantic-core/setup.py'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-r73yhlap/pydantic-core/

Even though the installation of PrintWatchAI_Backend was not successful, I installed the plugin on DWC, but once I enter the data in "settings" when I regenerate the page, they are not saved and you have to enter them again.
In the "preview" screen the webcam does not work.
DWC AI.jpg

For now I don't have any other tests I can do...

Peter Lebiedzinski

@Gianluca This looks like it might be an issue with the version of pip installed. Can you try updating it with:
pip3 install --upgrade pip
or
pip install --upgrade pip

and then re-running the installation commands that failed for the backend?

None of the DWC plugin settings or UI items will work if the Backend is not running so both of your issues should be fixed by doing this. Let me know if this fixes your issue.

Gianluca

@Peter-Lebiedzinski
I'm sorry, I tried to install PIP but I'm getting other errors. Unfortunately I am unable to continue..

pi@VORON-24-SBC:~ $ pip3 install --upgrade pip
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in ./.local/lib/python3.7/site-packages (23.2.1)
pi@VORON-24-SBC:~ $ pip install --upgrade pip
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already up-to-date: pip in ./.local/lib/python2.7/site-packages (20.3.4)
WARNING: You are using pip version 20.3.4; however, version 23.2.1 is available.
You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.

pfmoore created this issue in pypa/pip

closed ImportError in system pip wrappers after an upgrade #5599

pfmoore created this issue in pypa/pip

closed ImportError in system pip wrappers after an upgrade #5599

A Former User

This post is deleted!

Peter Lebiedzinski

@Gianluca said in AI Print Monitoring / Printer Health monitoring:

You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.

You may need to upgrade pip using python.

See if this post helps at all: https://github.com/pypa/pip/issues/5599

or if any of these commands work:
python3 -m pip install --upgrade pip
python -m pip install --upgrade pip

pfmoore created this issue in pypa/pip

closed ImportError in system pip wrappers after an upgrade #5599

Peter Lebiedzinski

@Herve_Smith This Forum post was made prior to us planning on supporting DWC-only, since then we have developed a plugin that works with DWC-only configurations, see forum comment:

@Peter-Lebiedzinski said in AI Print Monitoring / Printer Health monitoring:

@DNvM84 @jay_s_uk
I've created a Plugin for DWC-only configurations to be able to run the AI monitoring. It will work for DWC 3.4, 3.4.6, and 3.5.0-rc as of now. It can be found here: https://github.com/printpal-io/DWC_PrintWatchAI_Plugin

In order to use the plugin, you still need a device (Raspberry Pi, or any SBC) running the AI backend component, and it needs to be on the same network (Ethernet or WiFi) as the Duet mainboard. The backend component as well as how to set it up can be found here: https://github.com/printpal-io/PrintWatchAI_Backend

I would recommend using a Raspberry Pi for running the backend since they are simple and easy to use.
I have also tested the Plugin's Backend component with great success on an Orange Pi Zero3 board and it works just as well. The Orange Pi Zero3 is a little more economical ($22.99 on amazon: https://www.amazon.com/Orange-Pi-Allwinner-Quad-Core-Development/dp/B0CB1BYTT8/ref=sr_1_5?crid=BYJ6JUDXT64O&keywords=orange+pi+zero+3&qid=1694217352&sprefix=orange+pi+zero+3%2Caps%2C118&sr=8-5)

When testing, please run with Test Mode enabled, since the code that checks if the printer is 'Printing' has not been fully tested.