Setting tool offset using USB microscope

Danal

@tekstyle said in Setting tool offset using USB microscope:

i have a raspberry pi that i used to run octoprint with marlin. i might pick up a c270 to get this to work if i can't use the current raspberry pi camera with a phone macro lens.

My github has a script that interfaces with a Duet, that is it MUST run on a D3/Pi combo connected with SPI cable... and it has another script that will work for (almost) any printer, because that script asks you to type in the coordinates as it steps you through the alignment. Meaning the Pi need not be connected to anything (beyond the camera).

So you can use whichever, if you wish.

On the C270, take it apart and screw the lens in (or was it out?) until it focuses within a few inches. Works great.

tekstyle

@Danal

i got installopencv.sh installed on my raspberry pi. i have a Geany under programming. what do I do wiht the.py files? i have a duet2wifi

Danal

For duet 2, the only one that will really work is DuetPython/toolAlignImage.py

You will need either a physical console, or VNC. It has to be a screen that can display a video window. There are tons of tutorials on how to set those up. Once you have a session running on the graphics screen (not via ssh), change into that directory and enter:

python3 toolAlignImage.py

Within a few seconds, an image from your USB camera will appear (USB only, not PiCam). This camera should be a "close focus", and I still recommend the Logitech C270, removed from its case, and the lens screwed in (or out, I don't remember) until it focuses about 3 to 4 cm (inch and a half in barbarian units) from the lens. You can fine tune this once the camera is mounted.

Once you have an image up, follow the prompts in the script. It will have you do things, and enter numbers from the web console. In reality, this will work with ANY printer that mounts tools, you can jog, and it displays positions.

Roughly, it will have you jog each tool into place, and center it exactly, and enter the X and Y shown on the screen.
IMPORTANT: At the beginning, it will have you enter G10 Px X0Y0 for each tool. DO NOT SKIP THIS, it is easy to miss.

At the end, it will give you your G10 commands. NOTE: I MAY NEED TO UPDATE THE CALCULATIONS, I made some interesting discoveries lately. It might work... I will look at it tomorrow and let you know. (or update GIT).

Example camera mount. This really should be narrower, I quickly re-used it from another project.

IMG_0247 (1).jpeg

STLs for that camera mount:

upper_lid 2020 mount.stl
cam_holder.stl
inner_lid.stl

Danal

And... if the script doesn't work completely correctly, as long as it displays the live video stream with the green circle, you could write all the X and Y on a piece of paper and do the math yourself.

In fact, just added a script called videoOnly.py. If nothing else works, use that and pencil and paper.

tekstyle

@Danal said in Setting tool offset using USB microscope:

For duet 2, the only one that will really work is DuetPython/toolAlignImage.py
You will need either a physical console, or VNC. It has to be a screen that can display a video window. There are tons of tutorials on how to set those up. Once you have a session running on the graphics screen (not via ssh), change into that directory and enter:
python3 toolAlignImage.py

this is what i am getting now. I am connected to raspberry pi via keyboard, mouse, and a monitor.

pi@raspberrypi:~/Desktop/Duet $ python3 toolAlignImage.py
Traceback (most recent call last):
File "toolAlignImage.py", line 11, in <module>
import cv2
ModuleNotFoundError: No module named 'cv2'

Danal

@tekstyle said in Setting tool offset using USB microscope:

@Danal

i got installopencv.sh installed on my raspberry pi. i have a Geany under programming. what do I do wiht the.py files? i have a duet2wifi

Did you run that script? It takes upwards of an hour. Command to run it:

cd to the directory where it is.

./installopencv.sh

timcurtis67

@Danal said in Setting tool offset using USB microscope:

While this looks like the other pic, this is machine vision "finding" the end of the nozzle. This also produces a coordinate (in pixels), and you can see that averaging these over a bunch of reads can get it nailed down:

316.0 240.0
316.0 240.0
315.88 239.88
316.0 239.75
316.0 240.0
316.0 240.0
316.0 240.0

I moved the nozzle 0.1mm, and the readings (in pixels) changed to:

312.0 238.0
312.0 238.0

appears to be about -4 x -2 pixels, so about 4.472 on the diagonal. Part of the work that needs to be done is figuring out how to "calibrate" movement in Pixels to movement in MM, when this is installed on a bunch of peoples printers, with different focus, and maybe even different cameras.

Danal,
Just out of curiosity and for my own understanding.

I'm not sure if the nozzle is being moved by the Duet drivers or not? But if it is moved by the Duet drivers then after you lock in on the location of the nozzle can't you just get the XY location from the Duet, do the location (offset) calculations and then send it back into the Duet with the G10 command to set the proper offset?

Then the pixels to mm conversion would be irrelevant. Even different camera focus lengths wouldn't really matter. You can poll the Duet for it's XY location after the alignment is locked.

Danal

In the final alignment, yes, getting the position from the duet will work fine. No pixel/mm mapping required.

At the same time, lets assume the video image is 720 by 480. We find a circle with center, say 100 200. What is a good movement to get the nozzle closer to the circle being 720/2 480/2 (i.e. centered)? 1MM? 10mm? .1?

Perhaps the best way is to avoid camera to mm mapping at all, and just make a reasonable guess for the first move (1mm) and loop through successive approximations.

Perhaps the initial move could be 1mm, and establish a rough mm to pixel mapping, and the next move could be really close, and the final centering could be successive approximation.

These are all valid design ideas. Some involve the mapping, some don't. I'll need to play around with it and see what really works.

A separate, but related, challenge comes with the "out of the box" OpenCV "find a circle" function. It has several oddities.

First, it is REALLY good at finding circles. Glare on the nozzle will often result in several circles returned. We need it to find just the nozzle. One of the best ways to do that is to limit a min and max radius of found circles in pixels. I've hand tuned my examples, but I need to find a way to make this work for (almost) everyone.

Second, it is SLOW. The more constraints you add (like min/max from point 1 above) the slower it gets. A frame rate of about 2 a second is really good when searching with constraints. It gets even slower if it does not find circles (I assume it keeps looking). I am seriously considering modify the library to return a null if a certain amount of time passes and no circles have been found.

Combine these two things and getting it to find the actual nozzle, once and only once, is a challenge that I'm still researching.

One thought that I have not looked at at all: Are there other circle search functions built on OpenCV, beyond the one that comes with OCV?

The real key to all of this is time. At this moment, I'm rebuilding the printer with which I was doing most of these experiments. Maybe in a few days I can try some of this again.

tekstyle

@Danal said in Setting tool offset using USB microscope:

@tekstyle said in Setting tool offset using USB microscope:

@Danal

i got installopencv.sh installed on my raspberry pi. i have a Geany under programming. what do I do wiht the.py files? i have a duet2wifi

Did you run that script? It takes upwards of an hour. Command to run it:

cd to the directory where it is.

./installopencv.sh

Yes I did. It took a while to run. Then it would it was done, my monitor just turned off and I couldn't wake it up. Not sure if that's normal. Pulled the power and reboot. Went to the folder containing the toolalignimage.py file and that's what I got. I'll try to reformat, reinstall raspbian, and reinstall installopencv.sh again.

Danal

If at all possible, start with the image provided by Duet, the "full" not the "lite".

The install script does not turn off the monitor, it probably just "power saved" out. You may wish to search on "how to keep my screen on" or "how to keep my Pi from powering off".

tekstyle

@Danal said in Setting tool offset using USB microscope:

If at all possible, start with the image provided by Duet, the "full" not the "lite".

The install script does not turn off the monitor, it probably just "power saved" out. You may wish to search on "how to keep my screen on" or "how to keep my Pi from powering off".

The duetpi full and lite doesn't work with duet2. It's designed for the duet3 unfortunately.

Danal

@tekstyle said in Setting tool offset using USB microscope:

@Danal said in Setting tool offset using USB microscope:

If at all possible, start with the image provided by Duet, the "full" not the "lite".

The install script does not turn off the monitor, it probably just "power saved" out. You may wish to search on "how to keep my screen on" or "how to keep my Pi from powering off".

The duetpi full and lite doesn't work with duet2. It's designed for the duet3 unfortunately.

True... but... you can still run them. Just turn off the DSF, and you have a regular Pi.

Having said that, current base raspbian should be OK as well. We just might have to install something that the isntallopencv.sh script doesn't currently get, because I built and tested it on top of the Duet image. No big deal either way.

Let me know when you have it rebuilt, modified for no sleep, and opencv installed.

tekstyle

i followed this tutorial to set up duetpi on raspberry pi 3. took out the card and put it into the raspberry pi. powered it up (not connected to the duet2. not sure how i would do it). all i get is an endless list of EXT4-fs error lines after line. been doing this for the past 2 hours. not sure if I did something wrong?

droftarts

@Danal said in Setting tool offset using USB microscope:

Second, it is SLOW.

Are you doing any post-capture/pre-recognition processing of the image? You can convert colour images to greyscale or black and white (with a threshold level), and even use the black and white version as a mask. This will reduce the amount of image recognition processing massively. I can't say I've tried this myself, just read about it.

Ian

Danal

@droftarts said in Setting tool offset using USB microscope:

@Danal said in Setting tool offset using USB microscope:

Second, it is SLOW.

Are you doing any post-capture/pre-recognition processing of the image? You can convert colour images to greyscale or black and white (with a threshold level), and even use the black and white version as a mask. This will reduce the amount of image recognition processing massively. I can't say I've tried this myself, just read about it.

Ian

Yeah, that's all very fundamental, always grayscaling, trying different kinds of blur and threshold and mask and... Some do perform better than others, but the best still have highly variable "search" times that are extremely sensitive to the tuning parameters vs. the image.

Danal

@tekstyle said in Setting tool offset using USB microscope:

(not connected to the duet2. not sure how i would do it).

You can't. Don't connect it. It is really a stand-alone PI, just happens to be built with the Duet image as a start.

i followed this tutorial to set up duetpi on raspberry pi 3. took out the card and put it into the raspberry pi. powered it up, all i get is an endless list of EXT4-fs error lines after line. been doing this for the past 2 hours. not sure if I did something wrong?

Followed the Duet "etch" tutorial? And edited wpasupplicant.conf? And did NOT format when windows complained about the card after etching?

That last bit is the most important. The card is partitioned, with a windows readable "boot" partition and a much larger Linux partition. Windows tries to assign a drive letter to that Linux partition, balks, and asks to format it. Which you must always decline,

EXT4-fs is the file system that windows doesn't like, and asks to format. That certainly seems to relate to those error messages.

tekstyle

@Danal said in Setting tool offset using USB microscope:

@tekstyle said in Setting tool offset using USB microscope:

(not connected to the duet2. not sure how i would do it).

You can't. Don't connect it. It is really a stand-alone PI, just happens to be built with the Duet image as a start.

i followed this tutorial to set up duetpi on raspberry pi 3. took out the card and put it into the raspberry pi. powered it up, all i get is an endless list of EXT4-fs error lines after line. been doing this for the past 2 hours. not sure if I did something wrong?

Followed the Duet "etch" tutorial? And edited wpasupplicant.conf? And did NOT format when windows complained about the card after etching?

That last bit is the most important. The card is partitioned, with a windows readable "boot" partition and a much larger Linux partition. Windows tries to assign a drive letter to that Linux partition, balks, and asks to format it. Which you must always decline,

EXT4-fs is the file system that windows doesn't like, and asks to format. That certainly seems to relate to those error messages.

Yup. I followed the tutorial on duet exactly. Only thing was inside the boot partition, it already contained a wpasupplicant.conf file. I Openned itnin notepad and editted it with my ssid and PW. Pulled the card out and popped it in to the rpi. Then powered it up. Will try again tonight

tekstyle

@tekstyle said in Setting tool offset using USB microscope:

@Danal said in Setting tool offset using USB microscope:

@tekstyle said in Setting tool offset using USB microscope:

(not connected to the duet2. not sure how i would do it).

You can't. Don't connect it. It is really a stand-alone PI, just happens to be built with the Duet image as a start.

i followed this tutorial to set up duetpi on raspberry pi 3. took out the card and put it into the raspberry pi. powered it up, all i get is an endless list of EXT4-fs error lines after line. been doing this for the past 2 hours. not sure if I did something wrong?

Followed the Duet "etch" tutorial? And edited wpasupplicant.conf? And did NOT format when windows complained about the card after etching?

That last bit is the most important. The card is partitioned, with a windows readable "boot" partition and a much larger Linux partition. Windows tries to assign a drive letter to that Linux partition, balks, and asks to format it. Which you must always decline,

EXT4-fs is the file system that windows doesn't like, and asks to format. That certainly seems to relate to those error messages.

Yup. I followed the tutorial on duet exactly. Only thing was inside the boot partition, it already contained a wpasupplicant.conf file. I Openned itnin notepad and editted it with my ssid and PW. Pulled the card out and popped it in to the rpi. Then powered it up. Will try again tonight

you won't believe this. I used my other laptop running windows 8, win32diskimager to load the duetpi image. and that worked. i have no idea why using balena etcher on win 10 doesn't work.

dc42

@wilriker said in Setting tool offset using USB microscope:

@dc42 Would a G10 L10 to set tool offsets relative to current position desirable (making G10 even more complex OTOH)?

Not necessary, because you can already do that using the object model.

@tekstyle said in Setting tool offset using USB microscope:

Tool offset with limit switches would be great.

Already supported if you can configure the limit switch as a Z probe. See the G30 S-2 command.

Danal

@wilriker said in Setting tool offset using USB microscope:
@dc42 Would a G10 L10 to set tool offsets relative to current position desirable (making G10 even more complex OTOH)?

Also, any scripting environment that is complex enough to do multi-tool alignment with "one button" is going to have basic math. And that's all that is needed to produce the final G10 commands. Subtract T0's coordinates from T1s, etc, etc. So, no, G10 does not need any changes.

P.S. That's why Python at the moment... a better place might be conditional G-Code, but that does not yet work from the Pi DSF... so having a single script have access to the camera, and the printer, requires something on the attached Pi at the moment. Python seems very suitable for the required image processing.

@tekstyle said in Setting tool offset using USB microscope:
Tool offset with limit switches would be great.

The "Zonly.py" script in the https://github.com/DanalEstes/PythonDSF/ repository is touchplate based. It is still very much experimental 'under development'. Still maybe worth looking at for ideas.