DuetLapse available for Alpha testing

oozeBot

In case it went overlooked, I just wanted to toss out there that I came up with a solution for timelapse video when using a Duet 3 with attached Raspberry Pi..

https://forum.duet3d.com/topic/19225/timelapse-video-through-execonmcode

stuartofmt

@oozeBot

Thanks - a nice piece of work. I looked at it at one time but did not want to fiddle with generating the additional slicer code (likely an irrational reaction).

In my case I could replace the wget with ffmpeg for the stream output from the app I am using and also the modified aggregation ffmpeg.

I may end up back at your script - but for now I am wrestling with recompiling ffmpeg to get some additional filters that are not present in the somewhat old Pi version.

oozeBot

@stuartofmt I get it, but maybe there is something there that could be built upon. It would also be possible to put the trigger in daemon.g but I wanted to control the trigger within the gCode at precisely the same point, etc..

stuartofmt

@oozeBot

Just so you know - DuetLapse is working fine and has some nice features. I'm just embellishing around the edges.

Being unfamiliar with Python - some of it is trial and error. Also some of it (e.g. ffmpeg on Pi being quite old) would be the same in either approach.

I've written quite a few bash scripts in the past and it would be a bit of a slog to achieve the same flexibility as DuetLapse with bash - although not impossible by any means.

As of this afternoon - I have another embellishment done (required newer ffmpeg) and and will update my git repository shortly.

stuartofmt

@JohnOCFII

I'm pretty well there - doing some final testing now. I'll post the new DuetLapse.py later today / tomorrow. It has additional parameters to control the output directory and the extra time at the end. The defaults are the same as before.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

ffmpeg -r 10 -i /tmp/DuetLapse/IMG%08d.jpeg -c:v libx264 -vf tpad=stop_mode=clone:stop_duration=2,fps=10 ./out1.mp4

Here are the steps I went through to build a more up-to-date ffmpeg (well - the straight line version that avoids the various dead ends and failures to compile the optional packages). They were taken from here but shortened to omit the optional packages in the post and using a build that omits calling those packages.

https://pimylifeup.com/compiling-ffmpeg-raspberry-pi/

Remove existing ffmpeg

sudo apt remove ffmpeg

Make sure OS is up to date

sudo apt update
sudo apt upgrade

Install dependencies

sudo apt -y install autoconf automake build-essential cmake doxygen git graphviz imagemagick libasound2-dev libass-dev libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavutil-dev libfreetype6-dev libgmp-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libopus-dev librtmp-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-net-dev libsdl2-ttf-dev libsnappy-dev libsoxr-dev libssh-dev libssl-dev libtool libv4l-dev libva-dev libvdpau-dev libvo-amrwbenc-dev libvorbis-dev libwebp-dev libx264-dev libx265-dev libxcb-shape0-dev libxcb-shm0-dev libxcb-xfixes0-dev libxcb1-dev libxml2-dev lzma-dev meson nasm pkg-config python3-dev python3-pip texinfo wget yasm zlib1g-dev libdrm-dev

Build ffmpeg (takes a while)

git clone --depth 1 https://github.com/FFmpeg/FFmpeg.git ~/FFmpeg \
  && cd ~/FFmpeg \
  && ./configure \
    --extra-cflags="-I/usr/local/include" \
    --extra-ldflags="-L/usr/local/lib" \
    --extra-libs="-lpthread -lm -latomic" \
    --arch=armel \
    --enable-gmp \
    --enable-gpl \
    --enable-libass \
    --enable-libdrm \
    --enable-libfreetype \
    --enable-libmp3lame \
    --enable-libopencore-amrnb \
    --enable-libopencore-amrwb \
    --enable-libopus \
    --enable-librtmp \
    --enable-libsnappy \
    --enable-libsoxr \
    --enable-libssh \
    --enable-libvorbis \
    --enable-libwebp \
    --enable-libx264 \
    --enable-libx265 \
    --enable-libxml2 \
    --enable-mmal \
    --enable-nonfree \
    --enable-omx \
    --enable-omx-rpi \
    --enable-version3 \
    --target-os=linux \
    --enable-pthreads \
    --enable-openssl \
    --enable-hardcoded-tables \
  && make -j$(nproc) \
  && sudo make install

Check the ffmpeg version

pi@srsenderpi:~ $ ffmpeg -version
ffmpeg version git-2020-12-22-a7f9b3b Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --extra-libs='-lpthread -lm -latomic' --arch=armel --enable-gmp --enable-gpl --enable-libass --enable-libdrm --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libssh --enable-libvorbis --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-mmal --enable-nonfree --enable-omx --enable-omx-rpi --enable-version3 --target-os=linux --enable-pthreads --enable-openssl --enable-hardcoded-tables
libavutil      56. 62.100 / 56. 62.100
libavcodec     58.115.102 / 58.115.102
libavformat    58. 65.100 / 58. 65.100
libavdevice    58. 11.103 / 58. 11.103
libavfilter     7. 94.100 /  7. 94.100
libswscale      5.  8.100 /  5.  8.100
libswresample   3.  8.100 /  3.  8.100
libpostproc    55.  8.100 / 55.  8.100

ffmpeg - filters shows tpad being available.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I'm pretty well there - doing some final testing now.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

I'll try this tomorrow (US Central time - GMT-6).

stuartofmt

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

Greetings!

I was hoping that simply replacing ffmpeg and the new DuetLapse.py would "just work," but sadly, not the case.

I confirmed that my previous ffmpeg didn't have support for the pad filter, as you suggested.

I rebuilt ffmpeg according to your instructions. That all went well.

I then tried to execute the DuetLapse script but you addition of -psutil- wasn't working, so I Googled how to build that, and did so.

I then went to run DuetLapse, but received the message:

Process is already running...

Probably because I have DuetLapse starting from -systemd-

I'll have to see what else is different.

John

stuartofmt

@JohnOCFII

If you are using my DuetLapse.py then it will prevent starting twice - so you need to make sure it's stopped before starting it again
kill -s <pid> if you are running in background else CTL+C works.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

If you are using my DuetLapse.py then it will prevent starting twice - so you need to make sure it's stopped before starting it again
kill -s <pid> if you are running in background else CTL+C works.

Heh - yeah, I might switch back to my DuetLapse, but take your ffmpeg command line. I have mine set to run and restart continuously, that way, I'll get a timelapse created every time a print starts.

John

stuartofmt

@JohnOCFII

If you are using systemd and it’s set up correctly then you should be fine with my version. All my code does on startup is look to see if there is an instance already running and if so - not create an additional instance (which would case “confusion”). Similarly systemd should not try to start a new instance if there is one already running (if you have it set up that way).
Bottom line, they should be compatible with each other.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

If you are using systemd and it’s set up correctly then you should be fine with my version. All my code does on startup is look to see if there is an instance already running and if so - not create an additional instance (which would case “confusion”). Similarly systemd should not try to start a new instance if there is one already running (if you have it set up that way).
Bottom line, they should be compatible with each other.

Yes, you are correct -- but I also have a little test script that attempts to launch an additional copy of DuetLapse that I use when testing new features. And it was attempting to launch THAT that caused my issues.

So, yes, I stuck with your version but did comment out the process related bits for testing.

stuartofmt

I’ll add in a switch so you can control with / without the check. May be useful ...

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

I’ll add in a switch so you can control with / without the check. May be useful ...

Lovely idea!

JohnOCFII

@stuartofmt My short test runs succeed, but longer runs fail. Looking at syslog, the output from ffmpeg is a bit hard to follow. I'm curious if you've tried any tests with more than 50 layers.

I've attached a trimmed section of my Raspberry Pi syslog in case you are interested.

There is one point (line 73) where it mentioned it can't find the file that (I think) it should be writing? Then perhaps it gets confused after that... Not sure if my use of a subdirectory is causing an issue there -- I'm taking advantage of one of your new flags.

DLVideos/DuetLapse1227201756.mp4: No such file or directory

This is the command line I'm using to launch DuetLapse:

/home/pi/DuetLapse/DuetLapse.py -duet railcore.localdomain -camera web -weburl http://octocore.localdomain:8081/snapshot -detect layer -basedir DLVideos -extratime 4

Thanks!

John
faillog.txt [0_1609120917817_faillog](Uploading 100%)

stuartofmt

@JohnOCFII

I'll take a look in the morning but -- the basedir should be an absolute path without the trailing /

e.g /home/pi/DLVideos

stuartofmt

Made some more changes:

Here https://github.com/stuartofmt/DuetLapse

Changed from print commands to logging option -logtype [console, file, both] default is both. If -logtype console then only written to the console. If -logtype file then a logfile is written to basedir with the name DuelLapse.log If -logtype both then written to the console and the logfile. Messages are prefixed by the ip address set by -duet. This is to distinguish between printers if multiple are used (see also -instances (below)

added a switch for inhibiting multiple instances -instances [single, oneip, many] default is single. If -instances single then the highlander principle applies. If -instances oneip then only one instance per duet ip address (set by -duet ip address). If -instances many then it's the wild west.

stuartofmt

@JohnOCFII

I took a look at the log - I think it's just the path. So try it with -basedir set as an full path i.e. with leading / and no trailing / as I suggested above.

Also - as a side note - I saw your systemctl restarting the script. I'm not entirely sure why it would not have worked with the single instance being enforced but I changed the exit(x) commands (apparently more applicable for interactive use) to sys.exit(x). The documentation suggests the latter is better form ...

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I took a look at the log - I think it's just the path. So try it with -basedir set as an full path i.e. with leading / and no trailing / as I suggested above.

Also - as a side note - I saw your systemctl restarting the script. I'm not entirely sure why it would not have worked with the single instance being enforced but I changed the exit(x) commands (apparently more applicable for interactive use) to sys.exit(x). The documentation suggests the latter is better form ...

I'll give the new version a shot. I'll change -basedir to /home/pi/DLVideos

Thanks!

stuartofmt

@JohnOCFII

I just tried with 264 images - no problems creating the Video file.