duetbackup - CLI tool to backup your Duet SD Card [deprecated]

[wilriker]

@gtj0 Actually you discovered a bug in RRF rather than in my tool.

When RRF sends a ZIP file it will add

Content-Encoding: gzip

to the response header which simply is wrong (except it is a renamed gzip compressed file ). I will though see if I can work around that.

It won't be hard to create an uploader. Will sit down early next week and throw something together.

[dc42]

There is no reason that I can see for having any .zip files on the SD card. I guess they were either copied to the card manually on a PC, or uploaded some time ago before DWC was clever enough to unzip them and upload the contained files.

[wilriker]

@dc42 Still if the file is there Content-Encoding: gzip is wrong.

[wilriker]

Release 1.0.1

I just released v1.0.1 that works around the issue described by @gtj0, i.e. zip files on the SD card provoking an error.

[Dougal1957]

is the Darwin binary supposed to work on the latest MAC os when I download it it comes as a .dms file which my mac seems to object to? (it seems to think it is a document)

[dc42]

@wilriker said in duetbackup - CLI tool to backup your Duet SD Card:

@dc42 Still if the file is there Content-Encoding: gzip is wrong.

I will look into fixing this in the 2.03 release.

[Phaedrux]

@dougal1957 said in duetbackup - CLI tool to backup your Duet SD Card:

is the Darwin binary supposed to work on the latest MAC os when I download it it comes as a .dms file which my mac seems to object to? (it seems to think it is a document)

Same here.

[Dougal1957]

@phaedrux Thank God I'm not the only one I am not that good with the intricacies as yet of the Mac and not had much cause to tbh

[wilriker]

@dougal1957 @Phaedrux yes, the Darwin binary is supposed to work on MacOS 10.10 and later.

I don't know what you tried but for clarification: this is not an installer of any kind. It's the final binary and it's a pure command line tool with no GUI.

So after download open a shell and navigate to the directory the file sits in. Use

chmod +x duetbackup-darwin_amd64

to make it executable and then issue

./duetbackup-darwin_amd64

You should now get an error about missing parameters - but that also means it works.

P.S.: I'm on my phone and the filename was from memory so it might be a tiny bit different, please check that.

[Phaedrux]

@wilriker that makes more sense. I'll try again tomorrow.

[wilriker]

@phaedrux let me know if it works this way.

Also I will adjust packaging in the coming days so that a) the filename is the same fur every platform and b) it is already executable for Linux and MacOS. Right now on Windows you would also have to add .exe to the filename to achieve that - I should not do software releases when in a hurry.

[mudcruzr]

Under windows 10 I run:

.\duetbackup.exe -domain "192.168.0.53" -outDir ".\bkup"

(Tried it with & without the quotes and using ./bkup and .\bkup all with same error.)

and it errors with the following:
2019/06/01 20:15:32 Fetching filelist for 0:/sys
panic: time: missing Location in call to Time.In

The whole error message is here 0_1559416600853_Duetbackup_errors.txt

[wilriker]

@mudcruzr Thanks for reporting. I'll have a look into that on Monday. This part of the code is rather sketchy and even marked with a // FIXME comment already. So I guess it's time to follow my own advices.

[wilriker]

Release 1.0.2

I just released version v1.0.2.

• This release should fix the time zone issue @mudcruzr described.
• Also starting from this release the (callable) filename for all target platforms now is duetbackup - for this the files are package as either tar.gz or zip depending on target

[Dougal1957]

OK still not sure about this on the mac

I tried the following with the latest 1.02 executable and get the following

I suspect I need to pass some arguments to it but not sure how? any help appreciated

Doug

[wilriker]

@dougal1957 Yes, there are mandatory parameters. The names are given in the error message.

To find out all available parameters type

./duetbackup --help

The minimum valid command would look like

./duetbackup -domain "<your_duets_network_address>" -outDir <local_directory_to_save_files>

This will then sync all files in the SD card's /sys (default if -dirToBackup is not provided) directory to the specified directory on your hard drive.

P.S.: For applications implemented in Go it does not matter if you prefix parameters with one or two dashes, i.e. -domain and --domain are treated identically.

[mudcruzr]

@wilriker said in duetbackup - CLI tool to backup your Duet SD Card:

Release 1.0.2

I just released version v1.0.2.

• This release should fix the time zone issue @mudcruzr described.
• Also starting from this release the (callable) filename for all target platforms now is duetbackup - for this the files are package as either tar.gz or zip depending on target

Works for me, many thanks! (Windows 10)

I have a suggestion (only semi-seriously), if it's not too cheeky: an exclude switch, e.g.

./duetbackup -dirToBackup "0:/" -excludeDir "0:/gcodes" -domain "192.168.0.53" -outDir "./bkup"

(I didn't realise how many files I had accumulated in my gcodes directory, lol)

[wilriker]

@mudcruzr said in duetbackup - CLI tool to backup your Duet SD Card:

I have a suggestion (only semi-seriously), if it's not too cheeky: an exclude switch, e.g.

./duetbackup -dirToBackup "0:/" -excludeDir "0:/gcodes" -domain "192.168.0.53" -outDir "./bkup"

(I didn't realise how many files I had accumulated in my gcodes directory, lol)

I already have thought of that. I will see if I incorporate that - but on the other hand on successive calls it will only download new or changed files. GCodes files change rather seldomly so most of them won't get downloaded again.

[wilriker]

Release 1.1.0

I just released v1.1.0 and I think this will make @mudcruzr quite happy - as well as I hope that other users will also be happy with the new feature. Release notes below.

Exclude Feature

A new flag -exclude has been introduced to exclude remote paths starting with the given value. This flag can be used multiple times.

Simple Example

./duetbackup -domain duet.localdomain -outDir "sys" -exclude "0:/sys/c"

will skip all files and folders in /sys that start with "c" like config.g or cancel.g.

Multi Example

./duetbackup -domain duet.localdomain -outDir "sys" -exclude "0:/sys/c" -exclude "0:/sys/home

will skip all files and folders in /sys that start with "c" like config.g or cancel.g as well as all files starting with "home" like homex.g, homey.g, homez.g and homeall.g.

Example Full Backup excluding GCode Files

./duetbackup -domain duet.localdomain -outDir "backup" -dirToBackup "0:/" -exclude "0:/gcodes"

Other Changes

• Some internal code simplifications
• Logging has been reformatted and reduced - the former amount of logging can be restored with the new flag -verbose

[wilriker]

Release 1.1.1

I had to release v.1.1.1 to fix a bug where the local directory has to exist unless the corresponding remote directory has a sub-directory.

Now all missing local directories are being created independently of the contents of the remote directory.

Also the order of file handling has been adjusted to first handle all files in the remote directory before traversing into sub-directories. This way the order of log statements makes much more sense.