Custom gcode upload function in python - RRF HTTP server options

DuetUser

I try to write a simple python software to upload gcode to a machine in async manner.

I have it working synchronously by passing an iterator to request object, but I need async to send lots of files to different machines.

A file is divided into chunks and each chunk is an async task.
The problem is that subsequent chunks gets uploaded in separate POST requests as separate files, overwriting each other, so the uploaded file has last chunks size.

I've set the X-File-Id header but it does not seem to work.

So the question is is it at all possible to send one file in many POST requests to RRF?

T3P3Tony

@DuetUser this is one for @chrishamm for how the http API works.

chrishamm

@DuetUser Appending data to existing files isn't possible via the HTTP API AFAIK. If the payloads are really small, I suppose you could send echo "your line" >> yourFile as a G-code request to the Duet and/or use that command to append a line to the last chunk to start the next one via M32.

What exactly are you trying to achieve? Is it no option to write the file at once and to upload it to the Duet when it's ready?

DuetUser

@chrishamm There is and I successfully do it by passing an iterator as "data" to POST request, it feeds the request with chunks of data. Everything is fine but it is a blocking IO. I'm trying to make it async, ond one of the ways is to pass chunks sequentially in subsequent POST requests, only RRF treats each chunk as a separate file.
I have to dig into it more and perhaps make an async iterator and use it with async uploader, which I failed to do as of yet.

This is a working non async upload code:

import requests

class upload_in_chunks(object):
    def __init__(self, filename, chunksize=1 << 13):
        self.filename = filename
        self.chunksize = chunksize
        self.totalsize = os.path.getsize(filename)
        self.readsofar = 0

    def __iter__(self):
        with open(self.filename, 'rb') as file:
            while True:
                data = file.read(self.chunksize)
                if not data:
                    sys.stderr.write("\n")
                    break
                self.readsofar += len(data)
                percent = self.readsofar * 1e2 / self.totalsize
                sys.stderr.write("\r{percent:3.0f}%".format(percent=percent))
                yield data

    def __len__(self):
        return self.totalsize

def sendme():
    requests.get(baseurl+'/rr_connect?password=reprap')
    requests.post(url, data=upload_in_chunks(afile, chunksize=10))