Good practices to be memory efficient in firmware?
-
I am trying to collect the "NextStepTime" data associated with a move (say G0 X5) and "bin" them with a sample size/bin range of 10,000 i.e. if I have data such as
Array_NextStepTime = 6629<LF> Array_NextStepTime = 9375<LF> Array_NextStepTime = 11481<LF> Array_NextStepTime = 13258<LF> Array_NextStepTime = 14823<LF> Array_NextStepTime = 16237<LF> Array_NextStepTime = 17539<LF> Array_NextStepTime = 18750<LF> Array_NextStepTime = 19887<LF> Array_NextStepTime = 20963<LF> Array_NextStepTime = 21986<LF> Array_NextStepTime = 22963<LF> Array_NextStepTime = 23901<LF> Array_NextStepTime = 24803<LF> Array_NextStepTime = 25706<LF> Array_NextStepTime = 26644<LF> Array_NextStepTime = 27621<LF> Array_NextStepTime = 28644<LF> Array_NextStepTime = 29720<LF> Array_NextStepTime = 30857<LF> Array_NextStepTime = 32068<LF> Array_NextStepTime = 33370<LF> Array_NextStepTime = 34784<LF> Array_NextStepTime = 36349<LF> Array_NextStepTime = 38126<LF> Array_NextStepTime = 40232<LF> Array_NextStepTime = 42978<LF> Array_NextStepTime = 49596<LF>The no of bins would be 5, each having the following size :
BINS = 4<LF> BINS = 14<LF> BINS = 20<LF> BINS = 12<LF> BINS = 6<LF>so I define an array
BIN[]of arbitrary size but sufficiently large to accommodate the number of bins. The code I have written works fine for small moves such asG0 X10. but I run out of memory (?) if I define aBIN[100]and try to work on larger moves. I kind of worked-around the problem by increasing the sample size from 10000 to 60000 but I would still like to know if there is any way I can sample at 10000? What are some good practices that I should follow to reduce memory consumption? -
@jazbaatbadalgaye I didn't understand your BIN[] idea and how you want to program it in C++, but general ideas:
- analyze the biggest numbers and store in number formats as small as possible (8 bit, 16 bit etc)
- instead of storing absolute values, you could store the difference between the numbers, this may help to store in smaller space
- compression in memory may be a possibility by looking for a library which needs little memory itself
- M122 tells you how much memory is available. In the RRF source directory "src\Memory usage.ods" is a spreadsheet which shows how much some memory is used currently and where you could spare memory if you need it.
-
@joergs5 So basically what I am doing is this inside the step driver function:
if (dm != nullptr) { int index=0; index=(dm->nextStepTime/10000); dm->bin[index]++; }where sampling is done in intervals of 10000s and initialization is done as
bin[20]={0}. This works fine for small moves such as G0 X5 but not for larger moves. The maximum size of bin that I can work on isbin[40]but that too doesn't work for larger moves such as G0 X100. -
@jazbaatbadalgaye for me to understand, please tell me how you declared bin[]
-
@jazbaatbadalgaye said in Good practices to be memory efficient in firmware?:
maximum size of bin that I can work on is bin[40] but that too doesn't work for larger moves such as G0 X100.
bin[40] can only store values up to 399999, because 399999/10000 = 39 (int division cuts the rest), from nextStepTime higher values you'll get an overflow.
To avoid, check for maximum value and if it's too high, set it to an allowed value or ignore:
ignore:
index=(dm->nextStepTime/10000);
if(index <= 39) {
dm->bin[index]++;
}or add to maximum value:
index=(dm->nextStepTime/10000);
if(index > 39) {
index = 39;
}
dm->bin[index]++;Depending on the distribution of the step timers, it may be good to set a higher divider also.
It's strange that you cannot store a few variables, please tell me the declaration and you can make a M122 and publish to look at it.
-
@joergs5 Exactly! That is why it works when you increase the sampling to 60000 which prevents overflow.
I declared bin as
unint_32 bin[40]={0}; -
@jazbaatbadalgaye said in Good practices to be memory efficient in firmware?:
I declared bin as unint_32 bin[40]={0};
I understand. If you example above is real-world, you have not many steps in a 10000 interval. The code just counts and stores the steps, so you could declare uint8_t and get 4 times the array size to bin[160]. (8 bits with maximum 255 values stored). *)
The values from 10000 to 20000 is e.g. about 10 times, which means to encode you need 4 bits. This would allow splitting a byte into two sections, but this is "only" a doubling compared to unit8_t. This is probably not worth the effort. (and 10 to 20T may be acceleration phase, with full speed you have more steps).
*) there may be a trap, because some processors store variables at 16 bit boundaries. It may be better to store them in a char array.
-
@jazbaatbadalgaye it seems that you want to correct an histogram with frequencies of time for intervals. If memory is an issue can you decrease tree number of bonds but using exponential bin sizes? E.g each bin is 1.1x the size of the previous one. Later you can normalize the count of each bin by its size.
Another option is to track each time a small range of bins. If you run it multiple times on the same it input you will get all the bins you want.
My 2c
-
This post is deleted! -
@jazbaatbadalgaye I think the clue is in this line you posted above:
dm->bin[index]++;assuming "dm" above is a DriveMovement object, that means the "bin" array is also defined within the DriveMovement class. RRF creates a whole pool of DM objects initially, and looking at Move.h, this is going to be 83 or 125 depending on which board you are using.
So on a duet2, with a "bin" array size of 40, this would end up requiring an additional 13,280 bytes
-
@joergs5 I decided to take your advice by using 8bit int and now I am able to print for larger moves (G0 X50) but still running out of memory for realistic moves (e.g. G0 X120)
-
@jazbaatbadalgaye @sdavi 's comment is probably correct. You can set the array static so it's created only once. IMHO static is not good development practice in most cases, but for temporary debug/analyze I think it's ok. An alternative is to declare the variable in a singleton class and define setter/gettter functions. Platform should be one.
-
@sdavi Hey how did you get 13,280? I am getting 26,560 which is 2*13,280.
-
@jazbaatbadalgaye the uint32_t are 4 bytes, the array 4*40 = 160, 160 * 83 = 13280.
-
@jazbaatbadalgaye if you intentionally create the arrays for every DriveMovement instance (because you want eg know exactly whats going on in every instance), you can optimize the memory usage by:
at the moment when the DM instances are created, create your array with varying sizes. Depending on the associated drive numbers (your axes), you'll need different sizes, e.g. for axes you don't use, you'll need just a dummy array.
You can start with small arrays for every instance:
- create an array size variable in DM for every instance stepsize=1000000; and maxindex=5; and create the arrays with maxindex. The arrays need to hold bigger numbers, so you should use uint32_t again
- use code and check that index is always lower than maxindex:
index=(dm->nextStepTime/stepsize);
if(index < maxindex) { dm->bin[index]++; } - then check which instance uses a lot of steps. For those instances, you can set stepsize smaller and maxindex higher to get a more detailed view for specific instances. Instances of drives which you use will contain data, the other instances not. I don't know how it's programmed: useddrives-gap-usedextruders-gap-auxdda, or useddrives-usedextruders-gap-auxdata, but you'll see how the instances are filled.
An alternative to this is to store all values in a single array (in a Singleton class like Platform) and store additionally the information from which DM instance it is in this array. The total required space is much lower, because some DMs don't send data. To identify the DM instance, just store a little additional id variable in the DM instance at initialization date (I think an instance id is a good idea for the first method also).
