Host Engineering Forum

General Category => Do-more CPUs and Do-more Designer Software => Topic started by: Andrew S on July 29, 2019, 03:21:44 PM

Title: Processing larger binary files
Post by: Andrew S on July 29, 2019, 03:21:44 PM
I need to download a binary file to the SDCard. I'm using MQTT so my thought is to pull it via and MQTT SUBSCRIBE and write it into a byte array. My understanding is that byte arrays can't be larger than 56,000 bytes. This file may be larger than that - is my only option to create multiple byte arrays and then count received bits to push them into multiple byte arrays?

I was hoping there was a more elegant solution - does anyone have any better ideas about how to receive larger binary files so that they can be written to an SDCard file?

Thanks, Andrew.
Title: Re: Processing larger binary files
Post by: BobO on July 29, 2019, 03:45:31 PM
I need to download a binary file to the SDCard. I'm using MQTT so my thought is to pull it via and MQTT SUBSCRIBE and write it into a byte array. My understanding is that byte arrays can't be larger than 56,000 bytes. This file may be larger than that - is my only option to create multiple byte arrays and then count received bits to push them into multiple byte arrays?

I was hoping there was a more elegant solution - does anyone have any better ideas about how to receive larger binary files so that they can be written to an SDCard file?

Thanks, Andrew.

Files can be written progressively. No need to store the entire file in memory before streaming to the disk.
Title: Re: Processing larger binary files
Post by: Andrew S on July 29, 2019, 04:18:54 PM
Marvelous. Thanks Bob.