Feature request: MD5 checksums for zip package parts

[syncing_feeling]

Greetings,

I have a feature request that is a further improvement on a recent improvement. Since version 5.10: "Interrupted ZIP Package uploads will now always be resumed the next time the profile is run - even if the ZIP Package is split. Previously this feature was only partially implemented." As a result of this improvement I am experimenting with handling some large files differently when backing up over WAN. The VPN service that I use will occasionally, briefly lose connection, so I am wanting to minimize the amount of data that might have to be retransmitted in the event that a connection issue corrupts the file transfer.

So I am using the following settings:
-Zip packages, within unzipping performed by the Remote Service.
-1 file per zip package; max file size of 4GB.
-Limit Zip File Size to 50MB.

The last setting (Limit Zip File Size) is the most relevant. By keeping the parts small I hope to have less to retransmit if an upload of a part fails for some reason.

In trying out these settings I am noticing that it does not appear that the individual zip package parts have MD5 checksums computed for them. Further, it appears that once a zip package part (eg, .z01, .z02) has been uploaded it is deleted from the Temp directory on the local source computer. My concern with this approach is that a individual part could be corrupted, but if there is no checking of the integrity of each part then in the event one is corrupted they will all have to be resent (because it appears that the Remote Service will only discover that a part was corrupted when it tries to assemble all of the parts and the assembly fails or the assembled zip package fails its MD5 checksum check).

If each zip package part (eg, .z01, .z02, etc) could be verified (ie, by MD5 checksum) to be correct (after it arrives and before the source part file is deleted) then I can more easily transfer large files (1GB+) by having them split up into smaller chunks so that any time there is a connection issue SFFS only has to retransmit a small chunk (ie, a 50MB zip package part).

I hope that the above makes sense.

As always many thanks in advance for your consideration!

[superflexible]

Zip files always have a checksum, not MD5, but sufficient to verify integrity. If the zip file is corrupted, the Remote Service will not unpack it and will rename it from .zip to .err. The next time you run the sync, the file will be uploaded again.

[syncing_feeling]

Zip files always have a checksum, not MD5, but sufficient to verify integrity. If the zip file is corrupted, the Remote Service will not unpack it and will rename it from .zip to .err. The next time you run the sync, the file will be uploaded again.

Okay, right. I was confused about the nature of the checksumming for zip files. That is fine that it is not MD5.

However, is there any checksum for zip package PARTS? For example, with these large files being put into a zip package I have the following files that then get assembled into the final zip package at the destination:
SFFS ZIP Package 7344.536870914-2011-03-12 07.16.20.Z01
SFFS ZIP Package 7344.536870914-2011-03-12 07.16.20.Z02
SFFS ZIP Package 7344.536870914-2011-03-12 07.16.20.Z03
.....
SFFS ZIP Package 7344.536870914-2011-03-12 07.16.20.zip

Do each of the individual parts (eg, .z01, .z02, .z03, etc. up to the final .zip) have a checksum for each part? If they do not, then if one part gets corrupted won't SFFS have to reupload ALL the parts again (which is what I'm trying to avoid)?

The way I want it to work is that if a single part (ie, any .z0#, and/or the final .zip) gets corrupted in transfer that SFFS be able to detect that and just re-send THAT single part instead of having to re-send ALL the parts. Because I am working with large files (1GB+) it is not very efficient to have to re-send all that data.

I hope the above makes more sense.

[superflexible]

It will have to re-upload all parts. But that should be extremely rare. I am not currently planning any feature extension in this area, but I will consider your request for a future update, maybe not so soon but I will file it so it won't be forgotten.

[syncing_feeling]

It will have to re-upload all parts. But that should be extremely rare.

Actually, I've just now had a problem 2x where two different zip packages failed (both on the final file) and had to be uploaded twice, but this recent problem may be different than what I've been talking about (I'm not sure yet). I'll e-mail you with details since this is not the place for potential bug reports.

I am not currently planning any feature extension in this area, but I will consider your request for a future update, maybe not so soon but I will file it so it won't be forgotten.

Thanks for putting the ideas on your list. I understand and respect that you have to prioritize feature improvements. I appreciate your willingness to consider something for the future.