Performance of Synthetic Backup on SSD

[probspot3]

Hi,

I've recently upgraded to NVMe SSDs, hoping that this would vastly improve the backup performance of large files as well.

My setup is:

- Skylake Core i7 mobile CPU (4 physical cores and 8 hyper-threaded ones)
- 16 GB RAM
- 2 x Samsung 960 EVO NVMe SSD (one contains source data, the other one is the backup medium)
- Windows 10 and Syncovery 7.85h 64bit

My scenario is:

- Synthetic backup of a 60GB-large VM disk from one SSD to another one


After the initial backup, only block-level increments are written as expected. What I've noticed is that while reading the files (original and the backup one) in order to compare them and generate the delta, Syncovery does the reading (?) at approx. 160 MB/s (as shown by Windows resource monitor) whereas the SSD is capable of ~3 GB/s sequential read speed. Looking at the CPU it is loaded at merely 16% (no single core maxed out according to Windows 10 resource monitor, e.g. all of them are loaded at approx. 16%).

Now, yes, these SSDs are not very fast when writing. But my tests involved just starting the VM and shutting it down again in order to generate a small block-level difference between two disks. E.g. the total delta written by Syncovery after this test is approx. 5MB.

My question is what would be the bottleneck here. I was expecting the two 60GB files to be compared _very_ quickly.

What exactly is happening during the file comparison phase during synthetic backup? What kind of reading is involved (sequential?) and what kind of writing? Is the comparison multi-threaded (e.g. multiple chunks compared in parallel)?

Thanks

[probspot3]

Update: I played a bit with the option "Zip directly to destination". If I turn it off, then during the initial sync after the zipping is completed and the resulting file is copied to the destination, the copying is done with the native SSD write speed. Zipping itself is as slow as if I'd turn the option back on, therefore my current suspicion is that Syncovery is choking on its Zip performance, which is required for Synthetic backup. The selected compression level is "none".

I wasn't noticing this issue before because before the upgrade to SSDs I was backing up to an SD card with maybe 10 MB/s.

[probspot3]

Update 2: after disabling one feature after another I was able to narrow down the problem further.

Below is an overview of different read speeds of the source file with different features enabled (sorted descending):

- Native Windows copying with Windows explorer: ~3 GB/s read speed
- Syncovery with disabled partial update, disabled Zipping and disabled versioning (e.g. a simple copy): ~3 GB/s (native SSD speed)
- Syncovery with partial updating only (second sync): ~300 MB/s
- Syncovery with partial updating only (first sync): ~250 MB/s
- Syncovery with Synthetic backup (any run): ~160-180 MB/s

Enabling chunk hashing (e.g. partial update) introduces a major hit in performance. Interestingly, I don't see the CPU maxing out (it is at around 20% with balanced load across all cores).

Is only MD5 used for partial update? Would it be beneficial to introduce support for newer (configurable) hashing algorithms (possibly hardware-accelerated ones)? Intel seems to be hardware-accelerating SHA1 and SHA256 since the Goldmont architecture (https://en.wikipedia.org/wiki/Intel_SHA_extensions). For older platforms the GPU could possibly help out if available.

Just an example I found by googling: https://cyan4973.github.io/xxHash/

[superflexible]

MD5 calculation is slow. Thanks for the suggestion to look at other hash algorithms!

[probspot3]

I've just noticed that version 8 of Syncovery is upcoming with new features.

Are you planning to include faster hashing algorithms in some iteration of 8.xx?

Thanks

[superflexible]

Too late now for 8.00. It is feature complete.

The biggest feature is being able to compress and encrypt and much faster speeds and upload to a cloud server without the need of temporary files.

All this partial file updating is more geared at transfers over the Internet, not local copying. So the speed is fine.

Maybe I can do it in some update after v8.0.