Performance with RDX

[strophy]

Hi,

I am testing Syncovery 7 beta 16 (x64) with a Tandberg Data QuikStor RDX device. It has 8 slots for loading RDX cartridges, and I have access to 3 of them over iSCSI, so they just appear as a removable disk in Windows. Copy performance is around 35MB/s burst, 16MB/s sustained. I guess it has a big cache.

I am trying to use Syncovery to run 3 parallel backup jobs for 3 Thecus N7700 NAS devices on the "left-hand" side, accessed over a CIFS network. I love the inclusion and exclusion settings and the ability to run jobs at the same time, nothing else I have tested offers this. Unfortunately I am running into some performance issues.

The network copy performance from NAS to RDX for a single device is the same as stated above, but as soon as I run a job in Syncovery, the maximum speed drops to around 12MB/s. If I am running 3 parallel backup jobs, the maximum speed is still around 12MB/s, so ~4MB/s per job. This is too slow, I am trying to backup around 8TB of data from each NAS.

I have noticed in task manager that of the four cores in my 2.4GHz Intel i5 machine, one of them is at 100% during the job while the others are mostly idle. The system process occupying this core is "System", i.e. the NT kernel. Could the processor be limiting the backup performance?

Final question before this gets too long: each cartridge is limited to 2TB, but like I said each backup set is around 8TB. What happens when the cartridge is full? In traditional backup software it would normally ask for a new cartridge, is this true in Syncovery as well? Is there a feature like incremental backup so next time I don't need to copy the full set over the network again?

Thanks for this really innovative product, I look forward to purchasing soon!

Leon

[superflexible]

Copying does not normally take much CPU. If Syncovery had anything to do with it, the CPU load would be in the Syncovery process. Something else on your system must be causing it, for example anti-virus software, or the Windows file buffering etc.

What you can do in Syncovery is turn on or off the "Windows API Copying Function". The setting is in the profile under Files->More.

When the Windows API Copying function is off, you can also try "Bypass File Buffering by Windows", which is on the Files tab sheet. Finally you can specify a number of files to copy in parallel for each job, also on the Files tab sheet.

Syncovery cannot normally split backup sets over several volumes. Especially because updating the backup copy is normally done by synchronization, i.e. updating in place and copying only new or changed files to the same backup destination. It would be better if you could create your backup jobs in such a way that every part fits on the destination.

However if you run a job manually, you can confirm to continue anyway even if the destination space is insufficient. When the drive is full, it asks you what you want to do, and you can "insert a new disk". But it has to have the same drive letter, so not sure if that is possible.

[strophy]

Thanks for those performance tips, it did indeed turn out to be a combination of the problems you mentioned. I installed on an older P4 server with a clean version of Windows, changed some of the performance settings to suit the NAS and I can now reach 41MB/s sustained to 3 slots, very close to the theoretical maximum. The processor thrashing issue does not occur even on this old machine, it is steady around 30% on two cores.

The solution for spanning a ~8TB backup set over several drives is also fine for us. Because Syncovery writes plain files instead of filling the media with cryptic backup files, we can even reuse partially filled media for other backup jobs, since each job will be a full copy. The drive letters do not change in the QuikStor setup we have here.

Final question: when purchasing the Single Pro user for Windows, are we able to directly use the registration information with the 64-bit version 7 beta? I find the performance significantly better in this version for our setup. Will the license be usable when the final version 7 is released?

Thanks,
Leon

[superflexible]

Hello,

yes, license codes which are ordered now work with versions 6 and 7.

Cheers,
Tobias

[strophy]

Thanks Tobias,

now that the performance issues are solved, I have stepped up from testing to trying to run a full backup. I see now that you chose your words very carefully before - there is only a prompt to insert a new disk if you run the job manually.

Unfortunately, in order to get the performance gains offered by the QuikStation RDX, I need to run 3-8 jobs together using the "Run Profiles in Background And In Parallel" option. This is the only way to write to multiple slots simultaneously. Is there a way to run jobs in parallel but still get the job to wait for a new disk when the first one is full? If I manually re-run an aborted background job with the full disk already inserted, it will display the prompt immediately, but I don't think it recognises that the disk is already full of files that are part of the job, so it will write the same files again when I insert a new disk. Is there a solution to this situation? Would be great, because so far Syncovery is the ONLY solution that offers good performance with the QuikStation...

Thanks,
Leon

[superflexible]

You can open several instances of the software. Just click on the icon several times and each Syncovery instance can run a different job. In recent Windows versions, opening another instance may require right-clicking the icon (depending on which icon).

If you sync again to a full disk, yes it will see the files which are there and will only attempt to copy additional files. So that should work if you need to finish a job that has previously filled up one destination drive. It won't work if it has filled up two destination drives already and you need to write to a third one.

It might be helpful to mark backed up files by resetting their Archive Flag. That way, you could continue a job later on and even point it to a new destination because it remembers which files have been backed up already.

There are two archive flag related checkmarks on the General Filters tab sheet. One is called "Reset Archive Flags" and is used to mark copied files. The other one is called "Copy Only Files With Archive Flags" and can be used to take the flags into account when determining which files to copy.

Typically, the initial backup uses only "Reset Archive Flags" while additional (incremental) backups use both Archive flag checkmarks.