excessive avg disk queue length

[technokid]

When running Super Flexible File Sunchronizer I experience severe performance issues on the receiving server. While running SFFS I have the following specs
The receiving machine
W2K3 UDSS, 2GHz Quad Core Xeon, 4GB ram, RAID 6 (4, 1TB Drives)
CPU usage 3%
Phy Mem. 1.7GB
Avg disk queue length 20-30

The host machine
W2K8 WSS, 2.33GHz Quad Core Xeon, 16GB ram, RAID 6 (5, 1TB Drives)
CPU usage 3%
Phy Mem usage 4GB
Avg disk queue length <1

The host machine has 10 profiles setup. Each profile represents a root folder. Total data sync'd is around 350GB. I have one way real time replication setup. I like what it does but it seems to really hit the receiving machine.

[superflexible]

Hello,
is your profile really set up as a one-way sync? Are you only copying "left to right" and not "right to left"?

[technokid]

Yes, it is setup to copy left to right, exact copy (delete after 14 days), real time sync.

[superflexible]

Well then the program only copies the files. It does not do anything else that could disturb your machine. It looks like something else is actually to blame, such as a bad disk or security software working in the background. Maybe you need to specify a "minimum pause between actions" so that there is not a permanent operation.

I hope your hard disks are connected well, i.e. via short, direct SATA cables. Do not use USB.

In addition, real-time may not be appropriate for you. Maybe you need to switch to doing a regular schedule such as every X minutes or every 1 hour.

[technokid]

I would agree. Since it is a backup server I do not have any AV software on it and if I pause SFFS the disk queue goes down to <1.

Does SFFS read the destination directory(s) for any reason?

[superflexible]

Hello,

yes, the destination directories are read for comparison. You can turn this off by using the "Cache Destination File List" feature on the Special tab sheet.

[technokid]

How does it keep a record of changed files and deleted files?

Thanks for the reply
Bret

[superflexible]

In a database.

[technokid]

OK think I found the problem. One of the profiles holds client's PST files. I am assuming since the PST files changes every time an email comes in / out or is deleted the PST file needs to be written to the other side to maintian a mirror copy. Also I am assuming that since the PST file is a database file it is seen as a single file and is copied in its entirety. And since PST files can get large this could strain a systems resources copying multigigibyte files.

I am monitoring now and all profiles except the one where the PST files are located and the ave disk queue length is <1.

thanks for the help
Bret

[d033e22ae348aeb5660fc2140aec35850c4da997]

You may try partial file updating for the pst file. Then you will only copy pieces of it. Keep in mind this only works if you are not zipping the files before transfer. I have many clients doing this and it works well.