Synchronizing folders on multiple sites

[Ivangelion]

Hello,

I'm planning to synchronize two folders on more than two sites, maybe the number will grow to 5 or more. There will only be inserting files on each site and no updating nor deleting.

What method would you recommend? How about the circular method? Namely, SiteA to SiteB to SiteC to SiteD to SiteE to SiteA, or each site synchronizes 2 other sites, i.e. SiteA to SiteB & SiteC, SiteB to SiteC & SiteD, etc. in case one of the sites fails.

Star method or server-clients method might be infeasible, because when the number of sites grows too many, the topology will break.

Thank you in advance.

[superflexible]

It depends mainly on the Internet bandwidth on each site and how quick you need to files to arrive at all the other sites. You could do it in a circular fashion. I wouldn't recommend the "each site synchronizes 2 other sites" method because then the same file might be uploaded to the same site twice at the same time and that can cause problems.

There are definitely companies that use a central server and do everything from there, some run up to 100 profiles at the same time and it works. It all depends on the bandwidth.

[Ivangelion]

Thank you. But what if one server is down in this circular fashion? To avoid the sync failure, must each site in the circular topology sync with neighbor in a two-way clockwise (or counter-clockwise) direction?

[superflexible]

Yes I think a two-way sync should be a good solution.

[Ivangelion]

Does this mean that each site should sync with both of its two neighbors in bi-directional way in the circular method? Because if one site only syncs with its left or right neighbor and this syncing partner is down, the changing server cannot propagate its changes to the other on-line servers.

Also, if the same sync folder is set up on both servers, will there be any problem like disk/process contention or copy issue?

Third, if there are 5 servers set up using this circular method, and there are 500 small files (about several KBs) added to the server1, is there any problem w/ regard to server3? There will be updates both from server2 and server4 on server3. With this scenario I could see there are "Creating shadows (###s)" in "Next Run" in some of the syncing profiles.

[superflexible]

It's difficult. I don't have much experience with this setup because customers normally synchronize various servers or workstations with a central/main server. You may want to consider that. The central storage could be on the Internet, such as Amazon S3 or Microsoft Azure (but preferably not Google Docs).

You should turn off using the volume shadow copy on the File Access tab sheet unless you really need it. That way, locked files will be skipped until the next run.

An additional way to avoid copying conflicts is to use the checkmark "Auto-Resume" on the Files tab sheet. It will use interim filenames with .incomplete added until the copy is complete.

If you stick to the circular method, the scheduling can be tweaked in such a way that files don't propagate from server 1 to server 3 at the same time via 2 and 4. If using real-time sync, you could choose a different "Minimum pause between actions" on each server to help make it asymmetric. But again I don't have any experience with that.