Real Time Sync vs Schedule Sync

[RadicalDad]

We are processing a two-way sync against a NAS box. I've observed the following behavior.

When using Real Time Sync, the NAS box properly sleeps. Even when scheduled syncs come up, they don't seem to wake up the NAS if Real Time Sync is in use. However, temp files are improperly replicated to both sides of a sync when using Real Time Sync. Even when (properly) deleted by the app which created them, the temp file gets re-created by the sync. This is not desirable nor correct behavior.

So then we tried turning off Real Time Sync and using a scheduled sync. Unfortunately, in this scenario the scheduled sync wakes up the NAS drives. This is undesirable behavior.

If I schedule the syncs at long intervals in order to let the NAS hibernate properly, then I need to be sure at the end of the day that a sync is done at shutdown. However, this has proved to be a serious problem. The shutdown scheduler interferes with normal shutdown in an undesirable way. Numerous error messages are generated. Further, the shutdown sync causes the Windows dirty bit to be set. This results in a chkdsk being run at every boot. That is extremely undesirable.

Is there a way that one (or more) of these issues could be resolved quickly? And what is the difference between real time and scheduled sync such that real time sync properly lets the do its "green" thing and hibernate?

Thanks.

[superflexible]

Hello,

are you sure that two-way real-time work with the NAS? Does the program get a notification when a file is changed on the NAS? If so, you are lucky.

A scheduled sync does a full scan of all the folders, so that should certainly wake up the NAS. How come your NAS can sleep so much? Don't people add files all the time? Personally I think a NAS should never sleep.

Something that could be improved is if you manage to exclude the temp files. Temp files must be excluded because due to their short-livedness, they cannot be processed properly in real time.

Make sure that on the Real-Time Settings dialog, you have chosen the bottom checkmark (exclude temp files). If this does not exclude all the temp files, try to exclude them with file masks.

I have not ever heard about the kind of shutdown problems you are hinting at. You would have to post more details so I can look into it.

[RadicalDad]

Real time sync (RTS) does indeed work with the NAS - a Synology DS211. I've tested by modifying a file on the NAS side from a different computer than the one doing the sync. RTS picks up the change in 2 to 3 seconds. The problems with RTS don't have much to do with MS Word temp files. The little bit I've tested indicates they stick around long enough to sync and be deleted when finished. The problem is with very short lived temp files. Early on in the testing I noticed a log entry concerning a temp file with a name I'd not seen before, so I can't even say what process creates it. The file was created and then deleted so fast that RTS didn't get a chance to copy it to the NAS, and that generated an error message as a result. Since I don't understand the naming convention for that file, its impossible to generate an exclusion filter. Other than having to deal with the error messages, this isn't a big deal. However, the biggest problem I have is creating new folders. If I create a new folder, then immediately rename it as part of the creation process, I wind up with two folders, one called "New Folder" and the other with the name I created. These sync to both sides. I've watched RTS copy "New Folder" to one side, then ignore the obvious delete when "New Folder" gets renamed. Instead, I've watched "New Folder" get copied back to the other side. That really should not happen. This error occurs on the PC side, not the NAS side, so slow notifications shouldn't be an issue. Filtering on "New Folder" isn't an option as I use a digital recorder which creates "New Folder" folders by default.

As for NAS hibernation, our organization is small, and one of the NAS boxes is in my home. We are trying to do double duty with the NAS - both backup and making files available to the whole organization. That said, it isn't often that a file will be needed off the NAS. Hibernation is appropriate, not to mention environmentally responsible.

As for the sync at shutdown issue, it has been quite consistent on a Windows XP machine. Seems like it wouldn't be difficult to replicate. The problem is that Windows wants to shut down before the sync has done its job. I get the Windows "program is not stopping" dialog box, which sits in front of a dialog box from Super Flexible asking if I really want to sync. (No, I was kidding about that in the sync profile. Sheesh.) Eventually Super Flexible decides to sync anyway, but not before the rest of Windows is mostly gone, waiting for the sync to complete. When the sync is complete, it puts up its own dialog box, asking if I really want to shut down now. Haven't tried it, but I wonder what would happen if I said no, since most Windows processes are gone by this point. Meantime there is a 300 second (5 minute!) countdown timer on the Yes answer. All this is just foolishness. I've already said I want to shut down once. Why do I have to say so again and again? This is a pain in the butt for sophisticated users, and totally unacceptable for non-technical users. Meantime, a disk monitoring process of Windows must already be gone because the dirty bit is set, resulting in a CHKDSK at every boot. Eliminating the sync at shutdown eliminates the dirty-bit-CHKDSK-at-boot problems, along with all the others I have described.

Super Flexible is a pretty good program. In testing, I've tried to make it fail, and except for the temp file problem, can't make it screw up. However, ONE of the two problems I describe (either with RTS or shutdown sync) needs to be fixed in order for this to be viable in a non-technical environment.

[superflexible]

Concerning the "New Folder" problem, I think you have to go to the Real-Time Sync dialog and make sure that deletions are enabled and set the delay to 0. If problems remain, please send log files to support@superflexible.com.

None of this shutdown "foolishness" will happen on Windows Vista or higher, because finally Microsoft added APIs to properly support shut-down jobs.

On XP, the sync upon shutdown has to use some tricks, so you will see some of these dialog boxes. Except it's supposed to automatically start the sync fast enough so that the "program is not stopping" dialog should not appear. It appears after 5 seconds and by that time, Superflexible should already have canceled the shutdown and be syncing. Maybe on some machines the 5 seconds are too long and I should make it 4.

You can change the shutdown wait time on the Program Settings dialog.

Maybe it would be a better idea to create a shortcut with a job that invokes the shutdown after syncing. So you would not invoke a Windows shutdown yourself, which Superflexible needs to intercept. You can create such a shortcut with the following command line:

C:\Program Files\SuperFlexibleSynchronizer.exe /RUNX="Profile Name" /U /SHUTDOWN

Then you would just double-click that link rather than whatever you used to do for invoking a shutdown.

[superflexible]

By the way, you can entirely avoid the "New Folder" problem by telling people they have to finish typing the name within 5 seconds .

[superflexible]

I have changed the delay on Windows XP to 4 seconds. Hopefully this will resolve the dirty bit issue too. If not, you should watch it and see if the shutdown eventually works as normal. I've never heard about the dirty bit issue before. The 4 seconds change will be in the next update (i.e. after version 5.36a).

[superflexible]

The new version 5.36b is now available, hopefully improving the shutdown experience on Windows XP slightly.

[RadicalDad]

Thanks so much for all your help. I really admire your dedication to the product and to tech support. I will try the latest version, and also send you a log file showing what happens with the "new folder" temp file issue while the delay is set to 0. If you'd like to tell me how you would like logging options set (and where to set them) that will save me a little time looking for those options.

I've been buried in another project for the last 10 days. I'll try to get back on this today, but it might be Monday before further testing happens. If you send me what you want for log file options in the meantime, that will be helpful.

Thanks

[RadicalDad]

I have uploaded three logs showing the problem creating a new folder with Real Time Sync (RTS). The first log shows the folder is missing before SuperFlexible can copy it, the next two folders show it being recognized and copied. There are no delays in my settings. The new folder was created on the left side, which is the local machine where SuperFlexible runs. The right side is the NAS box. t would be nice to fix this problem.

As for running the sync at Windows XP logoff, the new settings don't help. I'm not so concerned here as it seems the DOS batch command you illustrated could be used instead without problem (haven't tried it yet). I'm wondering how necessary this is. My goal would be to run RTS all the time. The only reason to run a sync at either startup or shutdown is to catch possible errors in RTS. I'm doing this at the suggestion of something I read when first setting up my tests many weeks ago (but I can't recall where I saw that). Is this really necessary when using RTS?

Thanks

[superflexible]

You certainly do not need a log-off job when using real-time sync. It would be sufficient to choose the checkmark "Run profile entirely once, then use Real-Time Synchronization" on the Real-Time Sync dialog.

Your log files are quite interesting, I will try to improve it to avoid the "New Folder" problem.