S3 Sync problems

No more questions - please go to http://www.syncovery.com/forum instead

Moderator: SuperFlexible Administrators

S3 Sync problems

Postby Dfects on Wed Jul 06, 2011 3:44 am

Hiya,

What I'm trying to achieve is a 2 way sync between all my Pc's, with S3 as the central store exactly like DropBox. At the moment, I'm having some problems configuring this right.

I've created a small test syncing just one file to show you the problems i'm having. The first sync is fine:

Image

But then, running it again straight away it looks like the date hasn't changed on S3 to the actual modification time of the file so its trying to download it again which is completely pointless:

Image

Here are my comparison settings:

Image

I've tried with and without 'smarttracking' but same results.

I've also tried telling it to ignore timestamps, but then it doesn't seem to always pick up changes. I remember reading on here amazon syncs always use a file hash. I created a hash before (both sha and md5) then changed a character in the file, then another hash and as expected they were different. Running sync again didn't pick up the change at all though?!?

Any ideas?
Dfects
 
Posts: 3
Joined: Wed Jul 06, 2011 3:29 am

Re: S3 Sync problems

Postby superflexible on Wed Jul 06, 2011 4:00 am

Hello,

you must use Filename Encoding to preserve the timestamps on Amazon S3, because Amazon S3 does not provide any other means of preserving them. The Filename Encoding checkmark is on the Versioning tab sheet as well as on the Internet dialog where you specify your Amazon S3 credentials.

Please leave the Comparison settings at their defaults, i.e. do not ignore seconds altogether and ignore only small differences of up to 2 seconds, not more. The 2 seconds differences can occur between different file systems, such as NTFS and FAT32.

SmartTracking is only needed if you want to also propagate deletions to the other machines.
User avatar
superflexible
Site Admin
 
Posts: 2478
Joined: Thu Dec 31, 2009 3:08 pm

Re: S3 Sync problems

Postby Dfects on Wed Jul 06, 2011 4:17 am

Thanks for the reply.

Is there any way to do this without the filename mangling? Should it be checked on hash instead if the timestamps set to ignore? As it doesn't seem to be the case.

Could the timestamp also not be stored as metadata for the file on s3, rather than mangling the filename itself?
Dfects
 
Posts: 3
Joined: Wed Jul 06, 2011 3:29 am

Re: S3 Sync problems

Postby superflexible on Wed Jul 06, 2011 4:28 am

You must use Filename Encoding. But don't worry about it, it works great. The filenames become a bit longer, but you don't have to look at them as a person. The program will do it for you.

Hashes are only used to verify the integrity of a transfer. Not for comparison. Since Windows does not provide such hashes, it would be inefficient, and the timestamp would still be missing.

Timestamps could be stored as metadata, but Amazon S3 does not include metadata in the file listing, so metadata is totally useless.
User avatar
superflexible
Site Admin
 
Posts: 2478
Joined: Thu Dec 31, 2009 3:08 pm

Re: S3 Sync problems

Postby Dfects on Wed Jul 06, 2011 4:45 am

Thanks for the quick and helpful replies :)
Dfects
 
Posts: 3
Joined: Wed Jul 06, 2011 3:29 am


Return to Windows Support * new forum: www.syncovery.com/forum

cron