2 Quesions

[newsportalsite]

Hello.

We are a web development company and we are trying out Super Flexible File Synchronizer. I actually recommended this because I use this to backup my Laptop and Home PC's to S3.

Anyways, we are trying to backup to S3, which seems to be working. All of the files, that we have checked, showed up on S3 in their entirety. This feature is working fantastically.

We recently completed one server backup. We decided to run a test restore for one folder, so we moved one version of our software to a different folder, thereby simulating a "delete". We opened the interface, and opened the restore wizard.

From there, we selected all the default options. Once we got through that, it started to read our AmazonS3 listing.

It took about 30 minutes to read our listing, seeing how we have about 900k files on S3 (this is an approximation we got from the "Usage Reports" in Amazon S3. We based it off the "Puts"). When it finally finished that, it only showed the first 3 folders. When we closed it to try and run it a second time, we received a message that we ran into a memory limit, and it asked us if we would like to try again and show the next x amount of files in the restore wizard. We said yes.

It then proceeded to re-index Amazon S3. So there goes another half hour. And we still only got he next 3 folders.

We have since tried to "catalog" the file list by adding a "database", but we are unsure if we did this right. What is the steps needed to properly setup a cached or cataloged file list to eliminate the need to index Amazon S3 every time we need to restore?

Also, we are running Windows Server 2008.

What would be the best setup for our "job"? We currently run every night between 1AM and 9AM. What would you suggest, not just in terms of time, but in terms of the plethora of features this software has? What would be an optimal setup for us?

If we can get this version working, we will likely be buying up to 8 more licenses. We are currently running the newest V4 version available. When I tried V5, it would not authenticate with Amazon S3 under any circumstances. There is always a chance I did something wrong, but there is also a chance that the BETA software available for download has a few bugs in it. We will upgrade as soon as I can run successful trial run on my laptop.

Thanks

Nick w/ NewsPortalSite

[superflexible]

Hello,

the latest version 5 is no longer beta and has no known issues with Amazon S3. It is highly recommended especially if you want to use the database because the new database engine is more capable of handling this large amount of files.

But first, please go to the Program Settings, tab sheet Advanced and choose "Never split jobs due to low memory". Obviously the splitting is not beneficial in these cases. With that option, you should be able to do a restore in a reasonable amount of time.

Unfortunately listing items on S3 is slow because S3 only sends a listing of 1000 items at a time.

To use the database, in the backup profile you would go to Special and choose "Cache Destination File List". On the pop-up dialog, please make sure that only the first checkbox is chosen.

Then when you use the Restore Wizard from that job, it lets you choose whether to use the database rather than pulling a whole new listing. If you save your restore profile, you can check the settings that are responsible for this: on the Special tab sheet, "Use Cache Database For Source", and on the Database tab sheet, you should see the name of the backup profile (or the name of the database that the backup profile is using).

[newsportalsite]

Thanks.

the latest version 5 is no longer beta and has no known issues with Amazon S3. It is highly recommended especially if you want to use the database because the new database engine is more capable of handling this large amount of files.

We updated to the newest version.

But first, please go to the Program Settings, tab sheet Advanced and choose "Never split jobs due to low memory". Obviously the splitting is not beneficial in these cases. With that option, you should be able to do a restore in a reasonable amount of time.

We checked this option.

Unfortunately listing items on S3 is slow because S3 only sends a listing of 1000 items at a time.

We totally understand that the limit is stemming from Amazon, not SuperFlexible.

To use the database, in the backup profile you would go to Special and choose "Cache Destination File List". On the pop-up dialog, please make sure that only the first checkbox is chosen.

We have done this.

Then when you use the Restore Wizard from that job, it lets you choose whether to use the database rather than pulling a whole new listing. If you save your restore profile, you can check the settings that are responsible for this: on the Special tab sheet, "Use Cache Database For Source", and on the Database tab sheet, you should see the name of the backup profile (or the name of the database that the backup profile is using).

We checked "Use Cache Database For Source". Under the Database tab, "Open Database Read Only" is not checked, the "Database Name to Use (Profile Name):" is set to the exact same as the Profile Name, and the "Database Contains Paths Relative To:" are both blank. For some reason, we are still unable to see the Database from the restore tab titled "Use Backup Database". We see:

"The file list will be read in directly from the backup, because no database has been found.
Click Nest to continue."

I do want to clarify what we originally wanted to happen. We would go through all of the tabs in the restore wizard leaving everything to their default, and have the list of files generated in the nice file listing window we get after clicking "Show Available Files" to be generated from the Database instead of the S3 listing request(s). We currently can pre-select a folder under "Folder Selection", and that does help to expedite the listing request a little bit, but this does not suit our needs as we may want some files from some folders and no files from another.

Thanks. I am very satisfied with the product so far and I am eager to see what the software can do now with the new update.

Nick w/ NewsPortalSite

[newsportalsite]

Would anyone happen to have a solution to this? We are nearing the end of our trial and we need to make a decision on whether we are going to purchase or not.

[superflexible]

I may have said something misleading. There can be two different kinds of profiles:

A) Backup profile -> uses Cache Database for Destination ONLY!
B) saved Restore profile -> uses Cache Database for Source ONLY!

You only need to use the Restore button once. Then you can save the Restore functionality as a profile. This is offered in the last step of the Wizard. When you look at this profile, that's where the database is checked for the source side and that's where the database name must match the original backup profile's name. If you don't save your restore profile, you can forget about these options that I mentioned.

I just tried it with the latest version 5.12. I clicked on my Backup profile that uses the cache database for the DESTINATION. The wizard offered me to use the database. It did not say that there was no database. Maybe you can try that again.

[newsportalsite]

I want to take a minute and explain once again the problem we are having. I am afraid my example might have caused some confusion.

We would like to be able to restore everything as fast as possible. This is inherent to any disaster recovery plan. This is not, however, what I ultimately want.

As of right now, 10:21 on 10/29/10, our job status is:

10/13/2010, 2:00AM Building List ... 115180 Entries

This means that it has been building a list of files on the remote host for 17 days. Thus, no backup has occurred for 17 days. It is absolutely critical that this gets solved so it backs up as frequently as possible. Ideally, it should backup everyday.

Once we get the back up fixed, then we can move onto the restore. Restoring does nothing if we don't have recent files to restore.

If there are any config files I can send or something, just tell me. We are looking to get this solved as soon as possible and would like to be as much help as you need.

Thanks. Have a Happy Halloween.

Nick

[superflexible]

For the best results, you have to install the software on the machine where the data to back up is stored.

Even for the largest jobs, it should never take more than maybe an hour to build the file list. You seem to have a really bad setup.

While testing, you should run the job manually in the foreground so that you'll get better progress information.

It would be good if you sent screenshots and log files to support@superflexible.com so I can see what you're actually doing.

If you cannot run the job on the same machine where the data is stored, you need to use the ExtremeSync Remote Service on that machine to help build the file list fast. See also our FAQ "Building the file list takes too long" on http://www.superflexible.com/docs.htm.

[d033e22ae348aeb5660fc2140aec35850c4da997]

Are you using Volume shadow copies on the remote system? I have seen this before delay the file list building but not 17 days. VSS is only supported locally.

There is clearly a problem. What kind of settings did you choose in the profile? Maybe send the ini in an email as well. I also saw only once when I edited the ini file manually that it caused me major issues because the formatting of the ini was off.

[superflexible]

Except be aware that the INI file contains your Amazon S3 credentials. To send a profile to our support, it would be best to make a copy of it, remove the credentials, and then export it by right-clicking it in the Profile Overview.

[newsportalsite]

I sent an email for support.

Sorry for the delay. I thought I would receive notifications of thread updates, but I guess I do not. My apologies.

The email came from nick@newsportalsite.com . I think I forgot to place a link to this thread in it. Again, my apologies.

Nick