Fixing Duplicates

Though not common, over time there have been users who have reported entries duplicated either on their device or at Google/Plaxo.  As we've made continued improvements to squash duplicates, we've kept in touch with those users.  But, we want to broaden access to that information and make sure all our users know first, where duplicates can come from, and second, how to deal with them.

By far the most common cause actually isn't a NuevaSync problem at all, but a bug in the Apple synchronization client.  The precise details are rather convoluted, but the quick version is that if an error ever happened when syncing, it was possible for the device to get confused and start adding entries it already had.

In response to that, we’ve written and deployed code that actively intercepts duplicates before they get committed.  It isn't possible to intercept 100% of duplicates this way, but the results so far have been excellent.  And since the code is in our server, it applies equally to devices of all types and versions: iPod, iPhone, Windows Mobile, Nokia, etc.

Apple apparently noticed the problem as well, and another major step came with the release of the 2.1 firmware.  When we began testing the 2.1 beta releases a few weeks ago, we noticed an improvement in the devices' behavior; they did not create duplicates in the same circumstances as the 2.0 devices.  This is a great enhancement for Apple users, and though we are still a little reserved about 2.1 (cf. some of our earlier posts), we are very pleased with this particular fix.

While those changes (and several other, smaller ones we've made) work to prevent duplicates, what do you do if you already have them?  To answer that we have created some experimental tools to "dedupe" your account.  The tools for Google Contacts and Calendar have been available for some time, but today marks the release for Plaxo.  All these tools are specifically targeted at removing duplicates which might have been created through us.  If your service offers its own tool, you may want to use that instead.  Plaxo, for example, offers an advanced duplicate merging and removal tool which is well worth a look.  Our tools are experimental and by their nature designed to delete entries.  Make a backup first and exercise caution when using them!

They can be accessed at:
https://www.nuevasync.com/PublicSite/user/tools/dedupe.htm

Detection and removal can take a few minutes, so be patient when it runs.  If a lot of entries are removed, it is probably a good idea to resync your device afterward (cf. https://www.nuevasync.com/PublicSite/user/troubleshooting.htm), but that is entirely up to you.

By raw numbers, duplicates have never been common, but nevertheless they have been one of--and perhaps the--most serious of issues to us. With these new measures, we’re hoping most users will never have a problem.  If you do, e-mail us at support@nuevasync.com and we'll do everything we can to get it fixed and you back syncing.

As always, many thanks to our users!

2.1 Firmware Investigations

Just a quick update, but following information provided by diligent users, we have some lines of investigation which we are pursuing on the freezing issue which can sometimes occur with iPhones or iPods running the new 2.1 firmware.  We've also managed to reproduce this ourselves on a test device, and are working on a consistent reproduction case to nail it down.

If you encounter this problem, please do e-mail support with the subject 'iPhone Freezing Returned'.  We've had a number of users send in reports, but more information is always good.  Please include some details about--as best you can recall--the last operations you were performing before the freeze.  That information is especially helpful.  Things like, were you adding an entry,  changing one, or not changing anything?  If you were changing one, was there anything unique about the entry?  We may not respond individually to each e-mail, but we are studying them.

And our thanks again to all the users who have sent us information.  Thank you!

More on the iPhone Freezing Syndrome

So far unfortunately we have been able to identify any specific data returned by the service that might be upsetting the iPhone sync client. The best workaround we have is to use 'Settings\General\Reset\Reset All Settings' to clear the device sync state. This will also reset various other setting such as your icon positions and stock tickers. Most users who have used this method report success, although some have reported the freezing subsequently returns.
If you have had freezing return after device reset, please send us e-mail and mention 'iPhone Freezing Returned' in the subject so we can identify your message quickly. We want to check the server logs for those users to see if we can correlate us sending a specific event to their device with the onset of freezing.
Also, if anyone has the freezing problem and is also an iPhone developer, we'd be interested to see the log from your device (can be viewed using the XCode utility with the device docked).

iPhone 2.1 Freezing Syndrome

It would seem that all is not well with the 2.1 firmware update. We've had reports from several users (although by no means all users with the 2.1 update) that their phone freezes while attempting to sync. Reading the comments on the previous article and the support e-mail, it does appear that the problem can be resolved. We're working to determine the exact correct set of steps to un-freeze the 2.1 iPhone and will post it when it's ready. In the meantime it might be prudent to not update your phone/iPod to 2.1 just yet.
We were a little asleep on this issue unfortunately because although there were a few support tickets over the weekend that mentioned freezing, we had seen similar reports trickle in for the past couple of months (from users with earlier firmware) and so for a while we assumed this was just more of the same random iPhone hangs, and not something related to the use of our service. There were more reports on the problem, and more detail in the blog comments here than in the support e-mail, but we were focusing on the support e-mail.
Check back here for updates on this issue.

iPhone 2.1 Firmware Update

A number of users have written to report that they had problems after updating to 2.1. There are no known problems with the 2.1 firmware and Nuevasync. However, the firmware update process seems to reset the sync state on the device which results in a 'resync' the next time the phone contacts our service. The resync operation is quite expensive and can take a while. Also at times a resync can fail if Google is busy. The users who reported problems subsequently let us know that their iPhones picked up sync again after a while. During the time between the firmware update and the next successful resync, your iPhone will loose its calendar events and/or contacts. Therefore we recommend updating your firmware at a time when loosing sync for a time will not cause major inconvenience. If in doubt, check your device sync status.

Scheduled Maintenance Complete

As mentioned in an update to the earlier notice, scheduled maintenance has been completed for all servers and the service has been turned back on to full capacity.

A Post on Service Quality Issues

In the interests of preserving the useful comments on this post (which was originally about a scheduled downtime required to replace defective hardware and to deploy a faster database machine), I'm leaving this post up forever.

The hardware upgrade appears to have been a great success. Server load is quite low and so far there are no reports of serious problems.

The post originally said :

Nuevasync service will be unavailable for a time today while a major hardware upgrade is done. The planned outage time begins at 11am MDT (18:00 UTC) and is expected to last two hours. During this time the web site will redirect to a maintenance page and no syncing will be done. iPhone/iPod users : PLEASE DO NOT DO ANYTHING. Don't 'resync'. Just leave your phones alone and they will pick up sync again when the service is restarted. If you resync, all your data will be deleted from the iPhone and it may take a long time for it to be synced back.
Update : the upgrade went well, and it turned out we had a bad hard drive (bad in a bad way in that it didn't report any errors but silently hung the system when certain sectors were read). Dealing with the drive issue (which means re-installing the OS on the machine since the old drive couldn't be imaged) will take a little longer than the expected down time. We should be up in about 45 mins time (2:30 MDT, 21:30 UTC).
In response to the people who commented that the old service status posts should not be deleted : well I agree in so far as I'd like the comments preserved, but on balance it seemed messy to leave all those rather quickly written and often in hindsight incorrect posts on the blog. What we really need is a page that a) stays up when our connection or servers are down and b) isn't a blog. Right now we don't have that in place so the blog got abused with the status posts. It's not ideal but we've taken the approach that you the users would rather we fix the problems than make things all pretty and neat.