Product and service reviews are conducted independently by our editorial team, but we sometimes make money when you click on links. Learn more.
 

Cause Of The Dropbox Outage: Hacker Or System Update?

By - Source: Toms IT Pro

Dropbox, the file storage and sharing cloud service, had a hiccup on Friday that shut the Dropbox site down and continued to affect its user base over the weekend.

In a Dropbox tech blog entry posted at 6:40PM PST on Friday, the site claimed the problems were due to "routine internal maintenance, and was not caused by external factors." Slightly less than two hours later, the site claimed to be back up and running with a denial of claims of leaked user information.

Over the next two days, status updates assured DropBox's user base that progress was being made and no files were lost during the restoration process. Finally on Sunday evening, the Dropbox status report indicated that Dropbox was working through some final issues with its photos tab but the site was up and running for all of its users.

The claim of leaked user information originated from a Korean hacker group calling themselves AnonOpsKorea, an offshoot of Anonymous. The attack was meant to honor the death of Aaron Swartz, a programmer and tech activist who committed suicide about a year ago while awaiting federal trial for hacking, according to a report on the New York Times Technology Bits site.

According to DropBox, the data the hacker group claimed to have leaked on pastebin.com was not DropBox data and had been published on December 9, 2013. The indications are that the hacker group used DropBox's inconvenient outage as an opportunity to claim responsibility.

In an outage post-mortem on DropBox's tech blog, Akhil Gupta summarized the outage with the statement, "On Friday evening our service went down during scheduled maintenance. The service was back up and running about three hours later, with core service fully restored by 4:40 PM PT on Sunday."

The Dropbox site placed the blame on a "bug" in a script used as part of an Operating System (OS) upgrade process. According to Dropbox's post-mortem blog, there was never any risk to users' data, however as the company explains , "a subtle bug in the script caused the command to reinstall a small number of active machines. Unfortunately, some master-replica pairs were impacted which resulted in the site going down."

In its post-mortem, Dropbox described two core issues that occurred and what it is doing to keep them from happening again. Rather than relying on scripts to be bug free, DropBox is adding a verification layer on each machine that will allow it to check to see if the process it is ordered to execute is potentially destructive; if it is, the machine can refuse the command.

Another lesson learned was that disaster recovery and specifically restoring huge databases from backup systems is a slow process. To speed the process up, DropBox has developed a tool that will improve recovery time from MySQL backups. The company indicated they will open source the tool to give others an opportunity to learn from their experience.

Dropbox, founded in 2007, claims over 200 million people as customers of their service.

[ Get IT news updates right in your inbox -- Sign up for Tom's IT Pro's Weekly Newsletter ]

_________________________________________________________________________________________

ABOUT THE AUTHOR

Bill Oliver has worked in IT as a techie, trainer, manager, and in business roles supporting IT for 20+ years.  For the past 12 years his focus has been on the business side of IT Contracts, Software Licensing, and all things related to IT Purchasing.

More by Bill Oliver

Comments