The myth about how Amazon’s Web service started just won’t die
How AWS got started and what its co-founder is doing now that he says could be bigger than cloud
There’s a rumor that goes around cloud circles about how Amazon.com created what is now the multi-billion dollar infrastructure as a service (IaaS) cloud computing industry in the early 2000s.
Some people wrongly assume that Amazon had spare, excess computing capacity from their ecommerce site that was used as the basis for Amazon Web Services’ (AWS) cloud.
It’s something that Benjamin Black has heard a lot. But it’s not true. And he would know: Black is widely credited with co-authoring the initial proposal at Amazon that led to the creation of AWS.
“Why will that not die?” Black says about the rumor. “It’s totally false.”
Black, who recently accepted a new position at cloud company Pivotal, says from day one, every part of AWS has been purpose built for AWS. And now he’s hoping to work on a new project that he says could be even bigger than the cloud he helped create at Amazon.
How AWS actually got started
Benjamin Black co-authored a paper at Amazon.com in 2003 that helped kick off Amazon Web Services and the IaaS cloud computing market. Black now works at Pivotal.
In 2003 Black was running a website engineering team at Amazon. The company was growing fast and IT wasn’t keeping up. Black worked with Chris Pinkham, who he says is one of the best managers he’s ever worked with. Pinkham pushed Black to consider how Amazon’s infrastructure could more efficiently scale up. They explored how abstraction and decoupling the applications from the infrastructure could make it easier to manage.
“We realized there could be a lot of value in doing that, and a lot of value to others potentially outside of Amazon,” Black told Network World. “We could sell it (the infrastructure) as a service.” Black and Pinkham wrote up the idea, which made its way to Jeff Bezos, who greenlighted the proposal. Pinkham then led a team to build Elastic Compute Cloud (EC2), which are virtual machines as a service and one of AWS’s first products released in 2006.
“Right off the bat we just thought it would be an interesting thing to do. It took a while to get to a point of realizing that this is actually transformative.”
That seed of an idea turned into what is now the market-leading IaaS public cloud computing company. Amazon was estimated last year by Gartner to have a public cloud that is five times larger than its next 14 competitors combined. Needless to say, the idea Black helped start took off. Pinkham went on to found startup Nimbula, which Oracle bought and used as the basis for its cloud platform. Pinkham now works as an engineering vice president at Twitter.
How did Bezos receive the idea? Black recalls Bezos envisioning a platform that would give anyone, such as college kids in a dorm room, the tools they would need to start a new company.
“That’s still the idea people have about it,” Black says. “At the same time, it’s taking over the world.” He says the fundamental key to AWS, which remains today, is that it provides the undifferentiated technical infrastructure to anyone who wants it – whether that’s VMs, storage, or Hadoop as a service.
Some of the directions Amazon has taken AWS have surprised Black. AWS is moving further and further “up the stack” to provide application services, like virtual desktops and email. Not everything he and Pinkham proposed made it into the initial version, but every change was for the better, he wrote in a blog post describing the origins of EC2.
Did Black realize the idea he and Pinkham proposed to Bezos would turn into what is has today? Far from it. “Right off the bat we just thought it would be an interesting thing to do,” he says. “It took a while to get to a point of realizing that this is actually transformative. It was not obvious at the beginning.”
How the Internet of Things could be the next cloud
Black has a new gig now. After stints at Microsoft, VMware, advising the company Chef and starting his own monitoring company named Boundry, cloud company Pivotal hired Black as senior director of technology. Pivotal, which is a spinout from VMware, EMC and has substantial backing from General Electric, is behind the open source platform as a service (PaaS) Cloud Foundry.
Whereas an IaaS like AWS is a massive distributed system of virtual hardware and services – like compute, storage and databases – a PaaS is an application development and hosting service.
In his new role at Pivotal Black hopes to spearhead the company’s burgeoning Internet of Things (IoT) lab in Seattle, where he lives.
INSIDER: 5 ways to prepare for Internet of Things security threats
There’s an opportunity for a company like Pivotal to create a series of application components that can be used in IoT that serve as a basis for many other IoT apps, Black says. “There are some pretty basic patterns across all of these desired apps,” he says. “What we’re looking do is develop the primitives that would allow anyone to get into the IoT marketplace.”
When asked how the IoT market could compare to the cloud computing market that he helped usher in, Black said: “Bigger.”
The head of Amazon Web Services bashes IBM and launches a VDI service at this year’s AWS Reinvent conference
Private clouds offer “none of the benefits” of a robust public cloud, and are only a stopgap solution perpetuated by “old-guard” IT companies such as IBM, said Andy Jassy, Amazon senior vice president who heads up Amazon Web Services.
“If you’re not planning on using the public cloud in some significant fashion, you will be at a significant competitive disadvantage,” Jassy told a packed auditorium of nearly 9,000 IT pros Wednesday in Las Vegas, for the opening keynote of the AWS Reinvent conference.
Jassy split his time between extolling the benefits of using large public clouds such as Amazon’s and introducing new services.
While he spent much of his presentation discussing the benefits of cloud computing, arguing that it offers increased agility, better security and lower costs, he also took time to criticize private clouds, or cloud infrastructures that organizations have set up in-house for their own use.
To set up a private cloud, an organization still needs to invest a considerable amount of money in hardware and software, so it requires up-front capital costs that a public cloud doesn’t, he said. Private clouds don’t offer the agility of public clouds, in that the enterprise still can’t change to a new platform or set of software as quickly. It also doesn’t offer economic advantages of buying hardware in large amounts.
Some organizations, such as governments and health-care providers that have strict regulatory requirements, still need to run operations in private data centers, he said, but over time, these specialized-use cases will diminish as more of the features required will be available on public clouds.
Amazon offers a number of services that help organizations run hybrid clouds that are partially run on Amazon and partially in-house, including VPNs (virtual private networks), and identity and access management. The company also works with traditional enterprise IT management tool providers, such as Eucalyptus, CA Technologies and BMC Software, to provide a single view of both on-premises and cloud operations.
But AWS put these services and partnerships to help customers move almost entirely to the AWS public cloud.
“We have a pretty different view of how hybrid is evolving than the old-guard IT companies,” Jassy said. The approach popular with companies such as Hewlett-Packard, Microsoft and IBM, for instance, assumes an enterprise will want to run most of its operations in-house and use public clouds to augment operations when traffic is heavy.
“We believe in the fullness of time, very few enterprises will run their own data centers,” Jassy said to note the difference in the AWS approach. “That informs our approach in what we build. We will meet enterprises where they are now, but we will make it simple to transition to where the future workloads will be, in the cloud.”
“I think a lot of old-guard technology companies aren’t so thrilled about how fast things are moving to cloud,” Jassy said. He showed a slide of one of a number of advertisements that IBM has placed on buses this week in Las Vegas that claim that the IBM Cloud service hosts “30 percent more top websites” than any other cloud provider.
“It’s creative, I’ll say that,” Jassy said. “I don’t think anybody who knows anything about cloud computing would argue [IBM] has a larger cloud business than AWS.”
In June, IBM purchased SoftLayer to boost its public cloud offerings.
Jassy also took time to announce some new services.
Perhaps the most notable launch for the company is a new VDI (virtual desktop infrastructure) service, called Amazon Workspaces.
Workspaces provides a virtual desktop for an organization’s employees that can be accessed from Apple Macs, Microsoft Windows computers or Android devices. It provides a “persistent state,” Jessy said, meaning that the desktop’s contents will remain the same no matter what device the desktop is accessed from.
Despite the advantages it offers administrators in managing their users’ computers, VDI thus far has not made major inroads into the enterprise IT market, though Amazon is hoping Workspaces will prove cost-effective and easy enough to manage that it will be appealing.
Workspaces will cost about half of the expense of the current average VDI implementation, he said. The service, which is now offered in a limited preview, can be paid for on a month-by-month basis. A Workspaces desktop with one virtual CPU and 50GB of storage space will cost US$35 a month, and the “performance” desktop with 2 virtual CPUs and 100GB of storage will cost $60 per month.
With Workspaces, an organization can bring its own licenses for Microsoft Office and security software, or Amazon will offer these applications for an additional $15 a month.
AWS also launched a security service that can provide customers with detailed log reports of who is accessing their APIs (application programming interfaces) and what services they consume, as well as a streaming service for apps.
Technology is advancing day by day in fact the new technology is no killing the old technology in reality it is advancing the previous versions, peoples are more and more easy and secure way to in technology usage, Microsoft is always been a very fast detector how to reshape the new technology is all software’s like Microsoft Office, Operating systems like windows XP to Windows 7, Internet Explorer 8 IE8, and more,
Most of the bricks organizations are now becoming bricks and clicks organization, the requirement to advance these organizations required certified peoples to work with them and. A professional person holding Microsoft certifications in his hand is often valued over other workforce all around the planet. Among all on hand Microsoft certifications, one of the most accepted one is MCTS Training, Microsoft Certified Technology Specialist focus on emerging technological prospective and employing these concerns for progressing in Information Technology industry. If you have certain required abilities for this exam you can pass it quite effortlessly. These abilities take in the following:
Intro on MCTS Certification
The MCTS certification is the one, which helps the candidate to step into the IT industry. MCTS also helps the professional who are already in the IT industry to get into a good position in the field. The candidates who are applying for the MCTS Certification should have experience about the network connectivity, desktop operating system, security, and applications. Those who are very good in these areas can have the MCTS certification without any problem and they may be experienced in a particular filed. The future of the certification will be very good and more demand will be there for MCTS certified professional. There are lots and lots of products that are developed with Microsoft Technology. Microsoft develops products which is very helpful for the users.
What expertise and skills MCTS certification demands?
Though you can acquire a reputable status by obtaining this certification, but it obviously demands a few expertises’s that you must have. For this reason, you must be able in:
* Computer network literacy
* Solving logon related problems
* Creating as well as maintaining the desktop applications
* Executing password resets and others alike
MCTS certification will enhance your
MCTS: .NET Framework 2.0 Web Applications
Microsoft SQL Server technologies
Microsoft Exchange Server technology
To get this certification, you will require an experience of at least two years in implementing, troubleshooting, and debugging a given technology. One can say that this certification is the foundation for all the different Microsoft Certifications that are meant to validate your expertise in the functionality and features of Microsoft key technologies. As an IT professional, either you can demonstrate your in-depth knowledge in a given technical application or choose to earn as many MCTS training as you want to endorse your capabilities across a number of Microsoft products. However, it is all the more essential to constantly update your certification to enhance your competency under today’s robust IT scenario.
If your preparing for career change and looking for MCTS Online Training Certkingdom.com is the best online training provider that provide the all the and complete MCTS certification exams training in just one package, certkingdom self study training kits, save your money on bootcamps, training institutes, It’s also save your traveling and time. All training materials are “Guaranteed” to pass your exams and get you certified on the fist attempt, due to best training CertKingdom become no1 site.
IT AND Microsoft Certification At Certkingdom.com
Thought I would make this post to give people the feedback about my first IT certification MCSE 2003. As this is rather a large subject covering a variety of areas, I have attempted to break these down Microsoft Certified Systems Engineer 2003 preparation into different segments with timelines.
What is Microsoft Certified Systems Engineer (MCSE 2003)
Microsoft Certified Systems Engineer 2003 (or MCSE 2008) is the best-known and premiere Microsoft certification. It qualifies an individual as being able to analyze the business requirements for information systems solutions, and design and implement the infrastructure required. As of 2008, the MCSE is available for two different product lines; Windows 2000 and Windows Server 2003, each of which requires a different set of exams.
For the MCSE 2003 certification, candidates must pass six core design exams (Four networking exams, one client operating system and one design exam) and one elective exam, for a total of seven exams. For the MCSE 2000, a candidate needs to pass five Core Exams (Four operating system exams, one design exam) and two electives. For the MCSE NT 4.0 (retired), a candidate needed to pass four Core Exams (Networking Essentials, Windows NT Workstation, Windows NT Server and Windows NT Server in theEnterprise) and two electives.
Core Exams for mcse 2003 certification
70-290 Managing and Maintaining a Microsoft Windows Server 2003 Environment
70-291 Implementing, Managing, and Maintaining a Microsoft Windows Server 2003 Network Infrastructure
70-293 Planning and Maintaining a Microsoft Windows Server 2003 Network Infrastructure
70-294 Planning, Implementing, and Maintaining a Microsoft Windows Server 2003 AD Infrastructure
The topic of these exams include network security, computer networking infrastructure, Active Directory, Microsoft Exchange Server, Microsoft SQL Server, and other topics of both general networking interest as well as specific Microsoft products.
The following is MCSE specialization, Upgrade paths
MCSE on Windows Server 2003
• MCSE on Windows Server 2000
• MCSE on Microsoft Windows NT 4.0
• MCSA on Windows Server 2003
• MCSE: Messaging on Windows Server 2003
• MCSE: Security on Windows Server 2003
MCSE on Windows 2000
• MCSE: Messaging on Windows 2000
• MCSE: Security on Microsoft Windows 2000 Server
Train for your MCSA or MCSE 2003 Training on Windows Server 2003 and get closer to Windows Server 2008. The strength of Windows Server 2003 in the market today indicates that demand for related IT expertise will continue for years to come. The best way to demonstrate you have those skills—and to inspire confidence in a hiring manager, your team, and yourself on Windows Server 2003—is with the Microsoft Certified Systems Administrator (MCSA) and Microsoft Certified Systems Engineer (MCSE) credentials. These credentials will not retire.
The most efficient way for Microsoft 2003 exams training.
- MCQ’s Training (multiple choice questions)
- Case Studies Training
- Study guides Training
- Labs Preparation
- Online Videos Training
- Audios Training
- Exams Testing Engines
- Scenarios Bases Question and Answers
When I started in the first line role, one of my initial questions was ‘what do I need to learn to get the best online mcse 2003 training at my home?’ I was given feedback from my friends whom boiled down to IT skills, MCSE 2003 would be preferential, but more importantly are your willingness to learn, attitude and aptitude.
I knew from the moment I had finished my initial training, that I was different to the normal bread of Helpdesk personnel. Rather than spending my time surfing the web, I had my head in a book reading and learning.
I also vetted all of my calls as if I was second line (even though I wasn’t). This did ruffle a few feathers, but I cleared it with my friend first and also made sure that a second line person approved my comments, before it went to third line. The feedback from my Team Leaders was it showed initiative and willingness to learn.
If your preparing for career change and looking for MCTS Training the best online training provider that provide the all the and complete MCTS certification exams training in just one package, certkingdom self study training kits, save your money on bootcamps, training institutes, It’s also save your traveling and time. All training materials are “Guaranteed” to pass your exams and get you certified on the fist attempt, due to best training they become no1 site 2009 & 2010.
In addition I recommend Certkindom.com is best and No1 site of 2008 which provide the complete Windows Server 2003 certified professionals training, Microsoft MCITP, Microsoft MCTS, Cisco CCNA, Cisco CCIE, CompTIA A+, IBM, Citrix, PMP, ISC, and lots more online training self study kits, saving your time and money on all those expensive bootcamps, conventional training institutes where you have take admission pay fees first and if you don’t want to continue no refunds no transfer to any other training course, If you planed to take CCNA or specialization in MCSE 2003 all the process starts again; as for getting online training can be much beneficial and you don’t need to take for fill any from to switch your training on any desire certification.
Greatest Tech Battles Ever Told
In honor of the patent war heating up between Apple and Samsung, we’re looking back at epic tech battles. The one thing they all have in common: the future of the universe hung in the balance. (Okay, not the universe but a really big market.)
Oracle. Apple. Google. Facebook. Microsoft. SAP. We’ve seen some of the biggest names in some of the nastiest battles over the years. The balance of power shifts, markets move, and there’s a disturbance in the Force. Call it Tech Wars.
iOS vs. Android
It’s iOS vs. Android with the future of mobile as the prize. Want more drama? Throw in the fiery words of the most admired CEO in history, the late Steve Jobs: “I will spend my last dying breath if I need to, and I will spend every penny of Apple’s $40 billion in the bank, to right this wrong. I’m going to destroy Android because it’s a stolen product. I’m willing to go to thermonuclear war on this. They are scared to death because they know they are guilty.”
PC vs. Mac
This is the greatest tech battle ever, played out on the small screen pitting the geeks against the cool kids. It is the battle from which all other battles have been judged. The words “I’m a PC, I’m a Mac” have become part of our culture. So who has won? Like Star Wars Jedi vs. Sith, the tide turns with every generation.
Oracle vs. SAP
Quick, what software can cost millions of dollars and take years to integrate? Hint: This complex software has derailed many CIO careers. There can be only one, of course, and it’s enterprise resource planning, or ERP. Oracle and SAP have gone head-to-head for years at this high-stakes poker table.
Facebook vs. MySpace
In the super-hot social networking space, Facebook rules the empire. But it wasn’t always that way. MySpace used to be the most visited social networking site in the world, riding pop culture, music and teenyboppers to lofty heights. Then came Facebook. It appealed to the young, college-educated professional and ushered social networking into the mainstream.
VHS vs. Beta
VHS and Beta are pretty much gone now, but the two technologies sparked the first battle for the living room — specifically, home movies. VHS, of course, won. It was the machine that launched a thousand rental stores across the country.
But nothing lasts forever, and VHS itself became victim to the DVD, which, in turn, is succumbing to streaming movies. Meanwhile, rental stores are getting torn down as quickly as a bad VHS machine chewed up the edges of a tape.
Internet Explorer vs. Netscape Navigator
If you were following the tech scene in the 1990s, you’d remember the browser war between Microsoft Internet Explorer and Netscape Navigator — one that drew in the Department of Justice and put Microsoft in the crosshairs of a precedent-setting antitrust case. It led to the surreal sight of Bill Gates testifying and saying over and over, “I don’t recall.” That’s right, the same guy with the brilliant mind.
Only techie publications cared much about the great decade-long Database War between Oracle, Sybase, Informix, IBM and others. According to tech writer Eric Lai, the war started a fixation on performance measured by artificially enhanced benchmarks, which has “led to a distrust of benchmarks that lingers to this day.” Oh, Oracle won.
Bookstores vs. Amazon
Pity the humble, independent bookstore and even the mega bookstore. Book readers saunter in, explore different titles, gaze through books and then… whip out their iPhone and order it on Amazon. The massive online bookstore took a wrecking ball to the brick-and-mortar bookstore and upended an industry. The mayhem continues to this day. Heck, Amazon brought the phrase “brick-and-mortar” into modern-day vernacular.
Google vs. Yahoo
Remember when “search” was a neat little Web tool from companies with cute sounding names? It didn’t take long for search to become a powerful market driven by search engines with complex algorithms that generate tons of dollars of online advertising. Google stomped on Yahoo and became one of the biggest, baddest tech companies on the planet. Struggling Yahoo has had five CEOs in five years and now hopes ex-Googler Marissa Mayer can lead a comeback.
War Games (Nintendo, Xbox and Playstation)
Nintendo, Xbox and PlayStation have been battling it out in the gaming industry for years, from home video consoles to mobile platforms. It’s been fun to watch and play, and if you’ve got kids, you’ve probably paid for them all. The intense competition has led to grand advancements in gaming, including epic online adventures, awesome first-person shooting campaigns and the Wii. Gaming now is one of the biggest markets for consumer tech.
Google Apps vs. Microsoft Office
When Google Apps first appeared on the Web to go head-to-head with the venerable Microsoft Office suite, it didn’t look like a fair fight. Google Apps were quirky to use and didn’t feel ready for prime time. But tech wars can turn on a dime. Google Apps has since cut a swath out of Microsoft’s market share, although Office is likely to continue to dominate the all-important productivity market for the foreseeable future.
Jedi Yoda vs. Darth Sidious
Epic tech battles have the feeling of the universe hanging in the balance, kinda like when Jedi Master Yoda took on the Darth Sidious, Dark Lord of the Sith. In some tech battles, good did not always triumph over evil. In Star Wars, Yoda got his butt kicked, narrowly escaped, and slumped off into exile telling us what we already knew. “Failed, I have.”
I am currently engaged with mentoring some young technology start-up businesses. What strikes me about these companies is that they spend the majority of their time utilising their skills to deliver their product. They are agile, knowledgeable and very hungry to succeed and to create.
When do they find time to “learn” new emerging technologies? They seem to have learnt it “on the fly” as they go along – such is the pace of technology at the moment. With cloud computing, mobile computing and social media now becoming the current “bubble”, I realised just how easy it is for anyone in IT to become out of date quite rapidly.
There is an old saying which says “use it or lose it” and I will add “use it, grow it and keep your eyes open to what is happening around you, always”. In this process we must keep learning.
Stopping learning, even for a few months or a whole year can make a huge difference. It is like being having a motor car – use it regularly and it works fine (sure it may need a little maintenance), but leave it parked outside for a year unused and the battery will be flat, tires a bit softer, oil a bit tired, the gas will have lost its vitality etc. (Of course it does depend on where you park it – it may not even be there when you return!)
Learning is the same, especially in IT (and most other professions – like medicine, law, tax etc) we need to keep up to date, and even a few months “out of the game” will render us less sharp, and left with an uphill battle if we want to regain our status.
If “IT” is our career, then we need to learn on a regular basis, via personal learning, e-learning, books, attending classes, or as I am realising, by working with very sharp entrepreneurs who are leveraging the three technology areas listed above without even breaking into a sweat.
What are your experiences of keeping yourself in the best shape you can?
Amazon’s rumored tablet possibly due out this year will come with free streaming video service and be priced at around $399, according to a report from investment firm Detwiler Fenton.
“It is particularly interesting to note that AMZN is expected to include its movie service for free for an unspecified amount of time to buyers of the device,” states the report, spotted Tuesday by CNET. “This is the same movie service AMZN already offers for free to its Prime customers.”
The Amazon tablet will reportedly feature a 10-inch color screen and “have a more robust applications processor” than the chip expected to power color Kindle ereaders that Amazon will also reportedly launch by the end of 2011, according to Detwiler Fenton.
The Detwiler Fenton report also corroborates some details reported earlier about the rumored Amazon tablet. It’s said to be due out before the end of the year and is codenamed Hollywood.
Reports from Taiwan in early May indicated that Amazon is ordering enough tablet components from Taiwanese manufacturers to assemble up to 800,000 units a month and plans to slash prices for its Kindles when it releases the tablet sometime in the second half of 2011.
Established tiers in the tablet market could shake out more fully in the coming months.
Detwiler Fenton expects low-priced color ereaders to fill a “low-end tablet niche” situated well below the most expensive devices, particularly Apple’s iPad. In the middle will be sub-$400 tablets like the rumored Hollywood tablet, the investment house predicted, as pricing pressure builds on companies like Motorola and Research in Motion which make tablets that are similar in price to Apple’s but not nearly as popular.
Welcome to this morning’s edition of “First To Know,” a series in which we keep you in the know on what’s happening in the digital world. We’re keeping our eyes on four particular stories of interest today.
Apple Unveils iCloud, iOS 5 & More
At its developer conference Monday, Apple announced its much-anticipated iCloud service; that the next version of Mac OS X, Lion, will be out in July for $29.99; that its next-generation mobile software, iOS 5, will be deeply integrated with Twitter, offer wireless syncing, its own messaging service and Newsstand, and more.
Rep. Weiner Admits to Tweeting Lewd Picture
Admitting that he had “not been honest with myself, my family and my constituents,” Rep. Anthony Weiner confessed at a press conference Monday that he sent a lewd photo via Twitter and that he had since lied about his account being hacked.
HTC Reports Strong May Sales
HTC says it generated $1.42 billion in sales in May, which is more than double than the same month in 2010 and a solid improvement from April’s $1.35 billion.
Xbox Live Getting Live TV, YouTube & Voice Search
Microsoft unveiled a redesigned version of Xbox Live at E3 Monday, one that includes more voice commands, YouTube integration and live television.
* Tuesday is World IPv6 Day, and Google is leading the charge to test and adopt the new Internet Protocol.
* As expected, former CBS Evening News anchor Kate Couric has signed a multiyear contract with Disney/ABC Television Group to host her own nationally syndicated talk show and join the ABC News team.
* Sony’s next-generation handheld gaming system, the PlayStation Vita, made its official debut at E3 Monday.
* Microsoft has released a trailer for Halo 4.
* Note-taking platform Evernote now has more than 10 million registered users, up 67% from January.
Call it Cloudgate, Cloudpocalyse or whatever you’d like, but the extended collapse of Amazon Elastic Cloud Compute (EC2) is both a setback for cloud computing and an opportunity for us to figure out how to stop it from happening again.
Amazon may be best-known for its online shopping site, but it also has a substantial cloud computing business. It provides a scalable, flexible and particularly efficient solution for companies to store and deliver massive amounts of content. Its model of only paying for what you consume was a radical innovation when it launched in 2006.
In fact, Amazon Web Services has been so affordable and reliable that thousands of companies from Foursquare to Netflix utilize the company’s cloud computing technology and servers to run their businesses. They put their faith in Amazon’s cloud because there was no reason to think that it would falter. One of cloud computing’s key tenets is reliability through redundancy of both servers and data centers.
Then on Wednesday, Amazon’s northern Virginia data center started experiencing problems that caused major latency and connectivity issues. The trouble was apparently due to excessive re-mirroring of its Elastic Block Storage (EBS) volumes — this essentially created countless new backups of the EBS volumes that took up Amazon’s storage capacity and triggered a cascading effect that caused downtime on hundreds (or more likely thousands) of websites for almost 24 hours.
The collapse took its share of victims. Among the most prominent companies affected were Foursquare, Quora, Hootsuite, SCVNGR, Heroku, Reddit and Wildfire, though hundreds of other companies big and small were affected. Luckily, one of Amazon’s most prominent customers, Netflix, didn’t experience problems because it’s built for the loss of an entire data center, while companies relying on Amazon’s four other global data centers didn’t experience too many issues.
A Learning Moment
FathomDB founder Justin Santa Barbara has a detailed post on his blog about what may be the biggest problem to come out of this week’s collapse: Amazon’s cloud redundancies failed to stop a mass outage. Its Availability Zones are supposed to be able to fail independently without bringing the whole system down. Instead, there was a single point of failure that shouldn’t have been there.
This week’s disaster in the cloud is a reminder to startups to build redundancy into their applications and their own systems, but as Santa Barbara points out, most startups don’t have the time or resources to engineer for multiple cloud systems (each Amazon global region/data center has its own rules and features, making a simple “switch” to another center difficult). These companies trusted Amazon to keep them online, and Amazon failed to deliver.
Catastrophic issues will always occur, but in the pre-cloud era, downtime only affected a single computer or website. Today, a catastrophic event takes down thousands of websites, causing millions or even billions of dollars in lost revenue and productivity.
This incident is no reason for us to shun cloud computing, though. Its benefits (scalability, cost reduction, device independence, performance and more) far outweigh its cons. We do need to take a hard look at how we structure our cloud infrastructure though and find new ways to either prevent single points of failure or quickly move content off failing clouds faster, especially as the world’s computing power is consolidated into fewer and fewer systems.
Cloud computing is still in its infancy, and today’s events make it clear that we still have a lot of work to do. It could be a whole lot worse next time if we aren’t prepared.