Wednesday Nov 19, 2008

Response: Comment on "Save money with Open Storage"

This is in response to a comment on the previous post with regards to the benchmark demonstrating no penalty hit for using a NAS storage device like AmberRoad instead of a LocalFS for a Web 2.0 application. The comment stated:
"I respectfully disagree with your comment that there is no performance penalty  
with NFS...You show a 12% increase in Processing utilization which is an        
overall performance hit.  You are doing approx. the same amount of work 
but chewing up more CPU...while your users are still the same, if scaled to 100%    
Util, NFS wouldn't allow as many users as LocalFS, because you are turning Blocks 
into packets into blocks, which is SLOW."
As per the subject matter experts at Sun, this is a fairly common argument in regards to idle %. We don't know if it's wrong in this case or not - but idle is not always an indicator of future performance or perceived headroom, it is wrong to assume so.  The reality is you don't know how much more you can get from your system as configured unless you push it to do so.  % idle is  a wrong metric to focus on (very common).  The metric that matters in this case is supporting concurrent users with response times for all transactions falling within the guidelines.  The NAS solution does this just fine for less $$.  Also as per detailed data here, the io latency reported by iostat is the same for both DAS and NAS - therefore the statement that NFS is slow is not backed up by evidence.

To the question if NFS io ultimately cost more?  Sure - but the missing point here is that it doesn't matter with this test.  Might it hurt us down the road pushing to 100%?  Maybe - but with a cheaper solution, and load scaled to 75% of the DAS solution, user response time was fine (this is what matters).  And to use the analogy of idle, with over 25% cpu free, we believe it could scale another 800 users, equaling the DAS result. Once we arrange for hardware (better network and drivers) to push the load beyond 2400 users, we would know for sure. Pl. stay tuned.

Lastly, do most customer environments really run at 100%?  Will the purported "NAS penalty" really affect them?  This shows that even if they run at 75% (pretty high for most environments I've seen), it's not an issue. HTH.

Tuesday Nov 18, 2008

Save money with Open Storage

Open Storage helps you save time and money for Web-scale Applications.

Want Proof?

Check this excellent  benchmark for Web 2.0 run on Sun Storage 7410 (aka AmberRoad) and CMT technology based Sun Fire T5120 servers.

Its also evident from the benchmark data that you don't suffer a performance penalty for using NAS.  There is a fairly common impression that performance could/would be slower than DAS, this shows that it's just not true with this environment.

System Processors Results
Storage Ch, Cr, Th GHz Type users Util RU watts / user users /RU
Sun Fire T5120 Sun Storage 7410 Unified storage array (NFS) 1, 8, 64 1.4 UltraSPARC T2 2,400 72% 1 0.20 2,400
Sun Fire T5120 LocalFS 1, 8, 64 1.4 UltraSPARC T2 2,400 60% 1 0.20

2,400

Sunday Nov 09, 2008

EUCALYPTUS : Open Source Cloud Infrastructure, The Skies are Opening!

Last week I attended a fantastic talk on EUCALYPTUS at a cloud computing meetup.  The presentor Rich Wolski, is a a professor in the Computer Science Department at the University of California, Santa Barbara.  He created EUCALYPTUS, an open source cloud computing implementation, that is interface compatible with Amazon EC2.  It was a great educational talk, very relevant to the budding cloud computing industry. Rich Wolski put in perspective the popularity of Cloud Computing when he mentioned that the term "Cloud Computing" was only coined about a year ago, by Google on Oct. 8 2007 in a press release.  Today after 1 year and 1 month, a google search on "cloud computing" gives ~9 million results! This explosive growth in cloud computing got Prof. Rich Wolski interested in the subject for research and gave birth to the project EUCALYPTUS.

EUCALYPTUS is an acronym and expands to: Elastic Utility Computing Architecture Linking Your Programs To Useful Systems. The infrastructure is designed to support multiple client-side interfaces besides EC2. It is implemented using commonly-available Linux tools and basic Web-service technologies making it easy to install and maintain.

The fig. below illustrates the EUCALYPTUS architecture.

 


The Cloud Controller implements all the gory details of the Cloud backend provisioning, whereas the Client side API Translator emulates a Cloud interface like EC2. This design makes EUCALYPTUS modular and extensible to emulate clouds other than EC2. Eg., EUCALYPTUS plans to emulate the Google Apps Engine in the near future by adding yet another API Translation layer. In its current version available today, EUCALYPTUS translator is built to EC2 WSDL published by Amazon and is 100% interface compatible with EC2. When RightScale management and monitoring tools connected with EUCALYPTUS, they were not able to identify any difference between EC2 and EUCALYPTUS.

Security and authentication mechanism is very similar to EC2 except for its without a credit card. User signup is web based and ssh key generation and installation is implemented just like EC2. Since there are no published administration and accounting tools published by EC2, EUCALYPTUS defines its own tools for user management and cloud management.

 When EUCALYPTUS project was launched, the objective was to keep it simple, extensible, easy to install and maintain, and build it on widely available and popular open source technologies. Another objective was to ensure that it is a cloud indeed, given there has been a lot of confusion about what a Cloud really is or not is. To ensure that, the team decided to emulate an existing cloud and made the following design decisions:

  • EUCALYPTUS would be interface compatible with Amazon EC2 and S3
  • It  would work with command line tools directly from Amazon without any modifications
  • It would leverage exisitng EC2 value added services like RightScale

Given EUCALYPTUS was an open source project and would need to run on any hardware without prior knowledge of the underlying infrastructure, it was also designed  to function as a software overlay such that existing installation is not violated too much and no assumptions are made about the hardware.

Some of the Goals of EUCALYPTUS were:

  • Foster research in elastic/cloud/utility computing
  • Experimental vehicle prior to buying commercial services from EC2 and other clouds
  • Providing a debugging and development platform for EC2 and other clouds
  • Provide a basic cloud platform for the open source community. Might evolve into a Linux experience..
  • Not designed as a replacement technology for EC2 or other commercial cloud services. In its current form, it can scale upto 1000 nodes.

Some of the biggest challenges addressed by Rich Wolski and his team of 5 research students around building EUCALYPTUS were:

  • Extensibility
  • Client side interface (modular design so that its compatible with EC2 and other clouds)
  • Networking
  • Security
  • Packaging and Installation (One click install)

EUCALYPTUS is hosted as a public cloud and its free for use. However, only installed images can be run and usage is limited to 6 hours. EPC (EUCALYPTUS Public Cloud) configuration consists of:

  • 8 Pentium Xeon processors (3.2 GHz)
  • 2.5 GB of memory per image
  • 3.6GB disk space
  • 1GB ethernet interconnect
  • Linux 2.6.18-xen-3.1
  • Xen 3.2

Yes, its as big as an electron in the EC2 cloud. So clearly, even though its not a replacement for a commercial cloud, cloud vendors could learn a lot from its implementation if they wish to build their own cloud. Developers and end users could use it for testing and debugging purposes before deploying it on a real cloud. Given the popularity of cloud Computing, it could be the next Linux experience! Who knows.

IMHO, its a fabulous piece of work done by a team of 7 engineers at UCSB, using open source technologies, working with a limited budget in a duration of about 6-8 months. Check it out!!

<script type="text/javascript">var addthis_pub="geekyjewel";</script> <script type="text/javascript" src="http://s7.addthis.com/js/152/addthis_widget.js"></script>

Wednesday Nov 05, 2008

Barack Obama and MySQL

It is widely acknowledged that Barack Obama put together the best minds and team together to run his campaign. One of the proof points is that his website www.barackobama.com was driven by MySQL! 

From Jonathan's blog:
http://blogs.sun.com/jonathan/

" On behalf of Sun Microsystems, I would like to offer my sincerest congratulations to President elect Barack Obama. What an extraordinary accomplishment.

I would also like to extend my congratulations to his web team for having chosen MySQL as the platform behind their election web site, BarackObama.com. "


Open Social Networks built on Open Identity and Open APIs

This has been an year of Application Platforms and we have seen a number of Social Networks and XaaS flourish. Facebook, MySpace, Hi5, Friendster, Orkut, Google Apps Engine, Y!OS, EC2 to name a few. Everything is syndicated and social is everywhere. Its the people who are the center of the services, not the software. There is a seemingly infinite demand for applications, almost difficult to deliver. Social computing is becoming a pillar of mass culture. Software is increasingly mediating between people and Apps is a way of life.

However, its also evident that in the current world, these social networks and platforms live alone and isolated from each other and there is no easy way to maneuver seamlessly between different networks.  This limitation is giving birth to Identity platforms wherein every user has an Open Identity and the social network vendors are being encourged to implement their platforms on top of these open Identity standards. In the coming year, we are likely to see more "Open Social Networks" so as to lower the barrier to entry in a social network, ease user acquisition and make socializing a more vibrant experience.

An Open Identity is an entity built on open standards that would define some of the following user attributes:

  • My profile
  • What people say about me
  • Who my friends are
  • What is my content
  • Which sites I can go to
  • etc.

Basically this entity would define a user. If all platforms leverage the same standard identity platform, then it would be seamless for a user to enter new networks. This would enable the service providers to get valuable insights into new user base by allowing them to easily plugin to their networks using open identity.

Yahoo! , MySpace and other vendors in this space are preaching what they are calling "Open Stack" that would make user data portable and enable a user to carry their address books with them while signing on to a new network or platform. As an example, figure below illustrates the Y! Open Platform.

With the web becoming more open and social, standards are needed to power the social functionality without having to reinvent the wheel every time. It only makes sense to tap into the existing semantic information and participate in the echo system. Its important to understand this distributed landscape and plan intelligently so that one can best leverage the building blocks.

Wednesday Oct 29, 2008

My 2 cents on Microsoft Azure

Microsoft recently announded their Cloud offering called Azure. Azure is more of a PaaS Windows cloud, offering their proprietary closed Winows products  as a service. Seems to me like that traditional Software as a Service (Saas) model. Does not REALLY excite me! Marketing dollars well spent though in picking up the name Azure, meaning "clear cloudless sky". In addition, they have not yet announced anything around SLAs and price for the service, key for successful adoption of cloud based computing.

Yahoo announced yesterday their own PaaS Y!Open platform based on OpenSocial APIs that is geared more towards targeting Social networking audience unlike Azure. This is more interesting to me as they will immediately be able to capture new developers and expand on their current echosystem.

But yes, Azure is offering more than what Google Apps Engine offers today. To me, it would be interesting to see what M$ offers around industry standard APIs like OpenSocial which is how they can attract more of the Next-gen developers building mashup services around their PaaS. They seem to have endorsed OpenSocial anyways.

Facebook has gone up from 27 million users to 140 million users after opening up their Facebook API to developers in the last one year with 40 thousand third party applications hosted on facebook. Now thats revolution. Its all about expanding the echosystem of your developers in this era of internet computing and thus monopolising the market/making yourself indispensible. Its all about the open APIs, rather than spinning your own proprietary API clouds like Azure. 

Web-scale and Cloud Computing open APIs is almost like the Open Source model which we have all come to love and thrive on. Stay tuned on my next blog on Open Web Identity leading to Identity platforms and Open Social Networks.

Monday Oct 27, 2008

Open Solaris + MySQL + ZFS Success Story in Production at SmugMug

SmugMug , a photos and videos publishing site, goes into Production on Open Solaris + MySQL + ZFS. Check out this story on why a Linux geek decided to move his site from Linux To Open Solaris.

Don MacAskill, chief geek and CEO of SmugMug says"  ZFS is the most amazing filesystem I’ve ever come across. Integrated volume management. Copy-on-write. Transactional. End-to-end data integrity. On-the-fly corruption detection and repair. Robust checksums. No RAID-5 write hole. Snapshots. Clones (writable snapshots). Dynamic striping. Open source software. " He is also excited about the CoolStack 5.1 stack available in Open Solaris along with MySQL.

Full Story on SumgMug Powered by  Open Solaris  and  MySQL

http://smugmug.com 

<script type="text/javascript">var addthis_pub="geekyjewel";</script> <script type="text/javascript" src="http://s7.addthis.com/js/152/addthis_widget.js"></script>
About

alkagupta

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today