NOTE: The views expressed below are my own and do not, in any way, represent the views of my employer.
The week before last I attended the OpenStack Summit in Portland, Oregon. It was a chance for me to get the latest information about one of the most exciting open-source projects for the past few years. I’ve been involved with OpenStack for quite a few months now, mainly deploying environments and writing documentation to aid in my understanding as well as providing it out for others to consume; one of the problems with OpenStack is that it’s very difficult to get started. Whilst I knew that OpenStack had an enormous market-hype with the vast majority of ISVs and hardware vendors jumping on-board, nothing could prepare me for the overwhelming turnout at the summit; the event had actually sold-out, which for an Open Source conference is an achievement. The majority of the sessions were actually overflowing, if you didn’t turn up 20-30 minutes beforehand then you had little chance of getting somewhere to sit. It just goes to show the level of interest in this project, people from all over the world and of all career paths attended as they were either actively involved with OpenStack in some way or knew they had to learn more; the event had a very refreshing buzz about it, people wanted to be there, were passionate about the technology and could really see it going somewhere.
The conference, with the exception of the daily keynotes, was split up into multiple tracks, for the active contributors/developers they had the design summit, but for those of us that wanted to gain an insight into the latest and greatest we found ourselves sitting through presentations covering the widest variety of topics. They also provided hands-on workshops all week, something that I found extremely valuable- for example, as I’m not a networking guy I sometimes find myself confused over complex networking and the concept of virtual networks, the hands-on Quantum lab enabled me to gain a good understanding of it. Even beginners were catered for, there were 101 sessions most days allowing people with very little experience of virtualisation and cloud computing to come away with an understanding of the OpenStack project and where it is headed. As a Red Hat employee myself, putting faces to the names of the colleagues I work with daily was a great opportunity, especially given our recent OpenStack distribution announcement; RDO, our community-supported offering (http://openstack.redhat.com). The ability to network with other OpenStack users (and potential future users) was extremely valuable, receiving feedback about what they wanted to use it for and what features they really wanted to see. In fact, when you look at the attendee list it goes to show the variety of attendees; it seemed to be a mix between the stereotypical LinuxCon attendees and VMworld attendees, a very dynamic environment.
The keynote sessions were extremely useful, the Rackspace keynote being the headline for many was, as expected, really good. The statistic which they keep using is that whilst they’re not reducing the amount of contributions they make, their overall commit percentage continues to decrease; clearly proving the success of the project in the open-source community. It was also good to see that Red Hat is out on top now with the latest Grizzly release and they’re not trying to keep that quiet! Rackspace is using OpenStack in production (no surprises there!) but the way in which they use it is very interesting, they deploy OpenStack on OpenStack for providing “private clouds” on-top of the public cloud, eating their own dog-food at every level. They’ve done this with an extremely high level of reliability and API uptime, so when people say that OpenStack isn’t ready for production, I really do beg to differ. Canonical’s keynote by Mark Shuttleworth was very good too, I think they’re at a set of crossroads, moving from the traditional ‘desktop environment’ to a more strategic cloud play, I’m just not sure they’ve got the resources to actually fulfill what they try and say, despite the fact that OpenStack has been predominantly written to run on Ubuntu (historically, anyway). Aside from the OpenStack “vendors”, organisations such as Best Buy, Bloomberg, Comcast and others presented what they use OpenStack for, how they’ve implemented it at scale and what they’ve learned (and contributed back!), it further goes to show that there’s real interest in many different areas if the use case is right, i.e. scale out, fault tolerant applications.
What interests me is that OpenStack is clearly viewed by many as a threat, you take VMware for example; they are actively contributing to the project to enable their ESXi hypervisor as a compute resource for OpenStack Nova, they’re concerned that when people see the benefits of OpenStack and the fact that it abstracts much of the underlying compute resource, the requirement for ESX will drop. VMware are at risk of becoming irrelevant in the long term where applications are written in different ways, i.e. to be more fault tolerant and not requiring hypervisor-oriented technologies such as HA, they have to try and retain a piece of the pie and leverage the existing investments that organisations have made on their technology. In addition, the next-step of the virtualisation piece is virtual-networking; Quantum provides a virtual network abstraction service with multiple plugins for software and hardware based networking stacks. VMware is active in this space also, providing their Nicira-based NVP plugin for Quantum, this is an emerging technology and will likely be the later piece of the puzzle that gets adopted. VMware and Canonical have just gone to market with a fully-supported offering, this joint venture provides a complete OpenStack environment for customers currently using VMware. As VMware didn’t previously have an operating-system, Ubuntu is able to step in and plug this gap, a potentially strategic opportunity for both organisations.
It’s not just VMware either, the likes of HP, Dell, IBM are all jumping on the OpenStack bandwagon, all with sets of developers contributing upstream as well as to their own product offerings, seeing OpenStack as a way of making money. Many vendors are providing additional components that allow OpenStack to integrate directly into existing environments, bridging the gap between upstream vanilla OpenStack and the modern-day datacenter. The VMware/Canonical offering is too early to tell whether it will be a success or not, but in my opinion they are doing the right thing, I am a firm believer that any vendor looking to provide an OpenStack offering should be open about the partnerships they forge and the additional fringe components that they choose to support; because of the vast support model in the OpenStack trunk it will attract a wide variety of customers with a disprate set of requirements, picking and choosing technologies to support will be extremely important.
OpenStack has opened the doors to a wide variety of new-startups, all providing additional layers or extensions to help integrate and ease the adoption of the product into organisations. Examples of these include Mirantis and SwiftStack. Mirantis is a company based out of Russia, they, more than anyone, impressed me at the summit. They’ve written a tool called Fuel which aims to provide a ground-up management platform for OpenStack; currently to implement OpenStack requires quite a lot of time, knowledge and experience, Fuel is able to configure an array of the underlying bare-metal technologies including networking and storage as well as to provision and manage entire OpenStack environments from a web-interface. If I was in a position to acquire a company, Mirantis would be at the top of my list right now! SwiftStack provide management tools and support for deploying a Swift-based object-storage cloud, their presentations were fascinating and for anyone interested in how Swift works then they’ve written a free book (http://www.swiftstack.com/book/) which I highly recommend. The exhibition room was full of vendors promoting either their distributions, their consultancy/architecture services or their additional add-on components, very rare for an open-source project… even at LinuxCon it’s usually full of the normal open-source companies plus hardware guys, this represented a huge mix.
What last week made me realise is that there’s an enormous opportunity for OpenStack, a lot of the community work has been done for the vendors wishing to pursue an OpenStack strategy, some vendors being in better positions than others to make it a successful venture; integration with existing enterprise technology will be extremely important for vendors to get right. There’s a lot of overlap between the capabilities of some existing products in the industry, however OpenStack, in my opinion, is addressing the problem to the next-generation architectures. There are lots of offerings/flavours/distributions out there, the most important thing about OpenStack-based clouds is interoperability… they must continue to be open, i.e. open API’s, open standards to allow portability between clouds. Whilst I fear that some organisations (especially proprietary vendors) are getting involved in OpenStack because of the hype, it represents a turning point in the way that big corporations think- it’s yet again proving the power of Open Source and what can be achieved when we work together.
Long term, there are many areas in which we can improve OpenStack, there aren’t many organisations out there that are ready to implement an OpenStack environment unless they go greenfield with it; integration is key but the switch from traditional data centers to a fully software-defined environment is a big step to take and this step will take years to fully embrace. It doesn’t mean that traditional enterprise virtualisation will go away either, there will always be a requirement for legacy applications but the convergence of these technologies will be an intriguing concept to watch. Over time I think that OpenStack will become a lot more than just a set of tools to build cloud environments; we see this with the latest tool sets such as the Heat API and nova-baremetal, tools that are implementing features that have been traditionally excluded. It’s an exciting time to be involved with OpenStack, I look forward to seeing what the future can bring for the platform and making it a success.
As I attended about 10 sessions per day (usually 30-40 minutes each), I won’t comment on them all, but some personal favourites to recommend people look into if they’re interested in learning more about what’s coming up in OpenStack:
Orchestration of Fibre Channel in Cinder – The default out of the box block storage configuration in OpenStack is typically iSCSI, whilst there’s plenty of additional options now Fibre Channel support has taken quite some time to make it into the code. The problems are typically around zoning, as far as Nova is concerned this is almost irrelevant as all it has to do is attach the underlying disk to the instance in the same fashion as iSCSI. This technology brings OpenStack closer to enterprise adoption. (http://www.openstack.org/summit/portland-2013/session-videos/presentation/orchestration-of-fibre-channel-technologies-for-private-cloud-deployments)
Ceilometer Metrics -> Metering – So, Ceilometer is a new project within OpenStack, it was introduced in Folsom as an incubated project but has made it into Grizzly as a full component. It enables organisations to implement a flexible chargeback model on pretty much anything they want to plug it into. It vastly extends the very basic quotas and utilisation that Folsom used to provide. There are still some limitations but it’s extremely powerful. (http://www.openstack.org/summit/portland-2013/session-videos/presentation/ceilometer-from-metering-to-metrics)
OpenStack HA with Mirantis – This talk actually turned into a product demonstration/pitch, but it was the one I was most impressed by. This demonstrates Mirantis’ Fuel implementation for managing OpenStack environments from bare-metal. Many organisations keep asking “how can we deploy OpenStack to scale?” or “How can we make OpenStack Highly Available?”. Mirantis attempts to solve the deployment and high availability problems with their tools and ammusingly they say it’s so easy, even a goat can do it! (http://www.openstack.org/summit/portland-2013/session-videos/presentation/standup-ha-openstack-with-open-puppet-manifests-in-under-20-minutes-for-goat)
Deploying and Managing OpenStack with Heat – This talk discusses how the “triple-o” or “OpenStack-on-OpenStack” project uses the Heat API (and nova-baremetal) to deploy entire OpenStack environments automatically, blurring the differences between the cloud layer and the physical world. (http://www.openstack.org/summit/portland-2013/session-videos/presentation/deploying-and-managing-openstack-with-heat)
Software Defined Networking (Scaling in the Cloud) – One of the things I mentioned earlier was not being a networking guy, these sorts of presentations helped me understand how things fit in, where things were going and how virtual networking was solving real world problems with scale. Gone are the days where we provide L2 bridges (+ VLAN tagging) to virtual machines, in the world of software defined networking we can remove a lot of underlying complexity and control it all in software. This is an area that will become extremely important in the future. (http://www.openstack.org/summit/portland-2013/session-videos/presentation/scaling-in-the-cloud-the-hype-and-happenings-of-software-defined-networking)
Note, ALL of the Summit videos are freely available online at: http://www.openstack.org/summit/portland-2013/session-videos/