Can OpenStack Be Saved From Itself?

OpenStack has a broad and growing community driving its development. Which may, ironically, be its biggest problem. At least, that’s what I heard from a several industry experts last week at Amazon’s AWS re:Invent conference, and which former OpenStack leader Andrew Shafer suggests in a biting review of the current state of OpenStack. As Shafer (@littleidea) highlights, community is the heart of OpenStack’s biggest opportunity, and its biggest failings.

”A Mishmash Of Naive Ideas And Pipe Dreams”

One thing is clear: OpenStack has a great deal of community traction. OpenStack events now regularly attract thousands of attendees, and its code base, once dominated by Rackspace, now reveals a broad developer community.

Broad, yes. But perhaps not altogether healthy.

As I’ve noted before, the best open-source projects are those that are heavily modularized, allowing disparate contributors to focus on a discrete corner of the overall project. Modularity is what makes Linux work so well, and OpenOffice so poorly: the former invites developers to tackle individual sub-projects whereas the latter essentially imposes a duty to understand and tackle the entire system.

Not surprisingly, Linux has flourished while OpenOffice has foundered in attracting developer interest.

Even in a highly modular open-source project, however, it’s common to see inconsistent quality of sub-projects. Sadly, though community momentum continues to propel OpenStack, OpenStack may attract consistently bad contributions, according to Shafer, with the Ceilometer monitoring component baring OpenStack’s general faults:

Ceilometer is a tragedy masquerading as a farce. In my opinion, this project should not exist and as it exists should not be relied upon for anything, much less billing customers…

Ceilometer’s implementation is such a mishmash of naive ideas and pipe dreams without regard for corner cases and failure scenarios, that Ceilometer’s association with OpenStack should be seen as a negative and graduation of the project calls into question the literal foundation of OpenStack decision making. Ceilometer’s quality is bad, even by OpenStack standards.

And yet Ceilometer was promoted within the OpenStack community. This despite acknowledge problems. This despite a faulty, “bolted-on” approach.

Politicking Away OpenStack’s Future?

The reason, Shafer believes, is politics. Or, rather, the creation of the Project Technical Lead (PTL) role, and associated jockeying for the prestige of being a PTL.

The dynamics of the perceived prestige of a PTL supersede[] other considerations. This dynamic allowed for a splintering of vision and mandate. If there has been any one thing that I have seen waste time implementing and operating OpenStack, that would have to be trying to coax the disparate OpenStack services, often from the same ‘release’, to work together.

As for remediation, Shafer despairs of things getting any better:

Not that OpenStack doesn’t work, or at least that it can’t be made to, given certain competence and constraints, but that OpenStack doesn’t have the coherence or the will to do more than compromise itself for politics and vanity metrics.

Importantly, unlike projects like Linux that impose discipline through a “benevolent dictator” like Linus Torvalds or a small, core group of developers, OpenStack may be taking “community” too seriously and trying to command by committee. It just doesn’t work that way, which is why I’ve argued OpenStack’s community might be better served by welcoming a hegemon like Red Hat.

Focus On Users, Not Vendors

While Shafter doesn’t go this far, he does have some ideas as to how to improve OpenStack:

• Focus on users, not vendors. If there can’t be a benevolent dictator, there has to be an overriding conscience. The project setup solves vendor political problems more than user software problems.
• PTLs end up being trophies. Electing PTLs every cycle is distracting and impacts continuity. Everything about this hurts OpenStack….
• Define a core and stick to it, or acknowledge that your governance is broken and make OpenStack an explicit free for all.
• Stop declaring victory with vanity metrics. Divide OpenStack google trends number by the number of marketing dollars spent. The total number of committers is less meaningful if OpenStack is an excuse to sponsor parties around a collection of disparate projects.
• Make ‘OpenStack’ brand mean something to users. Stewardship is greater than governance. If OpenStack is just a trough for the old guard IT vendors to feed slop to their existing enterprise buyers without regard to the technology or user experience then OpenStack has completely lost the plot.

All is not lost at OpenStack. Given the tremendous interest in and developer focus on OpenStack, the project is more likely to need same-day surgery than a full-frontal lobotomy. Despite its quality concerns, enterprises like PayPal have been betting big on its present and future. This will undoubtedly continue.

But whether it will lead OpenStack toward a Linux-like future is still very much TBD. Someone within the OpenStack foundation needs to take ownership of the project’s vision and inconsistent quality. That “someone” could be Red Hat, now the dominant contributor to OpenStack, or it could be a smaller candidate like Mirantis. But someone, not someones, needs to take charge.