Introduction to the Apache® Ignite™ community structure

Introduction to the Apache® Ignite™ community structureAt a recent gathering of the Moscow Apache® Ignite™ Meetup, I talked about open-source communities, how to become a contributor and a committer and some reasons why you should participate. 

Specifically, I was talking about the story of the Apache Ignite community. The limited time of the talk did not allow me to give more examples, so that was the inspiration for this post -- which is by the way based on my personal experience and is not the official position of any company or organization.

What does an open-source "community" look like?

It may seem obvious, but let's clarify what I mean by a community. In this case, I'm talking about an online community -- one dedicated to the development of open source software.

Apache Foundation

The history of Apache began in 1999 with the HTTP web server. A group of people began to develop a web server, and then users started coming to them -- some of whom began to send patches -- because they wanted to improve something in this product or fix bugs. Developers gradually began to allocate the most active members of this community the rights to push to the repository, something which is now called "committer rights." 

Applying the same approach to the development of open-source software, Apache has now become the largest foundation that develops open-source software. The organization is divided into 319 (so far) diversified projects, of which about 200 are designated "Top-Level." There is no single process for all projects -- all participants are volunteers; their work is never paid for by the organization.

Apache requires all projects to follow:

  • Legal policy
  • Brand policy
  • Voting on releases
  • Usage of mailing lists
  • Information security

Apache Incubator

To build a community, all Apache projects must pass through the Apache Incubator. This is an intermediate stage of becoming not only a project, but also building a community around it. Nobody in Apache will dictate what technology to use. Moreover, they will not even prompt you as to what decision to make. The task of the Apache Incubator is to build a community that jointly makes decisions. It is essential that participants understand how and who decides where they can speak and where their voice will be heard. The ASF organization helps by attaching to the mentor project.

Top-Level Apache Project

If the project exits the Apache Incubator, it can then become a top-level project (TLP). Apache helps TLP projects participate in conferences, helps in brand promotion and supports the infrastructure.

The community of each top-level project guarantees users that:

  • The code can legally be used
  • High-quality code
  • Compliance with information security (for example, all releases are signed)
  • The project will always be available to users

Open source and decision-making

Now, let's talk about how decisions are made. This is not always obvious -- even for community members. In the Apache project, there are several roles and corresponding mailing lists:

As a participant, your role can grow within the Apache Software Foundation itself. You can also become a mentor of a project and help yet other projects build their own communities.

The higher the role, the fewer people are associated with it. The structure is an inverted pyramid with the point ending at the PMC. Usually, everyone is interested in helping all participants grow into a higher role.

Unlike commercial companies where staff and growth are limited by budgets, there is no such thing in the open source world. Nothing limits the number of committers or PMC members. The group regulates itself independently.

User

Users are all of us. For the community, it is important that the user not only uses or gives feedback in the form of bug reports or feature requests but also helps others. That is, from the point of view of the community, the participant becomes a user only when he or she subscribes and responds to the user-list and helps the others master the product and shares his or her knowledge.

Developer/Contributor

If the user is ready to contribute to the project, then with the first contribution he or she automatically becomes a contributor (and also a developer). The contributor participates in a mailing list for developers, which discusses everything related to contributions to the community. All the developers influence the decision-making community -- and all criticize the solutions and offer alternatives.

Committer

If the community believes that a person has made a sufficient contribution, then it can be granted the right to push the repository. That is to say, the minimum number of reviewers of its code is reduced to zero (although in Apache Ignite, the committers still go through the review of the code). The committers sign the ICLA Individual Contributor License Agreement ) license. The committer gets an email account with apache.org and can now make decisions related to each contribution to the project.

PMC Member

PMC (Project Management Committee) Member. This member of the community has already made a great number of contributions to the development of the product and now has the right to make long-term decisions on the development of the project. They also help members of the community come to a consensus. In Apache, PMC has a lot of responsibilities --- for example, they can vote for a release. The committer and the contributor can also vote, but unlike them, the PMC has a binding vote.

PMC Chair

PMC Chair is the connecting bridge with the Apache Software Foundation Board. The PMC Chair is tasked with resolving emerging problems as they pop up and reporting to the Apache Board about the status of the project.

Decision-Making: 'Do-ocracy'

At Apache, the principle of "do-ocracy" is dominant. The more you do, the more opportunities you have -- and the more you influence the project. If a coordinated position of all participants is required for some decision, a vote is taken. The voting window is open for a minimum of 72 hours to give everyone a chance to vote. 

Voters put: 

+1: "for the decision." 
0: "I do not care." 
-1: "against the decision." In this case, you need to offer something else and explain in detail why you voted against it. 

There are other types of vote fractions.

"Lazy Consensus"

Lazy Consensus means you can approve a solution if no one objects in 72 hours. Lazy consensus is simply an announcement of 'silence gives assent.'. You don't have to insist that people discuss and/or approve your plan, and you certainly don't need to call for a vote to get approval. You just assume that you have the community's support -- unless someone says otherwise.

Approval by majority and approval by consensus

Voting for a release is more of an all-hands-on-deck process. In this case, the approval of the majority of community members is needed, along with three binding-votes (from PMC) and more votes "for" than "against". Consensus means there is no "-1" vote.

Veto

A qualified contributor, usually a PMC member, can veto. He or she vote "-1" for modifying the code and writes a detailed explanation. They cannot veto a release, but if a modification is very bad -- for example, it creates a potential security hole or could greatly degrade performance -- then the PMC can vote "-1".

Meritocracy

Literally, "meritocracy" means "power of the worthy." In open source, this means that if the user (as the group believes) has done enough for the community, then they are raised to the committer status. This is an exclusively subjective decision of all PMCs in this community. In large communities, like Apache Ignite, this policy can be more or less formalized.

Open source and money

This important topic usually pops up eventually: how can you make money with Apache Software Foundation products? The organization provides a very commercial-friendly license to organizations. You can sell products based on Apache or support for Apache products, and you can use them in commercial products. An essential requirement: there should always be an option for the free use of Apache products, too.  

A committer can receive money from a third-party organization for contributing to the project. A committer can also be hired by a third-party organization. But the Apache Foundation never pays committers. This is a principled position. For Apache, a committer is always a volunteer and a private person. That is, a company cannot participate in Apache projects -- only individual developers can.

Why and how to join this open-source community? Why is it worth my time contributing to an open-source project?

Participating in an open-source project is an excellent opportunity to pump-up your skills and earn a professional reputation among your peers. Commercial companies also value participation in open-source projects. 

What prevents people from participating in open source? 

  • "Do you need to be a senior developer with 20+ years of experience?" No. Apache Ignite is a complex project, but even here you can find simple tasks. For example, fielding easy tickets and writing documentation that is designed to describe the project better. Nowhere in Apache does it say that to become a committer, you need to write code.
  • "I need be fluent in English. And I'm not." This is not the case. Those who participate in the dev and users lists can confirm: there is no need to master English. Also, communication is written and not in real time, so there is time to think and write a weighted answer.
  • "If I send my component to open source, I will never be able to use it again." In Apache, this is not the case. You can continue to use your component in a commercial product, and it just will exist within the Apache Foundation under the Apache license.
  • "It will consume all of my free time." This is partly true: participation in the community is addictive. If you start to participate in the discussions, it will take some of your time. And as you grow within the community, it will take more and more of your time. The more you know, the more you can help -- and the more you'll likely participate in the user and dev lists.

How to start

If you want to participate in Apache projects, you will need an account on Github, an email address, an account in Apache JIRA -- and also optionally one for the WikiAny community has an article for beginners, and the same is true for Apache Ignite: How To Contribute

It's a good idea to write a welcome letter once you do join. Nothing fancy. Something like: "Hello! I'm (your name). I want to help with ticket, and my nickname in JIRA is so-and-so .This message is important in terms of assigning the user with a contributor role.

Mailing lists

To interact with other participants, Apache requires the use of mailing lists. I sometimes hear people complain that mailing lists are way too old-fashioned. But they are used because each participant in an Apache project is a volunteer, it may be located in a different time-zone, and can't be online in a chat. Mailing lists work the best because they are simple and everyone has access to them.

Community is more important than code

One of the principles of Apache: the community is more important than code. The Apache project is aimed at creating open-source community --  after that, the magic begins with the code.

What to 'contribute'?

Each community has a list of what it needs done. Ignite has a list of simple tickets. If you find a bug and you fixed it in your fork, then you can create a JIRA issue for this case and share it with the dev list.

Talk about the project

The community is very much helped by blog posts and articles about the product. For example, you can also write about some exciting use-cases, student projects, etc.  And your experiences are always welcomed on the dev and/or user lists. Don't be shy!