Software Development Procedure

This Wiki is read-only for reference purposes to avoid broken links.

This document is in process to update ...

STATUS: This concept was accepted by the citizen by votation on 29th of November 2010.

This is a suggestion to replace the PMC Head Concept (http://www.adempiere.com/index.php/PMC_Head_Concept).

Preface

For quite a while, I have been participating in the ADempiere project as a committer. Throughout this time the group of developers has experienced the increasing complexity of managing this project as a mission-critical software development endeavour.

This process has been occasionally not only complex but also tedious, since the philosophy of our project is based on the Bazaar approach, that expects many contributors with different profiles, experience and ways of thinking. So the big question is: how to manage the project with all these contributors?

On the one hand we have contributors (aka committers) with great experience in the project. They want to and often do contribute with new features, generating a dynamic line of development. Many of these new features can be brought to our committee for approval and inclusion in the main development branch.

This process is very important because it lets us review and evaluate new features by experts both technical and functional.

Another important aspect is the incredible ability of contributors to create new features, usually beginning with a creative idea that evolves and matures. After this has happened, the contributor wishes to show his deeds to others to receive feedback and eventually to submit for trial.

The new features can be fully completed or in progress, but it is important to establish a policy of sharing functionality in order to evaluate the pros and cons of the feature.

The process nowadays is complex because there aren't tools and procedures enabling dynamic collaboration. As a consequence of that, the project may be or seem to be stalled or still worse, these new features are never included within the main trunk, limiting the creativity and innovation of the project.

On the other hand there are contributors who discover a problem/bug in the program. They create a solution to the problem in their private repository. In many cases, many of these bug fixes do not land in the main line of code.

Why? In my opinion, it is because of the lack of tools and of a process that explains how to share, how to facilitate collaboration, reviews and evaluation of bug fixes in a simple way.

Another aspect of our software development concerns the release of new versions that include bug fixes as well as new features are approved by the project committees. In the current approach, the main line of code must be frozen during the release process. This simplifies the process for the technical equipment, but has the disadvantage of stopping the momentum of a Bazaar-like project like ours, which is in constant evolution.

The release process also involves steps towards maintaining already released versions. These versions are not fault-free and so it is possible to find errors anew. Thus the versioning process is an additional task for the technical team, and using a tool to facilitate the creation of a service pack becomes a key factor.

Once the major issues were pinpointed, the technical committee analyzed and improved the current development process. As a result of this, changes were made in the current procedure in order to solve the current problems and thus improve collaboration in the project.

kind regards

Victor Perez

Member of Technical Team

Goals

This document is about establishing processes, rules and teams to achieve the following:

• Produce high quality ERP software
• Regular and schedule releases
• An all the time stable “master” branch
• Explicit and concise rules how to manage the ADempiere Code Repository
• Reduce overhead caused by reverts for unfinished features
• Documentation for all new features
• Teams are to be responsible not single persons
• Lean project organisation
• Continuous and reliable progress
• Increase collaboration among developers
• Facilitate the task of project management

Glossary

Here, a couple of important definitions are introduced shortly to help the understanding of the processes, but explained in detail below:

• Main branch
  • master Branch (before: stable)
    • Only for release and hotfixes approved
    • An infinite lifetime
  • development Branch (before: development)
    • Main line of code
    • An infinite lifetime
    • Never freezes.
    • Gets voted and approved commits from features branches.
    • No experiments
• Support Branches (before directly in trunk)
  • • Release
    • Feature
    • Hot Fixes
      • These branches always have a limited life time
      • Since they will be removed eventually.

Software Production Cycle

Development Phase (3 Months)

• “development” branch which is the main line of development.
• New functionalities are implemented in separate “feature” branches with the aim to get them into development branch
  • Developers have 3 months time for integrating new functionalities, changes, Bug fixing and Integration testing in their branch
  • Before integrating new/ changed Functionality in "development" the Developers/ Implementers have to apply for approval by Technical and Functional Team

Stabilization Phase (1 month)

• After development phase is over there is one month stabilization phase where the “release” branch is created and frozen for new features. In this time only bug fixing and User Acceptance Tests (UAT) are allowed.

Release

• When “release” branch is finished, it is integrated back into the development and master branches, which are the main line of development. Eventually, a release is done using a tag.
• The next stabilization branch cycle starts. The old branch is close.

Developer working in Development Branch

Responsibilities

• Assure that code has 100% compliance with Adempiere Best Practices (ABP)
• Code is formatted using the official formatting definition (need to be defined and agreed on, but in meantime you can take this as an example)
• Respect Mercurial Best Practices (see [1])
• Announce any data model change in order to discuss and vote it in Technical and/or Functional team (example: the placed were we store some well known data like contact phone changed)
• Announce structural changes (e.g. file, directory, rename, move, delete) should be announced and approved by Technical team before applying it
• Keep Master Data Models (organizations, business partners, contacts, warehouses etc) consistent and do not use different ways of defining the relations between this entities (examples: storing the organization details like addresses, phones, faxes in Organization Info tab or in the linked business partner)
• In case of complex functionalities, define ADempiere Generic Workflows in order to guide the user how to setup the functionality and which is the designed the business process (example: take a look at Manufacturing Management Setup workflow from your ADempiere main menu)
• Provide necessary documentation:
  • Forum post announcing the development in forum
  • Wiki page (template and content to be defined)
  • Feature request in tracker
  • Entry in Feature Development Collaboration wiki page
• Keep his/her branch up-to-date with “master” branch and therefore must merge hotfixes into his/her branch.
• Provide immediate effort if the integration harms the "master" branch.
• Delete "dead" or "unmaintained" branches created in the past.

Process

• Every Developer can open an own “feature” branch. The branch can be taken from ”master” or "development" branch. The developer who opened the branch is the maintainer of the branch.
• The Maintainer can prepare the reintegration of his branch into "development" branch by informing the Release Manager, Technical and Functional teams. This shall be done as early as possible, so that Release Manager can oversee the complexity of new Implementations and can give early consultancy about when to integrate.
• Once the maintainer gets the approval of Technical and Functional team he can merge his branch into “development” branch
• After integration into "release" branch Integration testing is performed

Technical Team

Responsibilities

• Maintains “master” and “development” branch
• Ensures that all contributions meet technical requirements in respect to quality:
  • Code is well documented
  • Backward compatibility is achieved
  • Migration scripts complete
  • technical documentation sufficient
  • backward comparability analysis
  • migration scripts are provided (when needed)
• Regular releases by providing and supporting a release manager from amongst their members

Note: More detail in main branches section

Process

• When a feature in a development branch is ready for approval the developer contacts the technical team and applies for approval
• Technical Team meets on a regular basis and discusses the applications
• If all criteria are met the team approves the feature from a technical perspective and allows the developer to merge his feature into stabilization branch

Initial Staff

• Teo Sarca
• Karsten Thiemann
• Tobias Schöneberg
• Trifon Trifonov
• Victor Perez (Maintaining Libero MFG and HR only)
• <please add your name when you want to join the team (4 hours minimum work per week)>

Release Manager

Responsibilities

• organize contributions to keep release schedule
• maintains release tags

Process

• creates stabilization from trunk when new development cycle begins
• produces a release from stabilization branch after stabilization phase

Functional team

Responsibilities

• ensures that all contributions meet functional requirements in respect to quality
  • requirement makes sense
  • feature is needed in trunk or to be kept as optional extension
  • requirement is solved
  • acceptance criteria are defined
  • test cases are complete
  • user documentation is sufficient
  • system documentation is sufficient

Process

• The Functional team has got the final decision on which feature is accepted and merged from development to stabilization branch.

Initial Staff

• Victor Perez
• Ramiro Vergara
• Mario Calderon
• Mark Ostermann
• Michael Judd (Accounting/BI/Processes)
• Paul Aviles
• <please add your name when you want to join the team (4 hours minimum work per week)>

Other teams

There are already other teams that could be integrated into this Integration Process where it makes sense. For example Usability, Security team.

Hotfixes

• In exception to the Software Production Cycle there is also the possibility to apply hotfixes
• Technical und functional team can approve hotfixes to go directly to trunk
• developers are not allowed to commit hotfixes to stabilization branch or trunk without prior approval

Reverting policy

• technical team has the responsibility to revert code that has been comitted without approval
• The revert should be motivated precisely and without any doubt.
• Any other person outside of the technical team (including the functional team) are not allowed to revert, but are advised to call for a revert where they specify the concrete reasons.

Team Policy

Voting

• In order to approve a contribution at least 2/3 of team members need to vote positively

Membership

• Everybody can become a member of one of the teams
• The team can expell a member when one of the following applies
  • member does not show up for three meetings without informing the other members in advance
  • 2/3 of the team vote for expelling the member

Flexibility in this concept

As there might be changes to the version control software (e.g. switching to mercurial, git) technical terms (e.g. trunk) or best practices might become obsolete. In this case the teams can advance this concept and adapt it to needs of the future. The same applies for duration of cycles that might become a subject to change.

The only thing that the teams are not allowed to change and that is hereby voted and enforced by the citizens is

• Features need always approval before they hit the main line of development to avoid excessively number of reverts
  => Goals "reduce overhead caused by reverts for unfinished features" and "clear rules how to get code into trunk"
• The aim should always be high quality of the software. If a feature is putting the quality of the adempiere to risk it should not be approved.
  => Goals "produce high quality ERP software" and "an all the time stable trunk"
• New features need to be applied with documentation for users and developers
  => Goal "documentation for all new features"
• The power to take decisions and organizing processes should be with teams and not single persons.
  => Goals "teams are to be responsible not single persons" and "lean project organisation"
• Team members need to dedicate a minimum amount of hours per week to the work. They need to agree on that before they become part of the team.
  => Goal "continuous and reliable progress"

If all of above is applied we will have "regular and plannable releases".