Making the transition to Open Source – a proprietary view

September 14, 2011

Recently had a fabulous conversation with Tony Galluscio Healthcare Solutions Product Line Manager for Harris about his work with David Riley, Vanessa Manchester and Brian Behlendorf on the Health Information Exchange known as Connect!  Purpose was to understand the journey he and his team took in making the shift to Open Source with a broad community.

Fair to say that there were some clear commonalities that popped out during discussion between our work at Gates on the Shared Learning Infrastructure and the Connect project.   You’ll also see themes that will remind you of some of Karl Fogel’s guidance from his book on OSS production.

Here are the highlights:

What were the key actions Harris took to organize for Open Source?

1. Assigned 2 specific members from each core dev team to act as point for the OSS Community. 1 on point.  1 as backup.
2. Partnered with a leading OSS advocate who acted as “gardener” to the community.  In Harris’ case this was Brian Behlendorf
3. Organized Codeathons where a majority of the core team could get together with developers, share info and bond.  Dev team planned to have lower velocity during those 1-2 day periods.  Proved critical to sustainability and ongoing innovation further down the backlog

When did Harris make the decision to open up?  Did they wait till they had a critical mass of code/stories addressed before doing so?

1. ~3 months into the work David Riley started to evangelize the idea to open up across the team and the project’s stakeholders
2. ~6 months into the work the code started to be fed out to select individuals/strategic partners like Mirth via email with invitation to focus on contributing code for patches and bugs
3. ~12 months into the work Harris launched a full SDN with code/commit functionality and ability to download branches.  This proved critical.  For example, Mirth downloaded the code into their own dev environment.  Mirth is an OSS shop with its own community of 6500 followers which helped A LOT.  Gave Harris a Huge leg up in terms of understanding OSS and gaining traction in the ongoing dev process  

How did Harris organize for distributed agile teams?

1. Wherever possible, put devs together into local teams and identified a leader among them who they could interface with in person.  They’d often form up sub-teams within the team
2. The broader group was handled with good tooling: Skype, GoTo Meeting.  Harris used TargetProcess + Teams Board for project and code management.  Story cards provided a common basis across teams

How hard was it for Harris to open up the
work, given they were developing a secure PII exchange network for 26 different
federal agencies?

Harris had some early concerns

1. They were initially very uncomfortable just taking requirements and posting them on the web.  However, Brian convinced them of the benefit and David managed the stakeholder’s expectations.  Harris made the switch as soon as they found a way to write the stories in a sufficiently generic way that they avoided disclosing a particular agency’s schedule.  They prioritized the stories, took direct ownership for those that were prio 0, cleansed all stories of agency names, stripped out sensitive data from the backlog and published it
2. They were afraid that if they opened up they would give up some of the rigor and certainty of being able to meet deliveries to agencies.  Maintaining velocity was a must.  Harris overcame this issue by:
   1. Introduction of the community point person role in the dev team who could manage community expectations while the core team stayed on track with the prio0 stories.
   2. Scrum leader controlling access
   3. Community having to earn commit privileges by submitting code for consideration by Harris dev team leaders.  Harris developers were all grandfathered in and did not
      have to go through this process.  First approved committer was Mirth.  Required ~6 months to achieve full commit privileges

How did the codeathons work out? Wasn’t it a challenge given there were competitors in the room?

Codeathons exceeded expectations and have since taken on a life of their own (for the good)

Key contributors to success:

1. Had reps for each agency attend – this was important to helping agencies and their technical staff come up the learning curve on what it means to actually work in open source
2. Made a point of inviting competing vendors. EOD the whole project was about commodifying data layer so value could move up the stack.  All able to differentiate based on individual strengths.  Result was an ecosystem where they could all do business.  That was the context behind every invitation.  Mirth was one of those “competitors” who emerged as a core
   ally and committer to the code

Once the code was made public, what actions did Harris take to sustain momentum among developers while continuing to advance the larger project?

1. Allocated 1-2 engineers to oversee community coding further down the backlog.  Once code public, it became quickly apparent that the community was prepared to tackle stories further down the backlog.  For variety of reasons Other OSS stakeholders in the project had different priorities from the core dev team.  Harris therefore invested in engineers to spend time further down in the prio stack because Harris knew that while it wasn’t an immediate prio for its agency stakeholders, it also knew that the broader industry needed
   it.  Using its in house community advocates, Harris was then transparent with the community about why Harris itself was making the prioritization choices it was.  Having a Harris advocate was crucial here
2. Harris made a distinction between Open Source vs. Open Program, e.g.  what needed to be shared broadly with the community and what didn’t.
3. Harris set up an area of the SDN where it could keep things private among core stakeholders
4. Harris adopted a principle of “Local autonomy”, e.g. each agency responsible for determining what info went into and out of that agency.  Harris simply ensured an audit trail was in place to show local autonomy preserved
5. Harris built in an adapter layer on top of the core stack.  This created a more “plug-and-play” environment that vendors could plug their solutions into. Also allowed Harris to make a distinction between what could afford to evolve rapidly vs. what should not

Finally, wasn’t there a risk that the community would take the code in a completely different direction from the core effort?

That is always a possibility with OSS.  However there was one instance which started out bad and ended very well.  This concerned development of a branch of the code to deal with query retrieve gateway capability.  The work quickly attracted a vocal base who felt that the branch represented a lighterweight solution to the entire project.  There was noise and debate in the developer community for a while, not always constructive.  EOD Harris ended up incorporating the code into their Florida HIE project. Led to a win-win.  Wouldn’t have happened without the fork