24 August 2010

What are the barriers to walking the MOTU/Developer path?

I asked this on Ubuntu.StackExchange but only got 3 answers so far, so I wanted to put this a bit more publicly too. What keeps you from packaging, patching, etc? If you do those things, what keeps you from applying for upload rights?

About 150 people are members of lp:~ubuntu-dev right now. There are also a handful of people who haven't yet become MOTU or joined another developer team but do good work. More hands are always needed, so what's keeping the rest of you folks away from this area of contribution?

For reference, 150 is a pretty small number given about 30,000 packages. Debian has more like 1000 developers.

Feel free to answer here in the comments or on U.SE.


Jef Spaleta said...

Just as another point of comparison. Fedora currently has 1062 approved accounts listed in the Fedora Packager Commit Group.

Please don't take this as a rub your noise in it sort of comment. I think the fact that both Debian and Fedora have similar numbers for packaging committers may indicate that there is an underlying shared policy which is helping to reinforce packaging contributor growth for both those communities.

I would hold up the concept of primary ownership/maintainership as the underlying policy at work in those projects. And to some extent the associated concept of package orphaning when a primary maintainer isn't willing to stand up and be account for a package. Its the stress on personal accountability for each and every package that boosts the packager numbers.

Right now the loose team nature of MOTU across the entire repository de-emphasizes personal accountability for any particular package and it makes it easier to grow an expectation that someone else is going to take care of the packages that are important to you. Ubuntu doesn't have a orphaned package concept which gives people a chance to jump in and care for a package that is not being cared for.

Mackenzie said...

Thanks for the stats, Jef.

I'm sure that Fedora and Debian both have had significantly more time to acquire developers than we have, but I still think 15% is darned tiny.

As far as accountability goes, I think we're trying to head things more that direction with the new packagesets. We now have per-package uploaders. There's only a few so far, but at least now it's possible for the Developer Membership Board to say "we trust you to upload this package even if you're not experienced enough to work on 2/3 of the archive." There are also smaller teams forming, akin to how Debian has its education team, KDE team, etc. I hope those will also contribute to lessening the effects of Somebody Else's Problem.

Jef Spaleta said...

I think you are using a longer timeline for Fedora than is probably warranted, even when including Fedora.us Fedora is no where near as old as Debian as a project open to volunteer contribution. But that's not really important. What matters is growth of contributor manpower in relation to the growth in the workload over any particular time window of interest.

I can dig out rate of growth for Fedora packager contributors if you are interested. I think last time I looked growth in package contributors were scaling linearly with growth in repository size across a multiple year view. But that was like a year ago so things could have changed in the short term.

Do you have a good data driven view of the packager/developer contributor growth rate in relation to the package repository growth rate in Ubuntu across multiple releases?

Inside that subgroup definitioning.. is there a mechanism to designate packages as orphan and to start a clock on expiring them from the repository if noone is stepping up and being accountable for them? Look at how the sparc release was dropped for lack of interest in maintaining it. Is the same sort of lack of interest metric being used in the package repository itself in order to reduce overall workload for the existing packager team?


Mackenzie said...

Right, I know Fedora's not as old as Debian, because not much besides Red Hat itself and Slackware are.

I don't know of any stats on rate of people joining ubuntu-dev, but I know that since we started out with having something the size of Debian package-wise (hmm, well I wasn't around for that, but I think Universe/Multiverse have been there since shortly after No Name Yet--they were certainly there before me, and Ubuntu was a year and a half old when I switched), there's no way our number of packagers is anywhere decent relative to that number.

Packages do get removed from Ubuntu if they are orphaned in Debian (I started taking care of a package to get it back in Ubuntu when this happened once, and now it's back in Debian too). They also get removed if they are in a state of disrepair.

Packages are binary-copied from the current stable to the current development release when the development release opens. This has the potential to leave some cruft, so in Lucid, there was a full-archive rebuild, and anything that didn't build was sent out in a list to the Ubuntu-Devel mailing list with a note that they needed to be dealt with or would be removed by $date. Archive rebuilds are common, but usually logged to a webpage which we're supposed to check on occasion. Sending the list directly to the mailing list was, AFAIK, a first.

Anything that doesn't work with Python 2.6 was also removed.

Jef Spaleta said...

yeah, don't try to account for initial conditions.. you got an injection of both contributor and packages in one big lump at the beginning for different reasons for different projects.

What I would humbly suggest, is to look at the growth rate of the manpower to workload over the last couple of years to get a good baseline sense of what is happening.

First cut is number of additional packager/developers (whatever makes sense in the context of the current process as you understand it) in say the last 24 months and look at the repository growth over the same time period. Preferable on a month to month basis so you can get enough points in the trend curve to construct a power law fit with reasonable statistics.

But you may want to look in more detail by pulling out Debian merge activity from REVU activity. You may find that one activity is more strongly correlated or benefits more with incremental additions to manpower than the other.

Mackenzie said...

REVU is very much ignored, for many reasons:
1. We'd really prefer if things went to Debian to be synced into Ubuntu. More users get it this way.
2. We don't really need more drive-by packages being dropped off by people who don't intend to stick around and care for them. When I mentioned to Emmett that I thought I needed to package a few more new things and put them on REVU before applying to MOTU, I got a "NoPleaseDon'tWeDon'tNeedMoreStuffToTakeCareOf!" response.
3. I doubt I'm the only person with upload rights who still feels the need to double-check with more experienced folks often enough to be uneasy about reviewing others' packages.
4. REVU's server is woahfully out of date and throws completely wrong lintian errors.

Anonymous said...

One of the barriers I encountered (still am encountering) is that, upon submitting a patch, I had to wait for a sponsor. And wait. And wait. No review was forthcoming. It missed a release, despite being submitted way before freeze. Then it missed another release.

Finally, the patch was reviewed. However, I wrote the thing a year previously. As a first time contributor, I was not encouraged to do further work as I felt that nobody actually cared.

As for packaging software; the wiki docs are not clear -- there seems to be outdated material on the site which doesn't help somebody new to the system. Also, while it may be obvious to those who have been doing it, the packaging system is not intuitive and newbie questions are met with a certain amount of hostility in #MOTU.

Anyway, that's my $0.02!

Mackenzie said...

What happened with your patch? Was it a .patch or a .debdiff? Did you subscribe the sponsors team to the bug? The sponsor queue is routinely emptied a few times during each development cycle, but only bugs where you've subscribe the sponsors team are considered to be waiting for a sponsor.

(Still, I do tend to just ask on IRC if I need something sponsored for main. It's faster.)

There shouldn't be hostility in #ubuntu-motu, so feel free to ping an IRC op if you encounter any. There's a channel specifically for packaging help now too: #ubuntu-packaging

Anonymous said...

Debian's developer community is actually in decline (mainly due to the recentish purge of MIA folks). W could use some more help from Ubuntu and other places, so please send us your 150 MOTUs.

Mackenzie said...

There is overlap with the 150-ish devs Ubuntu has already. I don't know if you were at DebConf, but Jorge Castro did a presentation on Debian-Ubuntu relationship, and his last slide listed the 20-30 Ubuntu people who were present at DebConf. Most were at least DMs if not DDs. I was on the list, but I'm not a DM yet. A MOTU I know became a DD about a month ago, though. His name's Luke Faraone.