Wednesday, December 10, 2008

more downstream fun

Today I talked a bit with Rex from Fedora's KDE team about their experiences with KDE's releases this last year and issues around backporting features. It really wasn't much different from what other downstream groups seem to be going through, and it is becoming ever clearer to me that we need some processes and tools in place to make the gathering of information and then channeling that into communication in place.

Nearly all our downstreams, while appreciating the improvements we've put in place over the last couple years, ask us to communicate even more. What do they want us to communicate, exactly, and how? That, it seems, is much harder for them to articulate clearly.

When it comes to them communicating in the opposite direction, there seems to be a lot of desire but little follow through on that in the good times and a scampering about without a lot of direction when challenges arise.

I'd like to know what the release goals, the release challenges and the release deviations (from upstream) are in the pipes. Preferably without having to visit each irc channel or set up conference calls with distribution representatives twice a year.

On the one hand I feel bad for our downstreams because we're obviously not their only upstream and nearly all of them could use more human resources as it is. On the other hand, it's pretty evident that we waste a lot of resources by not communicating more effectively.

It seems we could do well with an efficient mechanism of communication and tracking. Bugzilla's really not the tool for it, and it's not a matter of creating Gantt charts either. (Thankfully.) I think the patch server I wrote about a few years back is still a good idea, but that's really just for patches and doesn't address issues of coordination and communicating logistics.

9 comments:

Andre said...

I guess it's going to be hard to come up with a solution for a problem noone can articulate. Can it really be a problem that is impossible to solve by rather classical tools, like the possibility to share bugs/requests and a closed area in a forum to ask questions/discuss? Perhaps if that is running, you will discover over time what you really need.

Jaak said...

Every project just needs to appoint a contact person or two who are in charge of communicating with the contact people of other projects, about the downstream and upstream issues.

Put a fancy name for the position e.g. 'downstream/upstream relation and cooperation manager' and make sure they keep in touch with other projects' respective people. Contact people will collect information their project's members ask them to collect.

Of course, if more detail information is necessary, contact person should hand the communication over to the members. But having a single or two dedicated people will reduce the communication cost a lot, since they deal with many parties who have overlapping information needs.

Good luck with improving the communication among open source project, it is a noble goal.

Aaron J. Seigo said...

@Andre: well, i don't think it's hard to articulate, i think people haven't really tried to articulate it.

i can articulate rather succinctly what i'm looking for when it comes to communication upstream.

i can tell you exactly what's gone on with various situations, be it the so-far-successful network manager thing or the more-or-less disasterous bungles elsewhere.

it's not unarticulatable, but it does take some effort and thought to be able to do so.

at the same time, i've been looking for right now are tools that aren't wrong as a sort of process of elimination.

forums aren't the answer for the same reasons mailing lists aren't.

shared bug repositories would be a step in the right direction, but doesn't cover non-defect time sensitive logistical issues or longer term strategic planning.

simply throwing stuff at the wall and seeing what sticks is not likely to cut it, because that's essentially what we've been doing for years.

@Jaak: that's the sensible answer. but sometimes sensible doesn't always work. ;) reality is messy and the reality is that it isn't about having people that can meet up, it's about ensuring that there is a process that scales (1:1 relationship based contacts between distros and upstreams fails due to churn and sheer # of both down and upstreams) as well as a mechanism to record and track progress over time.

when we've tried the 1:1 thing even with relatively small numbers of well connected people, the spider web of people who need to get brought in and the lack of reliability of availability makes it very hard to work out.

i hate to sound so .. you know .. negative. we've simply tried these things in the past because they do seem like sensible solutions. i just wish they worked as well in reality as they sound on paper. =/

Dave said...

improving communications doesn't really mean talking more, it's more about making sure all communications are relevant and timely. I think ideally it would be a meta project probably hosted by one/more of the distributions, with a rating system of desires/needs matched up to specific priorities of the upstream projects. ratings can be from -1 to 5. It should be possible to push trouble tickets to upstream projects when they aren't in your direct control, ie, argb visuals, etc.

of course, that ends up being a highly structured bug/featurzilla which projects with limited manpower would have to keep up with.

Nathan said...

What about using blogs and/or RSS somehow?

2 suggestions for consideration:

1) Blogs of similar topics can already be grouped on planet.whatever. If someone from each project (a huge number, I know) was designated as the spokesperson, they could be added to some kind of planet feed.

2) Spokesperson for each project has rss feed for certain types of blog posts. Interested parties (other spokespeople) can subscribe to the feeds they are interested in.

ingwa said...

This is just standard selling: Listen to the customer and understand his problems and pain points; then solve them.

In the case of KDE, this isn't done for money so the internal process is more difficult to steer and control, but it's still exactly the same process. Big things like software the size of KDE has very long sell cycles, and the vendor and customer need many meetings before every problem have been communicated clearly so that both parties understand each other.

The real problem starts when we (KDE) understand what needs to be done. Who is going to do the real work? We don't control our contributors and most FOSS developers and other contributors are young and don't have the experience nor inclination to work with big organizations and often they have very different objectives than the distributors do.

I tend to agree with Jaak that we need somebody dedicated who can do this work. We also need to spread more the mantra of enterprise features and compatibility with non-unixlike operating systems within KDE. This is something alien to most KDE developers even if we do have a healthy mix of young and enthusiastic people and the older and more experienced ones who have done their share in the real world trenches (but hopefully still enthusiastic as well).

Lure said...

Most of the open projects (OpenSuse, Kubuntu, Fedora) plan their features for next release on their wikis. What if we would create a wiki page(s) on techbase.kde.org, that would link to these feature pages with additional description of what/how they want to implement (delta to KDE version: like backport, building own utility...)?

I think that each distro could maintain such page with minimal effort and interested KDE developers could subscribe to such techbase wiki page to get informed about planned changes.

Some backports were also shared efforts (or at least they looked like that from external view), and for those even a new subpage could be created and could be even used for coordination between downstream projects.

gamemank said...

Looks like my first attempt at posting got lost somewhere in the caverns of OpenID.

Would it be useful to have an email from a distribution at the beginning of the cycle of what is planned for the next release? Where would such an email go? Or maybe a wiki page as Lure suggested?

freitag said...

@Lure: openSUSE is features not tracking in a wiki, but in a free feature tracking tool called Fate.
It was developed over the last years at SUSE and serves us very well.
We learned that Wikis are not a good tool to use for this kind of stuff.

I will see how we can come up with a proposal how Fate could be usefull for the KDE community.