So for a good part of my days over the last while, I've had the experience of being downstream from KDE as well as the distributions who are KDE's downstreams. Some days it feels like I'm in a mobius loop as I shift positions in the production sequence. =)
It's been nice to be able to hold that perspective for extended periods of time, though. Being downstream (once removed!) from KDE (and the rest of the stack) is a rather different experience and mindset from being upstream to a distro. That's probably stating the obvious, but it's certainly another part of what has gotten me thinking about these issues more deeply.
Holding both experiences concurrently has given me the context to do things like explore negotiating with myself, role playing both sides of the conversation. (I don't actually self-negotiate when it comes to the real-world work issues; this is more a "mental exercises for hot showers" type material.)
More practically, I have to apply my goals and needs as a downstream without prejudice because I have a team that relies on and demands that; in a different point of the week (or sometimes the same day, even) I have to apply my goals and needs as an upstream without prejudice because, well, I have a team that relies on and demands that.
It is said that you can not slave for two masters, and at first I thought I'd run into issues with this until I realized that my master for whom I slave is one and the same: Free software. I just get to play the role of two different servants whose work must dovetail.
Given that these "supply chain" issues are kicking my ass in both of my servant roles, it's become something of an un-ignorable burning itch. Like athletes foot, or jock itch. Yeah, not pleasant.
I do also want to give a positive example (that I personally had nothing to do with) to show that this up/down coordination can work out well. The challenge was that a KDE4 user interface to network manager was needed.
Preferably it would be an improvement over the KDE3 one and it should work with the latest, greatest network manager stuff. Now, network manager is "upstream" to KDE and the distributions who really, really want this are downstream from us. Along comes Novell, one of our downstreams, and they say, "We really need both a better KDE network manager GUI for OpenSuse and it needs to use KDE4 libraries." They (Will Stephenson being huge in this role) start working on it. That work goes into the upstream repository while it is worked on, thus building a partnership through shared infrastructure.
Unfortunately KDE 4.2 goes into feature freeze before the network manager interface is ready. So people talk about it and determine that it could be in shape by the time 4.2.0 comes out, it just can't be part of the 4.2.0 release proper. OpenSuse could then ship this thing with their KDE4 packages even though it's not actually released via upstream yet. Sebastian Kugler, an "upstream" Plasma developer, starts pitching in to help make it happen. Everyone is informed and comfortable with the situation.
It seems to be working out well, but what happened differently here?
People along different parts of the production/supply chain got together and coordinated. Needs were communicated upwards in advance of crunch-time, and a coordinated effort emerged. Upstream was given a heads up about the inclusion of a non-released component that is just That Important(tm), and upstream actually got involved and puts resources on it to help ensure success. Upstream's happy because they get help from a much needed component from downstream, downstream's happy because they fill a need.
It's not a perfect world (perfect would have been inclusion with 4.2.0, or heck, even 4.0 if we really want to go for Perfect(tm)), but it's close enough (they are still using the KDE3 network manager in the meantime) and everyone is informed and comfortable with the plans. Needles to say, the user wins across the board and nobody is losing.
This all seems common sense, doesn't it?
Yet it is a much rarer occurrence in the FOSS desktop world than it ought to be. We are often chasing our upstreams, who are often running off on their own tangent, while we're doing the same thing to our downstreams. Every so often someone screws the upstream over because they don't feel they have a better choice, and the upstream gets miffed. All perfectly logical, but not very brilliant.
So why did common sense prevail in this case? In part because Will, Sebas and the Plasma team all know each other and get along. We talk often, have shared values and open lines of communication. Obviously this is not an easy dynamic to replicate.
So we know that with agreeable circumstances it can work out. We know that release schedules don't need to get in the way even if they don't line up perfectly. We know that resource allocation can be modified in response to voiced need. We know that different points in the chain can work together as if we were actually coordinated in some logical fashion. We do not have to guess at the plasusibility of it, then, we just need to figure out how to replicate the phenomenon.
What we are (or at least I am) missing are sure-fire ways to replicate these results when:
- We don't all know each other, or know each other very well
- We aren't in constant, regular communication with each other
- We rely on each other for different pieces of the puzzle
- We have independent bodies setting each group's agenda/roadmap
- The topic isn't always as limited and focussed as "an interface for the user to activate a network connection".
And it all needs to fit into 40-60 hours a week. Cue the Mission Impossible theme song? =) It may sound like it when it's put that way, but I have faith that we can construct an 80% solution that will blow away our current 20%-because-we-get-lucky track record.
14 comments:
I don't know if what I am thinking is realistic nor if it has any potential in solving at least partially the issue. I am not a developer (a few web thingies in spare time), probably a poweruser, so this may be far off.
What about having someone publicly identified as some sort of facilitator/focal point? Let's say I am appointed as the "KDE-fixer". Someone from distribution OpenMandiBuntIan emails me "we are 4 months ahead from planned release, and functionality F in app A is missing/bugged as hell, what to do?" or "Is this framework ready for use in that particular thing we're thinking of?".
If I am familiar enough with the issue (it's all over the mailing list, I talk with the developer everyday on IRC,...), I can give a provisional answer, but most likely, I'll raise the issue to the concerned people.
This ones know me, know why I'm here. Depending on the request, they can say "I can't be bothered" "I know the problem, but don't have time, I willing to welcome any help from them"... I can continue to be the fixer, making sure that the request gets an appropriate answer , ie: that they know what options they have. Sometimes I would not need to follow through each step, once downstrem and upstream have agreed on a solution, they can work directly together, if they want/need.
Similarly I spread the word that Plasma team is looking for real world implementation of this and that by third party apps or distros, and help coordinate this within the timeframes and resources of each part. Or everything else that need to be coordinated, and quickly becomes a burden for those who are already busy with something else.
This will not solve everything at once, and this should be prevented from becoming some sort of administrative management hell, of course... but whaddya think? Did I really drink too much?
Mobius loop, got to note that down briliant. :)
I know you aren't a big fan of Marks synchronicity proposal, but wouldn't it have helped in the networkmanager case?
Well, we all know that repeating things and giving solutions are what we keep computers around for. So I would argue for a technical solution.
Launchpad (if it were opensourced) have nice ideas (maybe other trackers also?) like integrating several stages of development (blueprint for ideas/designs, code, bugs, translations for other stages) and also being able to link to other trackers. And I think that this is the real power.
The basic idea is having all the relevant info of the projects in a machine readable form (this info is somewhat a way to broadcast your status without the need of emails or IRC) so it can be linked from other places.
I'll give an example:
Instead of writting an HTML page with release schedules, developers will write a machine readable file (the release schedule ontology) distributions link to those files.
Then when they can make queries as "status of projects as of 10 February" (their tentative freeze date, this search for KDE GNOME or any other project that exports that file). Compare it with their own Blueprints and decide they need a new network-manager-gui (for instance)
Of course, KDE can use the same technique to know the status of their upstreams.
Now the other way, distributions (or KDE to their uptreams) release a file with information about their design goals unmeet because of uptream.
In this example, the lack of an improved network-manager-gui-kde4
The same downstream problem can appear by one or more distributions what would make it a good candidate to bend slightly the rules to ship what they are asking for.
The main problems with this approach are:
- Long setup time (creating the software and the usage)
- A good definition of the files should be agreed.
What can I say :-)
I think this is a very good example of how things need to be done when something has to be finished by a firm deadline, not just eventually. It's a paradigm shift from "if" to "when". Back on Akademy in Malaga, I had a nice chat with a guy from the Canary Islands (mEDUXa project) who was quite persistent in demanding that KDE (as a project) should see distributions as customers. Seeing them as a "downstream" in your sense of a supply chain is just another word for it.
But it takes more than tools and people in focal positions, first of all it requires commitment and while it's easy (well, easier at least) to have this commitment from paid-for workers, the challenge comes from working with volunteers. If anything, it requires planning well in advance.
@Kragil
To some extend if distributions would ship but not enable a new version of such a service, allowing the developers of the user interfaces to catch on so that the next synchronized release could have the new service enabled and have GUI for controlling it.
However, seeing that some distribution even replaced a suported version with an unreleased one, I'd bet they wouldn't act that sensibly
So where did the connection manager application in Ubuntu Ibex come from. It's new as it is not the same as the one used in the KDE3 ubuntu and is a heck of a lot worse.
It refers to interfaces as eth0, wlan0 etc (these cryptic names should be long gone)
The only available connections are those previously used which is crap as if you are in range of a previously used network then it should automatically connect otherwise provide new alternatives (this is not as intuitive as the old design)
The old connection dialog used to request all the information needed in one screen and then other info in an advanced area rather than letting a user enter half the info press connect - have it fail without a pop-up (it used to request the passphrase later on in a pop up rather than just fail)
Well the list goes on - I don't know where Ubuntu found this slapdash GUI from but please send it back.
"Unfortunately KDE 4.2 goes into feature freeze before the network manager interface is ready. So people talk about it and determine that it could be in shape by the time 4.2.0 comes out, it just can't be part of the 4.2.0 release proper. OpenSuse could then ship this thing with their KDE4 packages even though it's not actually released via upstream yet. Sebastian Kugler, an "upstream" Plasma developer, starts pitching in to help make it happen. Everyone is informed and comfortable with the situation."
Bingo.
Since the freeze is 2 months before the release that its plenty of time to work to have at least a stable basic version of n tool not completed, plus, the time the distros has before its own release.
The problem I see is when the distro release day is before the official lets say 4.2 release day.
In that case distros should push their own release day a little farter, in the case it is a KDE centric.
Has KDE now has a regular 6 month release period? if that's the case there would be no excuse for distros to accommodate to those release times.
Hy aseigo, nice interesting post as always.
If I understand correctly, problem is that upstream, 'stream', and downstream are desynchronized, on which features they think is important to get to the next release, right?
My solution to this is quite simple, and I hope you'll find it usefull.
I know you've spent a lot of time negotiating with stream, and downstream, the current release system, but the way I see it it's just too artificial to ever work properly without anyone doing he's own thing and backporting unwanted bugs.
Now what we need to do instead of a 'whatever happens 6 month release' is a 'feature matrix release', in which you choose the features you want completed 6 moths from now and what needs to be done to get there, and when you actually get there you throw a party and a lot of good marketing around. This is very similar to what we have now, the only 'BIG' diference is that you actually have each feature developed independently and whenever they're beta quality, people who download kde 4.xyz.beta get those features too, and whenever they're production ready a new minor release comes out, of course if to be stable feature A needs updated feature B it stays in beta till B is also done.
Of course you'll have a hell of a lot more minor releases, one for each feature, almost, but that's what makes sense, and downstream will be a lot more compliant,
and users will be a lot happier.
this will obviously introduce more complexity, but then you could have someone permanently responsible for stability maintenance.
Hope to have given a good contribution to the debate, this is what makes sense to me and is actually the reason I don't like using *buntus and instead use arch or gentoo.
Kiberlynx
Another solution, when meeting someone of a distribution:
// continuous integration as debian sid
if (distribution_does_continuous_integration)
{
answer = "we will be great partners";
} else {
answer = "change your broken model";
}
this is the main part of a program that should be compiled and send to distributions xD
> Unfortunately KDE 4.2 goes into feature freeze before the network manager interface is ready. So people talk about it and determine that it could be in shape by the time 4.2.0 comes out, it just can't be part of the 4.2.0 release proper.
I see this as a failure in KDE's development model. If it will be ready in time for 4.2.0, it should be part of 4.2.0! If it will be ready only shortly after 4.2.0, it should be part of 4.2.1. I think less rigid freezes would benefit everyone and also reduce the amount of backporting being done by distributions.
@Kevin Koffler: "I see this as a failure in KDE's development model."
it's a failure in KDE's (and just about every other FOSS project's) *release* model.
i say this because *development* is continuing.
the fact that it happens out of main stream with no other recourse is a release management issue.
the fact that we will deliver downstream what's needed is a strength.
so while our release system can (and imho should) be more accommodating and flexible without sacrificing the ability to do actual releases, i'm satisfied if not ecstatic.
To answer, no.
Isn't this situation a rare occurrence where a series of 'almost ready's were able to come together?
I'll tell you what made it work. It was/is in everyone's interest to have it work.
If there is discord or friction between streams, then there is a conflict in interest. Sometimes the conflict is in timing that will eventually work out, which is a resource allocation issue. Other times the conflict is in direction.
I've seen processes similar to this in construction where a construction manager harnesses the interests of independent sub contractors to accomplish something. The difference, and it's a big one, is the availability of coercion which doesn't exist between free software contributors.
I suppose that is why distributions and implementers end up having someone they pay to do the stuff they really need to have done. The community at large does amazing things, but your precise need usually gets done by your self.
As for how the coordination is done, the best construction managers I've seen have deep knowledge of all the processes involved, and while there are broad goals laid out, the day to day getting things working together is done by the manager being on site, seeing what is happening, forseeing choke points and planning solutions, etc. Very hands on, communication intensive work.
Interesting being on both ends of the transaction. I have had the same experience in construction, and the sheer terror of being dependent on another entity whose interests are not yours is enlightening. Oddly enough, some people have the knack of being downstream or consumer of the process. Again, it comes down to them being skilled at aligning interests. In some cases, it is simply cookies that makes the difference.
Derek
Sorry to plug the blog up, but wasn't the freedesktop.org initiatives with the various standards being established a similar endeavor?
In that case it was someone with time and skill set convincing others to work with a standard. You know the time and effort involved because you did some of the coordinating/selling/coercing/cajoling, iirc.
Those were one-time efforts, but coordinating between multiple development efforts is similar.
Derek
Post a Comment