Over the past few months I've been quietly tracking our upstream and downstream interactions. The motivation for this has been my experience with the KDE 4 series, in which we have both been on more people's radar (when was the last time NVidia mentioned KDE 3 in a driver release?) and seen more trainwrecks both up- and down-stream from us.
In particular, the "distribution backporting" issue has really gotten out of hand. It's no longer just individual unbaked features that are being backported and shipped with stable KDE releases, but now entire applications are being pulled from pre-beta SVN and shipped with stable KDE releases. I know of two different applications this has now happened to.
Just as disturbing, some downtreams feel that they have the right to speak for us as an upstream to their communities and have, by doing so, misled people and abused KDE's relationship with users in doing so.
Of course, I'm still reeling from the issues Plasma has run into all across the X.org and Qt map. Features those projects have advertised, in some cases for years, simply were not mature or tested until we showed up and found out the hard way by running headlong into pointy skewers of buggy or simply featureless code bases which got that way due to not having real world applications at the ready to tease them into shape.
The losers in this are all around: users get sub-par product, KDE's relationships are damaged both with users and downstreams, upstreams look incompetent when really they just got caught out unawares and downstreams provide a muddled experience in their delivery phases.
This is not a healthy situation. I hope that's stating the obvious and that we're all in agreement thus far.
Note that I haven't really said anything about who's to blame for this. Take the case of lacking real world applications to test our frameworks with: who is at fault? Should KDE have been there earlier pushing on these areas back when these features were first being trialed? Should those writing these frameworks have ensured there were real world type applications doing meaningful things with their new fangled hotness before claiming stability and maturity? Not easy questions, are they? Personally, I'm not one for fingerpointing unless you really get it obviously wrong after having had a workable solution pointed out to you. We aren't at that point, however, with nearly any of the above.
While tracking these issues and trying to make heads and tails of them, often with great frustration draped across my brow, I have also been looking for analogs elsewhere. Most problems have already been at least identified, and most problems that have been identified have a history of attempted solutions. Just because I haven't had to deal with a particular problem first hand doesn't mean someone else hasn't, and so I've been searching for solutions (good or bad) elsewhere.
In talking with some of our downstreams, I've found it increasingly useful to talk about things in terms of supply chains and the kinds of relationships that end up forming around them. Unlike "traditional" software development where the supply chain is tightly managed and usually completely or nearly completely internal, the open source model seems to me to be a lot more like the free market dynamics seen in commodity markets and how supply trickles on through the machinery to eventually create products the consumer purchases. At least, that's been my impression. Feel free to correct me in the comments section of this blog entry. ;)
Mapping our current situation on to that of supply chain dyanmics does give us a well studied and highly optimized set of solutions that other people have already torn their hair, hearts and souls out over. So I'll be slowly edumecating myself about SCM (no, not that SCM, this SCM), something I'm only mildly versed in at best, and thinking about how to apply it sensibly, if at all possible, to improve our current state of affairs. It could be a dead end, but it could also be a source of inspiration. There's only one way to find out.
I'm devoting brain time to this because it is going to be increasingly critical to get our act together if we want to take on the next level of competition we face in the market without gutting our values and our resources. We have to communicate and coordinate much, much better between the different points in our software supply chain and get our roles correctly aligned with each other. Right now it's very ad-hoc and the network of people has grown beyond that which is sustainable based on a who-knows-who game masquerading as personal relationships.
I hope to meet with several of our partners at the Gran Canaria Desktop Summit in mid-2009 to discuss these things directly, and to have brought myself up to speed enough on the issues to be able to formulate and deploye systematic support solutions. If you have a great SCM book you think I should be reading in the next couple months, please comment below. I'm also wondering how I might gain access to the Supply Chain Council's full SCOR reference manual.
Subscribe to:
Post Comments (Atom)
74 comments:
Similar situation for NetworkManager.
Some distributors decided to ship something they called 0.7 months before the project itself did the actual release.
This new "compete on amount of new features" vicious cycle has basically redefined "bleeding edge"
@Kevin: yes, this is absolutely not unique to KDE; KDE is merely the context from which I feel most comfortable speaking, being a stakeholder in that community.
I believe you are 100% correct, though, and that this is an issue across the FOSS desktop.
I hope whatever we might come up with here can be utilized broadly. We're in this together, after all: the user doesn't see "networkmanager 7" they just see a user interface, even if that's not really what's going on.
(btw, server side FOSS doesn't seem to deal with this issue quite as much for various reasons; the embedded space is a whole other silly kettle of fish, though, that is even worse in various ways. does that mesh with other's experience as well?)
Features missing: KDE4 versions of Amarok and KOffice still meandering around in beta stages. Mythical fantasies like Nepomuk integration and meaningful ZUI will probably not make it into 4.2 either.
Feature parity: That is one joke I am still laughing over. The claim that KDE devs are working to maintain feature parity with KDE 3.5. WTF? Now with 4.2 we will have spent a whole year trying to attain this "feature parity" and in effect a whole year of new ideas has been lost.
Usability: In an effort to implement fancy bling that is already available in other platforms, looks like KDE devs are taking pains to make things unusable. Case in point, Desktop Cube in Kwin.
OS X default behaviour for desktop switching: ctrl + arrow
Compiz in Ubuntu or Mint: ctrl +alt + arrow. Extra keypress, but still OK.
Kwin default behavior (in Kubuntu SVN) : Ctrl + F11 to activate the cube. Arrow keys to rotate, and then enter to select. WOW
And in the meantime, one can have a coffee and return. /sarcasm
Is anyone bothered about usability any more? Aaron? Celeste? Pinhiro?
Oh shoot! I forget. They are busy developing glowing panels, text next to (or below) buttons and blue window shadows respectively!!!!
In the light of such glaring missteps, no wonder Distros find the need to tinker around.
I can understand why Mandriva chose to backport Ark, in KDE 4.1 you cannot even unzip a Zip file. For Mandriva users (many coming from windows), it is simply not acceptable to ask them to open a konsole and type "unzip file.zip".
Personally I feel that proprietary software companies actually have an edge here in a sense. Because they do their work in secret, users will not be clamoring for new features.
Free Software, developed in the open, has the "disadvantage" that users know what cutting edge features are being worked on and can become rather impatient due to that.
Fedora also has voiced concerns that other distros are stealing their thunder by releasing features they develop before them. There was a story on LWN on that. I'm a bit afraid that this has the possibility of leading to more closed source developmen a la Xgl and similar. I also believe that this is a good case for "release early, release often".
My mostly content-free 2 euro cents.
@Aaron:
"(btw, server side FOSS doesn't seem to deal with this issue quite as much for various reasons; the embedded space is a whole other silly kettle of fish, though, that is even worse in various ways. does that mesh with other's experience as well?)"
--
That matches my experience. Server side, things have to move a lot slower. There is a lot more caution regarding new technologies, and a much larger emphasis on backwards compatibility. I know several people who administrate large university and commercial *nix networks, mostly based on CentOS. The amount of custom scripts is very large. Large changes are a big no-no.
I have less experience with embedded systems, but from what I hear from friends working in the area people end up reinventing the wheel to have different spokes every other day... Really silly stuff too, like UI menu description files and similar.
Oh Wonder of wonders! The guys who started the whole vicious cycle of releasing alpha level software as a "point zero" release (AKA KDE 4.0) are now crying about others stepping in to make that software more palatable to end users.
The Irony.
@Jean: how was your comment even remotely on topic?
"The claim that KDE devs are working to maintain feature parity with KDE 3.5. "
that isn't the goal; the goal has been to ensure that no important feature gaps remain, and 3.5 is simply a nice baseline to measure against for a number of reasons, such as: it represents a complete traditional desktop that is a mature product.
"Now with 4.2 we will have spent a whole year trying to attain this "feature parity" and in effect a whole year of new ideas has been lost."
i completely agree that new ideas have been delayed; they haven't been lost, however. that would imply i somehow caught amnesia. i'd also point out that we'll only have spent 6 months on 4.2 and that 4.1 was a lot less about feature parity than 4.2 has been.
"In an effort to implement fancy bling that is already available in other platforms, looks like KDE devs are taking pains to make things unusable."
so it wasn't good enough that the KWin developers delivered a desktop cube/sphere, but you start going on about its usability now? why don't you back up and consider the usability of the cube as a concept in general? hm.
anyways... you're absolutely wrong in any case: it's Ctrl+F# to switch between desktops, and you can assign a next/previous if you wish quite easily in the shortcuts dialog. and if you have the cube selected it will "spin around" as you switch.
if you're going to bitch about off-topic things on my blog, please at least have courtesy to be right.
"Is anyone bothered about usability any more?"
yes, and if you look around 4.2 you'll see a number of incremental improvements.
"They are busy developing glowing panels, text next to (or below) buttons and blue window shadows respectively!!!!"
the idea that i can't work on the glowing panel (est time: 3 hours in total) and deliver useful things in the rest of the 6 months of time is pretty stupid.
i can't take you seriously when your blowing that hard.
"In the light of such glaring missteps, no wonder Distros find the need to tinker around"
oh, i see how it's on-topic! you're suggesting that the problem is that we have screwed up so badly that downstreams are forced to do *something*, *anything*.
you have utterly failed to comprehend the situation, what i wrote or how the dynamics of this situation goes far beyond KDE itself.
care to discuss the issues with X.org and argb visuals? or the network manager Kevin brought up?
actually, i think that's giving you too much credit and you just used the above connecting conclusion as an excuse to bitch here in the comments section. poor form.
Oh wonder of wonders! The trolls have already arrived!
@patcito: obviously there are motivations that are, or at least appear, rational and defensible at every point in the chain.
that doesn't obviate the fact that the results of all these defensible reasons is an epic fail at the end.
including ark from trunk with alpha quality features in it is amazingly boneheaded in terms of the end user and has had the added consequence of disenfranchising the upstream author.
not being able to open a zip file, however, is equally boneheaded.
the question is how to meet the varying needs at each step of the chain in a way that minimizes the compromises and maximizes the final product's impact.
we're missing the ingredients to do that right now, and it has nothing to do with individuals not having agendas, goals and reasons that are self-consistent. it has everything to do with a lack of communication, coordination and relationship respect.
reflect on how well yanking an unreleased product and shipping it to users would go down in the proprietary world? law suits would be flying thick and fast, and not just about IP protection but also about damage to brand and violation of contract. while the proprietary world has a lot of ills, mostly associated with enforced IP dominance, there are good reasons those other things are guarded and nurtured so carefully.
another thought experiment: imagine if Toyota started sending out press releases and sales materials to others in the manufacturing industry concerning the metal alloys they purchase and use from their supply chain in a way that made the apparent source (Toyota) indistinguishable to the recipient (the market) from the upstream company (the metals company)?
reversed, what if the metals company had no communication with Toyota and simply shipped them whatever it could "best guess" was required?
@slougi: yes, there's a bit too much destructive competition going on between distributions right now. it seems some of them have forgotten that we are cooperatively competing, by the very nature of the licenses we use.
the rules are different in such a scenario, and to win inividually we need to play by them together
KDE is the worst offender of delivering half baked applications. This isn't necessarily the fault of KDE. Its why pre-release network manager was shipped. It was better than having wireless that just flat out didn't work with a whole range of configurations. Not a way to make friends and influence people. Similarly, not being able to open zip files without a terminal is bad for users. Its not about bleeding edge, that's merely an unfortunate side affect of the fact that the only code out there that meets expectations isn't "released yet.
The problem is not limited backporting unstable applications/features/bugs. It seems to extend to the point where your downstreams are actively disabling features in your product (i.e. Zoom Controls).
Maybe you should take a hard look at the feature sets you are targeting when distributions are actively shipping features that you claim are unstable and actively disabling features in code that you claim is release worth.
@Jean: "The guys who started the whole vicious cycle of releasing alpha level software as a "point zero" release (AKA KDE 4.0) are now crying about others stepping in to make that software more palatable to end users."
your interpretation of what i wrote speaks loudly about what's on your mind, because that is not what i wrote at all.
i'm not going to get into the 4.0 issue with you as it's been discussed extensively already and if you still don't understand or agree with it, you never will and we will just have to agree to disagree and move on.
but it would be nice if you could understand this blog entry in spite of that, in which i'm looking for ways to increase the cooperation and communication across the FOSS supply chain, strengthening the linkages in so doing and finding a way to meet the needs of both downstreams and upstreams (note that KDE is both; we're actually sort of "midstream" if you will)
Regarding backporting:
I confess I sometimes have trouble descerning the intensity of the objections to backporting as opposed to any other kind of porting in so far as it serves the needs of the distro's userbase. Opensuse backported powerdevil for 11.1. Kubuntu ported guidance-power-manager to KDE4 for 8.10. Both distros clearly identified the need to provide their users with a power management solution. Both distro's, at least based on what I have seen, have been clear about how these solutions came about and that these solutions are distribution-specific features. Both run the risk of having their solutions be erroneously be identified with the KDE brand.
I'm a Kubuntu user and goodness know I was happy to see guidance-power-manager in 8.10. I can imagine many Opensuse users will be happy to find *some* power management solution in 11.1.
There is, credibly, more that can be done with respect to flowing bug reports up/down the supply-chain more efficiently.
That said, whether features are backported, forward-ported, ported from other distros or whatever just seems to be part and parcel of open-source development. The code is open, available and the distros are doing what distros have always done: putting together chunks of code to best satisfy what they believe their users need.
Yeah we should certainly try to improve the relationships up and down the supply chain, but I confess that I'm genuinely been confused by the intensity surrounding "bridge-burnings" and the like.
@Ryan: "KDE is the worst offender of delivering half baked applications. This isn't necessarily the fault of KDE. Its why pre-release network manager was shipped."
what does network manager have to do with KDE? (following up on your initial assertion)
(answer: nothing)
"not being able to open zip files without a terminal is bad for users"
obviously; that isn't the question here. the question is: given that that is bad for users, how do we fix it without causing a different set of equally bad problems? because that's what has happened here: in an attempt to fix problem 'A' (which you blame KDE for), problem 'B' was created (i'll wait for you to assign blame for that one where you will).
problem 'B' did not need to be created.
that's the issue we face.
"Its not about bleeding edge, that's merely an unfortunate side affect of the fact that the only code out there that meets expectations isn't "released yet."
it isn't an unfortunate side effect (as in one we are resigned to accept), it's an avoidable side effect.
"The problem is not limited backporting unstable applications/features/bugs."
that's correct
"It seems to extend to the point where your downstreams are actively disabling features in your product (i.e. Zoom Controls)."
yes, which is another symptom of the overall problem.
"Maybe you should take a hard look at the feature sets you are targeting when distributions are actively shipping features that you claim are unstable and actively disabling features in code that you claim is release worth."
ah, it's our fault. see, this is why i don't like playing "assign the blame" because then people like you come along and get wrapped up in that game and completely screw it up.
you mention the disabling of the toolbox, and "humerously" enough we've already received bug reports about the problems that is causing their users. why? because that decision was also a bungle. why? because it was made in a vacuum by downstreams who feel that:
a) they know better (which would be like me suggesting i know how to write rpm or deb spec files better than they do)
b) they have no recourse other than to act in a vacuum
i don't know what can be done about (a), but i do know a lot can be done about (b) and that's what this blog entry was about.
not that you cared to notice that.
so, let me pose a question to you:
in all of your knee jerk blaming, er, analysis .. how do es the issue of argb visuals in x.org being wonderfully broken despite it being a "mature feature" for a number of years now figure into your world view? how does NVidia's driver gaffes work into it? how do the various obstacles we ran into with QGraphicsView over the last year map to that?
i'm really not trying to shift responsibility off of KDE here, you see. i'm trying to set straight a jumble of individual groups that all rely and depend on each other and yet insist on a chaotic and disorganized approach to needs management best described as "random".
@Jamboarder: "Yeah we should certainly try to improve the relationships up and down the supply chain, but I confess that I'm genuinely been confused by the intensity surrounding "bridge-burnings" and the like."
the latter is directly related to the ill state of the former.
it is not just about backporting, though it seems you too have skipped over all the other bits of the blog entry and zeroed in on that like some of the other people commenting have.
instead of rehashing it all for you, let me simply quote myself:
"We have to communicate and coordinate much, much better between the different points in our software supply chain and get our roles correctly aligned with each other."
and that has nothing to do with whether you as a user like having power management and everything to do with how we meet those needs in the supply chain from end to end.
@ The comparison with Toyota
So it is OK to compare with "closed source" companies like the car-maker to prove your point?
And if comparison with non-free corporations is valid, then how about the brickbats MS is getting over overly focussing on bling in Vista and forgetting key features initially promised in Longhorn?
Is it then fair to blame KDE equally for similar gaffes?
@ Working in a vacuum
If anything KDE is giving the impression of working in a vaccum. Major apps like Amarok and KOffice are still not final in their KDE4 form. Basic features like unzipping, powermanagement etc are lagging, while "bling" features, exotic wallpapers, panels, system trays are seemingly seeing the light of day. End user expectations and their fulfillment by distros is being seen as circumventing the system, without realizing that is should all be about the end user after all.
Correct me if I am wrong. Your message seems to be:
"We will take our own sweet time providing basic feature sets. People who demand forking are wrong. End users who expect these "basic" features in the current release are wrong. Distros that try to provide these to end users by back-porting are wrong.Distros should then keep a KDE3.5 based release for their user expectations and a KDE4 based release to fine-tune new features" ?
LOL. On an different note, all this sounds more like GM and their "promised" Volt, rather than Toyota.
First: congratulations on shipping Beta 1, 4.2 will be a really exciting release :).
Second: I think a time based schedule will help this problem, not whole as the NetworkManager suggestion shows, but will effectively reduce that issue. Distro's can expect which features are coming and help the upstream project with maturing them and adding features before feature freeze (it is the responsibility of the project to accept or refuse them.) At my opinion a time based schedule itself is not better, but is better for open source projects. A big company - like Microsoft - can exactly plan and schedule their software as it is their property. A time based schedule doesn't make sense here, because they haven't the problem of very much different projects with different maintainers. But in the FOSS community there is actually a need for syncing schedules. That doesn't mean that you have to sync it to GNOME or whatever (even though I think that makes the most sense), it means that you will say: every 5 months or whatever you want and every month a maintenance release for that major release. By accident, if the project is not mature enough on the end of the cycle, you can postpone it just like Ubuntu did with their Dapper release.
By the way, I don't claim that it will THE solution, I just think that it makes collaboration easier with downstream.
@Jean: "So it is OK to compare with "closed source" companies like the car-maker to prove your point?"
i was comparing with an organization that we all probably know of which has a deep and well organized set of supply chains.
it had nothing to do with open or closed.
"Is it then fair to blame KDE equally for similar gaffes?"
besides your reasoning being really .. bizarre .. please read the paragraph in the blog entry that begins with: "Note that I haven't really said anything about who's to blame for this"
"If anything KDE is giving the impression of working in a vaccum."
i'm not sure you know the meaning of the phrase "working in a vacuum". either that or you are using it incorrectly on purpose.
"Major apps like Amarok and KOffice are still not final in their KDE4 form."
it takes time for such apps to get to the point they are releasable; we released kdelibs as soon as we could (Jan '08) to help speed that process up.
if your assertion is that we should make time go faster, that's not realistic.
if your assertion is that we should have held up the rest of KDE4 until Amarok2 and/or KOffice was done, then you need to realize that without a release of libs+runtime+workspace it would be taking even long for those apps to release.
"Basic features like unzipping, powermanagement etc are lagging, while "bling" features, exotic wallpapers, panels, system trays are seemingly seeing the light of day."
again, these things (creating new features you don't consider as requirements and features you do) aren't mutually exclusive.
powermanagement in KDE3 was lagging significantly, btw. care to complain about that?
as for the system tray being a "bling" feature, .. gah. . where do i begin?
having a system tray where the icons can communicate that they attention to the user is not "bling" it's basic functionality. or how we can resize the icons makes it uncomfortable in a number of form factors (both large and small).
the current system tray has lacked numerous basic functionality features that windows has had, to pick a non-random measuring stick, for years and years. nobody in the FOSS world has decided to fix it. we do, and you call it "bling"?
"End user expectations and their fulfillment by distros is being seen as circumventing the system,"
again, that's not what i've written at all.
there is no system to circumvent (that's kind of the whole problem), and if such a system existed it would be all about meeting needs in a useful way, e.g. not attempting to meet needs but failing due to shoving alpha features at users (starting with x.org, flowing through KDE, through distros and ending up on your desk)
"without realizing that is should all be about the end user after all."
we realized that years ago. it's why KDE was started in the first place. its why i care about topics like this enough to write about them despite knowing it will attract the attention of people of your ilk.
@Aaron: I should have been clear that I agree with substance of your posting regarding improving the ability of the supply chain to meet the needs of users.
What has drawn my attention is how we go about it. This blog posting is a great step in the right direction. I've simply been a little disappointed with the interactions I've seen with respect to backporting in particular. Where (most of) the substance of this blog more credibly and constructively points to the underlying problems that need to be solved, other interactions have, unfortunately I think, unreasonably pointed to simple irresponsibility on the part of distros.
I reiterate though that I'm happy to see the conversation move in the direction suggested by this blog entry.
"We have to communicate and coordinate much, much better between the different points in our software supply chain and get our roles correctly aligned with each other."
Ain't that what Mark Shuttleworth
propossed some months ago to you? and you rejected him?
You know, coordinated projects for a common realsease time?
@Daniël: "I think a time based schedule will help this problem"
we have one; other projects have them. the supply chain problems persist.
let's not make the mistake of saying that the answer to inveterate problems we face are things we are already doing.
if what we are already doing worked, we wouldn't have these problems.
instead, the problem is everywhere.
to go back to the Toyota example, just because the mine produces out N tons of high grade aluminum ore each and every quarter doesn't mean that Toyota now has the kind or amount of aluminum alloy it requires for its manufacturing process.
the timed release schedules thing is an interesting topic, but not one that i think offers much relief here.
even if it was, we still need to work with projects like x.org and companies like NVidia as upstreams that have their own ideas on these things as well as companies downstream from us like Nokia that also have their own ideas on things.
it's a much bigger problem in scope than "KDE and Kubuntu don't have aligned release schedules".
@Jean: "Correct me if I am wrong."
yes, you are wrong. i've already explained what i'm saying a few different times.
"all this sounds more like GM and their "promised" Volt, rather than Toyota."
you do realize that in that example KDE would be the mining company and the distros would be Toyota, right? LOL, yourself.
@Ramsees: "Ain't that what Mark Shuttleworth propossed some months ago to you? and you rejected him?"
no, it isn't. supply chain management is about a lot more than simply predetermining delivery dates. a LOT more.
Mark's proposal was, and is, also unrealistic given the realities of our supply chain.
so thank goodness that's not what SCM relies on!
"You know, coordinated projects for a common realsease time?"
ok, explain to me how that would have magically given Mandriva a version of Ark that could open zip files in November?
LOL, and you do realize that it is the battery maker (KDE) that is holding up the distro (GM), right?
LOL yourself now.
"ok, explain to me how that would have magically given Mandriva a version of Ark that could open zip files in November?"
I don't have the answer, but what I see is your idea of how distros should bow to upstream whims, dictating what release and when realease it,I don't see that happening.
The fact that Mandriva couldn't provide a more realible way to provide an unzipping tool is not a failure in downstream, is a failure in upstream that Mandriva had to fix.
But I better ask rocks to bleed than ask you to take a litle responsability on that, It would be more easier.
@ Jamboarder
Ditto. Agree totally. But the very start of the posting seems to be reflecting the distros in bad light, while they are just trying another way to fix a gaping hole.
@ Ramsees
True that. Never point to the fact that KDE might somehow be responsible for the mess. Aaron doesn't like that.
Can we try to be constructive and not aggressive? (note that it may not be the intention, but that's how I see it)
I don't like the backport issues because it tends to blame on the project (KDE in this case) a fault which is not its own (bugs in backported features).
@Jamboarder: Powerdevil isn't a good case as the author itself mantains a 4.1 version.
I have an idea on why there's this backport run, but it's not constructive enough to be discussed here.
Hi Aaron,
"KDE and Kubuntu don't have aligned release schedules"
I didn't argued that ;)
Ok, you are right, I didn't know KDE had already a time based schedule.
I used GNOME for a while as I didn't found a stable/good distribution shipping a good 4.x release and I don't like the 3.x release.
Happy Hacking!
"what I see is your idea of how distros should bow to upstream whims, dictating what release and when realease it,I don't see that happening."
Why? Distros use the products of upstream and release them for their own merit. Why should they be dictating things on to the projects that do the real work?
"The fact that Mandriva couldn't provide a more realible way to provide an unzipping tool is not a failure in downstream"
Of course it is! If the project isn't finished, and the distro needs it, then they should work on it upstream or include a different application, not just use something that the original producer hasn't approved and let the people blame the producer for it.
"Of course it is! If the project isn't finished, and the distro needs it, then they should work on it upstream or include a different application, not just use something that the original producer hasn't approved and let the people blame the producer for it."
So that's the problem? they are worried about people blaming them?
They could easy blame them by not having the zipping tool (ark), ready for 4.1, since its a very basic tool.
@ Andre.
"then they should work on it upstream"
Aah (enlightenment). Shame on me for not understanding this basic idea. Distro depends on KDE. Does really well with 3.5 series. KDE decides to send some dud releases (4.0 and 4.1) Distro gives up what they are doing and starts allocating all their resources to covering KDE's backside.
Sorry for not understanding this earlier.
/sarcasm
"Why? Distros use the products of upstream and release them for their own merit."
That's the problem? a popularity contest? give the credit to me and not to them?
Distros are the ones who ship the product, who download the source code of KDE and compile it?
Distros and upstream need to sit and dialog about realease dates, but upstream shouldn't in any way enforce its politics beyond that.
Sorry if this sounds rambly...
I'm wondering whether the use of SCM as an analogy might be flawed. I've taken enough classes that highlighted the mismatch between conventional manufacturing and software development that my Spider-Sense is tingling.
In reality we've got a different problem than what is defined in the Smelter-Parts Supplier-Toyota analogy. The parts supplier can't normally change what his input from the smelter is at a fundamental level. Toyota can't take what the parts supplier sent him and change it. That's what's happening right now. In an public development model like we have with {up|down}stream, Toyota gets to grab titanium ore right from the smelter and force it through the parts supplier whether the parts supplier can handle titanium slugs or not. This might work or it might not. Essentially the power relationships are significantly different. As soon as the code hits the VCS anybody can do anything they want with it. The same thing happened with Oxygen icons getting used before 4.0 came out, didn't it? By definition, this is a problem that closed-source doesn't have to deal with. They have control over the release.
Does this mean that the distros are being negligent in not waiting for a release is tagged before using the code? Does that mean we need a different development model to cut down on the amount of patching and backporting like the Linux kernel? Would that alleviate some of the stresses in the power relationships amongst all the streams?
The other possible resolution I can see is to go the route of Mozilla and stringently control the KDE brand if the amount of dissonance between a formal release and a distro-specific release is too high. But I know that will go over pretty poorly (and I think it ought to) with the community at large. However, when the distros are causing flooding into KDE's bug tracker due to their issues then there is something to be said for being a little hard nosed.
Aaron, don't let types like Jean push your buttons.
Look at it from their perspective; after reading some of your blogs it's not difficult for anybody to come up with some silly of topic nonsense that will set you off. And then keep you going by now and then putting another log on the fire.
Leaving everyone else, who are rooting for you and seeing the good you are trying to achieve, to watch in agony as you waste so much energy on them. And worry that you'll burn yourself out.
Please Aaron, just let them go and concentrate on the positive.
Thanks for all the great work you are doing. You are an inspiration.
i think packagers saw/see the need to backport features because the current releases are missing stability and features ..
this problem if it is will go away after kde4 matures ..when will kde4 mature enough for packages to stop backporting code to add desirable features/fix nasty bugs will depend on how fast kde coders can move and users expectations ..
it is a shared blame btw kde users who can have unrealistic expectations, kde coders who, my the nature of open source model, seem to pay little attention to areas uses see as critical and for packagers and distributors who try to meet users expectations and their own standards
you dont seem to acknowledge kde's part in this mess and i think some commenters have taken an issue in this ..
there seem to be a lot of angry people out there who read your block and use kde .. ha ha ha ..
i cant help it but wonder what if you were speaking in two halls instead of a block ..would you show up?
Woha.. stop the fire..
Anyway, let's rather discuss this. First of all, all the distros around provide a mess. Lots of them are just copies of each other, with a few enchantments which could have been done in other/original distributions.
First off, Linux/Gnu is getting popular. It has had remarkable sales in the netbook area, considering they have only been on the market for about a year. Actually, in the country I live, Acer announced that their Linux sales in the 3. quarter was more than the sale of Apple notebooks, and they have been around here for 15 years.
Second, Ubuntu is getting a lot of deals around. Dell, HP, Toshiba probably more. All those gonna sell with Ubuntu. That means Debian (deb) and Gnome probably gonna get a lot of attention in the future. I'm not sure were KDE is heading, but it's way more advanced than Gnome so I guess it'll happen some interesting stuff around there too.
Now I think Marks suggestion about a more agreed release schedule would help a lot. Things would probably not get as much delayed, and bug fixing might be easier, considering people are running mostly the same software.
What happens next? Lots of distros figure out they are actually exactly alike, and drop their project and merge with another. I'm thinking especially of the "big guys", Ubuntu, Suse, Fedora and Mandriva. Fedora is supposed to be fully open - unique. The rest is pretty much the same. Sure, Suse might have a better implementation of KDE (had before at least), but except for the package system, it's alike Ubuntu and Mandriva. Sure, the devs see big differences, but me, as an enduser, don't see any difference at all. The main argument for having lots of packagesystems (deb yum ++) is that it increases competition. However, I fail to see why foss developers wouldn't advance their software due to this. Come on, we're not Microsoft.
So what do we end up with? One or two enduser OS (goes on sale on computers by OEMs). One free OS, like Fedora and one bleeding edge OS with a big red warning, for powerusers.
So I dropped popular OS like Arch and Gentoo now. But that's because they are not important in this issue. Only powerusers install them anyway, and their rolling release so it's no big problem whatsoever. I mean, people who install Gentoo, they don't complain, they fill bugs.
Hell, there is probably a lot of stupid things I've said already. I'm no developer or anything, just an "advanced" endusers view on Linux/Gnu. I know stuff like "base" Debian and such also needs a place. Anyway, what we need is a simple map. This thingy which starts with a circel with a name inside (Linux, no doubt) then go into the next, Gnu or something. We just need some order guys, and I suppose you guys who are attending to those big meetings of important Linux folks, give them some preaching about it.
>> including ark from trunk with alpha quality features in it is amazingly boneheaded in terms of the end user and has had the added consequence of disenfranchising the upstream author.
>> not being able to open a zip file, however, is equally boneheaded.
I would go a step further here. An archiving program that doesn't open archives has a value of zero. An archiving program that has alpha-level features but can do some common operations has some value. It is far from ideal but infinitely better than a program with basic missing features.
When given a fundamentally broken piece of software, there is only one solution, and that is to pull in a newer version that works at least on some level.
Sure of course it would be ideal if downstream monitors all the applications before they are released and fixes them properly if there is danger of a critical feature omission, but that isn't exactly realistic.
Downstream has to work with what is released. The reason there are currently a lot of problems with distros shipping unreleased code is because of the period of instability in KDE4. Downstream doesn't do this for fun, they just know that their users will murder them if they release a distro containing an archiving program that is entirely useless, or a panel that doesn't autohide, or whatever program that does not have a feature some may consider crucial.
Same with networkmanager. Some people need 0.7 features to get on the network, period. The difference between 0.7 and 0.6 being the difference being some users having an internet connection and not having one.
I'm not necessarily blaming the upstream devs for this situation, but it would sure help not to release software that just isn't ready. If ark is not able to perform basic tasks which define an archive manager, then it should never have been part of a release. Better to ship the KDE3 version until at least the basics are working.
Anyway I think this problem is going to resolve itself for the most part. As KDE4 becomes more mature, there won't be a burning need to backport features. Donstream doesn't do this for fun, and if it isn't absolutely necessary then it won't happen.
@Ramsees: "but what I see is your idea of how distros should bow to upstream whims"
no, that's not what i'm saying. (how many times have i said that so far here?)
note that i talk about upstreams from KDE as well.
and i talk about supply chain management, cooperation, communication ... you are reading into what i wrote whatever your jaundiced view of the world wants you to. for a change, try reading what i've *actually* written.
"The fact that Mandriva couldn't provide a more realible way to provide an unzipping tool is not a failure in downstream,"
actually, i think they could have provided this in a much more reliable way. that's sort of what i'm getting at here.
"I better ask rocks to bleed than ask you to take a litle responsability"
ok, Ramsees. you've made your point, over and over. you don't like. you think i'm a turd. and you're not above personal insults and unfounded accusations.
that's not fine. so here's the deal: you can stop posting comments on my blog. this is the last time you get to use my goodwill (allowing comments here, responding to them) as your soapbox.
@Jean; "Never point to the fact that KDE might somehow be responsible for the mess. Aaron doesn't like that."
ok, you can join Ramsees. why don't you and he start a blog all about how evil i am =) you can rant about it all day in your own little corner of vileness.
but i do think it's amazing that when i step up to actually address issues that, quite literally, nobody else is visibly moving on, you say i'm not being responsible.
amazing.
So, since looks like you are in the good will to make this coordination happen, what is KDE dispossed to do for acomplish this?
@bigolewannabe: "I'm wondering whether the use of SCM as an analogy might be flawed."
yes, it is. at best, only an analog. it's not going to map perfectly 1:1 for reasons you mention (and probably others), and it may turn out as i study it more to be completely inappropriate (at which it's back to square one, because this set of issues *must* be addressed).
i do think, at least this point, that we have valuable things to learn from how other industries manage complex chains of input/output.
"Essentially the power relationships are significantly different. "
absolutely; however the end result (a finished product made through several steps of creation, design and refinement) is at least similar.
"But I know that will go over pretty poorly "
agreed; i don't think that's a reasonably viable option at all.
it's too adversarial for my tastes; we ought to be working together, not trying to lever each other against our will. (which is, at least in part, what's going on right now)
"Does this mean that the distros are being negligent in not waiting for a release is tagged before using the code?"
i think they are; but i also think they feel forced into this situation because they aren't sure how to communicate effectively upstream.
KDE has the exact same issues with our upstreams, btw.
"Does that mean we need a different development model to cut down on the amount of patching and backporting like the Linux kernel?"
it might. my gut tells me that changing the mechanics of production in just one part of the chain (it's much bigger than just KDE) without changing the relationships or mechanics of communication for needs and abilities is not going to work out.
Being one of those being quite displeased about KDE's premature 4.x "stable" release cycle, it's quite amusing to read that you're disturbed having to eat your very own dogfood. And please don't try to tell me that you've written about oh so different cases, this would be a cheap excuse, I won't buy.
Raise the quality of stuff you ship denoted as stable releases, set higher standards for inclusion and you may complain in future. Right now, I don't think so.
@mhogomchungu: "seem to pay little attention to areas uses see as critical"
i can tell you that is patently false. if it was true, KDE would be a very, very different set of software.
so my question to you is: what gives you this impression?
"you dont seem to acknowledge kde's part in this mess and i think some commenters have taken an issue in this"
if you read through the comments, you'll see quite clearly that i do no such thing.
i do think that you, and others, are putting far too much weight on KDE and giving our upstreams and our downstreams far too much credit. and i think i know why: they don't communicate with you, they don't put themselves out there, they aren't what you associate with what you see on the screen when you log into "KDE". this unduly removes responsibility from them. and so it's "KDE4 is slow" not "nVidia's drivers are amazingly bad with argb visuals and a lot of XRender paths".
but if i thought KDE were not part of both the solution and the problem i wouldn't be looking for ways to create better communications and cooperation across the board.
@Kasper Johns: "so I guess it'll happen some interesting stuff around there too."
KDE is already the market leader for Linux desktops in netbooks as well as retail sales.
"Now I think Marks suggestion about a more agreed release schedule would help a lot"
i addressed this assertion in an earlier comment, and it really does feel too much like a "magic wand" solution. just wave it and things get better! ignore the fact that it doesn't map to the realities of the software dependencies that exist between our upstreams and our downstreams.
"Lots of distros figure out they are actually exactly alike, and drop their project and merge with another."
perhaps (i'm doubtful, but ok, let's go with it..). we still need a way to coordinate and communicate with this smaller set of downstreams. for us it's not the # of downstreams, it's just that there are more than 0 of them.
it also doesn't address, at all, KDE's upstreams and the issues we see there either.
this is about a lot more than just downstream Linux distributions, and i wrote as much in the blog entry.
"what we need is a simple map."
i don't know if it'll ever be simple, but yes, we need that map. and yes, we don't have it. and yes, it's hurting us.
one of the challenges is that map changes with time, so it implies an ongoing process of cooperation and communication.
"just need some order guys, and I suppose you guys who are attending to those big meetings of important Linux folks, give them some preaching about it."
that is, at its core, what i am wanting. the open question is: if that is our goal, how do we *realistically* get there?
@Leo S: "An archiving program that doesn't open archives has a value of zero."
it opened certain archive formats just fine. let's not get carried away here.
"When given a fundamentally broken piece of software, there is only one solution, and that is to pull in a newer version that works at least on some level."
that's not in question here. (wow, another phrase i'm having to use over and over and over.)
the question is, if that's what is needed, how can it best be accomplished without screwing the user (and all of our reputations) over in the process by replacing one problem (can't open a zip file) with multiple others (crashes, leaks, whatever else is lurking in the alpha quality softwre)?
there was a break down in communication in this specific case. i happen to know how that communication went, and it wasn't all downstream's fault. emails went unanswered. and nobody, including myself, knew that was going on.
that's what needs focusing on, not quibbling about whether users need to open zip files. of COURSE they do.
and once we agree on that, can we now move on to discussing how to make such situations work *properly*?
"but it would sure help not to release software that just isn't ready"
so if we had never released ark, they wouldn't've pulled it from svn? if not, what would they have used?
why did mandriva release ark at all?
sorry, but your suggestion escapes the other half of the relationship.
"Anyway I think this problem is going to resolve itself for the most part. As KDE4 becomes more mature, there won't be a burning need to backport features."
and, to sound like a broken record, how does that fix future issues? how does that fix non-KDE4 issues? why did we have these same problems with KDE3? how does it address the issues KDE has with its upstreams?
your simplification doesn't address any of that in the least.
"Donstream doesn't do this for fun, and if it isn't absolutely necessary then it won't happen."
nobody said they did it for fun (or malice, for that matter). i disagree that it's happening because it absolutely must (tooltips on the panel were an absolute necessity?), but that's not really the issue either.
the point is that if we determine that such occasions arise, time and again, and we have no effective means of solving it with reasonable assurances of quality.
note that KDE provides qt-copy with patches Qt Software doesn't have yet (or, sometimes, ever ships) that we rely on. but most of our qt-copy patches are backports that will appear in a future version.
distros often ship our modified Qt, and this causes problems for Qt application developers because they usually test against Trolltech's Qt, not our bug-fixes-applied version and sometimes that changes behaviour and their workarounds for bugs break.
KDE is in that sense as guilty as the downstreams.
i really don't think many of you commenting here are either thinking about it thoroughly or have enough context to come to useful conclusions.
@fers: "Being one of those being quite displeased about KDE's premature 4.x "stable" release cycle,"
this has been discussed elsewhere in great depth. i'm sorry you lack the acumen to understand it.
however, how does your position fix the issues KDE has with its upstreams? or explain that we had the same sorts of issues with KDE3? or that other projects are having the same problems? (network manager has already come up, but there are others.)
in your striving to pour vitriol on my head, you've simply missed the point.
btw, the only reason i am replying to your comment is so that you don't confuse others.
"it's quite amusing to read that you're disturbed having to eat your very own dogfood."
i eat our dogfood every day. i run KDE straight from svn ... ... oh, wait, you're using a phrase inappropriately because you don't actually know what it means.
fail.
what you meant is "getting a taste of our own medicine" perhaps.
seriously, if you're going to insult me, do it with style or look the fool.
"And please don't try to tell me that you've written about oh so different cases, this would be a cheap excuse, I won't buy."
i don't expect you to buy it nor do i need you to. we just need people involved with creating the software we gift to people like you to Get It(tm)
@aseigo >> actually, i think they could have provided this in a much more reliable way. that's sort of what i'm getting at here.
Could you elaborate? Give the situation of a broken ark, what is the best option for Mandriva? The way I see it, there are a few:
1. Pull in unstable ark. Pro: You can stick with the same application but the basics work. Con: You're doing the author a disservice by releasing code that he does not feel is release quality.
2. Ship broken ark as released. Pro: no divergence from upstream KDE. Con: Users can't manager their archives and get angry.
3. Ship no archiving software. Pro: The user is not presented with a broken application. Con: User must choose and install their own archiving software to handle archives.
4. Ship one of the archiving programs from another desktop such as xarchiver or archive-manager from Gnome. Pro: Users get an archiving program that works. Con: That program doesn't fit into the look and feel of the KDE desktop.
So what should they do in this case?
@aseigo
Many of the problems you are talking about have no good solution. Yes they are a problem, but without actually suggesting some better processes, I really don't see what you're trying to get out of this discussion.
You talk about how it sucks that ARGB visuals and nvidia drivers caused problems for plasma. That sucks, but it's a fact of software. The ARGB guys tested their software to the best of their ability against the current use cases and thought it was reasonably stable. Same goes for nvidia and graphicsview. Then you come along with plasma which is suddenly a much more rigorous test case and exposes a bunch of bugs and corner cases. Sure it would be great if that didn't happen, but such is the reality of writing software. There will be bugs, and the first person to heavily use your code will discover those bugs and suffer for them.
There is no solution to this problem. New features are buggy until the bugs are fixed, and nothing can be done about it short of dedicating more resources to bugfixing before releases, which is just not an option for most projects (even well-funded commercial ones)
Then you talk about how backporting and changing software is a problem. For sure you are right. But there is no solution there that will please everyone.
The situation is: software A that you want to use is broken for whatever reason and is preventing you from delivering your software B. The ideal situation is when you can contact the author of software A with your concerns and they fix it, but there are a lot of very valid situations where that is not possible (software A is sporatically maintained, the fix does not fit into Software A's release schedule, the fix does not fit into Software A's design, etc etc)
So you go ahead and fix software A yourself (like KDE does to Qt), but that annoys the authors of software A. No way to make everyone happy in these cases. Either you have to compromise the quality of your software B (such as no ARGB support in plasma) to appease the authors of Software A, or you ship a better Software B, at the expense of causing problems for Software A which you've modified.
So what's your proposal? You invite discussion, then tell everyone they don't know enough to understand the situation when people respond. Not very useful overall.
@Leo S
"what should they do right now?"
The answer to your query is pretty simple.
As mentioned in the original post
"some downstreams feel that they have the right to speak for us as an upstream to their communities and have, by doing so, misled people and abused KDE's relationship with users in doing so."
Mandriva shouldn't speak for KDE. They should point users to KDE and tell them what a *wonderful* job KDE devs have done. They should then proceed to incur wrath of said users and not try and come up with fixes. They should wait for at least a year for such errors to be corrected. Meanwhile they should spend their time reading PlanetKDE postings about how the Kickoff start menu can now be seen in wonderful themes. If users then complain on their forums about lack of features, broken features etc, they should act in a manner that doesnt harm the *reputation* of KDE devs, which as you might have noticed is at its zenith right now thanks to the wonderful software they are releasing. ;-)
> in your striving to pour vitriol on my head, you've simply missed the point.
I'm far from it. There is no need to read my comment as personal attack or an attempt to do so. I simply expressed my feelings reading the upper halve of your blog entry.
> btw, the only reason i am replying to your comment is so that you don't confuse others.
Do I have to say "thank you" now or is this just a snidely reply on my comment. Ah, we won't be paranoid, I guess...
> what you meant is "getting a taste of our own medicine" perhaps.
No need to come across so arrogant with the few sentences above the quoted one. With "dog food" I meant the bugs you probably received, driving you to write this blog entry, just as downstream unnecessarily got, because of the questionable quality of KDE 4.0.x.
I had a look at Wikipedia and would still think that it's semantically acceptable, even though your's may fit better. As I'm not a native English speaker, it would be nice to know, if the usage of this expression in the context I had in mind was really that odd.
> we just need people involved with creating the software we gift to people like you to Get It(tm)
Well, it's not that I didn't participate improving KDE 3.x (definitely not to the degree you or others did), but KDE 4 isn't too compelling to me up to now.
@Leo S
It's silly that it needs to be a religious thing. If Ark isn't cutting it then let's use something else. If it's GTK based then so be it. I believe it was Aaron who once said that if Orca works best as a screen reader then great use Orca. Better that a screen reader works than if it's coupled to the DE. The bigger point is that the whole stack is better than the competition in that regard.
If a particular piece of software isn't ready or up to the desired level of quality then it's the role of the distro to decide on a proper alternative. They could help the developers of the software improve it (ignoring Brooks law FTM), they could package the alternative, they could delay the release until it's ready.
Some distros do that and I hope that everybody here could be enough of a realist to see that if an alternative isn't good enough then it's a realistic option to use the next best thing regarless of how integrated it will be into the DE.
@Aaron
Do you think ideas like Launchpad with it's way to integrate the distro's bug tracker into the upstream bug tracker is a positive? It would seem to optimize the communication channels and allow the distro to identify when their modifications are the cause of a problem so that upstream doesn't need to worry about it.
Should the number of Akademies increase to quarterly? Maybe hit a different continent each quarter? That would increase the chances for the major players to get in contact with the devs.
Something doesn't sound right when I say KDE Upstream Coordinator in my head. It sounds to rigid.
What other ideas for optimizing the supply chain have you thought of so far?
Oh dear, what a mess.
The "4.x came to early" discussion is done, over, beaten to death, sorry. Perhaps I'm getting too old to get my underwear twisted by an .0 release of KDE 4's quality and scope, but actively remembering the transitions between Gnome 1.x -> 2.x, KDE 1.x -> 2.x (I had konqueror aliased to krash back then and was quite pissed that kfm was no longer the default file manager), having used the Linux kernel at versions 2.0 and 2.6,
remembering how long distros shipped apache 1.x and 2.x packages simultaneously until everything settled, etc. (and working as an software developer with properitary vendors for sensors in embedded space) adds some (imho necessary) perspective. Looking at the (fantastic) work of Rasterman et al on Enlightenment and their perpetual "will release really soon" beta state adds another one. I don't know what press releases you were reading, but KDE 4.0 was - quite literarely - advertised as "will eat your children" in advance by the very same developers that are now accused of letting a half finished product "go gold" (which is a silly phrase all by itself, but I disgress).
With that being said (and acknowledging that people disagree about the minimal requirements to do a .0 release, and acknowledging that scratching an itch is not something that can be prevented from happening, be it down- or upstream, and taking different definitions of terms like "prime time" etc. into account ):
What exactly is wrong with shipping the KDE3 version of ark until ark4 is ready for prime time? Please appologize if this is a naive question, but the distro I'm currently using does/did exactly this: Ship Qt3/KDE3 based apps to fill the gaps until they are closed by Qt4 apps. (Again, not much unlike the situation when GNOME underwent the last radical redesign, years ago. I remember having GTK1 apps installed in parallel to GTK2 apps on my system for this very reason at the time of the infamous lets-go-spatial! release of - IIRC - GNOME 2.6 . Heck, GNOME was even forked because people felt that the GNOME guys went totally into the wrong direction, focused on the wrong things, didn't listen to their users, etc. . I wonder what ever happened to goneme, it had *so* much potential ;-) ).
So: whats wrong with using knetworkmanager3 or amarok 1.4 (to name two apps currently opened on my system) until the respective equivalents are done? Why not a GTK app like squeeze (which I use for the rare occasions when I need a graphical archive program. yeah, not a typical user me, I guess).
In contrast to discuss for the umptenth time the decision to denote a developing release with a major version number, I would be interested to discuss questions like
- How can distros better communicate with upstream to gauge when a certain feature is to be expected to plan for alternatives in advance?
- How can communications between (for example) X.org, fd.o and KDE be improved to provide additional bug testing in advance of features needed in the future (e.g. "Hi. Our upcomming release relies heavily on stuff like XRendere, RGBA, etc. .What is the status of the tests/features and how can we help to find bugs before they bite us ?").
Reading the article, I thought this might be the proper forum for such topics, but it seems to be another round of "lets see how long it takes before comments are again disabled".
@setec
Amen! Preach it {brother|sister}!
Does anybody know if there has ever been a graphical mapping (graph-theory map, not picture map) of all the potential dependencies of a project like KDE? It would be interesting to see how many different projects we are upstream to and how many projects are upstream to KDE. Or even all OSS projects out there to see what the landscape looks like. It might give an idea of the scope of the issues at hand.
@bigolewannabe: "Do you think ideas like Launchpad with it's way to integrate the distro's bug tracker into the upstream bug tracker is a positive?"
i think that's a strong step in a positive direction, yes.
"Should the number of Akademies increase to quarterly?"
we're doing them semi-annually now; one in Euope mid-year (Akademy) and one in the America at the start of the year (Camp KDE)
"What other ideas for optimizing the supply chain have you thought of so far?"
as i noted in the blog posting, i'm really at the start of thinking about this challenge. i don't have what i'd consider "good" answers yet, in the form of something i could implement today with confidence.
the problem is identified, i have an avenue of exploration that hopefully will bear fruit and we need to all talk about it.
as rough starts, however:
i do think it's pretty important to find a way for different interests to publish what they are going to be relying on in the near future. we need to know that Mandriva is looking for a zip tool, x.org needs to know plasma is going to rely on argb visuals .. etc.
upstreams are expected to publish their roadmaps and plans for downstreams, but the reverse communication happens sporadically if at all.
we also need to have a way to easily identify and find downstream changes; i often find out about modifications to my code by downstream only when bug reports start trickling in. that's a bit late. =)
as it is, we have no effective means of patch publication in the FOSS world that crosses project boundaries.
(i blogged about patch servers a couple years back, and i think that's as valid an issue then as it is now)
i think we need to be able to plan roadmaps in the same physical/virtual space, even if our plans don't dovetail. i talked with Jono from Canonical recently about this, and his thought was that we should really be having roadmapping meetings where we can discuss hot issues (as defined by the attendees) so we can actually coordinate things.
we need to have identifiable points of contact and trackable communications that are more robust than private email, more easily cataloged and tracked than the devel mailing lists, less rigid and cumborsome than bugzilla, but a bit more structure and process than a vanilla wiki...
there are large gaps in the problem that i don't have answers for yet.
and a lot of the above is pretty light on details. it's going to take me a while to come up with some comprehensive, and i'm hoping others will engage in the process as well so it's not just left up to me.
@setec: yes, with a bit more prudence (e.g. choosing ark from kde3 until the kde4 version is ready) downstreams could do a lot better job.
most aren't making those good decisions, and not just with KDE but all kinds of software.
so obviously they aren't receiving the kind of support and information they need. what support and what information is the question. i'm still figuring that out.
the questions you pose about how can we coordinate and communicate better are precisely the issues i am thinking about.
"I thought this might be the proper forum for such topics,"
that was my hope as well. =/
@Leo S: "Could you elaborate? Give the situation of a broken ark, what is the best option for Mandriva? "
here's what would have been the best scenario, imho:
* Mandriva identifies the needs: we need zip files to be uncompressable
* they have a place to register this issue (bugs.kde.org is not suited for this at all, btw
* upstream offers advice; in this case it would likely have been something like, "These are the changesets that address that need, let's apply them to the 4.1 release." or "Use the KDE3 version of Ark." it might have even been something else; i'll bet the author of ark knows better than either you or i. but if they aren't reachable or stop communicating, if the conversation thus far is done in the open and flagged as needing attention, someone else upstream can look after it.
* Mandriva then decides which path suits them best, and goes with it.
now, at that point, even if Mandriva did ship ark form trunk after all that at least there would have been a process in place and it would be plain what the options were they had to choose from. i think the odds of them shipping ark from trunk would be much smaller with the above process, however.
and hopefully that would be an approach of last resort and three months ago we would have had a meeting where the issue of unzipping files came up and it was identified.
and let's be fair: this isn't just Mandriva even though that example keeps coming up here. i can give you just as serious examples from Kubuntu, OpenSuse, Fedora, etc..
@bigolewannaba: something like this?
http://solaris.bionicmutton.org/graph.png
it's from Ade: http://people.fruitsalad.org/adridg/bobulate/index.php?/archives/670-KDE4-on-OSOL-mid-november.html
and it's obviously Solaris, and doesn't show our downstreams .. so.. not perfectly what you're considering.
i do know the OpenSuse guys have some wicked crazy graph generator thing too.
@setec
you asked "What exactly is wrong with shipping the KDE3 version of ark until ark4 is ready for prime time?"
one thing i can think of are the kde3 libs it will need as dependencies .. i have started using kde from svn in kde4 and i know you will need kdesupport, kdelibs,kdepimlibs and kdebase/runtime as minimum requirements to run any kde4 app ..having all these libraries just to run one kde4 app in a non kde4 environment could be too much ..i can assume having the necessary kde3 libs to run just one app to be problematic..especially considering size constrains in iso images
Getting extremely practical here.
The KDE community provides a product; that is, the KDE platform.
Distributions build a composite product on top of KDE (and the GNOME platform as well).
The problem in fact is the very idea of a release itself. If KDE X.x isn't ready as a product, then obviously the distribution will ship a flawed product themselves.
That's it. That's all there is to it. You can't communicate your way out of that; either KDE has the functionality that it needs to have, or it doesn't.
You can't communicate your way, for example, out of not having ZIP functionality in Ark; ship an alpha, ship nothing; either the way, the result is the same: an incomplete product.
The corollary here is solution C; that is, ship an alternative that provides the same functionality (Ark3, a GNOME tool, etc): but in this case we're admitting that the KDE product doesn't have the necessary functionality anyway.
Bottom line: if a distribution doesn't have a good product, should it ship it?
You can't force KDE to move development any faster than they do, or release in a different manner; strictly practically speaking, it doesn't matter what version numbers they give the platform: it either provides functionality a user needs or it doesn't.
So if a distribution cannot provide a product on top of KDE with this functionality, why attempt a half-arsed kludge? And yet this is what time-based releases do here.
Especially on the time-scale of free software: a new distribution, a new OS essentially, every six months? Your desktop environment (KDE, GNOME) seem to complete a development cycle every six months.
Syncing release to the cycle of your dependent products (KDE, GNOME) is a horrible idea. You think in the same six months that a product is made, a distribution can make a composite product of greater/equal quality on top of it?
That's no extra time there; there's literally no extra opportunity to add value, fix bugs, etc. It's no surprise there are so many rough spots in Linux distributions.
It's the culture. Development cycles of KDE and GNOME are advertised as "releases" and distribution "releases" are made over top of them even though they might not be ready for use, bug-tested, or polished.
The only real solution (for the USER) is to, quite simple, ignore any release that doesn't have the functionality you require.
Distro X 2007 doesn't open zip files? That's a bummer. Don't use Distro X 2007.
The associated problem, however, is that in free software releases, we have the advent of all new sorts of functionality (new drivers, bug fixes) coupled to a whole class of partially-implemented functionality.
Why can't complete-functionality be segregated from impartially-implemented functionality? _This_ is fact is the critical point. The idea of synergising everything into big commits from every project in the OSS world every six months, with pieces that are half-gold and pieces that are half-dirt, and expecting to have a coherent product and user-experience is completely counter-intuitive. It just won't happen, and everyone will be asking themselves why they have the same problem every six months...for a very long time.
There must be a new paradigm in quality-control design for free-software, I think. There must be a way to isolate, within a single project (ex, Amarok, Ark, Firefox), features and capabilities that are complete, and features and capabilities that are incomplete, and making sure that the latter are never allowed into "releases".
The segregation of development efforts into strictly-maintained, and into new-functionality, may require more man-effort than the OSS community has at present though. I was actually surprised that the KDE community had time to do maintenenance releases of 3.5.x while still going forward with 4.1.x. That's remarkable, that does give some hope that a solution to these quality problems may be found.
@Leo S: "Many of the problems you are talking about have no good solution."
every problem is a puzzle waiting to be solved.
"You talk about how it sucks that ARGB visuals and nvidia drivers caused problems for plasma."
yes, and i think that the downstreams (in this case KDE) can and should be doing a lot more to help them figure these things out a lot sooner. XRender has been around how long? argb visuals in x.org how long? why, then, did these problems still exist in 2008?
because their roadmaps and our roadmaps were not aligned. while x.org was working on argb visuals i was making kicker use some horrible hack to make it appear translucent (*cough*just don't drag a window behind it*cough*).
why weren't they aligned?
well, we weren't communicating our roadmaps to each other at all! sure, the odd blog here and there, running into each other at conferences ....
... but did we ever sit down and go, "Hey, this is where we want to head. Where are you going? Where do we overlap and converge?" maybe x.org would have said, "Look, without real world apps using this stuff, it's going to be pretty rocky."
and while nVidia's driver cock-ups really made KDE4 look bad, what did we feel was our only approach? yep, we mentioned in a widely publicized press release with the announcement of 4.1 that nVidia drivers were problematic. that really didn't help their reputation did it?
such vicious cycles where like crabs in a bucket we step on each other in a random scramble out that none of us can escape into the broad market.
"So you go ahead and fix software A yourself (like KDE does to Qt),"
which i dislike us doing, btw.
"but that annoys the authors of software A. No way to make everyone happy in these cases."
but Leo, we aren't even trying to make people happy. right now there's this huge assumption on so many people's part that it's all unsolvable. and as long as we tell ourselves that it's unsolvable, it will be unsolvable.
moreover, we tend to treat the patternistic challenges as localized problems: it's Mandriva with KDE4's Ark; it's nVidia with Plasma; it's network manager 7 with $DISTRO; etc.. we don't stop very often to discuss the systemic issues that cause these localized problems.
the result are lots of localized problems that each seem intractable on their own.
and that's why i'm looking to other *systems* and how they work; i'm not actually trying to figure out how to get rid of qt-copy. i'm trying to figure out how to make the system work better for all of our needs, and if we manage that then qt-copy will "magically" go away on its own as a result.
disease vs symptom.
"Then you talk about how backporting and changing software is a problem. For sure you are right. But there is no solution there that will please everyone."
i agree that there will be time that there is no reasonable solution that is identifiable in a reasonable amount of time/effort that pleases everyone (or even, sometimes, anyone), but i think that most of the time there are at the very least excellent compromises if not straight out win-win-wins that we are not even trying to consider.
and we don't need a 100% solution; after all right now we have a maybe 10% approach. (maybe even that's generous.)
there's a lot of room between 10% and 100% and the user will benefit from every bit closer to 100% we get. i'd be ecstatic with a 80% hit rate on issues like this.
"So what's your proposal? You invite discussion, then tell everyone they don't know enough to understand the situation when people respond."
no, i said you and some of the others commenting here aren't grasping the situation. i don't really see a lot of my downstreams or my upstreams here, do you?
when i invited conversation, i was hoping for people with insight who are in places to affect change to interact either here on their blogs. i really wasn't looking for the Ramsees or Jeans of the world.
i'm looking for quality discussion, not any discussion. i'm looking for that which will move us closer to solution, not closer to wanting to tell each other to fuck off.
@Jud: i think there are huge amounts of (hard to swallow, perhaps) truth in what you say.
it may be hard to swallow because it does run so counter to what is happening now.
you say we can't communicate our way out of it, but i don't see these mechanics changing until we actually start talking to each other about our needs, our requirements and coordinating them.
beyond the needed communication just to heal the currently broken systems, you said:
"There must be a way to isolate, within a single project (ex, Amarok, Ark, Firefox), features and capabilities that are complete, and features and capabilities that are incomplete, and making sure that the latter are never allowed into "releases"."
the isolation of those features, deciding which features should get attention before others, which are critical and which can remain incomplete a little longer ... that requires communication and coordination of some sort otherwise someone, or all of us, is just left guessing.
hopefully we can automate or make the communication of these things emergent properties of our processes, but these are still things that need to be identifiable as well as mutable when required.
other than that, yeah ... you've got some really, really good points there that i'll be considering for a while. thanks for sharing.
kde4 is great. It has left stuff to do, but at this time, is very strong, and i hope that with kde 4.2 plasma will be rock solid. The effects are not just bling bling, some are really useful, like present windows, and, is very easy to activate, i configured it, in a way, where i move the mouse to the righ corner and it is done, withoug keyboard.
Obviusly, kde4 promised a lot, maybe was the mistake, but not from developers, but community.
Aaron: I think blogger has the ability to delete comments. Much of what was said above is not worth even reading.
It is helpful to remember that what is being done here is impossible. Yet it is. So...
KDE4 is almost a 'throw away all the old, start from scratch' release. Not quite, but for purposes of discussion.
Considering KDE3's state as a mature desktop, and the breadth of options and ability, there is no way that it would be without disruption. In fact, the idea that KDE3 could be replicated in a rewrite is preposterous. Yet it is happening.
The fact that the distros are putting resources into plugging the obvious holes in KDE4 to release means that they and their users see something they want to use. This is good. Very good.
What KDE4 ran across in the desktop stack is the reality for whomever wants to develop a non trivial application for linux desktop. There are very large holes everywhere, which create a perverse incentive; the software that gets written stays within the bounds of what has already been written (and where associated problems in the stack have been solved). Very bad. Breaking out of these bounds is going to be painful for everyone involved :) But absolutely necessary. Reality has an odd way of introducing itself, and makes people uncomfortable. This is good :)
I don't know if supply chain management ideas apply. They imply extraordinary efforts in coordination and organization so that when a production run happens, everything is there. The work involved in setting these things up is enormous.
KDE and the free desktop generally is like producing and selling a car without the wheels designed yet. Or the interior. Or a set of pedals for motor with a space for the motor which hasn't been made yet can fit. Maybe.
The only thing that is hurt by these so called missteps is a brand, or better The Brand. Brands are maintained by tight and rigorous control. Which I think is antithetical to the process and philosophy of free software.
Every problem can be solved by centralized decision making. Except of course that centralized decision making doesn't work. The urge to control, the urge to organize, the urge towards efficiency is very tempting. Free software is none of these, yet has done the impossible.
I find that usually if there is a problem of coordination in a large project, the problem is not organization. The problem is interests. People will do things that favor their interests. Maybe the distributions and upstream projects have a clash of interests.
Derek
To Aaron:
You are correct; communication is definitely needed to fix the entire process.
I just meant to say that by the time we realize the flawed process has led us to a dead-end (a lame-duck release of a tool for example), that's really the point at which communication comes too late.
@Jud: yes, at that point it is indeed too late.
here's hoping we can be early enough for some future as-yet-uncommitted-oops
Reading the comments on this story makes me sad, the signal to noise ratio is really low. I'm also surprised that Aaron bothers to try to answer all of these people who are clearly hell bent on proving their point (non of which relate to the blog). It really shows how much of an upstanding guy he is and how much he just want to make things better for everyone. That is what FOSS is about.
Anyway, I think a summary here is:
There are problems on multiple levels of the software (supply) chain. These problems are different and have different goals and motivations. Problems in one level of the chain affect other levels in ways that the former might not understand.
So, you have one level saying here is my problem, I need to fix it; and another level saying here is my problem I need to fix it. The major breakdown is that they aren't working together to find a solution that works on both levels.
This might be unfair, but to take the Ark situation as an example. The distribution level could have contacted the author of Ark and said, "The current released version of Ark isn't working for us, what can we do to make this work" At that point they would discuss a solution and work together to achieve it. For example, maybe the distro will help the author stabilize the development Ark and get a release out in time for the distribution to include it.
Here's hoping that Aaron's blog reaches out all levels of the chain and helps them work together.
@mhogomchungu
There is a point in time when keeping Qt3/KDE3 incl. dependencies is probably not worth the trouble.
Up until now, KDE centric distros probably needed the whole KDE3 shebang not only for apps like ark, but also for amarok and koffice (both nearing their respective KDE4 releases, but probably not suitable for everyday production use), knetworkmanager3, etc.
The point when shipping a complete KDE3 base environment just for the sake of providing a dwindling number of legacy apps gets unjustified may have already been reached, I don't know.
But I maintain that the decision which software should go into a release should be made using more information than a three digit number behind the package. Ideally, this involves a two-way communication over long-term goals and mutally accepted milestones / test patterns before something is considered worthy of a distrobution - release.
And altough the blame game is a fun pastime for the whole family,
it is not really a single entity that has to take the whole blame here. Distros were not forced to ship KDE4 as their main desktop. KDE4 did not test the current state of affairs regarding X.org features before betting part of their reputation on using features advertised as ready for consumption. And so on, and so on, casting stones. KDE4 probably should maintain better, more finegrained communication channels to up- and downstreams and negoiate roadmaps with them more closely. Distros should seek ways to ensure a suitable level of feature completness before betting their brand, reputation and farm on the version number of an (so far) unrelated project. Providers of critical low level infrastructure should probably activly seek out "customers" for their shiny new tech to get more bugs busted. And these are only the things I (as in I, the armchair software coordinator) came about after thinking for approximately 2 minutes on this problem.
And regarding focus/priority (while keeping in mind that people designing "exotic wallpapers" not necessarily have the necessary knowledge and expertise to add functionality to apps like ark. I know that I would not have the expertise to design a proper exotic wallpaper without risking unreverseble damage to many retinas):
If the KDE4 devs would not get their low level framework out of the door (e.g. stuff like the taksbar et al) early on (4.2 seems a good candidate for introducing said mid-level API's), developers tend to get a bit nervous and drag their feet. If stuff like the menus, the Taskmanager, etc. receive no polish after implementing the necessary base functionality, people tend to point towards other desktop environments in their .20es point releases and say "see, thats the way it's done: stable, featureful AND polished!". If only new, exciting stuff is worked on and the "old school" desktop is neglected, people start to scream bloody murder. Release to early, people search for their forks and other lynch mob gear. Release to late, and developers start to move to greener pastures and users blame you for not doing a new-feature release in over a year.
It's a miracle there are any developers left in this segment of FOSS that seem to enjoy there job, come to think of it.
I understand there's not some magic wand, but I look upon it a bit like a test and try thing. Unless there's a big issue about it however..
Anyway I've been thinking about the Android project. It might be completely irrelevant, or maybe not. From what I've read, there's Android, and the members of the open handset alliance (google, nokia, samsung etc) will in the future build upon this, freecode.
I suppose these guys got some sort of smart plan so things don't get so messy lke it is on the desktop right now.
But then again, a desktop is more advanced than a phone, so it's probably more difficult to handle... But I guess you can think around it. Ask those Google folks (I suppose some of them attend to Linux meetings) what their plan is. Or maybe cooperate with them some way.
This will never be solved, and you'll just have to look at it positively that KDE and its applications are in demand. As long as you have an open source repository then varying states of applications quality are going to be seen without some focus.
It goes back to what I've said many years ago. KDE badly needs a completely KDE focused and sensibly packaged distribution of its own, which gives everyone else a benchmark as to how things should work and when things should be bundled with it. It would also get developers and users using much the same platform so esoteric downstream bugs are lessened and you can recreate what someone is talking about in a bug report.
Aaron, as a KDE end user, I ask that you ignore the small number of idiots attacking you in these comments. After all, it is inevitable part of the human condition that some people are just going to be negative. KDE 4.1 seems pretty cool to me in my limited needs. In my linux evangelicalism, I tell people that KDE 4.2 is going to be clearly superior to Vista. And that is what matters right now. On the other hand, understanding the issues with KDE 4.0 will help help the process of creating the inevitable KDE 5.0. Your blog is a resource for future development.
Some comments (there's no way I can reply to everything, and it wouldn't add much useful things to the discussion anyway, so I'll concentrate on the essential):
* @Jamboarder: guidance-power-manager is not distribution-specific. It's developed in KDE extragear and we're shipping it in Fedora 10. That said, it is not the solution which will eventually be the upstream default, which may (or may not) be why openSUSE decided on backporting that solution instead.
* Ark in KDE 4.1.3 does open ZIP files, at least it worked on the one I tried to test this. The big missing features are:
- context menu integration into file managers (Konqueror, Dolphin, Krusader 2)
- support for 7z archives
- support for password-protected archives
Having looked at backporting the context menu integration only, I quickly came to the conclusion that the changes were so many (the context menu requires batch mode, which touches a lot of Ark code) that shipping trunk wholesale would be a lot simpler and most likely also more reliable (selective backporting works well when the changes are small and targeted; when they're huge, the potential to introduce bugs gets high quickly), so I fully understand why Mandriva is doing it. (In fact, I suggested doing the same in Fedora even without knowing Mandriva did it, but we ended up not doing it for F10. We might do it in an update now that the new Ark is getting closer to release though. That said, we'll most likely just upgrade the whole KDE to 4.2 at some point, so the later, the less worthwhile it becomes to spend our time backporting.)
* In Fedora, we're very careful to backport only the pieces which are really needed to get the features we want to backport. For example, I ripped out some Plasma theme changes out of openSUSE's panel autohide backport because they weren't needed. As I said above, backports should be small and targeted. But what if such a targeted backport is not viable because the changes are just too big? That's exactly the case of Ark.
* I agree sharing our patches in a central place would be beneficial to everyone involved.
* In the case of Ark, I think the mistake was to merge the libarchive-based-ark branch just because it happened to be ported to KDE 4 already. A straight port of the KDE 3 version doing only the minimal changes (i.e. still using K3ListView etc.) would probably not have taken too long. It is what I did to Kompare, it just took me one night. Kompare also had (well, still has) an experimental development branch, the 3_way_kompare branch, but I quickly realized it wasn't feasible to get that up to task in a reasonable time, so I opted for a straight port and was given Kompare maintainership for that work. IMHO a similar solution would have provided a much better Ark. Sure, the same old minor bugs and missing features from KDE 3 would still have been there (Kompare has years-old open bug/wish reports), and sure the port can introduce bugs (there's one in Kompare I fixed quickly in the 4.0 timeframe and one I still need to fix because I just noticed it very recently and nobody else reported it), but still it's a better thing to ship than a rewrite with major missing features (for example, Ark in KDE 4.1 can't even sort file lists alphabetically).
(Oh, and I'm not the Kevin in the first comment.)
"distros often ship our modified Qt, and this causes problems for Qt application developers because they usually test against Trolltech's Qt, not our bug-fixes-applied version and sometimes that changes behaviour and their workarounds for bugs break."
Unfortunately, Gentoo doesn't, and this just bit me. Without one of the bugfixes in qt-copy, the new Plasma notification popup code crashes every time a popup with a button on closes. This means that, every time I got an IM in Kopete, Plasma crashed as soon as I viewed it. As you can imagine, this is very annoying.
(I should probably use qt-copy anyway, since I'm using svn trunk of KDE, but I prefer to use the distro-managed package if possible.)
Post a Comment