exhibit a
an artist's impression of what a nocturnal triager might look like
because of this the relative value of bug reports varies pretty wildly. often more information is needed from the reporter to bring a report up to par. sifting and sorting is usually a job for the bug triage person, but they are rare (and often odd, see exhibit 'a' for an example of what one might look like).
now, we have a few great triagers in the project (thiago, binner and maksim all spring to mind) but they also tend to be busy with things like writing source code. so what do we do? well, we could try and find some more bug triage type people, but my hopes for that are limited based on historical trends.
moreover, as you can see from the chart below (click for a larger version) we're fighting against a rising tide that's almost as scary as exhibit a (the black line is fixed bugs, all other lines are open bugs in various states):
exhibit b
graph showing one of problems of success: the ever growing bug report database
this continuous creep, even in the face of the impressive black line of fixes, is a problem of success. we have more applications (which means more code) and more capabilities in our libraries (which also means more code) and more users than ever before. more code and more users equals more bug reports. now, we want more applications that cover the set of things people need and we certainly want more users but the resulting inexorable flood of bugs is something we could do with out.
so that's the problem. or, if you're more positive minded, the challenge. how can we answer? i started by saying that not all bug reports are created equal. i think we ought to start taking that into consideration.
if a report comes from a packager who is responsible for building and grooming kde for a distribution, for instance, that report is statistically going to have more weight for me than if it came from a random end user. this is simply because those packagers tend to have more knowledge and time to commit to the problem.
likewise, if a bug report comes from a kde developer, it will statistically have greater meaning to me. if a bug report comes from a well known bug reporter who has done a great job in the past with such reports, it tends to carry more weight for me.
given that most kicker bugs get reported repeatedly, i wouldn't miss that much if i kept my eye primarily on well known bug reporters such as those listed in the previous two paragraphs. i could dip into the "general" pool of bugs whenever i had caught up with those "trusted" reports or when i was looking for something less challenging or just different. note that things were pretty different a few years ago when we had both fewer users and a smaller code base, but today such a system would make me far more productive when combing through bug reports.
so i put it out there for discussion: should we have a stratified bug database system, where some people have a greater level of priority than others so that while everyone should still be able to report bugs, not everyone demands my attention equally? is bugzilla capable of this?
it's also slightly frustrating that there are OS vendors who package and ship kde, even as their default desktop, don't have a means to upstream bugs that are truly kde bugs and not just the result of their customizations. but that (distributed bug databases with push-button up and down streaming) is another story altogether .......
11 comments:
First, I appreciate the idea to make a better balanced bug system - I know that I need some more experience to write good bug reports and o it should be more important what other people with more experiences are saing. But you have to be careful that you do not completly drop the other bug reporters.
One idea could be to improve the "status" system - the "confirmed by popular vote" is a nice start: some people should have he possibility to give more votes or more weightily votes so that a bug is faster confirmed if some well known votes for it.
But, I have to add: I reported some bugs in the former time, and in some cases I got the feeling that no one cares about it - I know that everyone (especially the developers) has much to do, but a small message like "is on my todo list" is much, much, much better than to hear nothing and just "talking" with normal users on the bug list or to be afraid of that the bug will find its way into the next release.
And if you then ask as the reporter "is there no one who could say a word?" then you annoy the developers (which is also understandable).
And, for the archive: Here the bug where I was wondering if no one looks at it, where I was afraid that it would find its way into the next release, and where I annoyed a developer asking (I annoyed you, and I am sorry for that, aaron ;-) ): ghns wallpaper problem
Bugs where you really miss someone with deep knowledge are the bugs around crazy konqueror - I really tried to give information, but when you get the feeling that no one is listening, it's a little bit sad :-/
Konqueror takes 100% cpu and Input fields slow down in konqueror.
But, as I said: These are only some of the bugs I reported - others get fast and good response, and it showed that the system works most times :-)
Btw.: Another idea would simply be to make a new user type in the bugzilla, like "trusted users", which are able to change the state of a bug and to pick out duplicates. Maybe that would help the developers, too...
liquidat
> but you have to be careful that
> you do not completly drop the
> other bug reporters.
of course, which is why i noted that everyone should still be able to input bug reports.
> but a small message like "is on
> my todo list" is much, much,
> much better than to hear nothing
this requires knowing that it'll be on your TODO and actually researching that it's a valid bug. that often takes a significant amount of time. take into consideration the sheer number of new reports we get, many of rather low quality and utility.
> "confirmed by popular vote" is a
> nice start
unfortunately i've yet to see this prove to be of any real practical use =(
i have seen it abused many a time by people posting to dot.kde.org or kde-look.org telling people to vote for BR#XXXX
> like "trusted users", which are
> able to change the state of a
> bug and to pick out duplicates
this is a "bug triage person" and as i noted in the blog entry we haven't historically been able to attract nearly enough such people. it's a tough and thankless job.
so instead of trying to fight against the trends, i'm suggesting that we should find ways to sort out quality bugs as they are entered rather than afterwards.
How about creating levels of maintainer? instead of created some trusted user which in my opinion will make some user dissappointed since they are not trusted enough, why not making a stratified maintainer.
So, roughly you'll have Maintainer level 1, level 2, so on. Maintainer level 1 should be responsible to the main development of the package. Level 2 will have less responsibilities, maybe applying trusted patches. And level 3 could be the filtering guy. And as you may have suspected, it's a pyramid model.
I just doesn't feel right with the term "trusted user" and "less trusted user". It may works though, I have pretty optimistic mind. But I'm thinking of this. What would you feel if you're a KDE user and someone tells you: "I don't trust you that much, so big chance that you'll be ignored."
Just a thought :)
If I'm understanding correctly, we need a way to tag some bugs as more "credible" or "important" than others? Based on who is submitting it?
This sounds like a reputation type system. The more that a person has proven themselves, the better reputation they have. So, if someone submits a single bug, the bug isn't ignored, but could give way to a bug entered by someone who has entered a number of quality bugs. (quality is a little subjective, but could be reflected by other people, or the coders who fix the bug).
The problem here is to NOT simply base a "reputation" on the number of bugs. That's a factor, but there are other factors - is the submitter a packager? Is the submitter a core developer? These can all lend weight to the reputation value for a submitter. And hopefully then get the right bugs addressed...
(Fuzzy logic might be useful in determining the reputation values...)
My thoughts, for what they're worth.
@Toni
> How about creating levels of
> maintainer?
that would be a "bug triage person" and historically we (and pretty much the rest of the open source world) have had a devil of a time finding such people.
@ca_grover
you pretty much described exactly what i'm thinking =) thanks
As I see it, the problem is that the developers are overwhelmed by the number of bugs reported, so what we need is a system to classify the bugs.
Since this is next to impossible without the triage person (which we are lacking) I tend to agree with the idea of classifying instead the bug reporters.
However I think it's a little bit reducing to classify only a few selected person as "power bug reporters" I would much rather have a system in place that would assign a punctuation to bug reporters. When a developers picks up a bug he would assign a number of points to the bug reporter and so, the developers could have the bugs sorted by the "relevance" of the bug reporter, however if a complete stranger posts a really good bug report he would be very rapidly bumped up, at least in the list of bugs of a given sub-project.
Naturally one could assign initial punctuations to the maintainers, distribution packagers etc...
I think this is more a less what you are thinking with a twist, it gives the normal bug reporter equal rights and this way no one can claim that kde developers don't ear its users.
@Helder
well, we have a few people doing triage, but not nearly enough to keep up with the flow of bug reports. we'd need a rather impressive staff of full time bug triagers to keep up, to be honest
and what you describe regarding classifying bug reporters is exactly what i'm thinking ... huzzah! =)
You'd have to be careful with Helder's plan. I think it's a good initial solution to the problem, but *could* lead to other problems if not managed properly.
For instance, take the example where a new submitter does submit a good bug, and the coders pick it up and give him/her lots of points for this bug. Does this mean that the submitter is now on par with a KDE developer for submitting bugs? Or over time they submit 2 or 3 more, do their submissions now have equal weight with a submission from a well known developer?
My thoughts on this is that the coders MUST be able to apply points, but this only serves as one factor among many that determine the weight given a particular submitter. Otherwise, we can quickly return to the same state we have now.
We should avoid a situation where a prolific submitter can add up points and be considered before a single bug report from someone who is a core developer and a packager. This developer likely would be submitting a very legitimate and/or serious but needing immediate attention. They shouldn't be given less weight than someone who submits a number of minor bugs....
My 2 cents worth... If I'm not stating the obvious...
(and it should be noted that ANYONE submitting a bug, or taking part in the process is valued...)
The #1 Selling Internetmarketing Software In The World free site submitter traffic web ...Accused Of Using Secret Intelligence To Optimize Your Website Rankings Fast.
Nice Blog!!! I thought I'd tell you about a site that will let give you places where
you can make extra cash! I made over $800 last month. Not bad for not doing much. Just put in your
zip code and up will pop up a list of places that are available. I live in a small area and found quite
a few. MAKE MONEY NOW
Great blog it really nice. To learn more about blogger visit
blog The best blog
Post a Comment