Writing opinion pieces about a field that one is, for all intents and purposes, a "lay person" in is a pretty gutsy thing to do and fraught with the danger of, well .. getting it really wrong. This is why those who are recognized critics of food, wine, theater, movies, architecture, etc. tend to have expert-level knowledge of their respective field. They usually know many of the people in the industry very well and can recite arcane facts about their topic with command, ease and accuracy. This is not to say that all professional critics are competent, but professional critics do tend to have a certain standard of competency applied to them that lay people offering their personal opinions, valid as they may be, just don't have. The resulting difference in quality of analysis offered tends to speak for itself.
Everyone certainly has an opinion and an experience to relate. This, however, is different from an analysis or a critique. By way of example: when eating a meal at a restaurant cooked by a professional chef (and their staff), most people can identify whether the food tastes "good" or not to them. Different people will say the same dish is better or worse depending on their personal taste, of course. However, relatively few people would be able to identify all the ingredients used or preparation methods employed in the making of that dish. Even fewer would be able to accurately identify things that would improve the dish, such as which spices/herbs/flavours may have been a better compliment to the meal, what cooking method would have produced better results or what variations in ingredients would improve the result. Any food critic who can't do that, however, has no business being a food critic and will likely never make it to the "big leagues" of food criticism.
The difference between having a personal experience or opinion and offering an analysis or critique is therefore significant. I believe that sharing one's personal experience is great. Creating an analysis that one is not equipped to do is the opposite of great. While I may like or dislike the way a certain software product does something, and I may share that opinion with others, I am hesitant to proclaim why the flaws exist or how the flaws ought to be addressed unless I have made an effort to understand the fundamentals at work beyond just my own experience with that software. It's not unlike being a food critic in that way.
Unfortunately, many people who write about technology rarely could pass muster as a critic if we used the kinds of standards employed in most other fields. There are bright lights in the darkness, including people who are quite critical of the results of F/OSS projects. Those people tend to know of what they write and I really enjoy (even when I disagree) with what they write. They tend to either be people with deep first-hand personal experience in the act of creation of what they are writing about or they are people who have done a lot of homework and research into the matter to ensure they have a deep understanding of the field. They do not simply shoot from the hip and hope it sticks.
When people in our community write with a nonchalant attitude and offer analysis without the basis to do so, they do us a significant disservice. After all, we don't need competition creating FUD when we are doing it ourselves by generating gross misinformation that undermines our own activities. The amount of inaccurate information in circulation in our community can quite often, in my experience, be traced back to such unfortunate "analysis" pieces.
To quote Radiohead: "You do it to yourself, you do, and that's why it really hurts."
What really sucks is that these same people really do mean well and are, with all honesty in their souls, trying to participate in a positive manner in our communities. As such, I can't bring myself to get upset with them as individuals, but I do wish we could improve the situation and avoid a lot of needless energy wasting. Perhaps having a minimum set of standards we hold each other to when publishing what amounts to technical critique and analysis, just like other fields with critics try to, would be useful.
What do you think? How can we improve this situation?
Subscribe to:
Post Comments (Atom)
7 comments:
The problem is it's difficult to tell somebody "Thanks, but you're doing it wrong." It's especially difficult if you're personally involved: they assume you're trying to protect your own agenda.
@notriddle: that's the point of having standards for critics. then it isn't a matter of "do you agree" but rather "is the critic qualified". there will always be disagreement, but we don't have to coddle displays of incompetence at our own expense.
Traditionally, critics were hired by traditional publishing agencies like newspapers and magazines. With the advent of the internet, everyone can have a blog and hence an opinion. And sometimes (more often than not), people forget the difference between a critique and an opinion. The real problem is, even if people proclaim whatever they say as an opinion, the distribution is so vast (Essentially not justified for opinions) that it does cause harm.
I guess it has become a problem not only for technology, but the likes of politics, religion etc.
I personally believe that this is not a problem which can be solved, but to be lived with.
There are many things that I would like to say here, but that would take too long and - frankly - is probably not too interesting for everybody else. Plus, as I have learned the hard way, writing lengthy comments after re-watched "V for Vendetta" is something to avoid.
About one year ago, I pretty much stopped to actively participate in F/OSS related discussions and forums, at least compared to my level of participation before that. I still "consume" opinion pieces, blogs and discussions and sometimes, typically if somebody is objectively wrong concerning one of the few areas I consider myself the equivalent of an ethusiastic hobby cook (to stay within the frame of your metaphor) , I try to provide some perspective. About one year ago, two conclusions seemed to cristalise themselves for me, both of them worthy of a Capt'n-Obvious-tip-to-the-helmet-award:
Passion is a poor substitue for scientific rigor or even competence when it comes to analysing complex systems like F/OSS software and/or communities.
And just because our communities (e.g. libre software, open source software, commenters, users, analysts, bloggers, developers and innocent bystanders) are on the forefront of utilising the latest technology for our communication does not necessarily imply that we have moved beyond the "software XY sucks" / "no, software XY is the best thing since indoor plumbing" / "the top ten reasons why software XY and indoor plumbing sucks" back and forth ritual.
Just to name one example from the top of my head(*): The results of usability research and user surveys may yield results I'm not comfortable with (they often enough do, so I'm not constructing an academic example here). Yet, dismissing these results just because they run counter old habits is not suitable for the basis of an analysis. Either the science behind the survey is flawed (may very well be, HCI is quite a young discipline and as an aspiring phycisist, I don't envy colleauges from the social sciences for the difficult nature and fuzzyness of their prime research-objects, e.g. humans beings.), the survey itself was not representative or poorly conducted or important aspects of the problem remained untouched.
But merely dismissing the whole process because the medicine tastes bitter is not a very adult or even rational mode of operation.
((*)I happene to have read an "opinion pice" yesterday which IIRC you also commented on. No idea if that article somehow triggered your blog post, but it surely made me shake my head, especially given that the author is somebody which I have/had a lot of respect for).
Damn, the post again grew too long, sorry for that.
Best regards
Martin
I totally agree that there often needs to be a lot more humility from commentators around F/OSS. The willingness to have a conversation. Even the understanding that they don't necessarily have the right to be heard by the people they want to be heard by (FOSS project leaders can't always afford to have a conversation with everyone who has an idea).
I do however think that it is dangerous to lean to much on the idea of the expert. Like @setec I'm a physicist and the strength of proper science is that, while experts are given more time they aren't indulged: Others will suggest problems with their ideas, improvements etc, and by debate, advance. Expertise is very relevant but it shouldn't be a "hard filter": expert=opinion accepted, non-expert=opinion rejected.
Science has all of the structure of universities, journals etc which means there is not the level of "opinion noise" there is in software. It is hard to deal with and it requires multiple fora with different rules for discussion, not everyone can sit at top table.
I guess something specific prompted this blog I don't know what that is, so this isn't an implied comment on the rights or wrongs of that.
P.S. I agree with @setec about HCI research. It shouldn't be dismissed but one also needs to be careful. A little bit of research is a dangerous thing. It's a difficult field of study and the results of even the best experiments need to be understood and not over-generalised.
First of all I definitely think that this is something we can live with. We all mix up our intuitive feelings and more or less objective observations. Sometimes we simply like something and try to defend it. Sometimes we say something that is terribly wrong and everybody can live with that as long as we try to see our mistakes.
Experts: I hate discussion in which somebody says "I have 20 years of experience and you not" (I am less than 20 years old ;)), that does not help to find a solution. The result of such statements is a very personal level and normally there will not be further results. (Or sometimes I hear something like "I've written really large programs, there's no .NET-alternative, you're a stupid idealist", but he just means that he has written a few thousands lines of code) First of all we should look at the arguments and forget about personal things. Then we should respect the personal opinion behind the statement. But we should not try to value the whole person ("you are not an expert").
I think the answer to poor information is to provide better information. This is where open source might learn from what the scientific community has gotten right and wrong.
It is not uncommon for the first few reports on a subject to be right in the general, but wrong in the particulars. Since entrenched interests and human egos are involved, this leads to denouncement of the results, public flame wars and other drama.
But it is not often pointed out that the system usually corrects its errors, given enough time. After the outcry, other scientists attempt to verify the results and publish corrections. This requires time and it's somewhat sloppy process; but then, most things are. Eventually, truth disruptively rise to the top and real expertise gets recognized.
While it can be slow moving, the Open Source world works in the same way. Just consider how perceptions of KDE 4.0 have evolved. When released, KDE 4.0 was vilified as a technical turd. There were very public denouncements and calls that the entire code base get scrapped. In time, however, the technical merits have become obvious and recent coverage has been glowing.
(It would be nice is scathing comments were retracted and narrow-minded fools owned up to their limited vision, but that might be too much to ask.)
And like in the scientific world, members of the open source community can help bring about such shfits. For starters, that might mean that they strive to be as accurate as possible in their own blogging the subjects they discuss. (In general, I think that you do a very good job of this.)
After that, they might promote the blog posts of others that are worthwhile or helpful. This brings that material to the attention of a larger audience and raises the overall level of discussion in the community.
Finally, if there is a misperception or someone is just wrong, they might take the time to explain why. While it might bring pain and embarassment to do this publicly, it is still the best way.
Science works because most discourse is public and scrutinized. While this opens the door to negative and nasty, it is still very important. Positive feedback and good feeling can only move a cause so far. Eventually, you have to face up to problems and mistakes.
Ultimately, the quest isn't: "Did they get it right the first time?" but, "Did they get it right?"
If someone is wrong, eventually that will be found out, regardless of the level of passion. Richard Feynmane summarized it well, "For a successful technology, reality must take precedence over
public relations, for nature cannot be fooled."
Post a Comment