The topic of WebKit is not the easiest one. It's had its share of controversy and until recently QtWebKit wasn't really, at least in my opinion, quite up to scratch for production usage outside of fairly simple use cases. With Qt 4.6, which came with improvements across the board and new API for things like DOM access, it has turned a very important corner and makes the topic of "What does KDE want to do with QtWebKit?" ever more pressing.
When it comes to a web rendering stack, there are generally three kinds of uses for them:
- Web browser: this would be things like Konqueror, Firefox, Chrome, Internet Explorer, Opera, etc. They have one purpose in life, and that is to wrap themselves around a web content stack and give people the ability to "browse the web"
- Displaying XHTML/JS content from Internet sites inside a non-web-browser application: examples include the web or Remember The Milk widgets in Plasma, the Wikipedia content for geographical locations shown in Marble or the plethora of web based information shown in Amarok. These applications are not web browsers, and the content shown is highly specific and even pre-selected by the application developer. Done well, the user doesn't even think "oh hey, I'm browsing the web!"
- Displaying XHTML/JS content in a non-web-browser application that is locally generated: this includes things like KDE's documentation reader and the email viewer in KMail. These applications are using the web stack as a rendering system and little more. That you can fetch content from web servers on the network isn't too relevant and, done well, there are no "visible seams" between the "native" parts of the application and the HTML rendered content.
When it comes to WebKit and KDE, it is quite evident that it is in the latter two categories where it could have the biggest "footprint": there are simply dozens of applications that could be using it (versus our one web browser). They are also the easier topics since the usage of a web stack is simpler and more straightforward than in a full blown web browser. The conversation around whether to use WebKit in a web browser is not nearly as clear cut and we already have that conversation happening in the form of projects like Rekonq and Arora, in any case. So I'd like to focus on the non-web-browser application usage of WebKit in this entry.
As 2009 closed out, a couple of very significant QtWebKit events came about for KDE: the API around QtWebKit expanded considerably (the DOM API, for instance, which is often used in our applications now) and KDE WebKit was added to kdelibs. KDE WebKit takes QtWebKit and implements all the platform-specific integration bits, where the platform in this case is KDE. Things like KIO (and therefor the proxies set up in KDE's control center), KParts, cookies and more have been made available to users of QtWebKit as a result. These developments have made QtWebKit a serious contender for usage in KDE applications.
Of course, just because it exists is no reason to jump on the QtWebKit bandwagon. If we are going to use QtWebKit (or not), we need have reasons for doing so (or not doing so). We already have KHTML and it works rather well and is still under development, after all. Is there any compelling reason to use KDE WebKit in our applications instead of KHTML?
The Network Effect
We all know that WebKit, which was birthed from KHTML, is now used in several top tier web browsers and even more applications. Apple's Safari and Google's Chrome are both well known examples. This means that there are, relatively, huge numbers of people working on WebKit. This matters because web technologies and standards are not standing still. In fact, it's moving faster than ever and it requires a large effort to keep up. While I have all the admiration and respect in the world for the team working on KHTML and believe that given enough time there isn't anything they couldn't implement, the reality is that the amount of person-hours are limited and it is a far more sure bet that WebKit will keep pace in a relevant manner relative to the standards given the companies and communities behind it.
This affects all applications that would like to use the newer XHTML features that are becoming available.
There is also the issue of bug and feature compatibility. It is not true that every usage of WebKit is bug compatible or even feature compatible. This is because different products that use WebKit often ship with a branch or fork of WebKit that is slightly different from other branches out there. In part the differences stem from the branches coming off of mainline at different points in time (so it's an artifact of different release schedules) but also in part due to some customizations that are application-specific that get made. Still, there is greater similarity than difference and due to the number of WebKit apps and web browsers out there in daily usage, WebKit variants get more testing by web developers than our own KHTML does or likely ever will.
This is something that primarily affects usage in a web browser, such as Konqueror, however. For applications that generate their own content (e.g. KMail's email viewer), it's a non-issue. It does still come into play for applications that are doing "mash ups" using content directly ripped from sites online.
Technical Advantages
WebKit also has a number of technical tricks that have given some projects, such as Plasma, to already start using it.
First, it's canvas friendly and is easy to make at home on a QGraphicsScene. The Plasma team already did up some basic integration for gestures and zooming, but the Qt devs have recently gone even further and cooked up a really nice, mobile-ready implementation with things like "over scrolling" where you can scroll into areas of the page that haven't been rendered yet and they are "grey" until filled. That's mostly a mobile platform issue, but it shows what is not only possible but available today.
Then there is the really nice support for "mashing up" C++ (or, I suppose, Ruby/Python/etc) elements from your app with the web canvas and control element presentation and content from the app's "native" language.
The JavaScript runtime from WebKit now also powers QtScript, which means that if the application is using WebKit and QtScript then the JavaScript runtime code is shared (not the actual runtime environments, of course :).
There are also things like the very nice Web Inspector that comes along with WebKit for free. There are efforts underway for things like accelerating animations and effects and web slices/scraps. These are all cool things above and beyond "just" WebKit and shows that even in the Qt layers there is really interesting progress that continues to be made.
The above are all reasons why QtWebKit was the best fit choice for Plasma: we use QtScript, we are doing "mash ups" fairly regularly and we need as many of the current web features as possible. The Silk project is also relying on QtWebKit for things such as the site-specific browser, Selkie.
Release Schedule Challenges
One challenge that comes up in regards to relying on QtWebKit is whether or not KDE's release schedules and our development needs (read: feature requests) can be met by relying on a project with a different release schedule and even different feature priorities.
Features we need in a web rendering stack essentially come down to this: what does the current web dev community expect to be available in a web rendering stack (WebKit generally has the upper hand here) and what integration points into KDE technologies does the stack have? When it comes to integration points, the QtWebKit team has shown themselves to be very responsive to this and solutions have been forthcoming. It it what enabled the work done for KDE WebKit to be possible. For this reason, I don't think that features are a reasonable objection at this point.
As for release schedule timing, we have turned the art of "when will Qt get feature $FOO" into a science in KDE. I'd rather coordinate with Qt's schedule to get the desired features and performance results than have a perfectly melded schedule. Yes, it's a trade off, but the benefits outweigh the costs here.
Of course, others may have different opinions ... but we really ought to stop waffling just because there are different opinions.
Embracing Evolution
WebKit came from KHTML. It is, in my mind, the next step in the KDE Web Experience odyssey. (Again, others will differ in viewpoint, I'm sure.) WebKit has "taken over the world" as much as that is possible, becoming probably the most widely deployed web stack in terms of usage in different projects. The user base is also huge. These are successes that used our own people's efforts as a springboard. We should embrace that success and make it our own again rather than be put off due to it.
To speak to the "elephant in the room": I don't want to see KHTML development cease, nor do I think it should be limited or even needs to be. However, KHTML's existence alone does not mean that KDE applications must use it if there are more appropriate alternatives. "More appropriate" is a somewhat loaded term; in my mind WebKit would need to meet the integration requirements (using technology stacks which are so foreign that they degrade integration is not a good answer). It's an unenviable position to have competing technologies to choose from, especially when they are so close to our hearts, but this is the kind of decision we are faced with.
As more KDE applications use QtWebKit, if only because it offers features and functionality that we can't expect to get elsewhere in reasonable time frames, we are faced with a poignant question: are we comfortable with having some applications using one web stack and other applications using a different one? (That is our current reality.) Should we be aiming for consistency here, using the web stack that has the most to offer, weighing integration together with features and developer resources? Or are we alright with the extra overhead of having multiple web rendering stacks required for a KDE experience?
This is a discussion that's been simmering for a couple of years now. I think it's been somewhat academic up until this point because we didn't have KDE WebKit and QtWebKit was still missing important functionality. The current state, however, is that we do have KDE WebKit and QtWebKit is ready for what we use it for in our applications. I know because I've spent time over the last year looking at how we use XHTML in our various applications, whether it's in the KDE Help Center, KMail, the application welcome screens or apps like Plasma which are a bit more dynamic in their XHTML usage. The questions are therefore no longer academic but practical.
KDE application developers should form a consensus around this issue as much as is possible. We should want KDE applications do what is best for the applications and therefore our users. I personally believe that KDE WebKit is our best option, but consensus followed by action is what will really count. This is not the easiest decision, but then again it isn't really the hardest of decisions, either. If we decide make KDE WebKit (already it is included with kdelibs) as part of our applications' evolutionary path I think we will gain enough from it in terms of features and performance to justify the efforts.
Beyond What We Could, And Recognizing What We Are Already Doing
WebKit is already an integral part of Plasma, Amarok, Selkie and Rekonq. I can only assume it's being used in even more KDE applications and I just haven't noticed it yet. Regardless of what other KDE applications do, usage of WebKit in these applications means WebKit will be making a significant impression within the KDE universe in 2010.
Using it to write Plasmoids, both via QtScript as well as directly using XHTML/JS in a WebKit container, is just one example of where it is taking us. Due to this emergence, and the question we now have in front of us regarding just how pervasively we want to use KDE WebKit in KDE applications, this will be one topic that will be a noticeable part of 2010 for us.
Epilogue
I'd like to close with a huge "THANK YOU" and an expression of my respect and gratitude to everyone who has worked on KHTML today or in the past. I think it's been a remarkable project from day one and remains a remarkable project right to this day. It blazed trails for F/OSS web stacks and has become insanely successful in various ways. This stands on its own, inviolable and untouched by the contents of this blog entry.
This also closes out the "Key Quests for 2010" blog entry series. I hope those who have read it have enjoyed it or at least been moved to thinking about some of the opportunities, challenges and ideas we who make up KDE are going to come across in the year ahead. I also truly hope that it inspires further discussion throughout the community and that they entries serve not as end points but as conversation starters, though provokers. None of us alone have all the answers, but together we can get pretty close. This has been key to the success of KDE.
There are so many compelling and exciting things afoot in KDE right now that 2010 is almost certainly going to be an amazing year for all of us. The 14 areas of interest I discussed in this series just scratch the surface, and mostly in areas of KDE-wide needs. There is so much more that happens at the "local" level within KDE and so many more things that are going so well that we hardly need to add to the discussions those topics already receive. That why KDE in 2010 is an experience I can't wait to be a part of along with all of you.
*hugs*, aseigo.
(This article is part of the "Key Quests for KDE in 2010" series)
18 comments:
Very insightful and courageous post.
I really wish that Webkit will be everywhere in KDE. KHTML should be preserved as an opt-in for Konqueror though.
That way a simple user wouldn't have contact with KHTML and only see Webkit.
It would also be good for resource usage and snappiness.
I really hope KDE can do such a transition in KDE SC4 without having to start a SC5. Gnome did a lot of stuff like that in Gnome2, maybe too much, but it should be an option and Webkit is a worthy undertaking.
I wish KDE the best.
Another important use for WebKit is its SVG renderer, which does (or should soon) support a wider set of SVG features compared to QtSvg classes. It could, for example, be used to render icons or other SVG artwork.
I am a professional web developer, and a leading developer for a major open source web framework. I use KDE as my primary desktop at home, including Konqueror as my main browser. It's what I am using to post this message. I think I am one of maybe 2 developers on the entire project that uses Konqueror, or for that matter even knows it exists.
Quite simply, we don't support KHTML. We never have. We never will. It will never have enough market penetration to justify the time to support it. And by that, I mean have any market penetration at all.
WebKit, between Chrome and Safari, commands a respectable chunk of the desktop browser market. More importantly, though, it totally pw0ns the mobile browser market. The iPhone, Android, and Palm's webOS all use WebKit-based browsers. WebKit is the browser engine of the modern mobile.
KHTML? I don't think it's ever had enough market share to even warrant its own slice on a pie chart separate from "other/not known".
I don't say that to belittle the KHTML developers. Quite the contrary, it should be a huge complement to their work that Apple chose KHTML over Gecko to base their own browser on, and then extend. In fact, they (and the others using WebKit) have pushed the KHTML-derived WebKit engine to new heights that seriously challenge Firefox's dominance in the "non-MS-suckage" browser market. Kudos to the KHTML developers for pulling that off!
And now it's time to move with it.
A friend of mine once described being declared a monopoly the greatest complement: Congrats, you've cornered the market. You've won the game. Now let's move on and do what's necessary for the health of the economy.
I see KHTML in the same way. Congrats, guys. You've won. The work you did is now the basis for 2 major desktop browsers and the 3 most advanced smartphone platforms on the planet. It will likely become the de facto standard mobile web browser for the foreseeable future. Really, kudos to you. That's a feat few could pull off, especially with something as hard to write as a web browser.
And now it's time to embrace that and move on, to do what's best for KDE.
Really, I don't see a move from KHTML to the now-superior WebKit for all of KDE to be a bad thing, or a sign of defeat. Quite the contrary, it's should be a badge of honor, a milestone of success, and a cause for great celebration.
It's also desperately needed, because as Aaron notes KHTML has not kept up with the fast-moving web as well as WebKit has and its market share is not, nor will it ever be, large enough for people to care about. That includes open source advocates and KDE fanboys like me. I know exactly one other person that uses Konqueror as his primary browser, and that's in a massive web development community. And he may have switched since we last spoke, I don't know. But I know our CMS's Javascript doesn't work right in KHTML, and it never will.
So yes, KDE application developers and browser developers, please, embrace what you have birthed and is now taking over the next generation of the Internet. There should be parties when KDE switches to WebKit fully, not because it "got rid of that old KHTML" but because "KDE is so awesome it's code is taking over the world, and now it comes full circle".
That's a great reason to celebrate.
I agree with Crell. The KHTML team has done a great range of things. I'm grateful for their work and contribution to the web. Because they made a base for Webkit and that's should be something that is to be proud of even if it was a fork. Because it's not based on MS's trident engine or gecko.
I have no intention to belittle the KHTML team either. I currently use QtWebKit in my application for chat rendering. And I'm happy with it. Though it has a few issues currently. A some what large memory footprint (maybe too many webkit windows running :รพ).
I hate to see KHTML fade. But it would be nice to see it keep up in the web rendering specs and pushed farther and harder. But as WebKit is more wide range than KHTML. I don't think KHTML will be able to keep up to it.
KHTML is not useless at all. Just to clarify before getting flamed. KHTML does have it's uses. I'd like to see it take over and replace KTextBrowser. But for big requirements I don't see KHTML taking place of WebKit for it.
I think Crell said it well what I think. The KHTML did something what no one else couldn't, it brought the web together on desktops and mobile devices.
I love Konqueror. But I do not have so much feelings about KHTML because I can not "see" it.
But I use currently RC1 of KDE SC 4.4. And I must say that Konqueror just feels much better than never. It is snappier, fast and still renders correctly.
Since KDE 3.0.3 versions, I have seen only few pages what have not worked with Konqueror. And those were on the worst "IE dominant" decade. On these days, I have only two sites what does not work correctly. Other is Logitech site what renders right but soon the whole page is loaded, it is shown as white page. There is easy fix for it, just fake using other browser.
The other is my bank online banking site. It has a drop-down list what is rendered to look like the system UI (Qt). It works that you just click it and you select the wanted service and it opens it. So you do not select service and click "Go" or anything else. The dropdown does not work that it would start loading that service. But it just selects it and you need to press enter to send the selection to the server. That is something what I can not fix with faking used browser.
But it is not hard once a week press the enter ;)
But otherwise, I use GMail, Slashdot and many other heavy JS sites very well. I do not care about JS speeds on benchmarks results. Only on sites what I use. And so far, KHTML has be great.
So I can not see reasons to actually abandon it at all.
Just because Webkit is more used?
The only reason what I can think now is that if webkit makes the development easier, then it is up to KDE do they want to make that change.
I am happy with Konqueror and I am a KDE! (Joke about MS "I'm a PC")
Oh, forgot to mention this.
We need to do something to get the SVG rendering possibility. It is one of features what I wait for KDE SC 4.5-4.6 that we can have SVG icons everywhere, not those bitmaps.
Do what you need to do to get those up and working!
"The iPhone, Android, and Palm's webOS all use WebKit-based browsers."
To this you can add all S60 based mobiles! (most Nokias)
Was using Konqueror a bit with webkit (in kubuntu lucid)
It works really well now, no comparison to some month ago, probably also due to newer webkit in qt, and the improved kpart.
Just ony uglyness in Konq is holding day-by-day use back now:
https://bugs.kde.org/show_bug.cgi?id=58040
Correction. The one other developer I mentioned that I knew of that used Konqueror? I just talked to him. He now uses Chrome almost exclusively instead.
So as the only known Konqueror/KHTML user developing for this web framework, I ask you to please switch to WebKit so that I can use my own work in a KDE-native browser. :-)
Re the technical stuff, KHTML works just fine on a QGraphicsView like Plasma's, the SuperKaramba folks did it and it was trivial.
And QtWebKit has a big technical disadvantage: it uses "fake widgets" (i.e. drawn using QStyle, but not implemented inside Qt), not native widgets! In other words, it isn't any more native than a GTK+ app using gtk-qt-engine. Those fake widgets behave in a subtly different way than the native ones, making for poor integration. KHTML uses actual Qt widgets.
IMHO, the KDE code currently using WebKit should be ported to KHTML, KDE should be buildable with a Qt built with QT_NO_WEBKIT so we can remove the code duplication from distros.
@kdepepo: The SVG renderer is also available in KHTML (it has been ported). And by the way, it originally comes from KSvg which was part of KDE 3's kdegraphics.
@Crell: Just fix your code for Konqueror. Screw the market share, do it for yourself. As a Free Software developer, you should have other ideals than market share! Or alternatively, fix KHTML if that's where the problem is, it may also benefit other web sites. Saying "just use one of the big oligopolies" is unhelpful and a very "proprietary / closed source software" attitude.
@Kevin: Why thank you for telling me what ideals I should have. I'd never have realized the True Meaning(tm) of open source without you.
However, the problems we usually run into are with Javascript. I am primarily a PHP developer, not a Javascript developer, and our Javascript mostly relies on jQuery anyway, which doesn't support Konqueror directly because there's all of 5 people using it. So in my limited time, I'd rather work on things that will be usable by 98% of my client base rather than 0.001% (vis, me).
I would hardly call WebKit a "big proprietary oligopoly". It's Free Software (LGPL specifically, I believe), same as KHTML. It's derived from KHTML, and even after this much time I'm sure shares much of the same underlying code.
It's not like I'm saying "just use IE" or "just use Opera" or even "just use Firefox" (also Free Software). I am encouraging KDE to move forward and improve its platform and the web in general, and make users' lives easier, too.
If you have a problem with QtWebKit's widgets, fix the problem. It will also benefit other applications. Saying "just stick with our home-grown engine when there's a more capable open source alternative" is unhelpful and very proprietary of you.
Just wanted to chime in from a user's point of view - I recently switched to Konqueror on KDE 4.3 from Firefox on Windows. So far, in about 3 months of use, the only "issues" that i've had with Konqueror loading or displaying pages incorrectly have all been fixed by changing the browser id to Firefox or Safari. Which suggests that the problem may actually be the web pages in question, not Konqueror/KHTML.
Also Konqueror does some pages better than Firefox (eg Facebook which uses lots of JS). I realise this has nothing to do with Webkit but I think it shows that KHTML is not in as bad a position as some would say.
In fact seeing it renders pretty much all the pages i visit identically to Firefox I would be surprised if it took someone such as Crell a significant amount of extra time to test sites with Konqueror. Or if it is even all that necessary with standards compliant sites?
So all in all if the KHTML devs choose to use their time to work on it, I think it can continue to be very useful for someone like me (who spends way too much time on the internet!)
I can live with KHTML (I'm currently using it), if it would implement new features and speed up their JS (html rendering is pretty fast, but I have sometimes problems with JS).
And I can live with WebKit, if the KDEWebKit guys would port all Konqueror features like the native widgets, kwallet (is it already implemented?) and the plugin system.
For now KHTML is somehow usable for me and WebKitPart is not. When the feature sets change I may switch or not. Of course there are the network effects for WebKit, I will simply wait and look if KDE-integration becomes better.
Jonathan
Another big problem with QtWebKit is that the HTML5 video support doesn't work with Phonon-Xine.
@Kevin Koffler: "KHTML works just fine on a QGraphicsView like Plasma's, the SuperKaramba folks did it and it was trivial."
what one can do is wrap it in a QGraphicsProxyWidget. KHTML does not let you simply render to a painter. as a result, it ends up with more overhead and lower integration; it just doesn't allow one to do much more than embed it "as is". take a look at the webkit graphics widget the Qt devs have done to understand what a difference this makes.
we actually have one Plasma widget that has been embedding KHTML on the canvas because WebKit didn't have the DOM API it needed. so we have experience with both.
'it uses "fake widgets"'
that's a reasonable point, though the differences in behaviour are not particularly overt and certainly don't outweigh the rest of the benefits we get.
note that for in-application use, use of form widgets is rare anyways.
"the KDE code currently using WebKit should be ported to KHTML"
until KHTML can render directly to the canvas, that quite literally will not even be an option for Plasma. i've been telling KHTML developers this since we started using WebKit over a year ago. i don't see it changing, and for understandable reasons.
given the other benefits of WebKit, i also disagree.
so, we disagree here, and that's unfortunate.
as for your reply to Crell where you say: "Just fix your code for Konqueror."
this is very typical of the disconnect with reality that most arguments in favour of KHTML embody. "just fix your code for Konqueror" isn't working for the majority of web developers (indistinguishably close to 100%, globally) and no matter how much we would love to close our eyes and ears and shout that phrase to the world: it isn't going to happen. market share does matter, because the web dev's time, energy and budget is as limited as ours is.
this is where i get really disappointed with the pro-KHTML sentiment: it requires we adopt a delusional viewpoint of the world.
@simon: "Which suggests that the problem may actually be the web pages in question, not Konqueror/KHTML."
yes, it is indeed the web pages. they could all be made to work with KHTML. that isn't going to happen, however.
"In fact seeing it renders pretty much all the pages i visit identically "
yes, it renders a great number of sites well. a great number isn't good enough when we can do better, particularly when the web standards continue to move forward.
"I would be surprised if it took someone such as Crell a significant amount of extra time to test sites with Konqueror."
knowing web developers personally, i can tell you it does. the one web dev i know who used to test against khtml no longer does because of this. sad, but a fact.
"So all in all if the KHTML devs choose to use their time to work on it,"
absolutely. i think the KHTML developers should continue to work on it for as long as they would like to. i have zero issues with that.
but just because they decide to work on it doesn't mean that i as an application developer must there use it.
i'd also note that i really am far less interested in the web browser usage of WebKit or KHTML as i stated in the blog entry.
the primary issue is with what renderer we are using in all of our other applications: Plasma, Amarok, Selkia, Kontact, KHelpCenter, the about screens, etc.
> note that for in-application use, use of form widgets is rare anyways.
But the scrollbars are also "fake"!
"Is there any compelling reason to use KDE WebKit in our applications instead of KHTML?"
Because it works where ktml doesn't. It happens to many times to me, when a site doesn't work in konqueror I switch to webkit kpart and works magnificently.
Post a Comment