plasma in 18-24 months?

In my last blog entry, I mentioned QtComponents, QML and the new Qt scene graph and then vaguely alluded to profound implications for Plasma. I will not tease you, dear reader, for longer than necessary: this blog entry contains my current preliminary thinking on the what this could all mean for Plasma going forward.

First, I'd like to note that none of this would be possible without the fantastic work going on at Nokia's Qt development offices. They are tackling hard and interesting problems with gusto and producing some very nice results in the process. QtComponents is being developed very much in the open right from the start: an open mailing list for all dev discussion, a public git repo that even contains experiments and early code sketches, a set of use cases and open tasks in Jira. Outreach to community members such as myself, which allowed me to join their design sessions last week, is just one more piece of this. This open from end-to-end, right from the beginning development model is part of the "new Qt" ecosystem that is the culmination of years of consistent effort on the part of many individuals involved with Qt. It's paying off now, and I hope that all new Qt components undertake a similar, or even the exact same, type of approach.

Enough about that, however... what does all this new stuff mean for Plasma?

There are two things that I'm really not very happy with when it comes to Plasma right now:

• libplasma provides some very elementary UI components that really belong in Qt, but because they aren't in Qt have ended up in libplasma out of necessity where they are more of a distraction (in terms of code clarity and design) than anything else


• QGraphicsView, particularly QGraphicsProxyWidget, is not nearly as performant as it could be. It's gotten pretty good in terms of being quite usable even on modest hardware with Qt 4.7 and KDE Platform 4.5, but it could be so much better.

The new scene graph would allow every paint operation in Plasma to be hardware accelerated on the GPU using OpenGL. Not only would effects like blur become cheap (the expense is why we don't use blur on the canvas itself, but only on top level window backgrounds where we can use KWin's OpenGL driven compositing effects to achieve blurring) but things like nice halos around text should be able to be rendered at high quality and low cost. Pixmap usage should drop, framerates skyrocket and client-side image manipulations all but disappear.

This implies moving away from QPainter, which only gets in the way of proper acceleration since the painter can do "anything" in "any" order. Which means moving all of our user interface to the declarative model, specifically QML. In turn that means needing proper declarative style widgets, specifically QtComponents. Finally, it means leaving QGraphicsView behind and moving to a scene graph only system.

This is not a small amount of work: every Plasmoid, Containment, popup and window dressing (e.g. the add widgets interface, the panel controller, the activity manager ..) that does direct painting needs to be moved over to QML. Thankfully we can do these one at a time with the results working very nicely with the current QGraphicsView based libplasma. This means we don't need to do a "massive porting effort with a huge pause between releases"; each release can contain more and more QML driven elements and fewer and fewer uses of QPainter. The result for the user while this happens is likely to be nicer looking interfaces, as QML usually makes this easier to achieve.

Marco has been working extremely hard on the first thing we need to make this process possible: Plasma <-> QML integration. This work is being merged into trunk right now and will debut in KDE Platform 4.6. This means we can start the process of QML-ification of Plasma now. Plasmoids written in C++ can create Plasma::DeclarativeWidgets, Javascript Plasmoids can load QML files from their package and Plasmoids can be written entirely in QML (using the Plasma Javascript API from within the QML).

We will also need to replace our current QGraphicsProxyWidget based pushbuttons, sliders, etc. with QtComponent based ones. This is mostly an issue for Plasmoids written in C++; Plasmoids written using Javascript should shielded from this set of changes. Other scripting languages such as Ruby and Python expose more of the implementation details and so those Plasmoids may require some adjustments. However, moving to QtComponents means waiting for QtComponents to be "ready" and dealing with the baggage of our existing C++ QGraphicsProxyWidget subclasses in libplasma.

Of course, we're not simply waiting for QtComponents to magically mature on their own. Several of us (Artur, Marco, myself) are engaged with the QtComponents project and its team already. Marco has also already provided the first QtComponent push button whose text is set using a source from a Plasma::DataEngine, all done in QML. So we know the basic ideas work in a very practical manner.

Here's the big bomb-shell, however: to fully complete the migration process, we'll need to create a "libplasma2" which is binary and source incompatible to the current libplasma. Corona will cease to be a QGraphicsScene subclass and instead become a scene graph manager; Containment and Applet will become some kind of QML item; the QGraphicsProxyWidget subclasses will be dropped.

The good news is that this is an almost completely internal set of changes. The design of Plasma lends itself very nicely to this set of changes, and the C++ classes in libplasma are nicely aligned as well. For instance, Corona right now is the QGraphicsScene and manages Containments and provides "global" management of things like configuration data, screen geometry hooks, etc. None of that would change, except it being a QGraphicsScene. DataEngines, Plasma::Services, Plasma::Svg, etc. would not change in the least, either. So the external impact looks like it will be surprisingly small for anything written in Javascript or which uses QML for its user interface. Note that Plasma Mobile is already 100% QML, modulo the components it inherited from other Plasma workspaces, consisting mostly of individual Plasmoids.

All of this means that it isn't going to happen in the next release, or even in the release after that. It will be a measured set of changes over the course of many releases to reach the final goal of "going scene graph". User disturbance will therefore be kept to a minimum and we'll be able to continue to make regular releases with noticeable improvements while we do this.

The end result of full hardware acceleration, a fully QML driven user experience and a much improved resource footprint are, in my estimations, worth all of this effort.

There are some potentially very exciting things this could mean beyond simply improving what we already do, though. One of the main ones involves threading. The scene graph is capable of rendering the scene in multiple threads. QML needs some adjustments to take advantage of this, but if/when that work gets done it would allow Plasma to run each Plasmoid in its own thread. (Note: this is different from multi-process.) That would mean that any pausing in a given Plasmoid or other user interface components would cause no annoyances in the rendering or interaction with any other part of the user interface. It would also enable fairly trivial resource management: we could then track per-Plasmoid memory and CPU consumption, for instance. Achieving such a multi-threaded Plasma will require some additional work on top of everything already discussed above, including things such as DataEngine access (which is not currently designed for multi-threaded access) and sharing of Svg renderers between Plasma::Svg objects (something that lets us gain significant performance benefits right now). My initial scan over things says that it isn't insurmountable: we can make rules about things like Plasma::Svg (can only be used in the same thread as the Plasmoid; so no sharing Plasma::Svg's between different instances of Plasmoids) and probably adjust things like DataEngine successfully (the idea of putting the DataEngines out of process, something I've toyed with conceptually since the 4.0 release, becomes even more attractive under this scheme).

It's an exciting set of possibilities, though none of this is certain yet. It's all still "work in process" research and none of the above may happen. I'm hoping it will, however, and the pieces are actually falling into place. Huzzah.