there are still many improvements to be made, of course. and there's still some very cool and challenging development being done, and yet to be done, in the linux kernel. there are still people waiting for some specific piece of hardware to be supported better, or at all. there are still features that aren't all the way there yet. but generally speaking, the linux kernel has matured and isn't something everyone runs out and compiles as soon as it's available like they used to.
it used to be that pretty much everyone was in the same boat you are in right now and had to go out and build the kernel by hand and pray to the penguin that it would work out right this time. but these days, for the vast majority of people, it tends to just work. so instead of it working for the minority, it now doesn't work for the minority. this is good progress.
i was speaking not so much about specific hardware coverage, but to the completeness of the framework technologies (from smp and firewalling to usb and pcmcia) and how the kernel devs themselves see the work "calming down", which is of course different than "slowing down" or being "finished".
most people are ok with last year's kernel. and even more people have exactly zero need to compile the kernel from source. IMO, we're not there yet with the desktop. we need to be (and are) moving towards that goal.