kio_httprc's needed

a bug was filed against konqueror today for a pair of websites that supposedly do not render correctly. but it turns out that the websites are buggy: bad javascript misdetects konqi and therefore feeds it junk HTML. switch the user agent and all is well. this is easy enough to do in konqi: go to the Tools menu, select Change Browser Identification, voila!

of course, the user who reported the bug still thinks konqi is broken as they didn't think to switch user agents, so all isn't well. this is an all too common event and results in unwarranted marring of konqueror's reputation.

so half an hour of putzing about later and i have a command line KDE/Qt app (weighing in at ~70LOC) called "agents" that rips through a kio_httprc file and spits all the domain-name to user-agent mappings into a postgresql database. this simple little app is aided by a short shell script that does some batch processing of kio_httprc files. sweet.

now i can do things like:

agents=# select count(id), d.id, d.domain 
                from domains d join agents a on d.id = a.domain 
                group by d.id, d.domain 
                order by count(id) desc;
 count | id  |         domain
-------+-----+------------------------
     2 | 179 | computerworld.com
     2 | 185 | sco.com
     1 | 177 | aec.at
     1 | 178 | coachella.com
     1 | 180 | marca.es
     1 | 181 | metica.se
     1 | 182 | msn.com
     1 | 183 | nickjr.com
     1 | 184 | pason.com
     1 | 186 | watchit.com
     1 | 187 | www.pcbanking.cibc.com

or

agents=# select distinct a.agent 
                from agents a join domains d on a.domain = d.id
                where d.domain = 'computerworld.com';
                       agent
----------------------------------------------------
 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

awesome. not sure what to do with the data yet, but i have some ideas. like ... generate kio_httprc add on files for people to download containing the top N sites; country specific kio_httprc add ons; a way to quickly check konqueror bugs for specific domains for known user agent quirks .... sounds like a nice application for HotNewStuff, too.

but to make it all work really well ... i need you! well... not really you, but your kio_httprc file where all these mappings are kept. the more of these files that get fed into the database, the better. so if you'd like to help out email me your `kde-config --localprefix`/share/config/kio_httprc file and i'll add it to the database.

(for the privacy conscious, it's all anonymized in the database. and if it makes you feel any better, i'll trade you your kio_httprc for mine upon request ;)