StupidFilter Software & Sites 14 NOV 2007

Because the internet needs prophylactics for memetically transmitted diseases.

StupidFilterRight, we have a couple of educated people (probably drunken students by my guess) with a lot of time on their hands and a slightly elitist view of the world, who have enough time to cobble together a little project they’ve slyly dubbed: The StupidFilter Project.

And as the name implies, the project is indeed one in which stupidity, particularly of the linguistic kind, is going to be rooted out from web content. – Idiocy on the web is rampant, a point with which I must agree. Originally the internet was a place where one could communicate intelligently with similarly clever people (no …normal’ people knew how to work it back then), but that was all lost when we were gifted with the best Internet present of all: user-driven web content.

The solution these guys are in the process of crafting is open-source filter software that can detect rampant stupidity in written English. In their own word’s, this filtering will be accomplished with weighted Bayesian analysis and some rules-based processing, similar to spam detection engines. The primary challenge inherent in the task is that stupidity is not a binary distinction, but rather a matter of degree – thus the harvesting and ranking of tons of user comments from public websites, all neatly slotted on their nifty little five-point scale.

The idea is then to eventually release core engine source code that can be used by developers for incorporation into content management systems, blogs, etc. as well a various platform plugins, like a WordPress or Firefox plugin.

Being a rule-based filter, StupidFilter would obviously target the most egregiously stupid comments, thing with too much or too little capitalizations, too many SMS abbreviations or even excessive use of LOL or exclamation points for example (I would love to see L33T speak tossed in there as well to be honest!). It doesn’t know squat about irony or context though, so if you are a smartass with a taste for the ironic, you might just find yourself on the wrong side of this particular filter.

Currently the initial research is running smoothly and the project is looking to release usable alpha code by the end of this year.

Pointless I know, but perhaps one of the coolest little tools I could use in the future in combating the evils of stupid people, usually easily identifiable thanks to their atrocious writing skills :)

Funnily enough, I’m not sure how someone like Karl would cope. The guy is brilliant as they come, but without an automatic spell-checker the man is lost to the world. It would be interesting to see how StupidFilter handles him :P

Note that the above smiley itself would probably rank me high on the Stupid list as well :)

Related link:

Related Posts:

About Craig Lotter

South African software architect and developer at Touchwork. Husband to a cupcake baker and father to two little girls. I don't have time for myself any more.