ExpanDrive

Just to give a taste of the sorts of things Jeff and I have worked on outside of Magnetk, my SIGGRAPH paper about an interactive preview rendering system I’ve been building with Industrial Light & Magic and Tippett Studio just made the top story on MIT Technology Review:

I’ve always hated MySpace. With a passion. Today, however, they rose one notch higher on my death list:

From:    MySpace Events
Subject: Girls Gone Wild DVD Collections has invited you to:
Girls Gone Wild - Free Videos! - Check it out

Girls Gone Wild DVD Collections has invited you to an event
on MySpace:

Girls Gone Wild - Free Videos! - Check it out

Click the link below to view the event details:

http://events.myspace.com/...

-------------------------

At MySpace we care about your privacy. We have sent you this
notification to facilitate your use as a member of the
MySpace.com service. If you don't want to receive emails
like this to your external email account in the future,
change your Account Settings to "Do not send me notification
emails."

Click here to change your Account Settings:

http://www.myspace.com/reloc.cfm?c=11

The content is sadly illustrative of much of what there is to hate about that tasteless also-ran (which has enjoyed an infuriating degree of success, given how completely unoriginal, not to mention monumentally user-unfriendly it is).

However, it’s the ending of the message that really got me.

This was sent directly to an address which I never gave them. Clicking on the nearest thing to an “unsubscribe” link, I can’t take any useful action because I have no account associated with this address. This is just straight-up spam, pure and simple.

Yet more than half of the message body (with which I was spammed) is dedicated to platitudes about their respect for me, my time, my attention, and my privacy.

Just one more reason to despise Newscorp.

Spammers seem to be suddenly getting much more successful in breaking through common filtering systems with increasing regularity of late. Either that or MIT’s spam filtering services suck compared to Stanford’s. I recall the first exciting distributed spam filtering system (which I encountered) was a simple hash database, containing checksums of known spams. Clearly the spammers have long-since conquered this one, but I wonder whether another axis of filtering might extend this idea using sliding window hashes to recognize content similarity. In particular, while spams often try to be somewhat unique, they generally have a very cut-and-paste feel, which suggests that greater success might be found through simply matching small regions of commonality through local fingerprints (a la LBFS).

Similarly, today’s most successful spams seem to rely almost exclusively on image text to sneak past filters. First Gmail and others seem to have added simple OCR to their filters, leading to italic, anti-aliased, non-fixed-width image text to break that next barrier. As these images get more expensive to generate (and they certainly seem to be identical across large numbers of spams in a given day, with only the markov-generated “poetry” varying from message to message), can’t we similarly use range hashes or other efficient fingerprints to recognize these oft-reused images from a distributed known-spam database?