Here is a depiction of the DQE system architecture (TEST and SET are the messages that the recipient uses to test stamps for freshness and then to cancel them):
DQE Architecture
Here is a depiction of the DQE system architecture (TEST and SET are the messages that the recipient uses to test stamps for freshness and then to cancel them):
|
|
Of course, there are many other spam-fighting proposals. Our conference
paper, below, mentions some of them. (A related comment is that DQE
probably fits into this
litany somewhere!)
Here are snapshots of the code that implements
the components described in Sections 5 and 6 of our NSDI paper.
Here is a HOWTO for
getting the code working. The HOWTO is still quite rough, so please
email us with questions, feedback, bugs, etc.
Michael Walfish J.D. Zamfirescu Hari Balakrishnan David Karger Scott Shenker
Goals and Technical Challenges
General DQE Requirements
No false positives Our high-level goal is reliable email.
We assume reused stamps indicate spam. Thus, a fresh stamp must never
appear to have been used before.
Untrusted enforcer We do not know the
likely economic model of the enforcer, whether monolithic (i.e., owned
and operated by a single entity) or federated (i.e., many organizations with
an interest in spam control donate resources to a distributed system).
No matter what model is adopted, it would be wise to design the system
so that clients place minimal trust in the infrastructure.
Privacy To reduce (already
daunting) deployment hurdles, we seek to preserve the current "semantics"
of email. In particular, queries of the quota enforcer should not identify
email senders (otherwise, the enforcer knows which senders are
communicating with which receivers, violating email's privacy model), and
a receiver should not be able to use a stamp to prove to a third party that
a sender communicated with it.
Challenges for the Enforcer
Scalability The enforcer must scale to current and future
email volumes. Studies estimate that 80-90 billion emails will be sent daily
this year (see, for example, a
report from IDC). We set an initial target of 100 billion daily messages
(an average of about 1.2 million stamp checks per second) and strive to keep
pace with future growth. To cope with these rates, the enforcer must be
composed of many hosts.
Fault-tolerance Given the required number of hosts, it is
highly likely that some subset will experience crash faults (e.g., be
down) or Byzantine faults (e.g., become subverted). The enforcer
should be robust to these faults. In particular, it should guarantee no
more than a small amount of stamp reuse, despite such failures.
High throughput To control management and hardware costs, we
wish to minimize the required number of machines, which requires maximizing
throughput.
Attack-resilience Spammers will have a strong incentive to
cripple the enforcer; it should thus resist denial-of-service (DoS) and resource
exhaustion attacks.
Mutually untrusting nodes In both federated and monolithic
enforcer organizations, nodes could be compromised. In the federated case,
even when the nodes are uncompromised, they may not trust each other.
Thus, in either case, besides being untrusted (by clients), nodes
should also be untrusting (of other nodes), even as they do
storage operations for each other.
Our conference paper at NSDI 2006, below, describes how DQE addresses the
above challenges.
Papers
The following papers give more information about DQE. The first is a
position paper for a workshop. The second is a full-length conference
paper that substantially updates the workshop paper with a much simpler
(and more practical) design. The conference paper also has
an analysis, implementation, and evaluation. The third paper is a
supplement to the conference paper; it contains a few explanations that we
were not able to fit into the conference paper.
If you would like to cite this work, please cite the NSDI
conference paper. Thank you!
ACM Workshop on Hot Topics in Networks
HotNets, San Diego, CA, November, 2004
Proceedings of USENIX Symposium on Networked
Systems Design and Implementation
NSDI, San Jose, CA, May, 2006
MIT CSAIL Technical Report, MIT-CSAIL-TR-2006-033, April-May 2006
Talks
Software
People
Funding
This work is supported by the National Science Foundation under grants
CNS-0225660 and CNS-0520241, by British Telecom, and by an NDSEG
Graduate Fellowship.
Contact Us
We welcome comments, questions, and feedback. Please send mail to