DQE HOWTO

Introduction

The DQE code is availabe from our snapshots directory. This code is in three pieces: The enforcer is a "best effort" key-value storage service; it knows nothing about stamps. The experimental client TESTs and SETs key-value pairs and cares only that the key is a SHA1-hash of the value (as in other systems, such as SFSRO, CFS, etc.) Actual clients of the enforcer do two things: (1) create stamps; and (2) TEST and SET key-value pairs (although these clients get their key-value pairs by hashing actual stamps).

Below we describe how to build and run these components.

This document is still a work-in-progress, so if something is not clear or is buggy or whatever, please email us at dqe at the domain nms.csail.mit.edu.

Enforcer

Compiling the enforcer

  1. Get the packages on which the enforcer depends:

    To get and install sfslite, follow these instructions; you'll be getting the code either using a command like:

    $ cvs -z5 -d :pserver:anoncvs@cvs.pdos.lcs.mit.edu:/cvs co -P sfslite1
    or else by pulling a tarball from this location.
  2. Getting decent performance with DQE depends on a few small performance improvements to SFS. To get these improvements, apply the patch dqe/enforcer/sfs-changes.patch to the sfslite source tree.

  3. When you run configure to get the build directories set up for sfslite, you should pass the following options:

  4. Now you need to hack the Makefile for the enforcer so that DQE builds properly. In dqe/enforcer/rules.Makefile, set

  5. Now you're ready to build dqe:

    $ cd dqe/storer
    $ gmake
    or
    $ cd dqe/storer
    $ gmake mode={optmz,profile}
  6. You may need to delete the commands in dqe/enforcer/Makefile that do chown.
  7. At the end, you should have a binary named dqm_node. Try doing:

    $ ./dqm_node

    The result should be a usage message.

Running the Enforcer

Each enforcer node takes as input a configuration file. This configuration file includes parameters like the local file storing the key-value pairs, the identities of the other enforcer nodes (every node is statically configured to know every other node), and the replication factor, r.

The config file also specifies which UDP ports the enforcer node uses. There are three UDP ports that are relevant: the first one, known as the portal_port, is the port that an enforcer client contacts (see the section below on running enforcer clients) to run TEST or SET operations. The second port is another port on which the enforcer node listens: it is called the storer_port and this is the port that other enforcer nodes contact to run PUT or GET operations. The third port is the source UDP port that the enforcer node uses when invoking PUT or GET at other enforcer nodes. (The reasons that we have all of these ports relate to prioritizing requests based on their type; see the paper for details.)

To run an enforcer node, you should choose three unused ports per enforcer node, with the understanding that the portal_port will be the "external" port that enforcer clients use.

There are a few ways to generate config files:

By this point you should have config files for the various enforcer nodes that you want to run.

Finally, run the enforcers themselves. On each enforcer node:

$ cd dqe/enforcer
$ ./dqm_node <config_file>

(To see a list of available command-line options for dqm_node, run it without a config file argument.) At this point, an enforcer client should be able to do TEST and SET operations at the enforcer nodes. See the next sections below for details on how to get enforcer clients running and talking to enforcer nodes.

Client Used in Experiments

To build and use the experimental client:

$ cd dqe/u
$ make

You should get a binary called u2. To run u2, you'll have to humor it with some number of command-line options, some of which should be self-explanatory.

Enforcer Clients

An enforcer client will normally be an email server. An email server needs to either print stamps (if it is the outgoing email server) or else check and SET stamps (if it is the receiving email server).

Compiling enforcer clients

Most of the enforcer client is written in Python, so there isn't much compiling, except for a Python wrapper of some crypto functions that are implemented in C++.

Anyway, here are the steps to getting the Python client going:

  1. Install the packages:

  2. Build the crypto wrapper:

    $ cd dqe/dqm/cryptkey_wrap
    $ make
  3. Set your $PYTHONPATH environment variable appropriately, e.g.:

    $ cd dqe
    $ MYDQEDIR=`pwd`
    $ PYTHONPATH=$PYTHONPATH:$MYDQEDIR
    $ export $PYTHONPATH    

Running enforcer clients

To do simple TEST and SET operations using a bare-bones client, do something like this:

$ cd dqe/dqm/user
$ ./portal-client.py myenforcerportal:8101 zebra set
$ ./portal-client.py myenforcerportal:8101 zebra test
$ ./portal-client.py myenforcerportal:8101 monkey test

With the commands above, portal-client.py is going to use "zebra" and "monkey" as strings to hash to get key-value pairs, and it will use as a portal to the enforcer the host myenforcerportal and destination port 8101. With a "real" DQE client, as described immediately below, the key-value pairs will be actual stamps.

To print, verify, and cancel actual stamps, you need to run, respectively:

$ dqe/dqm/user/stamp-printer.py
$ dqe/dqm/user/stamp-verifier.py
$ dqe/dqm/user/stamp-canceller.py

Each of these programs needs a config file that points the program to an enforcer portal and also gives the program the public key of a trusted quota allocator. To actually get these programs to work and to integrate them with a mail setup requires a bit of annoying setup; see here for slightly outdated instructions. Please feel free to send us email with questions. Our email address is dqe at the domain nms.csail.mit.edu.