Packet-level open-digest fingerprinting for spam detection on middleboxes

Abstract

This paper proposes a stateless open-digest spam fingerprinting at the packet level (layer 3) based on an open-digest fingerprinting algorithm Nilsimsa. Spam emails show several characteristics when viewed at gateway level, which are suitable for spam fingerprinting: (a) content invariance and (b) recipient address dispersion. In this paper, Nilsimsa is adapted to support both fingerprinting and fast email class estimation, on a per-packet basis. Email packets are incrementally fingerprinted on a per-packet basis, without the need for reassembly. Spam detection status is tagged to the last packet of each email. This in turn allows fast email class estimation (spam detection) at receiving email servers to support more effective spam handling on both inbound and outbound (relayed) emails. The work presented in this paper focuses on evaluating the accuracy of spam fingerprinting at the packet level with consideration on the constraints of processing byte streams over the network, including packet reordering, fragmentation, overlapped bytes, different packet sizes, and possibilities of random addition attacks. Results show that the proposed packet-level fingerprinting can detect spam with 100% random addition when the similarity threshold is set to between 36 and 59. This method gives 0% false positive and 100% true negative, which equals the performance attained for spam fingerprinting at full email abstraction (layer 7). This shows that classifying emails at the packet level can differentiate non-spam from spam with high confidence for a viable spam control implementation on middleboxes.