A hardware architecture of stateless open digest spam fingerprinting unit

Abstract

Spam has become one of the main problems for email users and servers. Approximately around 75 % of all sent emails is spam. Currently spam detection is being done at email servers on application layer which need to store all TCP packets of an email and then use spam detection methods after reassembling. The spam detection technique on the application layer is slow and cannot cope with the current speed of the internet. If email classes could be estimated before being accepted by receiving MTAs for queuing, better spam handling strategies could be utilized. The aim of this project is divided in two parts. First to develop an inline Internet Protocol wrapper that is capable for processing packets for stateless content classification. The second aim is to the development of a hardware spam detection unit based on Nilsimsa fingerprinting algorithm. The IP wrapper is implemented successfully on hardware. This unit increases the Ethernet speed from 25 Mbit/s to 85 Mbit/s and provides a simple interface to transfer and receive packets from Ethernet port. IP Wrapper also provides extra time to process received packets for the microcontroller. In this project we present hardware architecture of a modified form of Nilsimsa algorithm to be implemented on hardware for higher fingerprinting throughput while having the same accuracy as the original algorithm. The improvement is noticeable in the term of processing speed and hardware allocated resources, and the accuracy tested on false positive spam detection. The implemented algorithm works with the operation frequency up to 123 MHz and has the throughput of 1 byte per 2 clock cycles (4 bit per clock cycle). Our system also needs extra 512 clock cycles after receiving whole of the message to complete the computation of Nilsimsa fingerprint.