terena logo
18 September 2007
Mature LOBSTER has a wide reach
 

The LOBSTER project, which has developed a pilot European infrastructure for accurate Internet traffic monitoring, concludes this week with its final review on 18 September.

Within the past four months, the number of passive monitoring sensors installed in the LOBSTER infrastructure has risen to 50, with around 200,000 new cyberattacks being detected. This is in addition to the 400,000 attacks that were captured during the previous year, and reflects the expansion of the sensor network and improved detection methods.

Sensors are now deployed in several European countries, as well as Singapore and the United States. They continue to monitor more than 2 million IP addresses, with the aggregate traffic capacity on the monitoring links exceeding 75 gigabits per second. The locations of sensors in the network can be viewed using Google Maps, where a click on each icon allows their traffic data to be examined.

A selection of the cyberattacks captured by LOBSTER’s sensors have also had their payload traces anonymised and have been made publicly available for research purposes via an 'attack trace repository'.

LOBSTER software

The sensors use software developed by the project. It is available on a bootable Linux CD that allows users to try it out before installation. The project also released new versions of the Stager application - a system for aggregating and presenting network statistics - and the Ruler language that allows for high-speed matching and rewriting of network traffic based on regular expressions. (More details below.)

The LOBSTER infrastructure is unique in Europe and one of only three similar infrastructures that exist in the world today.

The project was comprised of nine partners including research organisations, commercial partners and National Research and Education Networks (NRENs).

More Detailed Information

LOBSTER's passive Internet monitoring sensors are deployed in Bulgaria, Cyprus, the Czech Republic, Greece, FYR Macedonia, Montenegro, the Netherlands, Norway (including Svalbard), Serbia, and Spain, as well as Singapore and the United States.

The sensors use commodity PC hardware with a variety of network adapters (which include the specialised DAG card as well as regular NICs).

The project's version of the Stager application is a system for aggregating and presenting network statistics, and although tailored for using NetFlow data, it is generic and can be customised to present and process any kind of network statistics. The back-end collects data with flow-tools, and stores reports in a database before automatically producing daily, weekly and/or monthly statistics. A web front-end can present data as tables, matrices or plots, with fully customisable reports.

The project's new version of Ruler language is simple to use, as well as being extremely powerful and fast (payload scanning at gigabits per second). Back-ends exist for general-purpose CPUs, Intel IXP2400 network processors and Xilinx FPGAs.

Rulerproxy is a complementary Linux-based program that allows one to apply Ruler filters to reassembled TCP streams (e.g. to scan for worms, unwanted content, or to rewrite/anonymise regex patterns).

Links

LOBSTER homepage

LOBSTER sensor network

The LOBSTER attack trace repository and report

Bootable CD of LOBSTER sensor software

New version of the Ruler language

New version of the Stager application