• Read Me
• Download
• NANOG 31 presentation

Introduction

The AddrTree toolchain is used by the AirCERT project to pre-process allocation data from the Regional Internet Registries (RIRs) for use as a supplemental data source for the analysis of security event data. Specifically, allocation data can be used to group event source and destination addresses by network, by contact/organization, and by location (currently, only country codes are supported by the AddrTree chain).

The pre-processing includes normalization of modification dates, resolution of conflicts between the RIRs, repair of range erosions (probable single-CIDR block allocations that are not properly aligned to CIDR block boundaries), and resolution of inversions (violations of allocation tree structure due to corrupted or stale allocation range data). After anomalies are corrected, each record is split into CIDR blocks. Additionally, AddrTree keeps a log of anomalies it found for later analysis.

Usage

AddrTree is a collection of Perl scripts whose workflow is managed by Make. It requires Perl (of course), GNU Make, and NcFTP (for automatic download of RIR databases). Additionally, because AddrTree does in-memory sorting of allocation records to build the tree, it also requires at least 1GB of available memory (on ia32 machines; requirements may vary on other architectures).

To use AddrTree, place your ARIN database access credentials in the arin_cred.txt file, then make download cidr. After an hour or so of processing, the IPv4 allocation tree in AddrTree format will be written to data/world_cidr.at, and the anomaly log to anomaly.log.

The resulting world_cidr.at file can be translated to tab-delimited format for import into a relational database using the atDumpTDF.pl script.

Data Format

AddrTree represents RIR allocation objects in the .at files using a line-oriented text format of colon-delimited records sorted in in tree traversal order (i.e. sorted by range start address ascending, then by range end address descending, then by allocation level ascending (allocation level is the number of intermediate allocations and assignments between a given object and its root allocation). The fields in each line are as follows:

The anomaly log uses a similar format with twelve columns; it is the AddrTree format above with an additional anomaly code column prepended. The currently supported anomaly codes are as follows:

Processing Stages

The following is a list of scripts making up AddrTree and their functions. All scripts read from standard input and write to standard output (and can be piped accordingly), and append anomalies to anomaly.log, unless otherwise noted. Note that the proper workflow for these scripts is already encoded in the included Makefile.