DOE/SciDAC High-Performance Network Research Project
INCITE:
Edge-based Traffic Processing and Service Inference
for High-Performance Networks
Rice University, Stanford Linear Accelerator
Center, Los Alamos National Labs
INCITE is a high-performance
networking research project funded by DOE SciDAC.
The explosive growth of high-speed computer networks, combined with
rapid and unpredictable developments in applications and workloads, has
rendered network modeling, control, and performance prediction
increasingly demanding tasks. Critical high-end applications such as
remote visualization and high-capacity data transfers routinely fail to
meet end-to-end performance expectations when deployed on high-speed
networks. Optimizing the performance of these and other demanding
applications requires that end-systems have knowledge of the internal
network traffic conditions and services.
Without special-purpose network support (at every router), the only
alternative is to indirectly infer dynamic network characteristics from
edge-based network measurements. Furthermore, the complexity of network
dynamics demands more advanced mathematical theory and methods in order
to develop scalable algorithms and accurate inference methods to
support high-performance computing infrastructures, such as
computational grids.
The INCITE (InterNet Control and Inference Tools at the Edge) Project
focuses experts from the fields of networking, supercomputing,
statistical signal processing, and applied mathematics towards the goal
of analyzing, modeling, and characterizing high-speed network services
based solely on edge-based measurement at hosts and/or edge routers.
This project develops an innovative framework for capturing the complex
dynamics and controlling the performance of networks driven by real
applications. Specifically, we aim to develop on-line tools to
characterize and map network performance as a function of space, time,
application, protocol, and service. This understanding will allow us to
devise low-complexity models and measurement methodologies amenable to
optimized on-line control and management.
Our effort consists of four closely inter-related research thrusts that
directly address the key challenges facing the DOE high-performance
networks program (Network measurement and analysis, High-performance
transport protocols, and Advanced traffic engineering tools and
services):
Thrust I. Multiscale Traffic Models and Analysis Techniques
We will design novel models that capture the multiscale variability and
burstiness of high-speed network traffic (Task 1). Leveraging the
powerful theory of multifractals, we will integrate model fitting,
synthesis, and prediction into one unified statistical framework. Our
new models, designed to match salient traffic characteristics at a
prescribed level of abstraction, will offer unprecedented realism while
remaining analytically tractable, statistically robust, and
computationally efficient. Using multiscale models, we will study how
large traffic flows interact and distribute their burstiness.
Furthermore, we will investigate, analyze, and characterize the
(adverse) modulation TCP/IP places on application-level traffic.
Thrust II. Inference Algorithms for Network Paths, Links, and Routers
We will develop theory and methods for understanding and inferring
network dynamics across time, space, and service class using
measurements only from the edge. Using multiscale traffic models, we
will characterize the dynamics of end-to-end paths and connections (Task
2). Our approach to infer the competing cross-traffic load on a
connection will utilize an innovative, exponentially spaced probing
sequence of "packet chirps" that balances the trade-off between
overwhelming the network with probes and obtaining statistics rich
enough for accurate estimates. Our tomographic framework for on-line
statistical inference of link-level parameters (losses, delays, service
strategies, and topology, for example) is also based solely on
end-to-end measurements (Task 3).
Thrust III. Data Collection Tools
We will enhance the PingER software suite for multiscale path, link, and
service inference using host-to-host measurements, which can be made
without direct access to the internal network (Task 4). We will
integrate these algorithms with the proposed ESnet NIMI infrastructure.
We will develop and deploy a Monitor for Application-Generated Network
Traffic (MAGNeT) that will allow us to more closely examine,
characterize, and model how the protocol stack modulates network traffic
(Task 5). We will develop an integrated suite of Unix software tools
for multifractal path inference and network tomography that operate on
host-to-host, passive measurements (Task 6).
The interdisciplinary INCITE team will develop both comprehensive theory
and efficient, scalable algorithms for the analysis, modeling,
inference, and control of complex, high-speed networks. Moreover, our
reference implementations will provide first-of-their-kind platforms for
obtaining a deep understanding of the complex dynamics of large-scale
high-speed networks and enable principled designs of future network
architectures, algorithms, and models.
INCITE home
Dec 1, 2001.
Richard G. Baraniuk.
Rudolf H. Riedi.