Resumen de Large-scale edge network monitoring
Network monitoring is vital for network operators, as it allows them to assess the level of Quality of Service (QoS) they offer to their users. To perform this monitoring, several techniques and protocols can be used.
At a high level, protocols such as the Simple Network Management Protocol (SNMP) provide a snapshot of the state of a network device or server. Devices are polled periodically to monitor their status and find issues or bottlenecks that may negatively affect QoS. A lower level approach is network traffic capture. With this technique, every network packet is inspected and the information is aggregated in the form of network flows that are used to detect problems in the network itself. Both techniques can help detect and fix network issues by themselves. However, when combined they provide a more complete view of the network that can lead to faster analysis and detection of the root cause of the problem.
Traditionally these measurements have been performed at the core of the network, where measuring at a few points can provide an entire view of the system. However, as the number of devices and speed of the networks increases, this approach is becoming unfeasible. This is due to the high cost of obtaining and maintaining monitoring equipment capable of dealing with the increased traffic.
This thesis proposes a solution to this problem that takes advantage of high and low level monitoring approaches. First, SNMP is used to measure the core network and devices close to the edge that support this protocol. On the other hand, network monitoring is performed on user devices at the edge of the network. Here the network speed is much lower and user devices can capture network traffic and generate flow records with a negligible effect on the user's experience.
To achieve this goal we designed and developed three different pieces of software. First, we developed a high performance SNMP measurement software. It is capable of obtaining information from the ever increasing number of network devices that can be found at the core and edge of the network. Next, we developed a network measurement agent called microsniffer capable of inspecting network traffic and generating flow records. This agent has a low footprint as it is intended to be run at user's devices without affecting their experience. Finally, client and server software was developed to control the behavior of the microsniffer and collect its flow records. The client can be configured to send just records that may indicate a problem. This reduces the footprint of the software on the user's device and decreases the storage requirements on the collection server.
During the design of these programs, we focused on high performance, security, and the user experience of the analysts tasked with configuring the software and reviewing the collected data.
In this thesis, we explain the design of each program together with the challenges that led us to choose one design instead of its alternatives. We explain their implementation details to show how they work and the way they achieve the desired performance.
Next, we performed exhaustive tests to check the validity of each program output and assess their performance.
Finally, we show examples of the information that can be obtained from each program and show how the high and low level approaches complement each other to achieve a more complete view of what happens in a network.
