Harnessing low-level tuning in modern architectures for high-performance network monitoring in physical and virtual platforms
- Autores: Víctor Moreno Martínez
- Directores de la Tesis: Francisco Javier Gómez Arribas (dir. tes.)
- Lectura: En la Universidad Autónoma de Madrid ( España ) en 2015
- Idioma: español
- Tribunal Calificador de la Tesis: Jaime Moreno (presid.), Iván González Martínez (secret.), David Fernández Cambronero (voc.), Sandrine Vaton (voc.), Mikel Izal Azcárate (voc.)
- Materias:
- Texto completo no disponible (Saber más ...)
- Resumen
Over the past decades, the use of the Internet has rapidly grown due to the emergence of new services and applications. The amount them available to end-users makes it necessary for their providers to deploy quality-assessment policies in order to distinguish their product among the rest. In this scenario, network processing and analysis becomes a central task that has to deal with humongous amounts of data at high-speed rates. Service providers must be able to accomplish such a challenging task using processing elements capable of reaching the required rates while keeping the cost as low as possible for the sake of profitability. This thesis analyses the main problems and provide viable solutions when applying commodity-hardware for high-performance network processing. Furthermore, diverse systems have been developed in this line, which have also been validated in industrial environments.
Traditionally, when the requirements were tight an eye was turned to the use of ASIC designs, reprogrammable FPGAs or network processors. This work is started with a study and evaluation of diverse architectural solutions for network processing. Those solutions offer great computational power at the expense of high levels of specialization. Consequently, they only address the performance half of the problem but they fail at solving the other half, which is the inexorably need to perform more diverse, sophisticated and flexible forms of analysis. Moreover, those solutions imply high investments: such hardware's elevated cost rises capital expenditures (CAPEX), while operational expenditures (OPEX) are increased due to the difficulties in terms of operation, maintenance and evolution. Furthermore, HW life cycles become shorter as technology and services evolve, which complicates the stabilization of a final product thus reducing profitability and limiting innovation. Those drawbacks turn specialized HW solutions into a non-desirable option for large-scale network processing. Nevertheless, this thesis has also evaluated the the use of this possibility using FPGA technology. Specifically, a prototype has been developed for network packet capture with accurate timestamping, reaching a tenths of nanoseconds precision and with GPS synchronization.
In this light, both industry and academia have paid attention to the use of solutions based on commodity-hardware. The advantages of those systems lay in the ubiquity of those components, which makes it easy and affordable to acquire and replace them and consequently reduces CAPEX. Those systems are not necessarily cheap, but their wide-range of application allows their price to benefit from large-scale economies and makes it possible to achieve great degrees of experimentation. Additionally, such systems offer extensive and high-quality support, thus reducing OPEX. Unfortunately, the use of commodity hardware in high-speed network tasks is not trivial due to limitations on both hardware capacities and standard operating systems' performance. Essentially commodity hardware is limited in terms of memory and internal bus throughputs. From the software side, limitations come from a general-purpose network stack that that overloads communications due to a prioritization of protocol and hardware compatibility over performance.
It is in this context in which the main contribution of this thesis, HPCAP, is presented. HPCAP is a high-performance capture engine that has been designed to solve the problems not yet solved by the state-of-the-art capture engines while keeping similar performance levels. The literature references capture engines are capable of capturing 10 Gb/s network traffic, but do not pay attention to the vital tasks, e.g.: storing this traffic onto non-volatile storage systems, accurately timestamp the traffic, or feed this traffic to diverse processing applications. Those are in fact the most relevan contributions of HPCAP to the field. We have empirically verified that if the network packets are not accurately timestamped when carrying out network monitoring tasks, the analysis made can lead to wrong conclusions. Packet timestamping accuracy is not only affected by the accuracy of the time source used, but also by the moment in which packets are timestamped: the more code is executed between the packet's arrival and its timestamping moment, the more variability and error appears. On the other hand, there are many scenarios in which after a high-level analysis over the network traffic, it is required to access to low-level packet information to identify problem sources. Consequently, keeping the packets stored for their subsequent access becomes a relevant issue. In this line, it seem reasonable to instantiate several network traffic analysis applications while an independent application is in charge of storing the traffic in non-volatile place. Nevertheless, this requirement is difficult to reach without paying the performance loss price, and this is the reason for which HPCAP has been designed taking this into full consideration. Furthermore, M3Omon has been created: a general-purpose network processing framework built on top of HPCAP, whose performance as also been exhaustively tested. M3Omon pretends to be a reference point for easily developing high-performance network applications, which has already been applied in the development of several project with a direct industrial application.
Other application domain with undeniable interest is the world of virtualized platforms. If those solutions based on commodity-hardware are to be applied in realistic highly-demanding scenarios, the increased demands for network processing capacity could be translated into a big number of machines even though if off-the-shelf systems were used. Such an amount of machines means high expenses in terms of power consumption and physical space. Moreover, the presence of commodity servers from different vendors empowers the appearance of interoperability issues. All those drawbacks damage the profitability that networked service providers may experience. This situation has pushed towards the applications of virtualization techniques in network processing environments, with the aim of unifying existing computing platforms. Techniques such as PCI-passthrough allow the migration of the results obtained in the physical world to the virtual one in a very direct way. This work has carried out a study regarding the impact of this technique on the previously presented systems, concluding that the performance loss experienced is appealingly low. Another approach is the on propose by an alliance composed of network operators and service providers which has introduced in the last years the concept of Network Function Virtualization (NFV). This new paradigm aims to unify the environments where network applications shall run by means of adding a virtualization layer on top of which network applications may run. This novel philosophy also allows merging independent network applications using unique hardware equipment. However, in order to make the most of NFV, mechanisms that allow obtaining maximum network processing throughput in such virtual environments must be developed. The contribution of this work in the NFV field is the evaluation of performance bounds when carrying out high-performance network tasks in NFV environment and the development of HPCAPvf as as NFV counterpart for the formerly presented HPCAP.
