HDFS scalability: the limits to growth

• Autores: Konstantin V. Shvachko
• Localización: ;login:: the magazine of USENIX & SAGE, ISSN 1044-6397, Vol. 35, Nº. 2 (ABR), 2010, págs. 6-16
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • The Hadoop distributed file system (HDFS) is an open source system currently being used in situations where massive amounts of data need to be processed. Based on experience with the largest deployment of HDFS, I provide an analysis of how the amount of RAM of a single namespace server correlates with the storage capacity of Hadoop clusters, outline the advantages of the single-node namespace server architecture for linear performance scaling, and establish practical limits of growth for this architecture. This study may be applicable to issues with other distributed file systems.