发布网友 发布时间:2023-12-24 15:58
共2个回答
热心网友 时间:2024-08-02 05:52
Hadoop is implementation framework of cloud computing platform, is an Apache open source software implementation based on Java language cloud development platform, and the data processing software platform, including the massive amounts of data distributed computing of large mass development and operation of computer cluster. By analyzing the Hadoop cloud computing platform in the face of a huge number of the problems existing in the small file shows the Hadoop master node NameNode put files and folders, etc all metadata manage memory, large-scale small file will make the NameNode winner node memory utilization greatly reduce or even lead to system crash. In order to solve this problem, based on the cloud platform Hadoop's own small File solution, is put forward based on the Sequence File and prefetching mechanism of combining the Hadoop mass solution for small files. Experimental results show that the application of the scheme Hadoop cloud platform, file access speed by about 34%, the master node NameNode server memory usage reduced about 15%.热心网友 时间:2024-08-02 05:49
Hadoop is a framework of cloud computing platform, is a Apache open source software calculation of development platform Java language based cloud, and data processing software platform, large-scale development and operation includes a large computer clusters of massive data in distributed computing. Hadoop platform in the face of massive small file through analyzing the problems of cloud computing, NameNode master node in the Hadoop all the metadata file and folder, in memory management, small file large-scale will make winner node NameNode memory utilization greatly reduce and even lead to system collapse. In order to solve this problem, this paper combined with small files with the Hadoop cloud platform solution, proposed a Sequence File and prefetching mechanism based on the combination of Hadoop mass small file solution. Experiments show that, in the Hadoop cloud platform application of the program, file access speed is increased by about 34%, use the main node of NameNode server memory to reduce the rate of about 15%.