论文部分内容阅读
MapReduce集群系统的单一控制节点结构使其存在着性能瓶颈问题,限制了集群的规模.为较好地解决系统性能瓶颈问题,提出了MapReduce架构的多控制点改进.新系统的实现基于Hadoop MapReduce,在对架构进行改进的同时,重新设计了信令系统,并增加了控制节点间的热备份机制.因此,新架构除了能解决瓶颈问题外,还可以通过控制节点间的热备份策略有效地应对控制节点故障,降低故障对集群性能的影响,保证应用的正常执行.
The single control node structure of MapReduce cluster makes it have the performance bottleneck problem, which limits the scale of the cluster.In order to solve the bottleneck of system performance better, a multi-control point improvement of MapReduce architecture is proposed.The implementation of the new system is based on Hadoop MapReduce, While improving the architecture, we redesigned the signaling system and added the hot backup mechanism between the control nodes so that the new architecture can effectively solve the bottleneck problem by controlling the hot backup strategy between nodes Control node failure, reduce the impact of failure on the cluster performance, to ensure the normal implementation of the application.