设为首页 加入收藏

TOP

Giraph基础介绍
2014-11-24 03:21:22 来源: 作者: 【 】 浏览:1
Tags:Giraph 基础 介绍

Giraph介绍:

Apache Giraph is an iterative graph processing system built for high scalability. For example, it is currently used at Facebook to analyze the social graph formed by users and their connections. Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in a 2010 paper. Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant. Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more. With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

原理:Giraph基于Hadoop而建,将MapReduce中Mapper进行封装,未使用reducer。在Mapper中进行多次迭代,每次迭代等价于BSP模型中的SuperStep。一个Hadoop Job等价于一次BSP作业。基础结构如下图所示。

\

Giraph 的基础架构

每部分的功能如下:

(1)ZooKeeper: responsible for computation state
partition/worker mapping
global state: #superstep
checkpoint paths, aggregator values, statistics

(2) Master: responsible for coordination
assigns partitions to workers
coordinates synchronization
requests checkpoints
aggregates aggregator values
collects health statuses

(3) Worker: responsible for vertices
invokes active vertices compute() function
sends, receives and assigns messages
computes local aggregation values

n块 i*讦… http://www.2cto.com/ym源码。

】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
分享到: 
上一篇数据库分组求和语句 下一篇Linux下MySQL的load_file常用路径

评论

帐  号: 密码: (新用户注册)
验 证 码:
表  情:
内  容:

·Shell脚本:Linux Sh (2025-12-25 05:50:11)
·VMware虚拟机安装Lin (2025-12-25 05:50:08)
·Linux学习教程,Linu (2025-12-25 05:50:06)
·工业机器人TCP校准中 (2025-12-25 05:19:17)
·opc 通讯协议与 TCP (2025-12-25 05:19:15)