# 同步图运算框架GraphLite（学习手札）基本思想 & 核心API

• 框架的基本思想
• 核心API的使用

GraphLite框架源码获取：https://github.com/schencoding/GraphLite

# 分布式存储

GraphLite图计算框架，将图划分为若干个子图，再进行分布式存储、计算。

• 单个机器都没有完整的图；
• 每个Worker存储着一个子图。而每个子图里又有许多的图节点，这里每个节点都有一个Compute。

# 节点通信

（是否在同一台机器上，已经不重要了，或者说可以做到对Compute是透明的）

# 存储结构分析

Input文件中数据格式

4039
88234
0 1
0 2
//…

## 建立图模型

• GraphLite框架首先读取了两行文本，先获得了图节点总数、图边总数；
• 然后一次性申请开辟了所有空间，这就有了Vertex array、Out-edge array；

• 该节点有多少条边；
• 第一条边的地址在哪里；
• 同时，每个图节点还挂接了一个指向消息链表（接收邻居发来的信息）的指针。

# 全局同步器分析

AggrValue m_global; /**< aggregator global value of AggrValue type */
AggrValue m_local;  /**< aggregator local value of AggrValue type */


## 主要的几个API介绍，

accumulateAggr ，统计单个Worker中所有节点的计算结果；（它更改的是m_local的值，在超步过程中会自动发送到Master；Master下发消息时会自动设置m_global值。）
getAggrGlobal ，获取Master下发的上一个超步中的全局计算值；
mutableValue ，修改Vertex图节点值；
sendMessageToAllNeighbors ，给所有邻居发送消息；
voteToHalt ，将本节点置为未激活状态。

# PageRank算法

• 首先，每个节点都初始化为1/N；
• 然后，根据计算式，循环计算每个节点的值；
• 何时收敛？当上一次计算值，与本次计算值相同时（或者误差在一定范围内）就认为收敛了。

## 可能的精度丢失

class VERTEX_CLASS_NAME(): public Vertex <double, double, double> {
public:
void compute(MessageIterator* pmsgs) {
double val;
if (getSuperstep() == 0) {
val= 1.0;
} else {
if (getSuperstep() >= 2) {
double global_val = * (double *)getAggrGlobal(0);
if (global_val < EPS) {
voteToHalt(); return;
}
}

double sum = 0;
for ( ; ! pmsgs->done(); pmsgs->next() ) {
sum += pmsgs->getValue();//此处为Message Value，非Vertex Value，也不是Edge Value。
}
val = 0.15 + 0.85 * sum;

double acc = fabs(getValue() - val);
accumulateAggr(0, &acc);   //统计单个Worker误差
}
* mutableValue() = val;
const int64_t n = getOutEdgeIterator().size();
sendMessageToAllNeighbors(val / n);
}
};


References:
[1] 陈世敏老师的《大数据管理系统与大规模数据分析》课程讲义
[2]王斌老师的《信息检索导论》课程讲义

2018-5-15 于北京 怀柔

# 附录

What is Aggregator？

Aggregators are a mechanism for global communication in synchronized graph computation. Aggregators are often used to implement sum, average, max, min, and other types of statistics. An Aggregator can be updated by vertex compute() in parallel. GraphLite guarantees the correctness. That is, all the updates will be correctly reflected in the Aggregator’s global state.

The basic idea is for each worker to keep a local copy of an Aggregator. The local copy is updated by the vertex compute() during a superstep. At the end of a superstep, the workers send their local copies of the Aggregator to the master. The master merges all the local copies and the previous global value to obtain the new global value for the Aggregator. This new global value will be sent back to all the workers, which can be read by vertex compute().

How Master and Workers use AggregatorBase?

1. Global value and local value
An Aggregator should have global states and local states.
We use global value and local value to denote them, respectively.

The value type is flexible and can be specified as T in the Aggrator.
The global value is m_global. The local value is m_local.

Master modifies the global value. Global value is read-only for all the workers.
A Worker modifies the local value. Master obtains all the workers’ local values and merge them to the global value.

2. Preparation:
Both Master and Workers will call Graph::init(). Please see PageRankVertex.cc for an example. This method should use regNumAggr(), regAggr() to register Aggregators with the system.

Master and Workers will use the Aggregator objects from the regAggr() to perform the actual Aggregator computaton.

Master and Workers will call AggregatorBase::getSize() to get the value size of the Aggregator so that the values can be communicated between Master and Workders.
3. In every superstep

• Master call AggregatorBase::getGlobal() to get the current global value. Then Master sends the global value to all the workers (in the superstep start message).
• When a worker receives the superstep start message, it extracts the global value, and calls AggregatorBase::setGlobal() to set the (read-only) global value in the local copy of the Aggregator.
• If vertex compute() calls getAggrGlobal(), then the read-only global value is returned. This value reflects the global value as of the end of the previous superstep.
• If vertex compute() calls accumulateAggr(), then Worker calls AggregatorBase::accumulate(), which is supposed to accumulate the new value to the local value.
• At the end of a superstep, every Worker sends an superstep end message including the local value of the Aggregator to Master. Worker calls AggregatorBase::getLocal() to obtain the local value for this purpose.
• When receiving a superstep end message, Master extracts the embedded local value of the Aggregator and calls AggregatorBase::merge() to merge the local value to the global value.
1. AggregatorBase::init()
Master will call AggregatorBase::init() after 3(1). Worker will call AggregatorBase::init() after 3(5).
AggregatorBase::init() often clears the local value.
AggregatorBase::init() may or may not clear the global value:
• Aggregator is cleared at each superstep:
If this behavior is desired, then clear the global value.
• Aggregator is accumulated for the entire computation across all
supersteps:
If this behavior is desired, then do not clear the global value.
The global value can be initiated in the class constructor.

The above is the aggregator class in the PageRank example. It implements a sum Aggregator. The value type is double. It is easy to understand the implementation of getGlobal, setGlobal, getLocal. accumulate() adds a new value as given by p to the local value. merge() adds a local value as given by p to the global value.

Here, we clear the global value in init(). So the sum is cleared in each superstep. Note that PageRank uses this Aggregator to compute the difference between the page ranks of two supersteps. So it should be cleared in each super step.

01-27
05-09 3331

12-13 1041
04-10 2761
05-14 1086
06-26 3026
07-28
05-12
09-18
10-15
07-27