reduce个数疑问

来源：4-17 流量统计实战升级之自定义Partitioner

慕粉1926212165

2021-07-27

 * map的输出到reduce端，是按相同的key分发到一个reduce上去执行
 * reduce1：(hello, 1) (hello, 1) (hello, 1)  => (hello, <1,1,1>)
 * reduce2：(world, 1) (world, 1) (world, 1)  => (world, <1,1,1>)
 * reduce3：(welcome, 1) => (welcome, <1>)
 * 
**上面的reduce个数和下面的numReduceTasks是一个东西吗**

 public class HashPartitioner<K, V> extends Partitioner<K, V> {

/** Use {@link Object#hashCode()} to partition. */
public int getPartition(K key, V value, int numReduceTasks) {
return (key.hashCode() & Integer.MAX_VALUE) % numReduceTasks;
}

}

写回答

1回答