前言
不同版本的Spark在yarn模式下的内存分配参数值有一定的差异,但是原理是不变的,本文就spark-1.6.0基于yarn的内存分配来说说为什么这样的问题。
参考:
https://spark.apache.org/docs/1.6.0/running-on-yarn.html
http://blog.javachen.com/2015/06/09/memory-in-spark-on-yarn.html
https://www.zybuluo.com/xiaop1987/note/102894
http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/
http://blog.cloudera.com/blog/2014/05/apache-spark-resource-management-and-yarn-app-models/
未完~待续