环境:spark1.4.0,hadoop2.6.0

1.安装好jdk

2.在spark的conf目录下找到spark-env.sh.template,打开,在后面加上

export SCALA_HOME=/home/jiahong/scala-2.11.6
export JAVA_HOME=/home/jiahong/jdk1.8.0_45
export HADOOP_CONF_DIR=/home/jiahong/hadoop-2.6.0/etc/hadoop

 

3.在eclipse的help->install new sofware上安装好scala插件

  地址为:http://download.scala-ide.org/sdk/e38/scala29/stable/site

  spark在eclipse上配置_scala

4.使用Scala语言来开发spark程序

  3.1 在eclipse中,依次选择“File” –>“New” –> “Other…” –>  “Scala Wizard” –> “Scala Project”,创建一个Scala工程,命名

  3.2 右击“SaprkScala”工程,选择“Properties”,在弹出的框中,按照下图所示,依次选择“Java Build Path” –>“Libraties” –>“Add External JARs…”,加入spark的lib目录下的spark-assembly-1.4.0-hadoop2.6.0.jar

  spark在eclipse上配置_spark_02