Skip to content

Commit

Permalink
Modify spark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ChengJie1053 committed Sep 8, 2023
1 parent c148682 commit 5c3f79e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/configuration/spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
Use spark yarn cluster mode,need to set label "engingeConnRuntimeMode": "yarnCluster",and need to upload the dependence of the spark to 'linkis.spark.yarn.cluster.jar'(the default value is 'hdfs:///spark/cluster')
spark dependencies include jars and configuration files,For example: '/appcom/Install/linkis/lib/linkis-engineconn-plugins/spark/dist/3.2.1/lib/*.jar','/appcom/Install/linkis/conf/*'
Precautions for using yarnCluster:
Eureka url if 127.0.0.1 should be changed to the real host, such as "http://127.0.0.1:20303/eureka/" should be changed to "http://wds001:20303/eureka/"
Eureka url if 127.0.0.1 should be changed to the real host, such as "127.0.0.1:20303/eureka/" should be changed to "wds001:20303/eureka/"

The spark-excel package may cause class conflicts,need to download separately,put it in spark lib
wget https://repo1.maven.org/maven2/com/crealytics/spark-excel-2.12.17-3.2.2_2.12/3.2.2_0.18.1/spark-excel-2.12.17-3.2.2_2.12-3.2.2_0.18.1.jar
Expand Down

0 comments on commit 5c3f79e

Please sign in to comment.