Is there a way to use a pom.xml file to update spark configuration?
I am trying to update my spark configuration to solve some dependency problems.
This pom.xml file seems to be useful for this purpose.
I am using a spark docker image.
ls /spark/conf
Gives:
docker.properties.template slaves.template
fairscheduler.xml.template spark-defaults.conf.template
log4j.properties.template spark-env.sh.template
metrics.properties.template
I've searched pom.xml in the container by using find / -name "pom.xml"
and got nothing.
Is there a way to use a pom.xml file to update spark configuration?
Topic scala apache-spark bigdata
Category Data Science