Is there a way to use a pom.xml file to update spark configuration?

I am trying to update my spark configuration to solve some dependency problems.

This pom.xml file seems to be useful for this purpose.

I am using a spark docker image.

ls /spark/conf

Gives:

docker.properties.template    slaves.template
fairscheduler.xml.template    spark-defaults.conf.template
log4j.properties.template     spark-env.sh.template
metrics.properties.template

I've searched pom.xml in the container by using find / -name "pom.xml" and got nothing.

Is there a way to use a pom.xml file to update spark configuration?

Topic scala apache-spark bigdata

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.