Article From:https://www.cnblogs.com/yuanyifei1/p/9122880.html

hortonworksThe source code can be found on the GitHub!

https://github.com/hortonworks/spark2-release

Find the corresponding version of the release source code and download it to local

Windows can be compiled directly, and Ubuntu needs to install zinc.

sparkThe project uses the private remote library of hortonworks!

Address: http://nexus-private.hortonworks.com/nexus/content/groups/public

The compiler tool is maven, the warehouse configuration method: open the spark project root directory pom.xml, find the repositories node, add

<repository>
<id>public</id>
<url>http://nexus-private.hortonworks.com/nexus/content/groups/public</url>
</repository>

Prevent memory overflow. Windows adds environment variables MAVEN_OPTS -Xmx2g -XX:ReservedCodeCacheSize=512m

linux Modify the /etc/profile file at the end to add export MAVEN_OPTS= MAVEN_OPTS= “-Xmx2g -XX:ReservedCodeCacheSize=512m”.

The environment variable will take effect after restarting the computer

Call./build/mvn -DskipTests clean package to start compiling

Leave a Reply

Your email address will not be published. Required fields are marked *