Installing Java Modules#
Contents
System Compatibility#
Java modules are regularly built and tested on macOS and Linux distributions.
Java Compatibility#
Java modules are currently compatible with JDK 8, 9, 10, 11, 17, and 18, but only JDK 11 is tested in CI.
Installing from Maven#
By default, Maven will download from the central repository: https://repo.maven.apache.org/maven2/org/apache/arrow/
Configure your pom.xml with the Java modules needed, for example: arrow-vector, and arrow-memory-netty.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>demo</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<arrow.version>8.0.0</arrow.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-vector</artifactId>
<version>${arrow.version}</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-netty</artifactId>
<version>${arrow.version}</version>
</dependency>
</dependencies>
</project>
To use the Arrow Flight dependencies, also add the os-maven-plugin
plugin. This plugin generates useful platform-dependent properties
such as os.detected.name
and os.detected.arch
needed to resolve
transitive dependencies of Flight.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>demo</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<arrow.version>8.0.0</arrow.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-core</artifactId>
<version>${arrow.version}</version>
</dependency>
</dependencies>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.7.0</version>
</extension>
</extensions>
</build>
</project>
When using Java 17 or later, some JDK internals must be exposed by
adding --add-opens=java.base/java.nio=ALL-UNNAMED
. Otherwise,
you may see errors like module java.base does not "opens
java.nio" to unnamed module
.
For example, when running unit tests through Maven:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M6</version>
<configuration>
<argLine>--add-opens=java.base/java.nio=ALL-UNNAMED</argLine>
</configuration>
</plugin>
</plugins>
</build>
Environment variables: To execute your Arrow Java main code.
_JAVA_OPTIONS="--add-opens=java.base/java.nio=ALL-UNNAMED" mvn exec:java -Dexec.mainClass="YourMainCode"
Installing from Source#
See Java Development.
Installing Nightly Packages#
Warning
These packages are not official releases. Use them at your own risk.
Arrow nightly builds are posted on the mailing list at builds@arrow.apache.org. The artifacts are uploaded to GitHub. For example, for 2022/03/01, they can be found at Github Nightly.
Maven cannot directly use the artifacts from GitHub. Instead, install them to the local Maven repository:
Decide nightly packages repository to use, for example: https://github.com/ursacomputing/crossbow/releases/tag/nightly-2022-03-19-0-github-java-jars
Add packages to your pom.xml, for example: flight-core (it depends on: arrow-format, arrow-vector, arrow-memeory-core and arrow-memory-netty).
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<arrow.version>8.0.0.dev254</arrow.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>flight-core</artifactId>
<version>${arrow.version}</version>
</dependency>
</dependencies>
Download the necessary pom and jar files to a temporary directory:
$ mkdir nightly-2022-03-19-0-github-java-jars
$ cd nightly-2022-03-19-0-github-java-jars
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-java-root-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-format-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-format-8.0.0.dev254.jar
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-vector-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-vector-8.0.0.dev254.jar
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-memory-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-memory-core-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-memory-netty-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-memory-core-8.0.0.dev254.jar
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-memory-netty-8.0.0.dev254.jar
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/arrow-flight-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/flight-core-8.0.0.dev254.pom
$ wget https://github.com/ursacomputing/crossbow/releases/download/nightly-2022-03-19-0-github-java-jars/flight-core-8.0.0.dev254.jar
$ tree
.
βββ arrow-flight-8.0.0.dev254.pom
βββ arrow-format-8.0.0.dev254.jar
βββ arrow-format-8.0.0.dev254.pom
βββ arrow-java-root-8.0.0.dev254.pom
βββ arrow-memory-8.0.0.dev254.pom
βββ arrow-memory-core-8.0.0.dev254.jar
βββ arrow-memory-core-8.0.0.dev254.pom
βββ arrow-memory-netty-8.0.0.dev254.jar
βββ arrow-memory-netty-8.0.0.dev254.pom
βββ arrow-vector-8.0.0.dev254.jar
βββ arrow-vector-8.0.0.dev254.pom
βββ flight-core-8.0.0.dev254.jar
βββ flight-core-8.0.0.dev254.pom
Install the artifacts to the local Maven repository with
mvn install:install-file
:
$ mvn install:install-file -Dfile="$(pwd)/arrow-java-root-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-java-root -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-format-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-format -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-format-8.0.0.dev254.jar" -DgroupId=org.apache.arrow -DartifactId=arrow-format -Dversion=8.0.0.dev254 -Dpackaging=jar
$ mvn install:install-file -Dfile="$(pwd)/arrow-vector-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-vector -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-vector-8.0.0.dev254.jar" -DgroupId=org.apache.arrow -DartifactId=arrow-vector -Dversion=8.0.0.dev254 -Dpackaging=jar
$ mvn install:install-file -Dfile="$(pwd)/arrow-memory-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-memory -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-memory-core-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-memory-core -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-memory-netty-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-memory-netty -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/arrow-memory-core-8.0.0.dev254.jar" -DgroupId=org.apache.arrow -DartifactId=arrow-memory-core -Dversion=8.0.0.dev254 -Dpackaging=jar
$ mvn install:install-file -Dfile="$(pwd)/arrow-memory-netty-8.0.0.dev254.jar" -DgroupId=org.apache.arrow -DartifactId=arrow-memory-netty -Dversion=8.0.0.dev254 -Dpackaging=jar
$ mvn install:install-file -Dfile="$(pwd)/arrow-flight-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=arrow-flight -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/flight-core-8.0.0.dev254.pom" -DgroupId=org.apache.arrow -DartifactId=flight-core -Dversion=8.0.0.dev254 -Dpackaging=pom
$ mvn install:install-file -Dfile="$(pwd)/flight-core-8.0.0.dev254.jar" -DgroupId=org.apache.arrow -DartifactId=flight-core -Dversion=8.0.0.dev254 -Dpackaging=jar
Validate that the packages were installed:
$ tree ~/.m2/repository/org/apache/arrow
.
βββ arrow-flight
β βββ 8.0.0.dev254
β β βββ arrow-flight-8.0.0.dev254.pom
βββ arrow-format
β βββ 8.0.0.dev254
β β βββ arrow-format-8.0.0.dev254.jar
β β βββ arrow-format-8.0.0.dev254.pom
βββ arrow-java-root
β βββ 8.0.0.dev254
β β βββ arrow-java-root-8.0.0.dev254.pom
βββ arrow-memory
β βββ 8.0.0.dev254
β β βββ arrow-memory-8.0.0.dev254.pom
βββ arrow-memory-core
β βββ 8.0.0.dev254
β β βββ arrow-memory-core-8.0.0.dev254.jar
β β βββ arrow-memory-core-8.0.0.dev254.pom
βββ arrow-memory-netty
β βββ 8.0.0.dev254
β β βββ arrow-memory-netty-8.0.0.dev254.jar
β β βββ arrow-memory-netty-8.0.0.dev254.pom
βββ arrow-vector
β βββ 8.0.0.dev254
β β βββ _remote.repositories
β β βββ arrow-vector-8.0.0.dev254.jar
β β βββ arrow-vector-8.0.0.dev254.pom
βββ flight-core
βββ 8.0.0.dev254
β βββ flight-core-8.0.0.dev254.jar
β βββ flight-core-8.0.0.dev254.pom
Compile your project like usual with
mvn clean install
.