Development Guidelines#

Logger Abstraction#

Apache Arrow Java uses the SLF4J API, so please configure SLF4J to see logs (e.g. via Logback/Apache Log4j):

  1. If no jar dependencies are added by the user via Logback or Apache Log4j then SLF4J will default to no-operation (NOP) logging.

  2. If a user adds any dependencies via Logback or Apache Log4j but does not configure/add/define logback.xml/log4j2.xml, then logs will default to DEBUG mode.

  3. To disable debug logs, the user must define their own rules within their logback.xml/log4j2.xml and define their own loggers.

Unit Testing#

Unit tests are run by Maven during the build.

To speed up the build, you can skip them by passing -DskipTests. .. code-block:

$ cd arrow/java
$ mvn \
    -Darrow.cpp.build.dir=../java-dist/lib -Parrow-jni \
    -Darrow.c.jni.dist.dir=../java-dist/lib -Parrow-c-data \
    clean install

Performance Testing#

The arrow-performance module contains benchmarks.

Let’s configure our environment to run performance tests:

In case you need to see your performance tests on the UI, then, configure (optional):

Lets execute benchmark tests:

$ cd benchmarks
$ conbench java-micro --help
$ conbench java-micro
    --iterations=1
    --commit=e90472e35b40f58b17d408438bb8de1641bfe6ef
    --java-home=<absolute path to your java home>
    --src=<absolute path to your arrow project>
    --benchmark-filter=org.apache.arrow.adapter.AvroAdapterBenchmarks.testAvroToArrow
Benchmark                              Mode  Cnt       Score   Error  Units
AvroAdapterBenchmarks.testAvroToArrow  avgt       725545.783          ns/op
Time to POST http://localhost:5000/api/login/ 0.14911699295043945
Time to POST http://localhost:5000/api/benchmarks/ 0.06116318702697754

Then go to: http://127.0.0.1:5000/ to see reports:

UI Home:

../../_images/conbench_ui.png

UI Runs:

../../_images/conbench_runs.png

UI Benchmark:

../../_images/conbench_benchmark.png

Integration Testing#

Integration tests can be run via Archery. For example, assuming you only built Arrow Java and want to run the IPC integration tests, you would do:

$ archery integration --run-ipc --with-java 1

Code Style#

Code style is enforced with Checkstyle. The configuration is located at checkstyle. You can also just check the style without building the project. This checks the code style of all source code under the current directory or from within an individual module.

$ mvn checkstyle:check