Hi all, I'm running zingg locally by mounting /data, /configs, and /tmp/zingg volumes locally on my machine. I'm simply using the zingg/zingg:0.4.0 image as a processing container and persisting the data locally. I'm able to extract the training data, run the labeling phase, and build the model, but when I attempt to run the match phase I'm getting a memory error:
25/01/17 20:36:51 ERROR Executor: Exception in task 0.0 in stage 97.0 (TID 294)
java.lang.OutOfMemoryError: Java heap space
I'm running this using this bash command:
#!/bin/bash
docker run --rm -it \
-m 14g \
-v ~/zingg/data:/data \
-v !/zingg/configs:/configs \
-v !/zingg/tmp_zingg:/tmp/zingg \
zingg/zingg:0.4.0 bash
which allocates 14g memory to the container. I'm attempting to process ~550k records in a parquet format. In looking at the documentation this should be enough memory to process these records. Is there a reason I would be running into this memory issue?
