Skip to main content

Zingg Community Icon

Zingg Community

Home
Events
Members

General
Help Zingg
Introduce Yourself
Zingg Databricks
Zingg Snowflake

⭐ Star Zingg
🐞 Submit an Issue
📚 Documentation

Powered by Tightknit

How to resolve no unmarked records error after running findTrainingDataset and label jobs

·Oct 24, 2024 04:01 PM

Hi! I’m trying to use the program on a dataset, but after running findTrainingDataset and label I got:

It seems there are no unmarked records at this moment. Please run findTrainingData job to build some pairs to be labelled and then run this labeler.

30 comments

· Sorted by Oldest

Sonal G.
·
Can you please share your environment details?
Nicolás N.
·
[GCP VM] processors=2 memory=4GB
I had to run the findTrainingDataset more than once. Then I went trough the label step. When I ran the training, i got another error. I think is due to RAM size, I have about 500.000 records in a table with 5 text columns.

·

24/10/28 12:34:08 INFO Heuristics: **Block size **100 and total count was 1692115
24/10/28 12:34:08 INFO Heuristics: Heuristics suggest 100
24/10/28 12:34:08 INFO BlockingTreeUtil: Learning indexing rules for block size 100
24/10/28 12:35:21 WARN Email: Unable to send email Can't send command to SMTP host
24/10/28 12:35:21 WARN Client: Apologies for this message. Zingg has encountered an error. Java heap space
java.lang.OutOfMemoryError: Java heap space

Nicolás N.
·
It happened here: zingg.sh --phase train --conf /entity-resolution/config.json
Sonal G.
·
How many records have you labeled?
Sonal G.
·
And is it possible to go to 8 cores, 16 gb RAM?
Nicolás N.
·
yes I went now for 64gb RAM 24 Core on my local server. I’m using now about 5.000.000 rows on a csv with just the names of the entities I’m grouping. I’ve labeled about 50 positive 150 negative.
Nicolás N.
·
the issue of running out of memory happened again when running the match phase. It ran for 2 hours and then failed.
Sonal G.
·
Hmm..Do you have a lot of nulls?
Sonal G.
·
Can you please share the match logs?
Nicolás N.
·
Not many null values.
Nicolás N.
·
I lost the logs
Nicolás N.
·
I’m running it on a docker container but it’s weird because it barely uses the resources of the server, and Still it runs out of memory
Nicolás N.
·
Sonal G.
·
Ah. Is the docker set to use the server resources? Also, are you passing the extra memory to Zingg through the --properties-file zingg.conf ?