Ryan J.

Commented on Help needed with starting customer entity resoluti...·Posted inZingg Databricks

The issue did turn out to be spark compatibility. I was originally attempting to utilize the old notebooks, so tried to keep the versions consistent. I was using 0.3.4 earlier. Moving to 0.4.0 did resolve the class execution issues. I’m using the 15.4 DBR on a private compute cluster.

Commented on Help needed with starting customer entity resoluti...·Posted inZingg Databricks

Ryan J.

I’m working with my Databricks SA on resolving Java class issues. Hosting the zingg jar in a volume, and have “installed” the library via the UI. Unfortunately the JVM can’t find the proper class. (Based on conversation this sounds like a common java issue in DBX) Troubleshooting so far:

The jar file is not currently found on the worker node local storage.
The java class path does not include the path where we’ve “installed” the jar.

Commented on Help needed with starting customer entity resoluti...·Posted inZingg Databricks

Ryan J.

Just popping back in momentarily to mention Luke’s notebook appears fairly detailed and I have high confidence that will answer some of my most basic questions. I’ll be spending some time with it today to validate and hopefully come back with some better questions (or a success story).

Commented on Help needed with starting customer entity resoluti...·Posted inZingg Databricks

Ryan J.

Thank you both. Luke, I’ll take a look at that notebook this morning. Sonal, It’s likely my unfamiliarity with zingg (and likely a little spark as an overall framework), however I have looked at that notebook and still have questions. It feels what I’m missing is basic documentation around zingg’s functional concepts. Examples of questions I currently have:

Are pipes required to get data in and out of zingg? If so, it seems like interactions within databricks may be best using the InMemoryPipe utilizing spark data frames?
How do I ensure the zing python bindings are able to utilize the java core?
Is there a simple method to test the binding?

Posted in Zingg Databricks·

Ryan J.

Help needed with starting customer entity resolution PoC using Zingg on Databricks

Hello, I'm fairly new to databricks (we've been using it organizationally for 2 years now) and glaringly new to zingg. I'm attempting to perform a PoC for customer entity resolution on databricks and ran across the provided notebooks. Unfortunately I'm struggling with initial implementation. I've noticed some steps in setup notebooks no longer work (downloading example census data from a repo comes to mind as one example). Not knowing where my deficiencies lie, and those of the notebook has left me starting to forge my own path using data that already exists within databricks, however I've not yet found a good place to start my work. Would someone be able to provide either of the following?

A link to documents which may allow me to work through the example notebooks or
A very basic example "quick start" that would allow me to understand the basics of how zingg integrates with the ecosystem?

8Comments

Commented on Ryan J. Has Joined the Conversation!·Posted inIntroduce Yourself

Ryan J.

That is the warmest of bot greetings. 😄 Thank you!