Skip to main content

Zingg Community Icon

Zingg Community

Home
Events
Members

General
Help Zingg
Introduce Yourself
Zingg Databricks
Zingg Snowflake

⭐ Star Zingg
🐞 Submit an Issue
📚 Documentation

Powered by Tightknit

What Scala version is included with zingg 0.4.0 Spark 3.5.0 release? | Zingg Community

What Scala version is included with zingg 0.4.0 Spark 3.5.0 release?

·May 13, 2024 05:36 AM

https://github.com/zinggAI/zingg/releases/download/v0.4.0/zingg-0.4.0-spark-3.5.0.tar.gz what version scala is installed w/ this?

84 comments

· Sorted by Oldest

Vikas G.
·
it is based on spark 3.5
Vikas G.
·
spark 3.5 requirements can be found at https://spark.apache.org/docs/latest/

·

Thanks Vikas G. Any idea for why:

contacts-1  | java.lang.NullPointerException: Cannot invoke "zingg.common.client.ZFrame.columns()" because "lines" is null

·

code is:


spark: SparkSession = SparkSession.builder.appName("myApp").getOrCreate()

df = (
    spark.read.format("mongodb")
    .option("database", "v1")
    .option("collection", "contacts")
....

filtered_df = df.filter(df["place_id"] == "65a603fac769fa16f6596b2e")
print(filtered_df)
inputPipe = InMemoryPipe(name="mongo-memory", df=filtered_df)
args.setData(inputPipe)

Brandon M.
·
How are you suppose to run findAndLabel pipe inside of python?
Vikas G.
·
https://github.com/zinggAI/zingg-vikas/blob/main/examples/febrl/FebrlExample.py
Vikas G.
·
just substitute phase options = ClientOptions([ClientOptions.PHASE,"match"])
Vikas G.
·
so instead of match you can e.g. give findAndLabel
Brandon M.
·
Vikas G. yes but how do you select 0,1,2,3 for match/nonmatch?

·

also, getting this error: any thoughts?

contacts-1  | py4j.protocol.Py4JJavaError: An error occurred while calling o81.execute.
contacts-1  | : java.lang.NoClassDefFoundError: scala/Serializable
contacts-1  | 	at zingg.scala.DFUtil$.addRowNumber(DFUtil.scala:21)
contacts-1  | 	at zingg.scala.DFUtil.addRowNumber(DFUtil.scala)
contacts-1  | 	at zingg.spark.core.util.SparkDSUtil.addRowNumber(SparkDSUtil.

Brandon M.
·
Vikas G. if you could answer two questions above, that'd be amazing. I've been struggling with this for the last 2 weeks 🙂
Vikas G.
·
When you the phase find and label it will ask for user input
Vikas G.
·
On error it must be due to some version mismatch on scala side. Alternatively you can try the docker image
Brandon M.
·
Vikas G. I unfortunately need scala version 2.13 because thats what the mongodb connector works for.
Brandon M.
·
Does Zingg not work w/ 2.13?