Error initializing SparkContext due to spark.kryoserializer.buffer.max set above 2048 MiB limit

2024-03-26 07:51:36,464 [main] ERROR org.apache.spark.SparkContext - Error initializing SparkContext. java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.spark.SparkEnv$.instantiateClass$1(SparkEnv.scala:285) at org.apache.spark.SparkEnv$.instantiateClassFromConf$1(SparkEnv.scala:296) at org.apache.spark.SparkEnv$.create(SparkEnv.scala:299) at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:189) at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:277) at org.apache.spark.SparkContext.<init>(SparkContext.scala:458) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2672) at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:945) at scala.Option.getOrElse(Option.scala:189) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:939) at zingg.zinggBase.init(zinggBase.java:76) at zingg.client.Client.init(Client.java:223) at zingg.client.Client.main(Client.java:181) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.IllegalArgumentException: spark.kryoserializer.buffer.max must be less than 2048 MiB, got: 8192 MiB. at org.apache.spark.serializer.KryoSerializer.<init>(KryoSerializer.scala:74) ... 29 more 2024-03-26 07:51:36,527 [main] INFO zingg.client.util.Email - Email message sent. 2024-03-26 07:51:36,527 [main] WARN zingg.client.Client - Apologies for this message. zingg has encountered an error. null