tag:blogger.com,1999:blog-4780450423910001617.post8393442894822558219..comments2022-02-19T05:26:16.579+01:00Comments on Marko's blog: Stream Processing With Spring, Kafka, Spark and Cassandra - Part 4Marko Švaljekhttp://www.blogger.com/profile/18275613894129227858noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-4780450423910001617.post-5755795837869553722017-06-01T22:07:47.874+02:002017-06-01T22:07:47.874+02:00I found your example useful even over 1 yr later. ...I found your example useful even over 1 yr later. Thank you.<br />I am using current version of Cassandra 3.10 which causes runtime exception. Without changing your code or updating org.apache.spark dependencies, I found that I had to update:<br /> compile('com.datastax.spark:spark-cassandra-connector_2.10:1.5.0-M3')<br /> compile('com.datastax.spark:spark-cassandra-connector-java_2.10:1.5.0-M3')<br />to:<br /> compile('com.datastax.spark:spark-cassandra-connector_2.10:1.5.0-RC1')<br /> compile('com.datastax.spark:spark-cassandra-connector-java_2.10:1.5.0-RC1')<br />Interesting that newer versions, such as 1.5.0+, would also cause runtime exception. I hope to update all dependencies to latest after understanding Spark better.MLLnoreply@blogger.comtag:blogger.com,1999:blog-4780450423910001617.post-87588844378849570532017-02-13T14:37:55.657+01:002017-02-13T14:37:55.657+01:00when i am implementin Spark projects i too faced s...when i am implementin <a href="http://www.bigdataanalyst.in" rel="nofollow">Spark projects</a> i too faced same problem. Add these dependencies in build.sbt file to solve above error.<br /><br />// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind<br />libraryDependencies += "com.fasterxml.jackson.core" % "jackson-databind" % "2.8.6"<br />// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core<br />libraryDependencies += "com.fasterxml.jackson.core" % "jackson-core" % "2.8.6"<br />// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations<br />libraryDependencies += "com.fasterxml.jackson.core" % "jackson-annotations" % "2.8.6"<br /><br />// https://mvnrepository.com/artifact/com.fasterxml.jackson.module/jackson-module-scala_2.10<br />libraryDependencies += "com.fasterxml.jackson.module" % "jackson-module-scala_2.10" % "2.8.6"sparktrainerhttp://www.apachespark.innoreply@blogger.comtag:blogger.com,1999:blog-4780450423910001617.post-36053275400807350592016-01-14T20:59:09.314+01:002016-01-14T20:59:09.314+01:00I didn't run into something similar. I googled...I didn't run into something similar. I googled around a bit ... looks like you might be having problems described here<br /><br />http://stackoverflow.com/questions/33815396/spark-com-fasterxml-jackson-module-error<br /><br />But then again, It works for me :(Marko Švaljekhttps://www.blogger.com/profile/18275613894129227858noreply@blogger.comtag:blogger.com,1999:blog-4780450423910001617.post-67181488819477110442016-01-14T16:37:51.336+01:002016-01-14T16:37:51.336+01:00Running into issue.
Exception in thread "ma...Running into issue. <br /><br />Exception in thread "main" java.lang.VerifyError: class com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final method withResolved.<br /><br />JavaPairInputDStream messages =<br /> KafkaUtils.createDirectStream(<br /> jssc,<br /> String.class,<br /> String.class,<br /> StringDecoder.class,<br /> StringDecoder.class,<br /> kafkaParams,<br /> topicsSet<br /> );<br /><br /><br />16/01/14 10:21:46 INFO VerifiableProperties: Property zookeeper.connect is overridden to <br />Exception in thread "main" java.lang.VerifyError: class com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final method withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;<br /> at java.lang.ClassLoader.defineClass1(Native Method)<br /> at java.lang.ClassLoader.defineClass(ClassLoader.java:760)<br /> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)<br /> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)<br /> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)<br /> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)<br /> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)<br /> at java.security.AccessController.doPrivileged(Native Method)<br /> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)<br /> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)<br /> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)<br /> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)<br /> at com.fasterxml.jackson.module.scala.ser.IteratorSerializerModule$class.$init$(IteratorSerializerModule.scala:70)<br /> at com.fasterxml.jackson.module.scala.DefaultScalaModule.(DefaultScalaModule.scala:19)<br /> at com.fasterxml.jackson.module.scala.DefaultScalaModule$.(DefaultScalaModule.scala:35)<br /> at com.fasterxml.jackson.module.scala.DefaultScalaModule$.(DefaultScalaModule.scala)<br /> at org.apache.spark.rdd.RDDOperationScope$.(RDDOperationScope.scala:78)<br /> at org.apache.spark.rdd.RDDOperationScope$.(RDDOperationScope.scala)<br /> at org.apache.spark.streaming.dstream.InputDStream.(InputDStream.scala:78)<br /> at org.apache.spark.streaming.kafka.DirectKafkaInputDStream.(DirectKafkaInputDStream.scala:56)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$8$$anonfun$apply$4.apply(KafkaUtils.scala:419)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$8$$anonfun$apply$4.apply(KafkaUtils.scala:410)<br /> at scala.util.Either$RightProjection.map(Either.scala:536)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$8.apply(KafkaUtils.scala:410)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$8.apply(KafkaUtils.scala:409)<br /> at scala.util.Either$RightProjection.flatMap(Either.scala:523)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:409)<br /> at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:532)<br /> at org.apache.spark.streaming.kafka.KafkaUtils.createDirectStream(KafkaUtils.scala)<br /> at com.ls.in2.kafka.SparkStreamingExample.main(SparkStreamingExample.java:55)JOHN PETERhttps://www.blogger.com/profile/04716575424339392929noreply@blogger.com