> how do I get the output of the MapReduce… Back to the basics. Since you don’t care to mention which version of Hadoop and which version of Oozie you are using, I will assume a “recent” setup (e.g. Hadoop 2.7 w/ TimelineServer and Oozie 4.2). And since you don’t mention which kind of interface … Read more
According to the explanation by Doug Cutting, Avro’s JSON encoding requires that non-null union values be tagged with their intended type. This is because unions like [“bytes”,”string”] and [“int”,”long”] are ambiguous in JSON, the first are both encoded as JSON strings, while the second are both encoded as JSON numbers. http://avro.apache.org/docs/current/spec.html#json_encoding Thus your record must … Read more
It took me a couple months of reading source code and testing things out. In a nutshell, Spark can only handle String and Binary serialization. You must manually deserialize the data. In spark, create the confluent rest service object to get the schema. Convert the schema string in the response object into an Avro schema … Read more