scala

What’s the easiest way to use reify (get an AST of) an expression in Scala?

by

A lot of things previously defined in package scala.reflect.mirror have moved to scala.reflect.runtime.universe: scala> import scala.reflect.runtime.{universe => u} import scala.reflect.runtime.{universe=>u} scala> val expr = u reify { 1 to 3 map (_+1) } expr: reflect.runtime.universe.Expr[scala.collection.immutable.IndexedSeq[Int]] = Expr[scala.collection.immutable.IndexedSeq[Int]](scala.this.Predef.intWrapper(1).to(3).map(((x$1) => x$1.$plus(1)))(immutable.this.IndexedSeq.canBuildFrom)) scala> u show expr.tree res57: String = scala.this.Predef.intWrapper(1).to(3).map(((x$1) => x$1.$plus(1)))(immutable.this.IndexedSeq.canBuildFrom) scala> u showRaw expr.tree res58: String … Read more

by

The <:< type is defined in Predef.scala along with the related types =:= and <%< as follows: // used, for example, in the encoding of generalized constraints // we need a new type constructor `<:<` and evidence `conforms`, as // reusing `Function2` and `identity` leads to ambiguities (any2stringadd is inferred) // to constrain any abstract … Read more

What is the difference between `##` and `hashCode`?

by

“Subclasses” of AnyVal do not behave properly from a hashing perspective: scala> 1.0.hashCode res14: Int = 1072693248 Of course this is boxed to a call to: scala> new java.lang.Double(1.0).hashCode res16: Int = 1072693248 We might prefer it to be: scala> new java.lang.Double(1.0).## res17: Int = 1 scala> 1.0.## res15: Int = 1 We should expect … Read more

Create new Dataframe with empty/null field values

by

It is possible to use lit(null): import org.apache.spark.sql.functions.{lit, udf} case class Record(foo: Int, bar: String) val df = Seq(Record(1, “foo”), Record(2, “bar”)).toDF val dfWithFoobar = df.withColumn(“foobar”, lit(null: String)) One problem here is that the column type is null: scala> dfWithFoobar.printSchema root |– foo: integer (nullable = false) |– bar: string (nullable = true) |– foobar: … Read more

Scala : fold vs foldLeft

by

The method fold (originally added for parallel computation) is less powerful than foldLeft in terms of types it can be applied to. Its signature is: def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1 This means that the type over which the folding is done has to be a supertype of the collection element … Read more

How to create a Source that can receive elements later via a method call?

by

There are three ways this can be achieved: 1. Post Materialization with SourceQueue You can use Source.queue that materializes the Flow into a SourceQueue: case class Weather(zipCode : String, temperature : Double, raining : Boolean) val bufferSize = 100 //if the buffer fills up then this strategy drops the oldest elements //upon the arrival of … Read more