Scalera tip: Keep your actor’s state with no VAR at all!

It’s pretty well known that using VARs is, apart from unethical, the evil itself, some kind of hell, it make kitties die and many other stuff you might probably heard before and that could eventually be the cause of a painfull slowly dead.

The essence of functional programming is therefore the immutability: every time I mutate an element, I actually generate a new one.

What about Akka actors?

When we talk about actors, we can define them as stateful computation units that sequentially process a message queue by reacting(or not) to each of these messages.

It’s always been said that, in order to keep state within some actor’s logic, it was ok to use VARs:

It’s impossible that concurrency problems happen: it’s the actor itself and nobody else who access that var and will only process one message at a time.

But maybe, we could renounce to this premise if we look for some way to redefine the actor’s behavior based on a new state.

Mortal approach

If we follow the previously described philosophy, the very first (and more straight forward) approach for keeping some actor’s state would be pretty similar to the following:

class Foo extends Actor{
  var state: Int = 0
  override def receive = {
    case Increase => state += 1
  }
}

Every time an Increase arrives, we modify the state value by adding 1.
So easy so far, right?

Immutable approach

Nevertheless, we could define a receive function parameterized by certain state, so when a message arrives, this parameter is the state to take into account.

If the circumstances to mutate the state took place, we would just invoke the become method that would modify the actor’s behavior. In our case, that behavior mutation would consist on changing the state value.

If we use the previously defined example:

class Foo extends Actor{
  def receive(state: Int): Receive = {
    case Increase =>
      context.become(
        receive(state + 1),
        discardOld = true)
  }
  override def receive = receive(0)
}

we can notice that the function defined by receive is parameterized by some state argument. When some Increase message arrives, what we perform is an invocation to become for modifying the actor’s behavior, passing as an argument the new state to handle.

If we wanted some extra legibility, we could even get abstract from every updatabe-state actor:

trait SActor[State] extends Actor {
  val initialState: State
  def receive(state: State): Receive
  def update(state: State): Receive =
    context.become(
      receive(state),
      discardold = true)
  override def receive =
    receive(initialState)
}

this way, we just have to define the initial state of some actor, a new parameterized receive function and a new update function that takes care of performing the proper become call as explained before.
With all these in mind, we now have some cuter brand new Foo actor:

class Foo extends SActor[Int] {
  val initialState = 0
  def receive(state: Int): Receive = {
    case Increase => update(state +1)
  }
}

Potential hazardous issues

Please, do notice that in the featuring example, we’ve used a second argument for become: discardOld = true. This argument settles whether the new behavior should be stashed on the top of the older one, or ‘au contraire’ it should completely substitute the previous behavior.

Let’s suppose we used discardOld = false. If every single time a new Increase message arrived we had to stash a new behavior, we could obtain a wonderful overflow issue.

See you in the next tip.

Peace out 🙂

More lazy values, the State monad and other stateful stuff

In the previous post, we talked about lazy evaluation in Scala. At the end of that post, we asked an interesting question: Does a Lazy value hold an state?

24195622

In order to answer that question, we’ll try to define a type that could represent the Lazy values:

trait Lazy[T] {

  val evalF : () => T

  val value: Option[T] = None

}
object Lazy{
  def apply[T](f: => T): Lazy[T] =
    new Lazy[T]{ val evalF = () => f }
}

As you can see, our Lazy type is parameterized by some T type that represents the actual value type(Lazy[Int] would be the representation for a lazy integer).
Besides that, we can see that it’s composed of the two main Lazy type features:

  • evalF : Zero-parameter function that, when its ‘apply’ method is invoked, it evaluates the contained T expression.
  • value : The result value of the interpretation of the evalF function. This concrete part denotes the state in the Lazy type, and it only admit two possible values: None (not evaluated) or Some(t) (if it has been already evaluated and the result itself).

We’ve also added a companion object that defines the Lazy instance constructor that receives a by-name parameter that is returned as result of the evalF function.

e9a2295b3db9b45c8f5484a09033c1c71cf88e3375bb7ff60456bc81c29a4e04

Now the question is, how do we join both the evaluation function and the value that it returns so we can make Lazy an stateful type? We define the ‘eval’ function this way:

trait Lazy[T] { lzy =>

  val evalF : () => T

  val value: Option[T] = None

  def eval: (T, Lazy[T]) = {
    val evaluated = evalF.apply()
    evaluated -> new Lazy[T]{ mutated =>
      val evalF = lzy.evalF
      override val value = Some(evaluated)
      override def eval: (T, Lazy[T]) = 
        evaluated -> mutated
    }
  } 

}

The ‘eval’ function returns a two-element tuple:

  • The value result of evaluating the expression that stands for the lazy value.
  • a new Lazy value version that contains the new state: the T evaluation result.

If you take a closer look, what ‘eval’ method does in first place is to invoke the evalF function so it can retrieved the T value that remained until that point not-evaluated.
Once done, we return it as well as the new Lazy value version. This new version (let’s call it mutated version) will have in its ‘value’ attribute the result of having invoked the evalF function. In the same way, we change its eval method, so in future invocations the Lazy instance itself is returned instead of creating new instances (because it actually won’t change its state, like Scala’s lazy definitions work).

The interesting question that comes next is: is this an isolated case? Could anything else be defined as stateful? Let’s perform an abstraction exercise.

Looking for generics: stateful stuff

Let’s think about a simple stack:

sealed trait Stack[+T]
case object Empty extends Stack[Nothing]
case class NonEmpty[T](head: T, tail: Stack[T]) extends Stack

The implementation is really simple. But let’s focus in the Stack trait and in a hypothetical pop method that pops an element from the stack so it is returned as well as the rest of the stack:

sealed trait Stack[+T]{
  def pop(): (Option[T], Stack[T])
}

Does it sound familiar to you? It is mysteriously similar to

trait Lazy[T]{
  def eval: (T, Lazy[T])
}

isn’t it?

If we try to re-factor for getting a common trait between Lazy and Stack, we could define a much more abstract type called State:

trait State[S,T] {
  def apply(s: S): (T, S)
}

Simple but pretty: the State trait is parameterized by two types: S (state type) and T (info or additional element that is returned in the specified state mutation). Though it’s simple, it’s also a ver common pattern when designing Scala systems. There’s always something that holds certain state. And everything that has an state, it mutates. And if something mutates in a fancy and smart way…oh man.

That already exists…

24314442

All this story that seems to be created from a post-modern essay, has already been subject of study for people…that study stuff. Without going into greater detail, in ScalaZ library you can find the State monad that, apart from what was previously pointed, is fully-equipped with composability and everything that being a monad means (semigroup, monoid, …).

If we define our Lazy type with the State monad, we’ll get something similar to:

import scalaz.State

type Lazy[T] = (() => T, Option[T])

def Lazy[T](f: => T) = (() => f, None)

def eval[T] = State[Lazy[T], T]{
  case ((f, None)) => {
    val evaluated = f.apply()
    ((f, Some(evaluated)), evaluated)
  }
  case s@((_, Some(evaluated))) => (s, evaluated) 
}

When decrypting the egyptian hieroglyph, given the State[S,T] monad, we have that our S state will be a tuple composed of what exactly represents a lazy expression (that we also previously described):

type Lazy[T] = (() => T, Option[T])
  • A Function0 that represents the lazy evaluation of T
  • The T value that might have been evaluated or not

For building a Lazy value, we generate a tuple with a function that stands for the expression pointed with the by-name parameter of the Lazy method; and the None value (because the Lazy guy hasn’t been evaluated yet):

def Lazy[T](f: => T) = (() => f, None)

Last, but not least (it’s actually the most important part), we define the only state transition that is possible in this type: the evaluation. This is the key when designing any State type builder: how to model what out S type stands for and the possible state transitions that we might consider.

In the case of the Lazy type, we have two possible situations: the expression hasn’t been evaluated yet (in that case, we’ll evaluate it and we’ll return the same function and the result) or the expression has been already evaluated (in that case we won’t change the state at all and we’ll return the evaluation result):

def eval[T] = State[Lazy[T], T]{
  case ((f, None)) => {
    val evaluated = f.apply()
    ((f, Some(evaluated)), evaluated)
  }
  case s@((_, Some(evaluated))) => (s, evaluated) 
}

iZcUNxH

In order to check that we can still count on the initial features we described for the Lazy type (it can only be evaluated once, only when necessary, …) we check the following assertions:

var sideEffectDetector: Int = 0

val two = Lazy {
  sideEffectDetector += 1
  2
}

require(sideEffectDetector==0)

val (_, (evaluated, evaluated2)) = (for {
  evaluated <- eval[Int]
  evaluated2 <- eval[Int]
} yield (evaluated, evaluated2)).apply(two)

require(sideEffectDetector == 1)
require(evaluated == 2)
require(evaluated2 == 2)

Please, do notice that, as we mentioned before, what is defined inside the for-comprehension are the same transitions or steps that the state we decide will face. That means that we define the mutations that any S state will suffer. Once the recipe is defined, we apply it to the initial state we want.
In this particular case, we define as initial state a lazy integer that will hold the 2 value. For checking the amount of times that our Lazy guy is evaluated, we just add a very dummy var that will be used as a counter. After that, we define inside our recipe that the state must mutate twice by ussing the eval operation. Afterwards we’ll check that the expression of the Lazy block has only been evaluated once and that the returning value is the expected one.

I wish you the best tea for digesting all this crazy story 🙂
Please, feel free to add comments/menaces at the end of this post or even at our gitter channel.

See you on next post.
Peace out!

Lazy values

Just in case you lived in a hole for the last ten years and you didn’t know: Scala allows managing lazy values.

image

In Scala, we can define a value that won’t be evaluated until it is explicitly invoked. For example:

lazy val myLazyInt: Int = { println("hi"); 2 }

As you can see, using lazy notation, we’ve defined lazily an integer that stands for the literal 2 and also prints a ‘hi’ when it’s evaluated.
Apart from violating the biggest functional programming law (referential transparency) due to the insidious println, side effects, dead, destruction, blah blah …

anigif_enhanced-1822-1407333641-6

notice that, if we execute the code block, the previously mentioned ‘println’ is not executed. The block is not evaluated until any other expression makes use of our lazy integer value:

val result = myLazyInt + 3
//woa! somebody printed 'hi' and I have a brand new 5 inside 'result'

Once myLazyInt is evaluated, its value won’t be calculated again, no matter how many times it’s invoked. Therefore, the mysterious impression won’t salute us anymore:

lazy val myLazyInt: Int = { println("hi"); 2 }
myLazyInt
//"hi"
myLazyInt //nothing special happened now ...
myLazyInt //no matter how many times you invoke it...
myLazyInt //seriously, let it go...

Curious. The question that could come up is, if I define a lazy value and I pass it as a method parameter, what happens? Is it evaluated at the very same moment that the method is invoked? Maybe inside the method? That’ll depend on the way you define your method’s parameters.

Call by name vs. call by value

When defining a method, people usually define its parameter ‘by-value’, that means, that we expect the parameter to be already evaluated when it is passed to the method:

def myMethod(someInteger: Int): Int = {
  println("begin")
  val result = someInteger + 2
  println("end")
  result
}

If we invoke our method with any integer:

val n = 3
val result = myMethod(n)
//"begin"
//"end"
require(result == 5)

We just print both traces and it’s not big deal. Nothing new so far.
What happens if we now pass to the method our lazy value? In which exact moment will it print the salutation? Before or after the method traces?
Let’s try:

myMethod(myLazyInt)
//"hi"
//"begin"
//"end"

It printed it out before the method traces, which means that our lazy value was evaluated just before the method was invoked. Why does this happen? Because the way that Scala usually works needs the exact value of someInteger in order to be able to execute myMethod
It’s a pity if we want to keep myLazyInt lazy until the very last moment. How do we fix that? We’ll pass the argument ‘by-name’, that is, indicating the way the value has to be resolved instead of explicitly passing the value:

def myMethod(someInteger: => Int): Int = {
  println("begin")
  val result = someInteger + 2
  println("end")
  result
}

This way (someInteger: => Int) we indicate that our method requires as parameter an expression that, in the end, returns an integer and not an integer itself. If we now execute the method passing our non-yet evaluated lazy value:

myMethod(myLazyInt)
//"begin"
//"hi"
//"end"

Voilà! We made it. The ‘hi’ trace is not printed until the exact value of our lazy guy is required inside the method.

Some other ways to express laziness

Another way to express a lazy evaluation, which could be extremely useful, is the Function0 type:

trait Function0[+R]{
  def apply(): R
}

It’s just a function that requires zero parameters and return an only output type. It’s expressed as follows:

val f: () => Int =
  () => 2
f.apply() //2

And that’s pretty much everything…Once understood in rough outlines how laziness works in Scala, let’s move on to more interesting questions. A Lazy value, does it represent something stateful?
The answer (or more extra questions) will be available in the following post.

Peace out!

Algrebraic Data Types in Scala

What a delightful idea to come back from vacation with batteries fully charged and with some wacky ideas around our minds to write about. Best of these came from the very influence of joints the moon.

ADT?

An Algebraic Data Type (TDA from now so we can save money for each word in WordPress) is just a way to express a data type (Cat, Dog, Prevarication) based on an algebra. And when we say ‘algebra’, we mean type sums and products (of Integers, Cats, Cars, Prevarications, …). For example:

Train = Locomotive + Wagon * Train

How do one read that? A train may be: a locomotive OR a wagon AND another train (that may be as well a wagon and another train, that may be as well a …).
Take a look at both disjunction and conjunction: the sum represents an OR, and the product represents an AND (like Boole algebra).

It’s also worthy to notice that, from this type definition you can infer a recursive pattern. With the Train type, the base case is definitively the Locomotive and, at the recursive case, we have a wagon and another train. As we’ll see later, this pattern is very frequent and makes easier the type definition.

And how are sum and product represented in Scala?

The easier way to represent the type sum (also called co-product), in a paradigm with polimorphism support (in general) and in Scala (in particular), is just the inheritance feature. If we have the following case:

sealed trait Animal
case object Cat extends Animal
case object Dog extends Animal

we’re indeed expressing a type co-product:

Animal = Cat + Dog

that is, an Animal can only be, a Cat, or a Dog.

Regarding the product, we could define it as the attribute set that compounds a certain type instance. For example,

case class Student(name: String, age: Int)

expressed as a product sum, would be as follows:

Student = String * Int

So, for building a Student instance, you need a String and an Int.

If we try now to materialize the previously exposed train model (with some additives) we’ll notice that

Wagon = String * Int
Train = Locomotive + Wagon * Train

is translated into Scala as

sealed trait Train
case object Locomotive extends Train
case class Wagon(model: String, passengers: Int)
case class Nexus(wagon: Wagon, next: Train)

So what is it good for?

hqdefault

…absolutely nothing, listen to me♩♪♫♬.
If you think, my fellow, that this is stuff that nobody uses, you haven’t thought about which scala.Prefef structures are defined this way. Lists, for example, as defined as:

trait List[+T]
case object Nil extends List[Nothing]
case class ::[T](head: T, tail: List[T]) extends List[T]

That is, a List can be, an empty one, or an element followed by another list.
If we express that idea in terms of products and co-products:

List[T] = EmptyList[T] + NonEmptyList[T]
NonEmptyList[T] = T * List[T]

Please, notice that, the case of the empt list (Nil) has a bizarre but beautiful implementation in Scala.

If we try to define an empty list for eeeeeeeeeevery single existing type, we would have to instantiate a Nil[Cat], a Nil[Dog], …
In order to avoid this, and having an only Nil, we make it extend from List[Nothing] that, as you’ll probably remember from other posts, Nothing extends from eeeeeeeeevery single existing type (both primitive and programmer defined). If we add the fact of List[T] being covariant at T, we’ll have an only object Nil that represents the empty lists for all types. Awesome, right?

odtUdEE

Example: Even numbers

In order to harden to this new way of thinking, let’s suppose the following challenge: how could we represent even numbers in Scala?

Requirements

If we’re not sophisticated enough and we trust a lil’ bit in runtime assertions, we could say that even numbers are defined as:

case class Even(value: Int) { 
  require(value%2==0, "it's not even")
}

But, if we try to create an Even with an odd integer number we’ll get a giant NOPE:

Even(1)
java.lang.IllegalArgumentException: requirement failed: it's not even
	at scala.Predef$.require(Predef.scala:233)
	at Even.<init>(<console>:7)

However this assertion won’t be verified until run-time, the moment when require is executed. Thus, our code could be compiled without being correct…
We can do it much better…

Next(Next(…))

Another option is to assume (and we won’t discuss about it) that zero is an even number, that we have infinite memory installed in our machine, that the overflow error doesn’t exist…

907958

In that, not so far, case (…), we could define even numbers as:

sealed abstract class Even(val value: Int)
case object Zero extends Even(0)
case class Next(previousEven: Even) 
  extends Even(2 + previousEven.value)

So, if we have a method that generate reservations for the Boat Love, that requires an even number of participants, we can use our brand new defined Even type:

def loveBoatReservation(
  peopleAmount: Even): Reservation = ???

Given there’s no way to build an Even from an integer that is not even, we avoid uncomfortable situations at runtime, where the amount of people that get on the Love Boat are odd. Otherwise, someone could be…

forever-alone-400x400

Recursive ADTs and its techniques

Once the data type is defined, let’s suppose we want to implement the sum of even numbers:

def sum(e1: Even, e2: Even): Even = ???

We handle several alternatives. One of them could be the quick-and-dirty one:

def sum(e1: Even, e2: Even): Event = 
  new Even(e1.value + e2.value){}

But take a closer look at the fact that we’re totally ignoring the constructors we’ve defined. If we want to use pattern matching over the result:

val four = new Even(4){}
sum(Zero, four) match {
  case Zero => 
    //it never gets inside this case!
  case Next(Next(Zero)) => 
    //OMG! IT DOESN'T FIT HERE EITHER!
}
scala.MatchError: $anon$1@649f2009 (of class $anon$1)

51067781

The other technique (much more sophisticated by the way) consists on invoking a recursive method that, for each call, it decreases the second even number, while it increases the first one. For doing so, we make use of Next apply(constructor) and unapply(extractor) methods:

def sum(e1: Even, e2: Even): Even = {
  @tailrec
  def rSum(ev1: Even, ev2: Even): (Even, Even) = {
    ev2 match {
      case Zero => (ev1, Zero)
      case Next(e) => rSum(Next(ev1), e)
    }
  }
  val (result, _) = rSum(e1, e2)
  result
}

Undeniably beautiful 🙂

Conclusions

Apart from becoming a lil’ bit crazier when reading back-from-vacation posts, we can extract several main conclusions from what we’ve read:

  • As we always say, every possible assertion that we can check at compile time instead of runtime, it’s saving time and headaches hunting bugs of software in production (which is more expensive and more keen to make heads roll).
  • Constructors are THE key. If we define an ADT, we can control that generated values of that type are correct by defining the proper constructors: Zero and Next. In both cases, we are certainly sure that even number rules are satisfied.
  • Methods that operate over recursive data types use to be recursive as well. And, apart from that, for generating values of the mentioned type (Even in our example) they should only use the existing constructor methods.

In a future post, we’ll talk about the relation between data types algebra and definition of formal grammars…or not.

Peace out!

Scala, Scala and nothing more

After so many posts on Scala, many snippets of code, and many weird concepts, there comes a post that had been pending since almost the beginning of this blog. What resources exist to learn Scala? Well … now we show our favorites.

yoda_meme

Books: a good starting point 🙂

  • Programming in Scala: a classic. Written by the very Odersky and with the third edition hot off the press.
  • Funcional Programming in Scala (also known, in an effort of originality and simplicity, as the red book). A book that opens the mind to the functional paradigm and it is advisable not only for people who want to program in Scala, but for people who are interested in functional programming.

Courses: to deepen a bit more.

  • On the online courses, Coursera and the new specialization of functional programming in Scala they have all the monopoly. It consists of 4 courses + 1 project and discusses in depth the most important aspects of Scala.
  • Furthermore, scala-exercises allows us to practice in a faster way, but no less effective. Fully recommended.

Events: not only to drink beer  😉

  • Scala Days : Undoubtedly the most important annual convention about Scala. Since a year, there are two conventions at year: one in America and one in Europe. In addition, all the videos of the talks are published 🙂
  • Scala World: UK convention with a warm welcome and where you can hear and see some of the biggest Scala gurus.
  • Scala eXchange: another of the best known conventions. It takes place on December in London.

These are the resources that we know and that we found interesting. Without doubt, there are many more that we left unsaid. If you missing some resource on this list, any contribution is welcome 🙂

Scala: Code interpretation at runtime

With today’s post, we’ll dive into Scala code generation on the fly: at runtime. We have to be cautious of not mixing concepts with Scala macros, which generate code at compile-time. These make use of Scala type system, which is much safer than generating code at runtime.

House-of-cards-but-why

When is it useful to make use of this mechanism then? We’ll try to shed light on this by following a very simple example, getting abstract of real implementation (that you may find at Scalera’s Github).

The problem: “da” serializer

Let’s suppose a not-so-wild case, which consists on communicating two services via an event bus (we’ll get abstract of its implementation: it could be a message queue like Kafka, Akka streams, …).

The main idea is the following:

Sender-receiver-schema

The producer knows how to send and the consumer has an associated callback for message arrivals:

trait Producer{
  def produce(message: Any): Try[Unit]
}
trait Consumer{
  val consume: Any => Unit
}

Both producer and consumer services know the message types that may arrive. In our example, they could be one of these:

case class Foo(att1: Int, att2: String)
case class Bar(att1: String)

If the producer wants to send a message using the event bus, it will have to serialize it somehow (JSON, Byte array, XML, …) so, by the time it reaches the opposite end, the consumer will start the inverse process (deserialization) and will get the original message.

…nothing weird so far.

Whydoeseverythinghavetobesocomplicated

If we have a JSON serializer …

trait JsonSer[T] {
  def serialize(t: T): String
  def deserialize(json: String): T
}

and we serialize our message …

implicit val fooSerializer: JsonSer[Foo] = ???
val foo: Foo = ???
producer.send(implicitly[JsonSer[Foo]].serialize(foo))

How do we know which deserializer to use when the consumer gets the message?

Option 1: Try every possible serializer until one of them works

In our consumer, we’d have:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = {
    case message: String =>
      Seq(barSerializer, fooSerializer).flatMap { ser =>
        Try(ser.deserialize(message)).toOption
      }.headOption.fold(ifEmpty = println("Couldn't deserialize")) {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

A lil’ bit coarse, right? If the proper serializer is the last of a 100 list, we would have tried and failed with 99 serializers before (Such a waste of CPU!).

4d8

Besides, we could also consider the case of having a different type deserializer, but it fits with the received message, so it would partially or wrongly deserialize the message.

Option 2: Add the message type to the message itself

We could add an extra layer wrapping the message for indicating the message type that it contains. This way, when receiving it, we could determine the serializer type we have to use.

MessageWrapper

For writing the type, we’ll make use of Scala’s TypeTags, getting info about the T contained message type.


//Wrapper for the message (and its serializer)

import scala.reflect.runtime.universe.{TypeTag, typeTag}

case class Message[T: TypeTag](content: T){
  val messageType: Message.Type = typeTag[T].tpe.toString
}
object Message {

  type Type = String

  def typeFrom(msg: String): Message.Type = ???

  implicit def messageSer[T:TypeTag:JsonSer]: JsonSer[Message[T]] = ???

}

//We'll make use of it for sending

producer.produce(messageSer[Foo].serialize(Message(foo)))

//And we redefine the consumer

lazy val consumer = new Consumer {
    override val consume: Any => Unit = {
      case message: String =>
        Message.typeFrom(message) match {

          case "org.scalera.reflect.runtime.Bar" =>
            println("it's a bar!")
            val value = messageSer[Bar].deserialize(message).content
            println(value.att1)

          case "org.scalera.reflect.runtime.Foo" =>
            val value = messageSer[Foo].deserialize(message).content
            println("it's a foo!")
            println(value.att2)

          case _ =>
            println("it's ... something!")
        }
    }
  }

As you can see, we don’t have to try every possible serializer anymore. Instead of that, from the message type we’ve extracted from the Message wrapper we have added, we’re able to use the proper deserializer.

But it is also true that we have to add an extra case for each string that represents the message type. Wouldn’t it be nice to deserialize somehow and to have the defined case only for the already deserialized objects? (Something similar what we first tried but without trying all posible serializers). Something like:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { msg =>
    genericDeserialize(msg) match {
      case f: Foo =>
      case b: Bar =>
      case _ =>
    }
  }
}

Option 2-cool: Serializers ‘under the hood’

For achieving something similar, we have to get focused on that genericDeserialize method: Which signature should it have? Initially, something like this:

def genericDeserialize(msg: String): Any

An Any? Seriously? My fellows, at runtime, we have no idea about the type we can get. We just know that, from a String, we’ll get ‘some….thing’. The match that applies to that Any will allow us to go from something totally abstract to more concrete types.

At this point is where both reflect library and Scala compiler appear.

reflect.Toolbox

The Toolbox API allows parsing strings and getting the resulting AST (abstract syntax tree). From that AST, it is able to evaluate the expression and returns an instance of an Any as well.

For instantiating a Toolbox and use type references, we hace to add as SBT dependencies the following:

libraryDependencies ++= Seq(
  "org.scala-lang" % "scala-compiler" % "2.11.8",
  "org.scala-lang" % "scala-reflect" % "2.11.8")

For example, if we wanted to parse the "2".toInt + 4 operation,

import scala.tools.reflect.ToolBox
import scala.reflect.runtime.{universe => ru}
import ru._

//  Scala compiler tool box
val tb = ru.runtimeMirror(
  this.getClass.getClassLoader).mkToolBox()

println(ru.showRaw(tb.parse("2".toInt + 4")))

we would get the abstract syntax tree generated as a String (by using showRaw):

Apply(Select(Select(Literal(Constant("2")), TermName("toInt")), TermName("$plus")), List(Literal(Constant(4))))

If we use the toolbox for evaluating the parsed expression,

println(tb.eval(tb.parse("2".toInt + 4")))

we’ll get an Any that represents the resulting value of the sum:

6

“Da” serializer

Once we’ve seen how it generally works, we apply the same principle to our serializer, so the expression we’re going to try to interpret is similar to:

{
  import scala.reflect._;
  import spray.json._;
  import org.scalera.reflect.runtime._;
  import MySprayJsonImplicits._;
  import MyJsonSerImplicits._;

  implicitly[JsonSer[Message[$messageType]]]
}

where MySprayJsonImplicits and MyJsonSerImplicits represent the objects that contain both the Spray implicits for JsonFormat and the JsonSer implicits that we have defined before.

$messageType represents the concrete type to deserialize that we would have got by using the TypeTag (as seen before).

If we adapt it to our code, we’ll get something similar to:

object GenSer {

  import scala.tools.reflect.ToolBox
  import scala.reflect.runtime.{universe => ru}
  import ru._

  //  Scala compiler tool box
  private val tb = ru.runtimeMirror(this.getClass.getClassLoader).mkToolBox()

  def genericDeserialize(msg: String)(serContainers: Seq[AnyRef]): Any = {

    val messageType = Message.typeFrom(msg)

    val serContainersImport = serContainers.map(container =>
      "import " + container.toString.split("\\$").head + "._").mkString(";\n")

    val expr =
      s"
         |{
         |  import scala.reflect._;
         |  import spray.json._;
         |  import org.scalera.reflect.runtime._;
         |  $serContainersImport;
         |
         |  implicitly[JsonSer[Message[$messageType]]]
         |}
        ".stripMargin

    tb.eval(tb.parse(expr))
      .asInstanceOf[JsonSer[Message[Any]]]
      .deserialize(msg).content
  }

}

If you take a look, we’ve empowered the generic deserialization method notation to hold a sequence of objects to import, so we won’t make explicit which object contains the Spray implicits and which one contains our JsonSer‘s.

val serContainersImport = serContainers.map(container =>
  "import " + container.toString.split("\\$").head + "._").mkString(";\n")

It is also noteworthy that, when deserializing, we get a Message[Any]; so we’ll have to get the ‘content’ field that represents the raw value of Any type that holds the deserialized message.

The result

So finally, we can now make use of our function for generically deserialize and let our consumer code be ‘swaggy’:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { 
    case msg: String =>
      genericDeserialize(msg)(Seq(case3,Message)) match {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

Conclusions

Being able to evaluate code at runtime let us do a lot of interesting things when we want to interpret String based types. However, these kind of techniques won’t care about the so worthy and safe Scala type system, so these are tools to be used with a lil’ bit of care.

It is also kind of expensive in time terms to evaluate these expressions. I would recommend to use a type cache in this case. Something really simple like a map:

type TypeName = String
var cache: Map[TypeName, JsonSer[Message[_]]]

And when invoking the method, we’ll check if the type evidence (JsonSer) we’re looking for is already stored in the map. If so, we’ll use it, otherwise, we’ll create and store it in the cache, using it as result.

post-28553-Steve-Jobs-mind-blown-gif-HD-T-pVbd

Easy peasy…
Peace out!