Scalera tip: Handling sticky implicit contexts

A couple of days ago (translation for the masses: like a month ago) I noticed Viktor Klang was tweeting about removing the annoying implicit evidences from methods. And some things I read seemed so elegant to me that I was forced to share some related ideas with all of you that don’t follow him at Twitter (@viktorklang).

Setting some context

Imagine the typical polymorphic method where we need an execution context for evaluating some Future:

import scala.concurrent.{ExecutionContext, Future}

def myMethod[T]
  (element: T)
  (implicit ev: ExecutionContext): Future[Boolean] = ???

You could say it’s as typical as disgusting, having to repeat the same exact words in the following 10 method definitions: (implicit ev: ExecutionContext).

Playing with type alias

The happy idea that is being proposed is to define a type alias like the following one:

type EC[_] = ExecutionContext

This way, by adding some syntax sugar, we would re-define the method signature:

def myMethod[T:EC](element: T): Future[Boolean] = ???
myMethod("hi")

Beautiful, isn’t it?

Some other possibilities

Non-polymorphic methods

In case our method isn’t parameterized, we would have to add some boilerplate (by adding a wildcard for the type that parameterizes the method). In essence, it should be working the same principle:

def myMethod[_:EC](element: Int): Future[Boolean] = ???
myMethod(2)

Multiple implicit contexts

The not-so-crazy case in which we needed several implicit parameters of different natures, we would have to define as many type alias as different type parameters we required:

type EC[_] = ExecutionContext
type MongoDB[_] = MongoDBDatabase

def myMethod[_:EC:MongoDB](element: Int): Future[Boolean] = ???

But what if …?

Multiple implicit parameters with same type

In case we have several implicit parameters that share the same type,

def myMethod
  (element: Int)
  (implicit ev1: ExecutionContext, ev2: ExecutionContext): Future[Boolean] = ???

it turns out that …

Well, by definition that’s impossible given that it would incur in some ambiguity issue when resolving implicits. It’s true that Scala allows having these kind of signatures, but we could only invoke them by making explicit the arguments contained in the second parameter group.:

myMethod(2)(ec1,ec2)

which is kind of…

Type-constructor implicit contexts

When we have implicit parameters that are also type constructors like List[T], Future[T], Option[T]

…well, it actually depends.

Case 1

If the type that parameterizes the method and the one that parameterizes the evidence are not related, there’s no big deal: we define another type alias and move on:

type EC[_] = ExecutionContext
type MongoDB[_] = MongoDBDatabase
type IntOpt[_] = Option[Int]
type StrList[_] = List[String]

def myMethod[_:EC:MongoDB:IntOpt:StrList](
  element: Int): Future[Boolean] = ???

Which would be equivalent to:

def myMethod(
  element: Int)(
  implicit ev1: ExecutionContext,
  ev2: MongoDBDatabase,
  ev3: Option[Int],
  ev4: List[String]): Future[Boolean] = ???

Case 2

If the type that parameterizes the method and the one that parameterizes the evidence have to match …

Well, it’s not possible. The syntax sugar we’re using here implies that both types have to match. Maybe it was too pretty for our bodies 🙂

See you in the next post. Peace out!

Scalera tip: contextos implícitos pegajosos

El otro día (para la gente normal: hace cosa de 1 mes) vi que el gran Viktor Klang twiteaba acerca de como quitar las molestas evidencias implícitas en definiciones de métodos. Y me pareció tan elegantes algunas de las cosas que leí, que me vi en la obligación de compartir algunas ideas al hilo de dichos consejos con aquellos de vosotros que no le sigáis aun en Twitter (@viktorklang).

La situación

Imaginad el típico método polimórfico en el cual necesitamos un execution context para ejecutar un futuro:

import scala.concurrent.{ExecutionContext, Future}

def myMethod[T]
  (element: T)
  (implicit ev: ExecutionContext): Future[Boolean] = ???

Es tan típico como feo, el tener que repetir la coletilla de (implicit ev: ExecutionContext) en 10 métodos seguidos…

Jugando con type alias

La idea feliz que se propone es definir un type alias del siguiente tipo:

type EC[_] = ExecutionContext

De esta forma, re-definiríamos la cabecera de nuestro método como sigue:

def myMethod[T:EC](element: T): Future[Boolean] = ???
myMethod("hi")

¿Bello o no?

Otras posibilidades

Métodos no polifórmicos

En el caso en que nuestro método no esté parametrizado, tendríamos que añadir algo de boilerplate (añadiendo un wildcard para el tipo que parametriza el método), pero en esencia debería seguir funcionando el mismo principio:

def myMethod[_:EC](element: Int): Future[Boolean] = ???
myMethod(2)

Múltiples contextos implícitos

En el no-tan-descabellado caso en el que necesitáramos varios parámetros implícitos de distintos tipos, necesitaríamos definir tantos type alias como tipos distintos de parámetros requiriésemos:

type EC[_] = ExecutionContext
type MongoDB[_] = MongoDBDatabase

def myMethod[_:EC:MongoDB](element: Int): Future[Boolean] = ???

Pero, ¿y si…?

Múltiples parámetros implícitos del mismo tipo

En el caso de que tengamos múltiples parámetros implícitos del mismo tipo,

def myMethod
  (element: Int)
  (implicit ev1: ExecutionContext, ev2: ExecutionContext): Future[Boolean] = ???

ocurriría que …

Bueno, por definición eso es imposible ya que incurriría en un problema de ambigüedad a la hora de resolver implícitos. Es cierto que Scala nos permite este tipo de signaturas, pero sólo podríamos invocar al método haciendo explícitos los argumentos del segundo grupo de parámetros:

myMethod(2)(ec1,ec2)

Lo cual es un tanto…

Contextos implícitos que son constructores de tipos

Cuando tenemos parámetros implícitos que son constructores de tipos como List[T], Future[T], Option[T]

En realidad depende.

Caso1

Si el tipo que parametriza el método y el que parametriza la evidencia no están relacionados, no hay mucho problema: definimos otro type alias y a correr:

type EC[_] = ExecutionContext
type MongoDB[_] = MongoDBDatabase
type IntOpt[_] = Option[Int]
type StrList[_] = List[String]

def myMethod[_:EC:MongoDB:IntOpt:StrList](
  element: Int): Future[Boolean] = ???

Lo cual sería el equivalente a:

def myMethod(
  element: Int)(
  implicit ev1: ExecutionContext,
  ev2: MongoDBDatabase,
  ev3: Option[Int],
  ev4: List[String]): Future[Boolean] = ???

Caso 2

Si el tipo que parametriza el método y el que parametriza la evidencia tienen que concordar …

Bueno no es posible. El syntax sugar implica que el tipo que parametriza el método vaya en concordancia con el tipo que parametriza nuestra evidencia. Quizás era todo demasiado bonito 🙂

Hasta el próximo post. ¡Agur de limón!

Scala: Code interpretation at runtime

With today’s post, we’ll dive into Scala code generation on the fly: at runtime. We have to be cautious of not mixing concepts with Scala macros, which generate code at compile-time. These make use of Scala type system, which is much safer than generating code at runtime.

House-of-cards-but-why

When is it useful to make use of this mechanism then? We’ll try to shed light on this by following a very simple example, getting abstract of real implementation (that you may find at Scalera’s Github).

The problem: “da” serializer

Let’s suppose a not-so-wild case, which consists on communicating two services via an event bus (we’ll get abstract of its implementation: it could be a message queue like Kafka, Akka streams, …).

The main idea is the following:

Sender-receiver-schema

The producer knows how to send and the consumer has an associated callback for message arrivals:

trait Producer{
  def produce(message: Any): Try[Unit]
}
trait Consumer{
  val consume: Any => Unit
}

Both producer and consumer services know the message types that may arrive. In our example, they could be one of these:

case class Foo(att1: Int, att2: String)
case class Bar(att1: String)

If the producer wants to send a message using the event bus, it will have to serialize it somehow (JSON, Byte array, XML, …) so, by the time it reaches the opposite end, the consumer will start the inverse process (deserialization) and will get the original message.

…nothing weird so far.

Whydoeseverythinghavetobesocomplicated

If we have a JSON serializer …

trait JsonSer[T] {
  def serialize(t: T): String
  def deserialize(json: String): T
}

and we serialize our message …

implicit val fooSerializer: JsonSer[Foo] = ???
val foo: Foo = ???
producer.send(implicitly[JsonSer[Foo]].serialize(foo))

How do we know which deserializer to use when the consumer gets the message?

Option 1: Try every possible serializer until one of them works

In our consumer, we’d have:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = {
    case message: String =>
      Seq(barSerializer, fooSerializer).flatMap { ser =>
        Try(ser.deserialize(message)).toOption
      }.headOption.fold(ifEmpty = println("Couldn't deserialize")) {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

A lil’ bit coarse, right? If the proper serializer is the last of a 100 list, we would have tried and failed with 99 serializers before (Such a waste of CPU!).

4d8

Besides, we could also consider the case of having a different type deserializer, but it fits with the received message, so it would partially or wrongly deserialize the message.

Option 2: Add the message type to the message itself

We could add an extra layer wrapping the message for indicating the message type that it contains. This way, when receiving it, we could determine the serializer type we have to use.

MessageWrapper

For writing the type, we’ll make use of Scala’s TypeTags, getting info about the T contained message type.


//Wrapper for the message (and its serializer)

import scala.reflect.runtime.universe.{TypeTag, typeTag}

case class Message[T: TypeTag](content: T){
  val messageType: Message.Type = typeTag[T].tpe.toString
}
object Message {

  type Type = String

  def typeFrom(msg: String): Message.Type = ???

  implicit def messageSer[T:TypeTag:JsonSer]: JsonSer[Message[T]] = ???

}

//We'll make use of it for sending

producer.produce(messageSer[Foo].serialize(Message(foo)))

//And we redefine the consumer

lazy val consumer = new Consumer {
    override val consume: Any => Unit = {
      case message: String =>
        Message.typeFrom(message) match {

          case "org.scalera.reflect.runtime.Bar" =>
            println("it's a bar!")
            val value = messageSer[Bar].deserialize(message).content
            println(value.att1)

          case "org.scalera.reflect.runtime.Foo" =>
            val value = messageSer[Foo].deserialize(message).content
            println("it's a foo!")
            println(value.att2)

          case _ =>
            println("it's ... something!")
        }
    }
  }

As you can see, we don’t have to try every possible serializer anymore. Instead of that, from the message type we’ve extracted from the Message wrapper we have added, we’re able to use the proper deserializer.

But it is also true that we have to add an extra case for each string that represents the message type. Wouldn’t it be nice to deserialize somehow and to have the defined case only for the already deserialized objects? (Something similar what we first tried but without trying all posible serializers). Something like:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { msg =>
    genericDeserialize(msg) match {
      case f: Foo =>
      case b: Bar =>
      case _ =>
    }
  }
}

Option 2-cool: Serializers ‘under the hood’

For achieving something similar, we have to get focused on that genericDeserialize method: Which signature should it have? Initially, something like this:

def genericDeserialize(msg: String): Any

An Any? Seriously? My fellows, at runtime, we have no idea about the type we can get. We just know that, from a String, we’ll get ‘some….thing’. The match that applies to that Any will allow us to go from something totally abstract to more concrete types.

At this point is where both reflect library and Scala compiler appear.

reflect.Toolbox

The Toolbox API allows parsing strings and getting the resulting AST (abstract syntax tree). From that AST, it is able to evaluate the expression and returns an instance of an Any as well.

For instantiating a Toolbox and use type references, we hace to add as SBT dependencies the following:

libraryDependencies ++= Seq(
  "org.scala-lang" % "scala-compiler" % "2.11.8",
  "org.scala-lang" % "scala-reflect" % "2.11.8")

For example, if we wanted to parse the "2".toInt + 4 operation,

import scala.tools.reflect.ToolBox
import scala.reflect.runtime.{universe => ru}
import ru._

//  Scala compiler tool box
val tb = ru.runtimeMirror(
  this.getClass.getClassLoader).mkToolBox()

println(ru.showRaw(tb.parse("2".toInt + 4")))

we would get the abstract syntax tree generated as a String (by using showRaw):

Apply(Select(Select(Literal(Constant("2")), TermName("toInt")), TermName("$plus")), List(Literal(Constant(4))))

If we use the toolbox for evaluating the parsed expression,

println(tb.eval(tb.parse("2".toInt + 4")))

we’ll get an Any that represents the resulting value of the sum:

6

“Da” serializer

Once we’ve seen how it generally works, we apply the same principle to our serializer, so the expression we’re going to try to interpret is similar to:

{
  import scala.reflect._;
  import spray.json._;
  import org.scalera.reflect.runtime._;
  import MySprayJsonImplicits._;
  import MyJsonSerImplicits._;

  implicitly[JsonSer[Message[$messageType]]]
}

where MySprayJsonImplicits and MyJsonSerImplicits represent the objects that contain both the Spray implicits for JsonFormat and the JsonSer implicits that we have defined before.

$messageType represents the concrete type to deserialize that we would have got by using the TypeTag (as seen before).

If we adapt it to our code, we’ll get something similar to:

object GenSer {

  import scala.tools.reflect.ToolBox
  import scala.reflect.runtime.{universe => ru}
  import ru._

  //  Scala compiler tool box
  private val tb = ru.runtimeMirror(this.getClass.getClassLoader).mkToolBox()

  def genericDeserialize(msg: String)(serContainers: Seq[AnyRef]): Any = {

    val messageType = Message.typeFrom(msg)

    val serContainersImport = serContainers.map(container =>
      "import " + container.toString.split("\\$").head + "._").mkString(";\n")

    val expr =
      s"
         |{
         |  import scala.reflect._;
         |  import spray.json._;
         |  import org.scalera.reflect.runtime._;
         |  $serContainersImport;
         |
         |  implicitly[JsonSer[Message[$messageType]]]
         |}
        ".stripMargin

    tb.eval(tb.parse(expr))
      .asInstanceOf[JsonSer[Message[Any]]]
      .deserialize(msg).content
  }

}

If you take a look, we’ve empowered the generic deserialization method notation to hold a sequence of objects to import, so we won’t make explicit which object contains the Spray implicits and which one contains our JsonSer‘s.

val serContainersImport = serContainers.map(container =>
  "import " + container.toString.split("\\$").head + "._").mkString(";\n")

It is also noteworthy that, when deserializing, we get a Message[Any]; so we’ll have to get the ‘content’ field that represents the raw value of Any type that holds the deserialized message.

The result

So finally, we can now make use of our function for generically deserialize and let our consumer code be ‘swaggy’:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { 
    case msg: String =>
      genericDeserialize(msg)(Seq(case3,Message)) match {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

Conclusions

Being able to evaluate code at runtime let us do a lot of interesting things when we want to interpret String based types. However, these kind of techniques won’t care about the so worthy and safe Scala type system, so these are tools to be used with a lil’ bit of care.

It is also kind of expensive in time terms to evaluate these expressions. I would recommend to use a type cache in this case. Something really simple like a map:

type TypeName = String
var cache: Map[TypeName, JsonSer[Message[_]]]

And when invoking the method, we’ll check if the type evidence (JsonSer) we’re looking for is already stored in the map. If so, we’ll use it, otherwise, we’ll create and store it in the cache, using it as result.

post-28553-Steve-Jobs-mind-blown-gif-HD-T-pVbd

Easy peasy…
Peace out!

Scala: Interpretación de código en runtime

En el post de hoy vamos a ver cómo generar código de Scala al vuelo: en tiempo de runtime. Hemos de ser cautos de no confundirlo con las macros de Scala, las cuales generan código en tiempo de compilación. Estas últimas hacen uso del sistema de tipos de Scala, lo cual es infinitamente más seguro que generar código en tiempo de ejecución.

House-of-cards-but-why

¿Cúando es útil hacer uso de este mecanismo entonces? Trataremos de arrojar luz siguiendo un ejemplo muy sencillo, abstrayéndonos de la verdadera implementación, la cual podéis encontrar en el Github de Scalera

El problema: “el” serializador

Supongamos un caso (no tan descabellado) en el que queramos comunicar dos servicios vía un bus de eventos (nos abstraeremos de la implementación: podría ser una cola de mensajes tipo Kafka, un stream de Akka, …).

El punto de partida es el siguiente:

Sender-receiver-schema

El productor sabe enviar y el consumidor tiene asociado un callback para cuando llegue un mensaje:

trait Producer{
  def produce(message: Any): Try[Unit]
}
trait Consumer{
  val consume: Any => Unit
}

Tanto el servicio productor como el consumidor conocen los tipos de mensajes que pueden llegar, en nuestro ejemplo serán estos:

case class Foo(att1: Int, att2: String)
case class Bar(att1: String)

Si el servidor quiere enviar un mensaje usando el bus de eventos, deberá serializarlo de alguna forma (JSON, array de bytes, XML, …) para que, al llegar al otro extremo, el consumidor realice el proceso inverso (deserialización) y obtenga el mensaje original.

…hasta aquí nada extraño.

Whydoeseverythinghavetobesocomplicated

Si tenemos un serializador para JSONs …

trait JsonSer[T] {
  def serialize(t: T): String
  def deserialize(json: String): T
}

y serializamos nuestro mensaje…

implicit val fooSerializer: JsonSer[Foo] = ???
val foo: Foo = ???
producer.send(implicitly[JsonSer[Foo]].serialize(foo))

¿Cómo sabemos qué deserializador usar cuando el consumidor reciba el mensaje?

Opción 1: Probar con todos hasta que uno encaje

En nuestro consumidor tendríamos:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = {
    case message: String =>
      Seq(barSerializer, fooSerializer).flatMap { ser =>
        Try(ser.deserialize(message)).toOption
      }.headOption.fold(ifEmpty = println("Couldn't deserialize")) {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

Un poco burdo, ¿no? Si el serializador correcto es el último de 100, estaríamos probando y fallando con 99 antes de dar con el correcto (¡qué desperdicio de CPU!).

4d8

Además, también podría darse el caso de que hubiera un deserializador para otro tipo, pero que encaja y es capaz de deserializar parcial o erróneamente el mensaje.

Opción 2: Añadir el tipo al mensaje

Podríamos añadir una capa por encima al mensaje e indicar el tipo de mensaje que es. Así, al recibirlo, podríamos determinar el tipo de serializador que necesitamos usar.

MessageWrapper

Para escribir el tipo, nos apoyaremos en los TypeTag de Scala, obteniendo información sobre el tipo de mensaje contenido T.


//Wrapper for the message (and its serializer)

import scala.reflect.runtime.universe.{TypeTag, typeTag}

case class Message[T: TypeTag](content: T){
  val messageType: Message.Type = typeTag[T].tpe.toString
}
object Message {

  type Type = String

  def typeFrom(msg: String): Message.Type = ???

  implicit def messageSer[T:TypeTag:JsonSer]: JsonSer[Message[T]] = ???

}

//We'll make use of it for sending

producer.produce(messageSer[Foo].serialize(Message(foo)))

//And we redefine the consumer

lazy val consumer = new Consumer {
    override val consume: Any => Unit = {
      case message: String =>
        Message.typeFrom(message) match {

          case "org.scalera.reflect.runtime.Bar" =>
            println("it's a bar!")
            val value = messageSer[Bar].deserialize(message).content
            println(value.att1)

          case "org.scalera.reflect.runtime.Foo" =>
            val value = messageSer[Foo].deserialize(message).content
            println("it's a foo!")
            println(value.att2)

          case _ =>
            println("it's ... something!")
        }
    }
  }

Cómo podéis ver, ya no tenemos que ir probando con todos los serializadores posibles hasta que demos con uno que funcione; sino que, a partir del tipo de mensaje (extraido del envoltorio de tipo Message que hemos añadido), somos capaces de usar el deserializador adecuado.

Pero también es cierto, que tenemos que añadir un case por cada string que representa el tipo. ¿No sería mejor poder deserializar (como sea) y que el case solo esté definido sobre los objetos ya deserializados (como antes pero sin probar a lo loco con mil serializadores)? Algo de este estilo:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { msg =>
    genericDeserialize(msg) match {
      case f: Foo =>
      case b: Bar =>
      case _ =>
    }
  }
}

Opción 2-guay: Serializadores ‘under the hood’

Para conseguir algo parecido, debemos de centrarnos en ese método genericDeserialize: ¿qué signatura debería tener? Inicialmente algo del siguiente estilo:

def genericDeserialize(msg: String): Any

¿Un Any? ¿En serio? Amigos, en tiempo de runtime, no tenemos ni idea del tipo que nos puede llegar. Solo sabemos que, a partir de un String, vamos a obtener un ‘algo’. El match que se aplica sobre dicho Any nos permitirá pasar de algo totamente abstracto a tipos más concretos.

Es en este punto, donde entra en juego la librería de reflect y el compilador de scala.

reflect.Toolbox

La api de Toolbox permite parsear cadenas de texto y obtener el AST (abstract syntax tree) resultante. Del mismo modo, a partir del AST es capaz de evaluar la expresión devolviendo una instancia de un tipo determinado.

Para poder instanciar un Toolbox y usar referencias a tipos, es preciso añadir como dependencias a nuestro proyecto las librerías de scala-compiler y scala-reflect:

libraryDependencies ++= Seq(
  "org.scala-lang" % "scala-compiler" % "2.11.8",
  "org.scala-lang" % "scala-reflect" % "2.11.8")

Si por ejemplo quisiéramos parsear la operación "2".toInt + 4,

import scala.tools.reflect.ToolBox
import scala.reflect.runtime.{universe => ru}
import ru._

//  Scala compiler tool box
val tb = ru.runtimeMirror(
  this.getClass.getClassLoader).mkToolBox()

println(ru.showRaw(tb.parse(""""2".toInt + 4""")))

nos devolverá el árbol sintáctico generado como un string (usando showRaw):

Apply(Select(Select(Literal(Constant("2")), TermName("toInt")), TermName("$plus")), List(Literal(Constant(4))))

Si a partir de la expresión parseada, usamos el toolbox para evaluarla,

println(tb.eval(tb.parse(""""2".toInt + 4""")))

obtendremos un Any que representa el valor resultante de la suma:

6

“El” serializador

Una vez visto el funcionamiento general, aplicamos el mismo principio a nuestro serializador, de manera que la expresión que vamos a intentar interpretar es algo similar a:

{
  import scala.reflect._;
  import spray.json._;
  import org.scalera.reflect.runtime._;
  import MySprayJsonImplicits._;
  import MyJsonSerImplicits._;

  implicitly[JsonSer[Message[$messageType]]]
}

donde MySprayJsonImplicits y MyJsonSerImplicits representan los objetos que contienen tanto los implícitos de Spray para JsonFormat que hemos definido, como los implícitos de la type class JsonSer que hemos definido nosotros.

$messageType representa el tipo concreto a deserializar que habremos obtenido previamente usando el TypeTag (como hemos visto antes).

Si lo adaptamos a nuestro código obtendremos algo similar a:

object GenSer {

  import scala.tools.reflect.ToolBox
  import scala.reflect.runtime.{universe => ru}
  import ru._

  //  Scala compiler tool box
  private val tb = ru.runtimeMirror(this.getClass.getClassLoader).mkToolBox()

  def genericDeserialize(msg: String)(serContainers: Seq[AnyRef]): Any = {

    val messageType = Message.typeFrom(msg)

    val serContainersImport = serContainers.map(container =>
      "import " + container.toString.split("\\$").head + "._").mkString(";\n")

    val expr =
      s"""
         |{
         |  import scala.reflect._;
         |  import spray.json._;
         |  import org.scalera.reflect.runtime._;
         |  $serContainersImport;
         |
         |  implicitly[JsonSer[Message[$messageType]]]
         |}
        """.stripMargin

    tb.eval(tb.parse(expr))
      .asInstanceOf[JsonSer[Message[Any]]]
      .deserialize(msg).content
  }

}

Si os fijáis, hemos permitido en la notación del método de deserialización genérica, que se le pase una secuencia de objetos a importar, de manera que no se explicita qué objeto es el que contiene los implícitos de Spray y qué otro contiene los de nuestro JsonSer.

val serContainersImport = serContainers.map(container =>
  "import " + container.toString.split("\\$").head + "._").mkString(";\n")

También cabe destacar que, al deserializar, obtenemos un Message[Any]; por lo que posteriormente hay que obtener el campo ‘content’ que representa el valor plano de tipo Any que representa el mensaje deserializado.

El resultado

Ahora sí, podemos usar nuestra función para deserializar de manera genérica y dejar el código de nuestro consumidor de manera ‘reshulona’:

lazy val consumer = new Consumer {
  override val consume: Any => Unit = { 
    case msg: String =>
      genericDeserialize(msg)(Seq(case3,Message)) match {
        case bar: Bar => println("it's a bar!")
        case foo: Foo => println("it's a foo!")
        case _ => println("it's ... something!")
      }
  }
}

Conclusiones

Poder interpretar código en tiempo de runtime da bastante juego cuando queremos interpretar tipos en base a String’s. Ahora bien, interpretar código en runtime se pasa por el forro el sistema de tipos que tanta seguridad aporta en Scala, por lo que son herramientas a usar con algo de cuidado.

También es algo costoso en tiempo la evaluación de estas expresiones. Os recomendaría que usarais una cache de tipos. Algo tan sencillo como un mapa:

type TypeName = String
var cache: Map[TypeName, JsonSer[Message[_]]]

Y al ejecutar el método, comprobar si existe ya un JsonSer almacenado en el mapa. De ser así, usamos ese y en caso contrario, lo creamos, lo almacenamos en la caché y lo devolvemos como resultado.

post-28553-Steve-Jobs-mind-blown-gif-HD-T-pVbd

Easy peasy…
¡Agur de limón!

Dynamic types

Today we bring you some extra black magic. I was reading the other day a little bit about the Scala Improvement Processes (SIPs), I found something that really got my attention: dynamic types.

What are they?

Dynamic types allow to define the behavior of some object when a method or a member that doesn’t exist is tried to be accessed. I.e.:

class MyClass {
  def someMethod(): String = "hi"
}
val myInstance = new MyClass
myInstance.nonExistentMethod

will complain at compile time. With dynamic types we can decide that, in case some whatever-its-name-is method is invoked, it will return certain value.

How do they work?

In order to use dynamic types, it’s necessary to import scala.language.dynamics and extend trait Dynamic:

class MyClass extends Dynamic {
  //...
}

The class (or trait) that we will define may have all the attributes and methods we want, but we can also define the following:

applyDynamic

this method is in charge of managing the invocations to the instance methods.
If we want our instance to know how to invoke ‘myMethod’ method with a ‘hi’ parameter, we can do it as follows:

class MyClass extends Dynamic {
  def applyDynamic(methodName: String)(args: Any*) = {
    val beginning = 
      args.headOption.getOrElse("hello")
    if (methodName=="myMethod") 
      println(beginning + " you")
    else println("Dunno...")
  }
}

If we now create a new instance and we invoke another made-up method, we’ll see that it will print out that it doesn’t know what you’re talking about:

val myInstance = new MyClass
myInstance.anotherMethod()//Dunno...

And if we invoke the myMethod method we’ve considered:

myInstance.myMethod("hi")//hi you
myInstance.myMethod(2)//2 you
myInstance.myMethod()//hello you

applyDynamicNamed

It works exactly like the applyDynamic method, except that it only allows to get by name the parameters that are used when invoking the method:

class MyClass extends Dynamic {
  def applyDynamicNamed(name: String)(args: (String, Any)*) = {
    val (argName,argValue) = 
      args.headOption.getOrElse(("unknown","hello"))
    if (name=="myMethod") 
      println(s"$argName : $argValue - you")
    else println("Dunno...")
  }
}

If the method is invoked without explicitly naming any parameter, this parameter’s name will be an empty string. The thing is, to make applyDynamicNamed work, at least one of the parameters has to owe a name.

myInstance.myMethod(test=2)//test : 2 - you
myInstance.myMethod(true,test=2)// : true - you
myInstance.myMethod(true)
/*
error: value applyDynamic is not a member of MyClass
error after rewriting to myInstance.<applyDynamic: error>("myMethod")
possible cause: maybe a wrong Dynamic method signature?
*/

selectDynamic

It is similar to a ‘dynamic getter’. I mean:

class MyClass extends Dynamic {
  def selectDynamic(name: String) = 
    s"My $name equals this string"
}
val myInstance = new MyClass
myInstance.someAttribute//My someAttribute equals this string

updateDynamic

And with this last method, we define the updating behavior of all members of the instance that are not statically defined:

class MyClass extends Dynamic {
  def updateDynamic(name: String)(value: Any) {
    println(s"you updated your attribute $name with $value value")
  }
}
val myInstance = new MyClass
myInstance.myAttribute = 2
//you updated your attribute myAttribute with 2 value

NB: It doesn’t work really well in the REPL (actually it is compiled as a getter). If you encapsulate it inside an App block, it should work.

Easy peasy.

Why this in a statically typed language?

haters

Official motivation of this SIP was:

Static types are great, but sometimes we would like to do without them. This could give better support for flexible DSLs that do not require defining the members of a type. It could also provide simpler interfaces with dynamic languages. Scala already provides a number of ways to escape from the straightjacket of the type system: The universal type Any, type tests and type casts, as well as pattern matching. However, one thing is missing: When selecting a member qual.sel, sel must be defined in qual, or be addable with an implicit conversion. This means that we can select members or call methods only if it is statically known that the receiver type supports that member.

Even it’s true that the purists may see it like an aberration of the statically typed language nature; for the jugglers, it could be seen as another way to risk their necks defining DSLs without compile-time checkings. You judge

Peace out!

Tipos dinámicos

Hoy traemos un poco más de magia negra. Leyendo el otro día un poco sobre los procesos de mejora de Scala (SIPs) me encontré algo que realmente me llamó la atención: tipos dinámicos.

¿Qué son?

Los tipos dinámicos permiten definir el comportamiento de un objeto cuando se invoca un método o se intenta acceder a un miembro que no existe. Por ejemplo:

class MyClass {
  def someMethod(): String = "hi"
}
val myInstance = new MyClass
myInstance.nonExistentMethod

dará un fallo en tiempo de compilación. Con los tipos dinámicos podemos decir que en caso de que se invoque a un método, se llame como se llame, devuelva cierto valor.

¿Cómo funcionan?

Para poder utilizar los tipos dinámicos, es preciso importar import scala.language.dynamics y extender del trait Dynamic:

class MyClass extends Dynamic {
  //...
}

La clase (o trait) que definamos puede tener todos los atributos y métodos que queramos, pero además podemos implementar los siguientes métodos:

applyDynamic

este método se encarga de gestionar las llamadas a métodos de la instancia.
Si queremos que nuestra instancia sepa como invocar el método ‘myMethod’ con el argumento “hi”, podemos hacerlo como sigue:

class MyClass extends Dynamic {
  def applyDynamic(methodName: String)(args: Any*) = {
    val beginning = 
      args.headOption.getOrElse("hello")
    if (methodName=="myMethod") 
      println(beginning + " you")
    else println("Dunno...")
  }
}

Si ahora creamos una instancia e invocamos otro método inventado, veremos que imprimirá por pantalla que no sabe a que se refiere:

val myInstance = new MyClass
myInstance.anotherMethod()//Dunno...

Y si invocamos al método myMethod que tenemos contemplado:

myInstance.myMethod("hi")//hi you
myInstance.myMethod(2)//2 you
myInstance.myMethod()//hello you

applyDynamicNamed

Funciona exactamente igual que el applyDynamic solo que permite obtener por nombre los argumentos con los que se invoca al método:

class MyClass extends Dynamic {
  def applyDynamicNamed(name: String)(args: (String, Any)*) = {
    val (argName,argValue) = 
      args.headOption.getOrElse(("unknown","hello"))
    if (name=="myMethod") 
      println(s"$argName : $argValue - you")
    else println("Dunno...")
  }
}

Si se invoca al método sin indicar el nombre de algún parámetro, este será cadena vacía, pero para que se invoque a applyDynamicNamed al menos uno de los argumentos debe tener nombre:

myInstance.myMethod(test=2)//test : 2 - you
myInstance.myMethod(true,test=2)// : true - you
myInstance.myMethod(true)
/*
error: value applyDynamic is not a member of MyClass
error after rewriting to myInstance.<applyDynamic: error>("myMethod")
possible cause: maybe a wrong Dynamic method signature?
*/

selectDynamic

Sería el equivalente a un getter dinámico. Es decir:

class MyClass extends Dynamic {
  def selectDynamic(name: String) = 
    s"My $name equals this string"
}
val myInstance = new MyClass
myInstance.someAttribute//My someAttribute equals this string

updateDynamic

Y con este método definimos el comportamiento de actualización de miembros que no están definidos de manera estática:

class MyClass extends Dynamic {
  def updateDynamic(name: String)(value: Any) {
    println(s"you updated your attribute $name with $value value")
  }
}
val myInstance = new MyClass
myInstance.myAttribute = 2
//you updated your attribute myAttribute with 2 value

NB: En la REPL no funciona de manera muy católica (lo interpreta como un getter). Si lo encapsulais dentro de un App debería funcionar.

Easy peasy.

¿Por qué en un lenguaje de tipado estático?

haters

La motivación oficial de la propuesta de mejora es:

Static types are great, but sometimes we would like to do without them. This could give better support for flexible DSLs that do not require defining the members of a type. It could also provide simpler interfaces with dynamic languages. Scala already provides a number of ways to escape from the straightjacket of the type system: The universal type Any, type tests and type casts, as well as pattern matching. However, one thing is missing: When selecting a member qual.sel, sel must be defined in qual, or be addable with an implicit conversion. This means that we can select members or call methods only if it is statically known that the receiver type supports that member.

Si bien es cierto que los más puristas pueden verlo como una aberración hacia la naturaleza de tipado estático del lenguaje, para los más malabaristas puede dar margen de maniobra a la hora de jugarse el tipo definiendo DSLs sin comprobaciones en tiempo de compilación. Juzguen ustedes.

¡Agur de limón!

ClassTag, Class and war stories…

Sometimes, when working with Scala, the need of working with type metadata just comes out. Even though macros might seem something too far, theorical and tricky (we’ll speak about them later), sometimes getting some info about runtime classes is more than enough.

ClassTag

You could see a ClassTag as some ‘wrapper’ that Scala adds over java.lang.Class runtime classes.

In order to work with ClassTag we need to import:

import scala.reflect.{ClassTag,classTag}

Whereas the first member imported is the class, the second one is just a method that allows to get implicitly the ClassTag of a certain type. I.e.:

scala> classTag[Int]
res0: scala.reflect.ClassTag[Int] = Int

Actually, it’s just syntax sugar for:

def classTag[T:ClassTag]: ClassTag[T] = 
  implicitly[ClassTag[T]]

scala> classTag[Int]
res0: scala.reflect.ClassTag[Int] = Int

There are methods (currently deprecated by type checking via TypeTag) like <:< or =:= that allow checking at compile time the type bounds.
For example, this:

def myMethod[T,U](t: T,u: U)(implicit ev: <:<[T,U]):Unit = 
  println(t,u)

allows constraining the relation between T and U types:

scala> myMethod(new Animal,new Car)
<console>:11: error: Cannot prove that Animal <:< Car. myMethod(new Animal,new Car)

scala> myMethod(new Cat,new Animal)
($line15.$read$$iw$$iw$Cat@350aac89,$line14.$read$$iw$$iw$Animal@1c5920df)

Where are my type parameters?

One of the main problems working with ClassTag is the information loss about the parameter types that you class might have. For example, if we ask for the ClassTag of List[Int] we’ll get that:

scala> println(scala.reflect.classTag[List[Int]])
scala.collection.immutable.List

As you can see, information about Int that parameterizes the list is lost. This process that removes parameter types info at runtime is well known as type erasure.

62517760

Nevertheless, if we need info about the static type, we can use TypeTags (which should be properly explained at future posts, when talking about macros).

scala> import scala.reflect.runtime.universe._
import scala.reflect.runtime.universe._

scala> val tt = typeTag[List[Int]]
tt: reflect.runtime.universe.TypeTag[List[Int]] = TypeTag[scala.List[Int]]

Frequent problems

How to get the Class of a known type T?

scala> classOf[Int]
res0: Class[Int] = int

I have a Class, but I need a ClassTag

As easy as:

scala> val clazz = classOf[Int]
clazz: Class[Int] = int

scala> val ctag = ClassTag(clazz)
ctag: scala.reflect.ClassTag[Nothing] = Int

scala> val ctag = scala.reflect.ClassTag[Int](clazz)
ctag: scala.reflect.ClassTag[Int] = Int

Notice that, unless we explicitly point the type, ClassTag builder will infer the most generic possible type (Nothing).

Is T a subtype of U?

We’ve seen before that, at compile time, we can use <:< and so for constraining type bounds but, what if it takes place at runtime? How can we check it?

We can use isAssignableFrom method, from class Class that indicates whether a class is the same class (or superclass) compared to another one:

object A{
  def apply[T:ClassTag] = new {
    def isA[U:ClassTag]: Boolean = 
      classTag[T].runtimeClass.isAssignableFrom(classTag[U].runtimeClass)
  }
}

We could try this code snippet with:

scala> A[Int].isA[String]
res0: Boolean = false

scala> A[String].isA[String]
res1: Boolean = true

Infer the type of an Any

This tip comes out of a question that a colleague made me. ‘Is there any way to, having the folling function:

def isA[T](t: Any): Boolean = ???

check if t value is T typed?’
First obvious step for checking it would be something similar to:

scala> def isA[T](t: Any): Boolean = t.isInstanceOf[T]
<console>:7: warning: abstract type T is unchecked since it is eliminated by erasure
       def isA[T](t: Any): Boolean = t.isInstanceOf[T]
                                                   ^
isA: [T](t: Any)Boolean

Compiler is warning us (Forewarned is forearmed) that we won’t have information about T at runtime, that it’s getting lost. What if we try with different types? It eats them all:

scala> isA[Int]("")
res2: Boolean = true

scala> isA[Int](3)
res3: Boolean = true

In this case, we said ‘okay, what if we just try to check if classes are assignables, getting the info from the ClassTag‘?
It happened that,

scala> val x: Any = 3
x: Any = 3

scala> x.getClass
res2: Class[_] = class java.lang.Integer

scala> classOf[Int].isAssignableFrom(classOf[Int])
res3: Boolean = true

scala> classOf[Int].isAssignableFrom(classOf[java.lang.Integer])
res4: Boolean = false

The x class is java.lang.Integer, which is not assignable from Int class (the same happens with many other primitive types).

So we finally…

…realized that ClassTag has a wonderfull unapply method that allows to get a typed value if this belongs to the ClassTag parameterized class. Just what we needed:

def isA[T:ClassTag](t: Any): Boolean = 
  classTag[T].unapply(t).isDefined

When we tried it at the REPL, we could gladly check that:

scala> val x: Any = 3
x: Any = 3

scala> isA[String](x)
res5: Boolean = false

scala> isA[Int](x)
res6: Boolean = true

Conclusions

It’s not very common to deal with all these isses unless you’re building a generic enough system. However, it doesn’t harm you to have a little idea about what the dark side is about…

137134463824621

Peace out!