Type-safe query builders in Scala revisited: shapeless

Not so long ago, I wrote a post about creating type-safe query builders in Scala from scratch. In it, I also suggested using shapeless library to do what was described. Now, I decided to write how it could be done. The code is in this repository.

Problem reminder

Without going into much details, the problem was to provide a type-safe way to build queries (to an abstract database, for instance) with parameters of different types. Something like

val query = beginQuery()
  .addParameter[String]("param1")
  .addParameter[Int]("param2")
  .addParameter[Boolean]("param3")
  .build()

// Compile:
query.execute("some string", 1, false)
// Won't compile:
query.execute(42, true, "another string")

shapeless and HList

There is a Scala library called shapeless, which purpose is to give a convenient way to do type-level generic programming. It has many interesting features, but we are going to use one - heterogeneous lists (HLists). HList is a list that contains elements of different types and handles them in a type-safe manner (i.e. without casting to Any).

import shapeless.{HNil, ::}

val hlist: Int :: String :: Double :: HNil = 12 :: "hi" :: 42.24 :: HNil
val head: Int = hlist.head
val second: String = hlist.tail.head
val tuple: (Int, String, Double) = hlist.tupled
val reversed: Double :: String :: Int :: HNil = hlist.reverse

Actually, we need to do computations like reversing and converting to a tuple on the type level, and it is possible with shapeless.

Reused parts

We are going to reuse pieces of code from the original post. Empty tuple, Tuple0, which is derived from Product:

package me.ivanyu.common

class Tuple0 extends Product {
  override def productElement(n: Int): Any =
    throw new NoSuchElementException()
  override def productArity: Int = 0
  override def canEqual(that: Any): Boolean = false

  override def toString: String = "()"
}

TypedParameter for demonstration:

package me.ivanyu.common

import scala.language.existentials
import scala.reflect.ClassTag

case class TypedParameter(name: String,
                          classTag: ClassTag[_])

The Query itself:

package me.ivanyu.common

class Query[P <: Product] (parameters: Seq[TypedParameter]) {
  def execute(parameterValues: P): Unit = {
    val zipped = parameters zip parameterValues.productIterator.toList
    println(s"Much magic: $zipped")
    // Your code
  }
}

Scala automatically converts "str", 1, false into Tuple3("str", 1, false) when the only parameter of a function is Tuple3[String, Int, Boolean]. This allows us to write execute calls in a natural way, without an explicit tuple.

There are two exceptions: no parameters and one parameter. We create implicit conversions for these cases:

package me.ivanyu

import me.ivanyu.common.Tuple0

import scala.language.implicitConversions

package object typed_queries_shapeless {
  val beginQuery = QueryBuilder.begin _

  // Implicit conversions to make execute()
  // and execute(value) more convenient.
  implicit def unitToTuple0(u: Unit): Tuple0 = new Tuple0
  implicit def valueToTuple1[T](v: T): Tuple1[T] = Tuple1(v)
}

QueryBuilder

Here is the code:

package me.ivanyu.typed_queries_shapeless

import me.ivanyu.common.{Query, TypedParameter, Tuple0}
import shapeless.ops.hlist.{Reverse, Tupler}
import shapeless.{HList, HNil, ::}

import scala.reflect.ClassTag

class QueryBuilder[L <: HList] private(parameters: Seq[TypedParameter]) {

  import QueryBuilder._

  final def addParameter[T : ClassTag](name: String): QueryBuilder[T :: L] = {
    val classTag = implicitly[ClassTag[T]]
    val param = TypedParameter(name, classTag)
    new QueryBuilder[T :: L](param +: parameters)
  }

  final def build[P <: Product]()(implicit aux: Aux[L, P]): Query[P] = {
    new Query[P](parameters.reverse)
  }
}

object QueryBuilder {
  // About Aux: http://gigiigig.github.io/posts/2015/09/13/aux-pattern.html
  sealed trait Aux[L <: HList, Out0]

  implicit def implicitAux0 = new Aux[HNil, Tuple0] {}
  implicit def implicitAux[L <: HList, RL <: HList, P <: Product]
    (implicit reverse: Reverse.Aux[L, RL],
     tupler: Tupler.Aux[RL, P]) = new Aux[L, P] {}

  final def begin(): QueryBuilder[HNil] = {
    new QueryBuilder[HNil](Nil)
  }
}

Clarification is needed. A query is built by adding parameters to the current QueryBuilder. We store them both on the type level (in a form of HList in QueryBuilder generic parameter) and on the value level (in a form of sequence of TypedParameters). Initial QueryBuilder contains no parameters, i.e. HNil on the type level and Nil on the value level.

A more interesting part is final producing of Query instance (and type). There are two important moments. The first is that the natural way to grow HList is prepending (like in normal Scala List), so our parameter type list will be in the reverse order by the moment of call of build. Therefore, we must reverse the type list (on the type level).

The second moment is that Query works with tuples (actually, Product) and not HList, so we need to convert an HList to the correspondent tuple type. Additionally, default shapeless implementation converts empty HList (HNil) into Unit, and we need it to be converted into our special type Tuple0.

To achieve this, we will use implicit values of special Aux type, which will help the compiler to figure out the types in compile time. See an implicit parameter of build method: (implicit aux: Aux[L, P]). When build is called, type L is known (it is some HList), and the purpose of this implicit trick is to provide consistent advice to the compiler about the correspondent type P for this type L, which is the output type of the method.

For the empty list, we provide the evidence implicitAux0 of type Aux[HNil, Tuple0]. So, when the compiler sees that L is actually HNil, it knows that P is actually Tuple0.

It is more interesting when we need to deal with non-empty lists. To provide evidences for these cases, we use implicit parameters provided by shapeless. reverse: Reverse.Aux[L, RL] helps the compiler to find the type of a reversed list, and tupler: Tupler.Aux[RL, P] helps it to find the correspondent tuple type for this reversed list. So, implicitAux[L <: HList, RL <: HList, P <: Product] is a way to, given a known heterogeneous list type L, find the target product type P through some unknown reversed heterogeneous list type RL.

It should be said, that in client code (where we call build) it is not necessary to import this implicits to make the call work, because the compiler looks for the implicits in many places, including implicit parameter types scope (i.e. object QueryBuilder in this case).

That is all: now we can build queries and executed them in a type-safe way:

package me.ivanyu

object Runner extends App {
//  import me.ivanyu.typed_queries._
  import me.ivanyu.typed_queries_shapeless._

  // Better, to remove deprecation warning: execute(())
  beginQuery()
    .build()
    .execute()

  beginQuery()
    .addParameter[String]("param1")
    .build()
    .execute("'some string'")

  beginQuery()
    .addParameter[String]("param1")
    .addParameter[Int]("param2")
    .build()
    .execute("'some string'", 1)

  beginQuery()
    .addParameter[String]("param1")
    .addParameter[Int]("param2")
    .addParameter[Boolean]("param3")
    .build()
    .execute("'some string'", 1, false)
}

Conclusion

Thus we have implemented a type-safe query builders using shapeless, which helped to get rid of lots of boilerplate code. As I said in the previous post on this topic, in most cases, shapeless should be considered as a default choice for such problems.