# Scala in North Lakes

A Scala conference, far from any significant technology hub? With no call for papers? With hikes planned in what is one of the wettest parts of the country? Without Martin Odersky? What could possibly go wrong? A lot. But, apart from a few of the speakers not making it, nothing did, and Scala World was probably as good as a Scala conference can get.

Instead of summarising individual talks, I will focus on the major themes. The usual disclaimers apply: it is a personal impression based only on the talks I saw, I might not have selected the best and most representative ones, and in general I understood very little of what was being said.

## Functional Programming

The conference schedule made it clear that functional programming will be the leading theme at Scala World. It was the subject of the keynotes on both days. Rúnar Bjarnason presented a philosophical argument on how the least powerful construct gives us most power of reasoning in a way that was digestible to those moderately versed in FP, finishing with a concept of Galois connection. Daniel Spiewak dove right into the algebra of Scalaz Stream, the shortcomings of existing one and promises of the new implementation, $$FS^2$$.

Practical functional architecture, presented by Raúl Raja, offered Kleisli arrows for composition, free monad for interpretation and delimited, monadic, dependently-typed, accumulating, checked exceptions for fault tolerance. John de Goes explained how free appliatives, being less powerful than free monads, are desirable where dependency on previous step is not required, i.e. where the program builds a static data structure, not another program. What we get in return is the ability to introspect on the structure before runtime. In the process, we saw how the sausage was being made (with a natural transformation), and boy, it was not a pretty sight. John offered a nicer solution based on finally tagless representation, which perhaps was a way of encoding GADTs in Scala, but I’m not sure – that bit was over my head. Anyway, even that effort was heckled by Daniel Spiewak, who insisted he could do it in half a line of code with Coyoneda. Contrasting with those fairly advanced talks, Chris Marshall’s workshop on error handling in Scalaz, which ended up with a monad transformer stack that offered Reader, Writer, Either and IO, was quite approachable to someone vaguely familiar with the subject – if not particularly enlightening.

If that all seems hermetic to you, you are not alone. Hard-core FP community sometimes does appear like a closed club talking in strange language, whose chief purpose is to make the outsiders feel stupid and inadequate. Changing this perception was at the heart of Erik Osheim’s talk. Erik is the founder of Cats library, which positions itself as a friendler (both to newcomers and to seasoned functional programmers) alternative to Scalaz. In contrast to Scalaz, Cats treats comprehensive documentation and supportive community as first-order concerns.

## Compiler Hacking

The sessions devoted to compilation of scala can be divided in two groups: the messy present and the bright future. In the former, we had an intro to scalac structure delivered by Chris Birchall, complete with a demo of compiler plugin that transpiles Scala (presumably, a subset) to Basic, and Simon Ochsenreither’s series of nicely structured tutorials on scalac bug fixing. The procedure always starts with text search for relevant terms in the entire compiler codebase, then hacking away until it compiles, finishing with a test case that demonstrates our fix or feature works as intended. Be prepared for long waits while your changes to the compiler are being rebuilt.

It is telling that new tools for working with Scala code and experimenting with the language tend to start from scratch rather than extend the infrastructure provided by scalac. It can hardly be construed as a glowing endorsement of the existing compiler’s codebase. If even Martin Odersky, the original author of the compiler, and Eugene Burmako, the person who gave us macros, move away from scalac when working on new Scala-related projects, one cannot help but share Paul Phillips’s opinion that scalac is doomed. That said, the new compiler, dotc, shows promise. Dmitry Petrashko, as of now the most prolific contributor to the new compiler’s codebase, discussed ways in which dotc is better than the predecessor: a number of smaller cakes instead of single giant one (?), global mutable state replaced by context object passed between phases, handling of symbol denotations that preserves the full history of a symbol, and mini-phases replacing the old maxi-phases. This last improvement should, to some extent, address the main gripe of many Scala developers: compilation speed. While scalac phases transform the entire AST, generating lots of garbage in old generation in the process, mini-phases work on the level of individual nodes, offering a better memory locality and shorter-lived objects. There is also hope for the elephant in the room, binary compatibility, in the form of typed trees that are saved in .class files and allow dotc to generate bytecode after the library is deployed. This feature also opens up exciting possibilities of whole-program optimisation after deployment, such as removal of virtual calls and limited automatic specialisation.

The dialect of Scala supported by dotc is actually a slightly different language, called Dotty. Those of us with substantial existing Scala codebases need not feel left behind: help is (about to be) at hand in the form of scala.meta. This new project from Eugene Burmako purports to provide a faithful representation of a program in a chosen dialect of Scala (2.11, 2.12 etc.) or Dotty. That AST representation can then by used for translating the program to a different dialect and emitting source code with even minor details, for example formatting, preserved. The ambition of scala.meta goes beyond syntactic manipulation; however, re-implementing Scala type checking outside of scalac turned out to be a monumental task. At present the project offers limited support for semantic information by calling out to scalac.

## Formal Methods

Those not satisfied with the assurances offered by run-of-the-mill FP devices, such as isolating effects and handling errors with monad transformer stacks, were treated to a preview of other techniques for making our software reliable. Sleep-depraved Jon Pretty showed CLI and internationalisation API of his Rapture libraries, where types were used to specify the required structure of command line arguments and the set of supported languages. While the usefulness of being able to express in the type that a string has English and French version, or require that both English and German messages must be provided to a function, is debatable, the possibilities offered by Scala’s type system, and Jon’s ingenuity in exploiting them, seem immense.

Getting the machine to write the program for us never fails to impress. On top of that, Edwin Brady is a very entertaining speaker, so his Idris demo, where the expressiveness of dependent types allowed the Idris runtime to infer the implementation, and the practicality was demonstrated by Space Invaders clone, met with well deserved cheers and applause. Lars Hupel led another excursion beyond Scala-land, into the realm of Isabelle proof assistant, to finally return to home soil by mentioning Leon – a proof assistant that supports a subset of Scala. Similarly, Paul Snively ran a workshop on formal logic in Coq.

It all appeared very elegant, sometimes mind blowing, and for sure gives the developer who has proven their sort or insert function formally correct a sense of satisfaction and accomplishment. What I am still waiting for is some evidence that those techniques do matter when it comes to delivering software in domains more mundane than nuclear plant control software or airplane avionics.

## Performance and Scalability

At the risk of gross oversimplification, one can say that Scala users are primarily interested in either correctness or performance. While the former theme was very prominent at Scala World, the latter was mostly mentioned in passing, when discussing Spark’s “Project Tungstent”, equality in Spire, and the speed of dotc and Scalaz streams. I only recall two talks specifically concerned with performance: one on Akka Streams (which I did not attend), and another on debugging performance, delivered by Konrad Malawski. In summary: do not hand-roll microbenchmarks. JVM is a complex beast and making sure that the performance characteristics of the code while benchmarking are the same as in a production setting is far from trivial. Use JMH instead.