Scala in North Lakes

A Scala conference, far from any significant technology hub? With no call for papers? With hikes planned in what is one of the wettest parts of the country? Without Martin Odersky? What could possibly go wrong? A lot. But, apart from a few of the speakers not making it, nothing did, and Scala World was probably as good as a Scala conference can get.

Instead of summarising individual talks, I will focus on the major themes. The usual disclaimers apply: it is a personal impression based only on the talks I saw, I might not have selected the best and most representative ones, and in general I understood very little of what was being said.

Functional Programming

The conference schedule made it clear that functional programming will be the leading theme at Scala World. It was the subject of the keynotes on both days. Rúnar Bjarnason presented a philosophical argument on how the least powerful construct gives us most power of reasoning in a way that was digestible to those moderately versed in FP, finishing with a concept of Galois connection. Daniel Spiewak dove right into the algebra of Scalaz Stream, the shortcomings of existing one and promises of the new implementation, \(FS^2\).

Practical functional architecture, presented by Raúl Raja, offered Kleisli arrows for composition, free monad for interpretation and delimited, monadic, dependently-typed, accumulating, checked exceptions for fault tolerance. John de Goes explained how free appliatives, being less powerful than free monads, are desirable where dependency on previous step is not required, i.e. where the program builds a static data structure, not another program. What we get in return is the ability to introspect on the structure before runtime. In the process, we saw how the sausage was being made (with a natural transformation), and boy, it was not a pretty sight. John offered a nicer solution based on finally tagless representation, which perhaps was a way of encoding GADTs in Scala, but I’m not sure – that bit was over my head. Anyway, even that effort was heckled by Daniel Spiewak, who insisted he could do it in half a line of code with Coyoneda. Contrasting with those fairly advanced talks, Chris Marshall’s workshop on error handling in Scalaz, which ended up with a monad transformer stack that offered Reader, Writer, Either and IO, was quite approachable to someone vaguely familiar with the subject – if not particularly enlightening.

If that all seems hermetic to you, you are not alone. Hard-core FP community sometimes does appear like a closed club talking in strange language, whose chief purpose is to make the outsiders feel stupid and inadequate. Changing this perception was at the heart of Erik Osheim’s talk. Erik is the founder of Cats library, which positions itself as a friendler (both to newcomers and to seasoned functional programmers) alternative to Scalaz. In contrast to Scalaz, Cats treats comprehensive documentation and supportive community as first-order concerns.

Compiler Hacking

The sessions devoted to compilation of scala can be divided in two groups: the messy present and the bright future. In the former, we had an intro to scalac structure delivered by Chris Birchall, complete with a demo of compiler plugin that transpiles Scala (presumably, a subset) to Basic, and Simon Ochsenreither’s series of nicely structured tutorials on scalac bug fixing. The procedure always starts with text search for relevant terms in the entire compiler codebase, then hacking away until it compiles, finishing with a test case that demonstrates our fix or feature works as intended. Be prepared for long waits while your changes to the compiler are being rebuilt.

It is telling that new tools for working with Scala code and experimenting with the language tend to start from scratch rather than extend the infrastructure provided by scalac. It can hardly be construed as a glowing endorsement of the existing compiler’s codebase. If even Martin Odersky, the original author of the compiler, and Eugene Burmako, the person who gave us macros, move away from scalac when working on new Scala-related projects, one cannot help but share Paul Phillips’s opinion that scalac is doomed. That said, the new compiler, dotc, shows promise. Dmitry Petrashko, as of now the most prolific contributor to the new compiler’s codebase, discussed ways in which dotc is better than the predecessor: a number of smaller cakes instead of single giant one (?), global mutable state replaced by context object passed between phases, handling of symbol denotations that preserves the full history of a symbol, and mini-phases replacing the old maxi-phases. This last improvement should, to some extent, address the main gripe of many Scala developers: compilation speed. While scalac phases transform the entire AST, generating lots of garbage in old generation in the process, mini-phases work on the level of individual nodes, offering a better memory locality and shorter-lived objects. There is also hope for the elephant in the room, binary compatibility, in the form of typed trees that are saved in .class files and allow dotc to generate bytecode after the library is deployed. This feature also opens up exciting possibilities of whole-program optimisation after deployment, such as removal of virtual calls and limited automatic specialisation.

The dialect of Scala supported by dotc is actually a slightly different language, called Dotty. Those of us with substantial existing Scala codebases need not feel left behind: help is (about to be) at hand in the form of scala.meta. This new project from Eugene Burmako purports to provide a faithful representation of a program in a chosen dialect of Scala (2.11, 2.12 etc.) or Dotty. That AST representation can then by used for translating the program to a different dialect and emitting source code with even minor details, for example formatting, preserved. The ambition of scala.meta goes beyond syntactic manipulation; however, re-implementing Scala type checking outside of scalac turned out to be a monumental task. At present the project offers limited support for semantic information by calling out to scalac.

Formal Methods

Those not satisfied with the assurances offered by run-of-the-mill FP devices, such as isolating effects and handling errors with monad transformer stacks, were treated to a preview of other techniques for making our software reliable. Sleep-depraved Jon Pretty showed CLI and internationalisation API of his Rapture libraries, where types were used to specify the required structure of command line arguments and the set of supported languages. While the usefulness of being able to express in the type that a string has English and French version, or require that both English and German messages must be provided to a function, is debatable, the possibilities offered by Scala’s type system, and Jon’s ingenuity in exploiting them, seem immense.

Getting the machine to write the program for us never fails to impress. On top of that, Edwin Brady is a very entertaining speaker, so his Idris demo, where the expressiveness of dependent types allowed the Idris runtime to infer the implementation, and the practicality was demonstrated by Space Invaders clone, met with well deserved cheers and applause. Lars Hupel led another excursion beyond Scala-land, into the realm of Isabelle proof assistant, to finally return to home soil by mentioning Leon – a proof assistant that supports a subset of Scala. Similarly, Paul Snively ran a workshop on formal logic in Coq.

It all appeared very elegant, sometimes mind blowing, and for sure gives the developer who has proven their sort or insert function formally correct a sense of satisfaction and accomplishment. What I am still waiting for is some evidence that those techniques do matter when it comes to delivering software in domains more mundane than nuclear plant control software or airplane avionics.

Performance and Scalability

At the risk of gross oversimplification, one can say that Scala users are primarily interested in either correctness or performance. While the former theme was very prominent at Scala World, the latter was mostly mentioned in passing, when discussing Spark’s “Project Tungstent”, equality in Spire, and the speed of dotc and Scalaz streams. I only recall two talks specifically concerned with performance: one on Akka Streams (which I did not attend), and another on debugging performance, delivered by Konrad Malawski. In summary: do not hand-roll microbenchmarks. JVM is a complex beast and making sure that the performance characteristics of the code while benchmarking are the same as in a production setting is far from trivial. Use JMH instead.

Advocacy

One of the major themes at Scala conferences, experience reports from the industry and language advocacy, was notable for its conspicuous absence. That made sense, since anyone attempting to convince Scala World attendees of the benefits of trying out Scala in place of Java, Python or any other lesser language, would be preaching to the choir. The only talk that touched upon the subject of “why Scala?” was Dean Wampler’s overview of new in the Spark-land.

* * *

Scala World 2015 was a brainchild of Jon Pretty, who came up with the formula, selected location, hand-picked speakers, organised hikes (including transport to and from), food, dinner, and ensured a nice touch to every aspect of the conference. To start with, both Saturday’s walk in Central Fells and Sunday’s half day hike up and down Helvellyn offered some spectacular views and variety and were paced exactly so that everyone could complete them in the time allotted. Booking Paul Phillips as the entertainer for conference dinner (excellent) was a master stroke. Last but not least, Rheged Centre, where the conference took place, is a beautiful building, resembling a grassy mound, with some of the rooms offering views over the pastures and rolling hills of Cumbrian countryside, and hosting a collection of shops with exquisitely designed items. I genuinely cannot think of a nicer location for a conference, whatever the subject.

Will I be coming back? Unlikely.

That has nothing to do with the organisation (faultless), location (beautiful and reasonably accessible), attendees (friendly, although quite homogeneous). It’s the subject: Scala. Over the years of participation in various conferences, I have realised that language-centric ones tend to generate a bubble where said language, ultimately just one of many tools, takes a central stage, and other software development concerns are forgotten. It is the concepts, techniques and all-encompassing approaches to building software that are what I personally would prefer to focus on. Scala is neither the best vehicle for presenting advanced FP concepts (Haskell being the lingua franca), nor for type-level programming (languages with first-class types, such as Idris, are more suitable), and once that is taken away, then what we are left with is a compromise: a tool, that with quite a bit of effort allows us to apply the advanced programming techniques and run the code on JVM. It is perhaps a sensible thing to do once one is comfortable with the underlying concepts. I’m not quite there yet, so I will skip the “FP in Scala” type of events until I have learned enough to appreciate and accept that the price paid in bending over backwards to get a type lambda is justified by the engineering properties we thus obtain.