My three–monthly ritual, or the road to OCaml

My journey to functional programming #

I’m starting to write again, and my writings are likely to be very much on a theme - OCaml (and other FP languages), and this inaugural entry explains why they fascinate me.

I’ve worked as a software engineer, and latterly a systems architect, for the last decade or so. In the process I’ve used many, many languages - from my hobbyist days with Pascal, Basic, C/C++, through to PHP, C#.NET, Java, Python, Javascript, Go, and a little Ruby. I even worked almost exclusive on mobile apps, when that was a thing, with Objective C and Java on Android. Perhaps it’s unusual to have worked with so many technologies, but there’s perhaps one language that changed my viewpoint on everything, clichéd as it is – Lisp.

More specifically, I was introduced to Clojure. I’d heard about functional programming, and felt I’d dabbled a bit using Linq in .NET. I’d tried playing around in Scala around 2010, but the steep learning curve and horrendous compile times on my ageing PowerPC iBook G4 seemed to outweigh any esoteric benefits gained from so-called functional programming. Clojure, however, introduced me to so many things I had never seen:

I still remember the moment my jaw dropped the first time I hooked up a REPL via Clojurescript and Figwheel and I was able to manipulate my application running in the browser in realtime from my shell, and it all felt like the most natural thing in the world. The development cycle was no longer a full stop-recompile-reload, but a single continuous process. Macros introduced me to a style of meta-programming I’d never thought possible. From knowing no Clojure or Lisp, I bashed out a full prototype of a complex and rich single-page app in a two week stint that would’ve taken me much longer using any of the technologies with which I was previously familiar.

Ultimately, I moved away from Clojure, as for various reasons at work we decided not to pursue it. LISP doesn’t appeal to everyone (parenthesis fatigue), but my personal experience of it left a deep impression.

After that, I went full-in on a personal exploration of functional programming. Ultimately, I discovered OCaml (and relatedly F#, and Elm). While not as minimal as Lisp, the (inferred) static typing was a revelation, and they introduced me to a couple of language essentials that I miss greatly in languages that don’t have them:

OCaml is also pretty fast! Applications compile very quickly, and performance-wise it’s within an order of magnitude of C, which for a garbage-collected high-level language is respectable and frankly, it’s easily good enough for an awful lot of use cases.

I quickly found myself reaching for OCaml whenever I wanted to quickly prototype an idea. OCaml has an excellent REPL called Utop, and quite often I can work on some data structures directly in that without even opening an editor and get a feel for the problem I’m working on. The type system is flexible and powerful and makes it easy to solve complex problems with static guarantees. It seemed like the pinnacle of everything I’d be looking for.

The road to using it professionally #

I really wanted to use OCaml (or its .NET sister, F#) professionally. Unfortunately, OCaml has a fair few well-known sticking points.

In F# land, the tooling is generally better, the standard library is very good (you have access to the whole of .NET, and interop with C# is mostly okay), and multicore is not a problem, but it’s definitely a second-class citizen in the .NET world, the third-party library ecosystem is not so strong, and it doesn’t have a macro system as flexible as OCaml’s PPX (Type Providers, though, are a fascinating idea. The killer for me, at the time, was that it didn’t work so well on non-Windows platforms. .NET Core has changed this to a certain degree.

Personally, I’ve used OCaml for a lot of stuff, including my own Hindley-Milner typed-checked ML style language that I got to cross-compile to Lua and Erlang. OCaml particularly excels at this sort of thing. Half the time it felt that once I’d gotten my data structures down, the accompanying code almost wrote itself.

Thinking that I could bring the same joy to my professional work, since 2016, every three months I’ve checked out the latest situation with OCaml and F# and tried to see if I can set up a simple app that can do the following:

Not a large ask, and undoubtedly has been possible for a long time, but my idea was to sell it to others as well as to myself. F# has Postgres drivers, but the type providers I’ve used (when I could get them to compile at all on Linux…) would tend to bork at any advanced features of Postgres such as custom types. So mostly I’ve concentrated on OCaml.

Every three months I would try the same thing, and perhaps would even succeed, but find the result lacking, the effort in getting it all working too hard, or I’d hit roadblocks such as out-of-date libraries.

I can imagine that if you are large organisation going all-in with OCaml like Jane Street or Bloomberg, the raw ingredients of OCaml are such that you can tailor it to your use-case and build the surrounding tooling, libraries and documentation yourself to make it worthwhile.

But as of now, so much has happened over the past few years that have improved the situation in OCaml greatly.

I’ve been following projects such as PGOcaml with great excitement, as with the PPX macro extension it allows embedding SQL in a type-safe way in your application. It had languished for some years but a recent flurry of activity has modernised it and, while it is still lacking in a few areas, makes it very serviceable for use. Lwt, a cooperative threading library, makes using it much easier with a simple PPX extension. Library providers solved the great Async/Lwt divide (and other such fragmentation in the ecosystem) by functorising their libraries, so a library such as PGOcaml can easily be made to work with Async, Lwt, or even (the default) OCaml’s built-in system threads. (There will be more on functors in later posts – OCaml’s module system is one of its most powerful properties).

So all-in-all, OCaml seems increasingly viable for my use case.

What’s next #

I really want to invest in this ecosystem because the raw constituent parts of OCaml have such potential to make programming a cleaner, faster and more joyful pursuit. I fundamentally believe that where tools can make your life easier, they should – and good type systems, ADTs, pattern matching, powerful module systems, all go very far in achieving this, but it’s all what I call the ‘meta stuff’ that is lacking with OCaml – size of community, quality of documentation / tutorials, and advocates. So I want to be a part of improving that situation, and I’ll be writing on here about it as I go.

So, my three-monthly ritual has come to an end. I doubt I’ll be able to use OCaml professionally soon (I work a lot in Elixir[2] and Rust these days, which cover some of the space that OCaml does but do much better with concurrency and performance/safety respectively). But I really think longer term OCaml could become highly competitive in the start-up space where rapid, joyful, high quality software engineering could provide a real edge. The rituals has ended, and now the work begins.

[1] One counterpoint to this is that you often encounter libraries of such high quality that even if they haven’t seen update in years, they still work flawlessly.

[2] Elixir and Erlang are also something I’m passionate about and I’m likely to reference in this blog. Elixir really demonstrates how the ‘meta-stuff’ can really accelerate language adoption.

 
6
Kudos
 
6
Kudos