Javascript Callback Hell, how to solve it?

If you use node.js or some other similar technology, based upon some react-to-events paradigm, you are almost certainly paying a TAX over your developments, the so-called Callback Hell. Today we will se how to solve Javascript Callback Hell. 

Specifically, it’s a TAX you pay every time you want to separate concerns and give some more layer of indirection, such as when you apply some basic DDD layering. DAOs, Domain Services, Infrastructure Services, and stuff like that.

You want to add some details-hiding interface, and you end up adding steps in the arrow of callback.

Otherwise you are paying a “Code Duplication TAX” which is much bigger, to which the javascript Callback TAX is insignificant. But this is a matter for another post.

Another bad side-effect of the Callback TAX is that you have to spread callbacks all over the application, just to be prepared when an implementation goes from sync to async. For instance, say you can have some DAO that is hiding an in-memory, sync I/O, object cache. If you don’t prepare it for async flow of execution, the migration to an on-disk persistence propagates through all the upstanding layers of indirection.

Javascript Callback Hell: solved

ASYNC

Async is a library that provides a mean to “verticalize” arrow code:

 <img class="alignnone wp-image-1318 size-full" src="/wp-content/uploads/2016/01/Screen-Shot-2016-01-09-at-02.20.19.png" alt="javascript callback" width="1356" height="850"> 

The main problem with async is that the resulting code is complex. Yes, it’s very “functional style”, it respect the “lambda” aspect of javascript architecture, it allows collaborative “parallel” executions. And.. it’s ugly. Avoiding nested async is an investment in complexity, lowers readability, and the result is that programmer is more focused on how to avoid “arrow code” than putting his/her effort in proper, encapsulated design.

PROMISES

The other solution, common in many technologies, are Promises, Futures, Promises/A+, etc.

Let’s see Q promises, a very good implementation that permit chaining, with something like this:

 <img class="alignnone wp-image-1316 size-full" src="/wp-content/uploads/2016/01/Screen-Shot-2016-01-09-at-02.13.09.png" alt="javascript callback" width="1364" height="914"> 

It appears to be a better solution than async. Cleaner, simpler. But also tightly coupled. I have to change all the contracts of my classes, not only the implementations, to use that library of promises. Try to imagine the cost of changing my promises library to another one. That’s why the Promises/A+ specification. From this point of view, Promises are a worst solution than async.

But both libraries (async and promises) have an added, deeper, hidden cost that can be dramatic, because of the strong coupling: testability.

There are some more options, like an extensive use of nested closures, or the use of reactive programming extensions, but I think they are solution to different problems; in fact, I think you should not be forced to use them because of the technology you are programming. And in addition, they could be impractical for everyday programming.

The same is true, IMHO, for a massive use of “Tell, Don’t Ask” as an architectural style, like this one; indeed, I see it almost utopical in the vast majority of the codebases I know, and it’s not so clear that is always the best solution, though it is in a world of pure theory. See for example the related article of Martin Fowler.

Another point of view for javascript callback

All these solution make big efforts to make the asynchronicity of node.js code easier to understand, to “mitigate” the problem at the same time being respectful to the inner nature of node.js reactor.

My proposal goes in another direction: is it really important for a programmer to repeat the javascript callback syntaxis over and over again, only because it’s “node.js programming”? There’s some point in that, some useful side-effect I cannot see? This comes with a cost, that I named here the “Callback TAX”. OR the reactivity of the underlying technology should tend to be nothing more than an implementation detail, like many others? What happens if I want to hide the asynchronous nature of an operation behind a common interface, some kind of “syntaxis encapsulation”?

Let’s give a “theoretical framework” to properly name the problem. It’s perfectly feasible to imagine a “compiler” doing the following, from:

</p>
<p><img class="alignnone wp-image-1319 size-full" src="/wp-content/uploads/2016/01/Screen-Shot-2016-01-09-at-02.13.18-1.png" alt="javascript callback" width="1360" height="190"></p>
<p>

 <img class="alignnone wp-image-1320 size-full" src="/wp-content/uploads/2016/01/Screen-Shot-2016-01-09-at-02.13.24.png" alt="javascript callback" width="1364" height="66"></p>
<p>

 <img class="alignnone wp-image-1321 size-full" src="/wp-content/uploads/2016/01/Screen-Shot-2016-01-09-at-02.13.31.png" alt="javascript callback" width="1362" height="68"> 

So we can say this is a mere problem of syntaxis, namely, syntactic noise. If we could solve the syntactical problem, the code in its whole would be perfectly portable between sync and async technology or implementations. In fact, in general we can state that every sync code can be written in async style. The contrary is not true.

Syntaxis encapsulation would give me the chance of changing from a sync implementation to an async one, for instance, inside a DAO class, without changing anything of the client code. Not even the syntaxis.

Besides, in the specific case of node.js, I could completely avoid huge (syntactical) problem in testability, and the cost of a bigger codebase.

A logical framework for the solution

Now, I got the bigger part of the work done: identifying a logical framework in which to achieve an optimal solution. From here it’s only a matter of finding the technology/library/framework nearest to the solution I propose.

Many technologies have a standard solution that goes in this direction. Think for example of C# async/await, Java/Scala continuations, and some Coffee Script solution (IcedCoffeeScript).

In node.js pure javascript style, I found a good solution in a library called wait.for, based upon fiber. An example:

Obviously, it is non-blocking.

No javascript callback syntax, no contract dependencies between client code and server code implementation details, above the logical contract. Sounds like encapsulation. As you can see, it respects all the constraints of our “theoretical compiler” but the wait.for construct, which it the only coupling (compare it with promises!!); furthermore, it can and should be encapsulated behind some adapter. If this is a good idea, it won’t be necessary to spend more time in implementing and understanding solutions like promises or futures. Until the time of writing, this extension shown no problems at all to me, and gave me back the supremacy of design over implementation in node.js without the big associated TAX I was paying. Remind that we are talking of codebase size and evolution, and its cost of change, which is not hot air, but true loss of productivity 🙂