Sunday, September 7, 2008

The Closures are Coming

Last month, Chris Lattner at Apple announced support for closures in C for Clang, the llvm-based C compiler Apple has been working on as a potential replacement for the venerable gcc. Because Apple is working on this extension, it means that closures are coming to both C and Objective-C.

This is a sorely needed feature C has been missing for quite a number of years. The need may not have been pressing two decades ago, in the era of smaller programs on early 32-bit CPUs with kilobytes or maybe a couple of megabytes of RAM. But those days are long behind us, and constructing programs without closures feels clunky, cumbersome, and a throwback to the era of 8-bit home "computers" running some quirky BASIC dialect.

Nearly every major general purpose programming language in use today supports closures or soon will. There are a few exceptions, like PHP, Tcl, Cobol and Fortran that don't have them, and it's debatable whether these languages are either "major", or will get closures "soon". (Update: PHP 5.3 has/will have closures. See below.)

C# has them, and has made them easier to use in the most recent version the language by providing lambda expressions. Java is likely to get them, and the BGGA proposal looks pretty strong. And even if these languages didn't, F# (which is basically OCaml by another name) and Scala are merging the functional and object oriented disciplines on the CLR and JVM, bringing features like closures and currying to these platforms.

The only major language that doesn't have closures is C, and thanks to Chris Lattner and the llvm team, this is changing. Hopefully their changes will be accepted widely, and this feature will be added to the next C standardization.

Adding closures to C introduces exposes a few leaky abstractions, though. Local variables in C (including parameter values) live on the stack, which doesn't do much good if a closure doesn't live long enough to be invoked. Therefore, this change adds a few new features that allow closures to be copied to the heap (via persistent = _Block_copy(stack_closure)), reference counted while on the heap, and freed when no longer in use (via _Block_release(persistent)). There are a few other wrinkles, and the usage is slightly cumbersome, but, well, it's C, and you should be used to that by now.

However, the implications of this change are much more profound.

Adding closures to C is tantamount to saying it is unacceptable for a general purpose programming language to omit support for closures. All of the old arguments for withholding them are now void: it can be done in a compiled language. It can be done without using a virtual machine. It can be done in a mainstream language. Closures are useful beyond the functional ghetto where the started (and offering just pointers to functions simply doesn't cut it).

If closures are available in C, one of the oldest and most widely used languages without closures, what's your excuse for not providing them in your pet language? It may be hard to do, but the techniques are proven, well known, and open for inspection. The utility of providing programmers with the means to use idioms like map, filter and fold (or reduce) is too overwhelming to claim it's not worth the time, cost or complexity to implement.

The writing is on the wall. If you intend to develop software in the 21st century, you will be using closures, or going to great lengths to avoid them.

This may sound like a bold statement, but it really isn't. Programming languages have co-evolved in fits and starts for decades. Some feature, like recursion, proves itself in academia, but is wildly regarded as unnecessary in industry. Slowly, industry adapts, and that feature becomes a necessity in all future programming languages.

Recursion may sound like an esoteric feature to require in a programming language. After all, it's not something that you expect to use every day. But what about lexical scoping? Believe it or not, block scoped variables weren't standard practice until the mid-1980s or so; having only global variables or dynamically scoped variables was once deemed "sufficient" until the consensus formed around lexical scoping as simply the right way to do it. A similar history surrounds object oriented programming, and garbage collection.

What's next? Who knows. The jury is still out on static vs. dynamic programming environments, live code vs. dead code, metaprogramming, macros, virtual machines vs. native compilation, type inferencing, strong static typing, prototypes vs. metaclasses, and a dozen other features where languages differ. Closures, at least, are on their way to becoming a required feature for every important programming language.

8 comments:

Neal Gafter said...

Actually, PHP does have closures now.

Adam Turoff said...

Thanks, Neal.

Apparently, they were added in PHP 5.3 alpha 1. Their status in the language came earlier this year.

Anonymous said...

Can't agree with your assessment of recursion, I'm just a lowly programmer of business apps and use it on a regular basis.

Praki Prakash said...

In the context of closures in languages targeting JVM, if the underlying JVM doesn't support closures, how can the language do it -- with any efficiency?

And the jury will be out there for a long time :) There is no one true way to build software. There are too many diverse requirements and matters of taste. I doubt if I will ever see a clear judgment on static and dynamic languages let alone the other things you mention.

D said...

This is an awful blog post.

1) Your language is vague: talk of leaky abstractions without getting into details.

2) You use red herring arguments. You say that C was more relevant when computers were less powerful, but a huge amount of software is written in C or C++, neither of which have closures, and this software is used today. For instance the Python programming language is written in C.

3) You mistake work on a new language feature (adding closures to C) for a value judgment on the original language (C is "missing" closures and as such is an incomplete language).

4) Incredibly, you argue against diversity in programming languages: you argue that languages without closures are somehow not worthy of the term "programming language"

mfeathers said...

C++ is getting them too.

http://herbsutter.spaces.live.com/Blog/cns!2D4327CC297151BB!785.entry

The syntax is scary, though.

Neil Bartlett said...

Your statement that Java will get closures is actually quite a stretch. Although the BGGA proposal looks strong, nobody expects it to be in the next version of the Java language, due in 2009 or perhaps 2010. There is no JSR (Java Specificiation Request) yet for Java 7 so nobody really knows the timetable, but it looks like we will have to wait for Java 8 to get closures.... maybe in 2012 or 2013?

This means we will have a world where Java is the only popular language without some form of first class functions. Even C will have closures... what an embarassment!

Of course, we have Scala for the JVM, and there's a compiler for Java+BGGA already. Probably more and more people will move away from the official Java language as it gets increasingly obsolete.

George Talbot said...

D: Don't be cranky.

Interesting post. I've noticed in C++ I've been using the pattern of using functors as closures quite a bit, without realizing I was using closures. The syntactic sugar that folks are proposing for C++ are intriguing in this respect, and would frankly save me quite a bit of hand-coding.

Binding environment and code anonymously and using it is more important than one might realize, and I agree with Adam's sentiment that a language without a way to do that in one form or another is really missing something.