Closures for Java, is it really a good idea?
June 30th, 2008 at 7:17 pm by Jesper de JongThere’s a very good article on JavaWorld by Klaus Kreft and Angelika Langer (the latter is the author of the well-known Java Generics FAQ) with the title Understanding the closures debate. I heard about this article on JavaPosse episode 194, and it’s also being discussed on Artima.
I knew that there were proposals to add closures to Java, but I hadn’t yet looked into the details of the proposals. The article explains very clearly what closures are exactly and gives a good overview of the three main proposals that there are today: CICE + ARM, BGGA and FCM + JCA. Read the article if you want to know what the alphabet soup means exactly. In short:
- CICE + ARM, proposed by Joshua Bloch and others, is not much more than a new, shorter syntax for anonymous inner classes.
- BGGA, proposed by Gilad Bracha, Neal Gafter, James Gosling, and Peter von der Ahé, is the most ambitious proposal and almost makes Java a functional programming language.
- FCM, proposed by Stephen Colebourne and Stefan Schulz, also adds functional programming paradigms but doesn’t go as far as BGGA in some aspects.
What’s so controversial about adding closures to Java? Adding any new feature to the programming language makes it more complex. There are people who fear that adding a powerful but complicated feature such as closures might make Java too complex - which means that people will turn away from Java in favour of easier programming languages. This talk by Joshua Bloch from JavaPolis 2007 makes clear how complicated Java could become if BGGA closures would be added.
So, the question is, is it really such as good idea to add closures to the Java programming language? I’m not sure that it is worth the trouble. After reading the article and seeing Bloch’s talk, I’m convinced that adding far-reaching closures such as BGGA and FCM will not be so great. What I especially don’t like in the BGGA proposal are the non-local returns: a return statement in a closure means you’re returning from the method that the closure is called in, not from the closure itself. This makes code very confusing and hard to understand.
I can see how CICE closures would be useful and at least it looks like that proposal is a lot simpler than the other two. But regardless of how much complexity closures would add to Java, there are reasons for me to doubt whether it’s a good idea at all to add closures to Java.
Java is an imperative programming language. Because of this, Java programmers generally use an imperative programming style. For example, we write a loop like this:
List<String> names = ...;
for (String name : names) {
System.out.println(name);
}
Adding closures will encourage a functional programming style, which is completely different from the imperative style. In the functional style, you’d write a loop very differently; you’d call a forEach method on the collection, passing it a closure to execute on each element of the collection (this is what it would look like with BGGA closures):
List<String> names = ...;
names.forEach({ String name => System.out.println(name); });
What are the consequences of this? Does this mean that for example there are going to be lots of new methods (such as the forEach() method shown above) in Java’s collections API to support this style of programming? That would bloat the collections API. Can you imagine what a mess you would get if you’re working on a project with multiple developers where some developers prefer the imperative style and others the functional style?
I’m not opposed to the concept of closures itself, they work very well in other programming languages such as Ruby. The difference is, however, that Ruby has had closures since the beginning, so that Ruby’s standard library was designed to work with closures from the start. If (part of) the standard Java library, such as the collections API, which was designed for the imperative programming style, is going to be extended to add support for closures, it will just grow into a complicated and bloated piece of patchwork.
The cost of adding a new feature to the programming language is very high. Millions of Java developers around the world will need to invest time and money to learn the new feature. The authors of the thousands of libraries and frameworks will need to think about the consequences for their products. And closures is not just another minor feature, it will add a whole new programming paradigm to Java. Are closures really worth all that effort? So far, I haven’t heard any convincing reason why it would be a good idea to add closures to Java. Sure, they are a cool and powerful feature. But how are closures going to make things easier for me as a Java programmer? I’m not convinced that they are worth the extra complexity and API bloat that they will add to Java.
Popularity: 1197 points
June 30th, 2008 at 8:29 pm
Two primary arguments are: (a) it would simplify APIs where resources must be cleaned up, such as IO, transactions etc and (b) they may simplify the fork-join API (JSR 166). The latter is not often stated but big: anything that helps simplify the use of multicores is worth considering, no?
It is up to you to decide if these are _convincing_ arguments. Absolutely, any closure proposal will have drawbacks and a learning curve, but there are undoubtedly positive aspects as well.
re: Ruby/beginning. Java is adaptive. Look at the AWT and then the correction (Swing). As is well-documented, anon inner classes were a half-way answer to closures way back in the beginning. It can be argued that it is only natural that Java continue to evolve and “close the loop” on the “hole” in the language.
That said, there are legit concerns as well, as you mention. It is a tricky, subtle debate that will ultimately rest with one’s priorities.
July 1st, 2008 at 5:43 am
You’re misinformed about the effect the closures proposal would have on the APIs. No changes are required or needed to the java.util Collections framework. Although it is true that some languages with closures have such methods, that does not imply that it is a requirement for a language with closures. There are many other ways in which closures are beneficial, and other ways in which they interoperate with collections.
July 1st, 2008 at 8:23 am
Interesting. It’s a lot like the transition from C to C++. That proved very popular in the short term, but it turned out that you just don’t need to do your low-level and high-level programming in the same language. Likewise maybe most people should be making a clean break to a modern language, but in a sense, Java is to the JVM and its libraries as C is to non-virtual machines. There are other languages available on the platform but they’ll “always” be second-class citizens to some extent. And if that ties you to Java for the forseeable future, the lazy (shortsighted? or just easiest to justify to management?) approach is to get the features you want shoehorned into Java as best you can.
July 1st, 2008 at 9:47 am
Neal: I understand that it’s not absolutely *necessary* to add functional programming style methods such as forEach() to the Java API. But closures do promote a functional programming style, and people are going to want these kind of methods when closures would be added to Java.
July 1st, 2008 at 10:28 am
The examples mentioned in the article on JavaWorld are indeed good candidates for closures, on the other hand we seem to be trained to use the workarounds (innerclasses, aspectj or alternative algorithms).
I’m not sure if adding any of the variants would do much good.
If closures would be added; People new to Java will eventually find different variants online and probably get confused on which variant should be used and in some situation. That is… after they finaly grasped the syntactical sugar. And using multiple styles(with and without closures) for the same purpose in one application would not benefit the maintainability I guess.
On the other hand, we had a similar situation when generics were added. Although this was only new syntax, not a new control flow. And even the succes of this change is still under debate.
So I would agree with you Jesper that they should not be added to Java. If one wants to do functional programming, even the addition of closures will probably not suffice.
July 1st, 2008 at 4:26 pm
Actually, foreach is not at all a functional construct. It’s just a normal imperative/OO thing you might want to do with a list. There is an advantage in implementation of having foreach a method instead of a language construct - Java’s foreach depends on your ability to write an external iterator, the difficulty of which has been blogged by Neal Gafter.
Unless you introduce more language features to make that convenient, as C# did (quite elegantly) with yield return.
A functional thing you’d want on a list or any other container type would be a filter, a map, or their grandaddy, foldLeft.
listOfStrings.foldLeft(0, {int acc, String s => Math.max(acc, s.length()) }) would give you the maximum of the lengths of all the Strings in a list. Every single for loop you can think of can be written using foldLeft or its children, and I haven’t come across a case that’s best as a for loop yet.
Closures don’t make a language functional. Not even function types do. A functional language is one where calling the same function with the same inputs always yields the same outputs. Java will never be that, though you can improve your code by programming in a subset of Java that is.
July 1st, 2008 at 5:24 pm
A while ago (at least a year), I read a debate on the BGGA proposal discussing the difference between “return” and “return;” (note the semicolon). One person said something to the effect that “it’s obvious, because of the presence of the semicolon.” IMHO, if the omission of the omnipresent end-of-statement token changes the meaning drastically, the code is going to be in for a world of hurt.
I look at BGGA, and I want to hurl because it tries to do too many things at once. As Bloch pointed out, CICE+ARM is a much better framework for doing what BGGA primarily seeks to accomplish.
As far as I can tell, these are the problems that we want to solve with closures:
* A simple one-function callback (e.g., the argument to invokeLater) takes a lot of syntax to complete.
* We want that |using| construct or other pet construct.
The former is handled quite nicely with pretty syntax (IMHO) with CICE; the latter by ARM. And neither requires mucky workings under the hood.
But how does BGGA solve it? Add function pointers, the syntax of which is ugly, and I foresee worse documentation as a result (a benefit of inner classes that few admit is that they are easy to document, while function pointers require a bit more work to do so). The under-the-hood-really-compiler-hacks are scary. I know Bloch pointed out some problems with auto{un}boxing (I don’t know how relevant they still are). Non-local returns in closures blow things up considerably in complexity for very little benefit.
In short, you take two wanted features, try to solve both of them at once, and at the same time attempt to satiate the “perl/python/ruby is GOD!” crowd and end up with a bloated carrion that tries to solve problems I’m not really sure exist and puts Java firmly down the path of C++ whereby it ceases to become a language in favor of a cacophony of different styles masquerading as a single language.
@Ricky:
“listOfStrings.foldLeft(0, {int acc, String s => Math.max(acc, s.length()) }) would give you the maximum of the lengths of all the Strings in a list. Every single for loop you can think of can be written using foldLeft or its children, and I haven’t come across a case that’s best as a for loop yet.”
for (Node n : graph.getBFSView()) {
block = blockMap.get(n);
for (Node child : n.getChildren())
AnalysisBlock.makeChild(block,
blockMap.get(child));
}
I don’t really see how a foldLeft or map or filter can make that better than as it is as a for loop.
July 1st, 2008 at 5:53 pm
I would go with BGGA or nothing at all. CICE & ARM are not nearly aggressive enough.
July 1st, 2008 at 8:17 pm
re: non-local return/transfers.
To be fair, note that BGGA now has the option of restricted closures which disallow ‘return’ and other transfers. The proposal has evolved, in response to Bloch and others.
I’m not a language expert, but from what I have read, Tennent’s Correspondence Principle is well-respected guideline that basically says an expression in a closure should have the same meaning as it did before it was wrapped. This is the motivation for non-local return (see Neal’s blog). I believe the argument is that if one does _not_ follow TCP then history has shown that things can get truly hairy.
This may or may not be enough for readers, but it is wise to know the underlying ideas.
re: function type syntax. It’s interesting how it looks harsh in Java and yet ‘exotic’ if it were something like Scala. It gets better with time.
July 2nd, 2008 at 7:08 am
People seem to focus on how it might make the language more complex, but let’s be clear… people are *leaving* Java in droves; you are losing them! Why is the language not growing as the community grows?
To even do nothing is to lose (!!!!), as people are waking up to the facts like: 100+ line single-method anonymous inner classes, ridiculous separation between primitive and Object types, try-catch blocks that you must repeat over and over without a closure to pass to the inner part of these blocks, terrible media support, triple-redundancy typing information on declaration with initialization, and I could go on….
Java could be improved; I fear it will end up like Delphi one day — innovative but no longer popular.
I don’t really understand this argument — if there is room to improve, why would we not improve it!? Backwards compatibility breakage will already happen, since the same people you’re saving backwards compatibility for will end up switching to a new language and breaking it anyway!
The sky isn’t exactly falling, but for Java, it is certainly getting lower.
July 3rd, 2008 at 3:06 am
I have suggested a more ‘aggressive’ version of CICE, C3S (http://www.artima.com/weblogs/viewpost.jsp?thread=182412). It uses type inference to shorten the syntax further and makes braces optional. But the proposal uses inner classes rather than introducing the new, and less powerful, concept of a closure.
The closure require boxing into objects to interact, there is no auto-unboxing, they can only box into an interface with a single method (no classes, no multiple methods), no inheritance, no fields, no partial implementations. A closure is in a strict mathematical sense a subset of an inner class. The closure has a this pointer back to the enclosing class, an inner class has two this pointers one to the inherited class and one to the enclosing class. Hence Closure = Poor Man’s Inner Class.
As the BGGA proposals stands many existing APIs would require modifying to conform to the above restrictions. Also in many cases you will need to change interfaces to extend RestrictedClosure so that the error of passing a non-restricted closure is caught. There are two types of closure in BGGA, the restricted closure uses => and the non-restricted closure uses ==>.
The BGGA proposal doesn’t conform to Tennant. Tennant himself pointed out that return, break, and continue are incompatible with his principle. Yes I know proponents of BGGA say it conforms but consider:
int f() { return 1; }
No wrap this using BGGA:
int f() {
{ () => return 1; }.invoke();
// Syntax error here, f does not return an int
}
Hence BGGA, as all languages with a return statement do, fails Tennant.
Lets turn the question round another way, what can you do with a closure that you couldn’t do with an inner class with short sytax. I am not against improving inner classes but I think it would be a mistake to introduce another thing, that like a primitive, is not an object and requires boxing.
July 6th, 2008 at 8:53 pm
Closures are the least of Java’s worries.
July 22nd, 2008 at 2:34 am
Hi All!
There are several discussion around like this one, talking about what went wrong in Java and what is to be made wrong with Java next. I suggest let us try to look at the core of the problem, not at cosmetics to be put on. IMHO the core of the problem is that way before the Java and, actually even before C this world did already exist. And there were plenty of languages at that time already (more than 200) and plenty experience of had been accumulated. Then, and we can not change that anyway, the vortex of life went towards enormous C popularity. Which is a pretty nice but primitive language. At this point a lot of experience was thrown away and, as I see, much forgotten. C developed into C++. Truly not on its own but incorporating some really good ideas developed earlier, like object orientation. Some other same good already researched and confirmed by successful implementations ideas were drooped.
In relation to closures: C++ did not add anything new to manipulation with functions compared with C. Yes, assignment of pointers to external functions in both. When Java emerged from C++ even that was lost. So what did exist on this Earth in 1960s and does not in 2008? The brief list of features of procedures we knew:
1) Procedures could be declared inside other procedures.
2) Procedures could be passed as parameters
3) Procedures could be returned as other procedures result
4) Procedures could be assigned.
The most generalized form of procedures plainly meant that procedure body is procedural type is one of the basic types of the language. Did not that cover closures? The best example of such a language is ALGOL 68 (for those who never head of it - not 89).
If we for a moment take that language as a starting point and apply (of course!) all we got to know since then, then we may imaging the following picture: Yes, classes are added. But now methods are just field values. Constant if marked as final, but changeable if not.
Sub-classing, which currently implicitly does it anyway is equivalent to assigning a procedural field. At the class level you simply assign:
MyClass = BaseClass.method1 = void(int x){ … }. But same me be applied to a single instance!: myInstance.method1 = void(int x){ … }. Note that ALL the existing Java things stay as they are.
There is no need in closures, as ANY procedural value is the one. Of course, return remains untouched.
The C# delegate already with us and in less clumsy form. It is simply myInstance += void(int x){ … }.
To build a reliable building we have to see first if all the necessary cornerstones are present. If not one should be added to the foundation. In our case it is about procedural values.
July 22nd, 2008 at 2:37 am
Sorry, typo: MyClass = BaseClass.method1 + void(int x){