Closures in Ruby and Python
Edit: This post has come up in a recent (2008-04-27) post on reddit. For an account of where the term, closure, came from look here.
While reading the the Django/RoR comparison by Sam over at magpiebrain the "Ruby has closures, Python doesn't" argument was mentioned. I admit to not being really familiar with the concept, but I believe I understand what it is. The thing is, I don't see what's important about it.
A link was given in the post to an interview with Yukihiro Matsumoto where "Matz" is asked about what a closure is and how it's beneficial. Here's the most relevant portion of the interview:
I respect cool implementations of higher-order concepts as much as the next guy, but I can't help but feel cheated by that response. I've seen some bickering between Python and Ruby clerics that try to demonstrate why closures are important and how Python doesn't have them; nothing I saw gave a practical example.
Anyone have any?
Edit: I may have found a practical example, but I'm not sure what's so special about it.
Here's a wiki entry about the benefits of closures in Ruby and here are the Pythonized versions that work the same way. This is just a special case of closures, I'm still looking for something completely different.
While reading the the Django/RoR comparison by Sam over at magpiebrain the "Ruby has closures, Python doesn't" argument was mentioned. I admit to not being really familiar with the concept, but I believe I understand what it is. The thing is, I don't see what's important about it.
A link was given in the post to an interview with Yukihiro Matsumoto where "Matz" is asked about what a closure is and how it's beneficial. Here's the most relevant portion of the interview:
Bill Venners: OK, but what is the benefit of having the context? The distinction that makes Ruby's closure a real closure is that it captures the context, the local variables and so on. What benefit do I get from having the context in addition to the code that I don't get by just being able to pass a chunk of code around as an object?
Yukihiro Matsumoto: Actually, to tell the truth, the first reason is to respect the history of Lisp. Lisp provided real closures, and I wanted to follow that.
Bill Venners: One difference I can see is that data is actually shared between the closure objects and the method. I imagine I could always pass any needed context data into a regular, non-closure, block as parameters, but then the block would just have a copy of the context, not the real thing. It's not sharing the context. Sharing is what's going on in a closure that's different from a plain old function object.
Yukihiro Matsumoto: Yes, and that sharing allows you to do some interesting code demos, but I think it's not that useful in the daily lives of programmers. It doesn't matter that much. The plain copy, like it's done in Java's inner classes for example, works in most cases. But in Ruby closures, I wanted to respect the Lisp culture.
I respect cool implementations of higher-order concepts as much as the next guy, but I can't help but feel cheated by that response. I've seen some bickering between Python and Ruby clerics that try to demonstrate why closures are important and how Python doesn't have them; nothing I saw gave a practical example.
Anyone have any?
Edit: I may have found a practical example, but I'm not sure what's so special about it.
Here's a wiki entry about the benefits of closures in Ruby and here are the Pythonized versions that work the same way. This is just a special case of closures, I'm still looking for something completely different.
posted by Matt at 17:08
12 Comments:
I'll take a shot at it from the Ruby-side.
Ruby's closures (or anonymous code blocks) allow you to create domain specific languages and control structures with ease.
So for a simple example, you can define your own 'if' statement like so:
def my_if cond, &b
if cond
b.call
end
end
a = 5
my_if(a < 10) {
puts "a is less than 10"
a += 1
}
#prints=> "a is less than 10"
#returns=> 6
It's a trivial example, and just a very small taste of how you can create DSL's in Ruby as well. I've even created a DSL for Describing Quantum Circuits using Ruby. For example, here's a QDL description for a simple quantum circuit:
1 require 'qdl'
2 include QDL
3 circuit = QCircuit.new{
4 io = qBusWidth(5)
5 l0 = qLayer {
6 feynmann(1,0)
7 toffoli(2,3,4)
8 }
9 l1 = qLayer{
10 feynmann(:in=>1,:out=>2)
11 }
12} #end of circuit
...It's all pure Ruby code, but the user doesn't need to know that.
(OK, neither the code nor pre tags seem to work here so you'll have to imagine the indentation... gee, what do Python people do in cases like this? ;-)
Take a look at: http://ivan.truemesh.com/archives/000411.html
for a tutorial on the limitations on Python's lambda and the effect that has on the ability to use closures they way they're used in Ruby.
Anonymous, thanks for the example. I wonder if the block syntax in Ruby was designed with DSLs in mind or if it was just a beneficial side effect. =)
ade, excellent link. The cases Ivan mention all seem to have a simple Python implemention that is just different than Ruby's. Many times, the Python way looks simpler, even if it's not as technically elegant.
Aside from DSLs and collections, Ruby's closures are used very often to make sure that acquired resources are released correctly:
File.open('foo') do |f|
# do something with the open file
end
The File.open makes sure that the file is closed when the closure is exited. Naturally this idiom extends to other types of resources as well (sockets, db connections, locks, transactions, ...)
There are other useful ways to use closures in Ruby as well. After spending too many hours writing tests for exception throwing using JUnit, I'm particularly fond of Test::Unit's assert_raises:
assert_raises SomeException do
# code that should raise SomeException
end
The standard library has some nice uses for closures too. Such as String#gsub
or String#scan
(With my very limited Python knowledge, I think that String#scan could be easily implemented using generators, but what about the block form of String#gsub?)
The examples sections of
PEP 343 has more examples on possible uses for closures.
ville aine, the File.open() with block support is a cool shortcut, but seeing an open method without a close method isn't Pythonic. It's a good example of the differing design philosophy between the two languages.
I don't know of a clean way to get the functionality of a gsub with blocks in Python either. But shouldn't a global substitution just substitute one pattern with one string?
I could see a few applications for using a block (e.g., the first pattern gets substituted with the text that matched the pattern concatenated with a "(1)", the second a "(2)", etc.). Doing that in Python would be ugly: a loop, a str.find, and a string concatenation.
Shame about crappy code pasting here.
str = "I have 6 eggs in 5 baskets"
def num2word(n)
return "five" if n == 5
return "six" if n == 6
# this is a simple example
# implementation, obviously
end
str = str.gsub(/\d+/) { |n| num2word(n.to_i) }
# str is now:
# "I have six eggs in five baskets"
Pretty cool, IMO.
Regarding seeing an open method without a close method isn't Pythonic, that's a poor excuse for avoiding some very useful resource management techniques. Ruby's DBI database access package will auto-close database connections, prepared statements, cursors, etc. The great thing is that they will also be auto-closed if an exception is thrown. You don't need they Ruby equivalent of try/catch everywhere just to ensure that resources are handled properly.
First-class closures, in the form of blocks, mean that they are used everywhere in Ruby, giving great convenience to the coder. To get the effect in Python, you have to create a separate (named) function, thus breaking the flow of your code. Further, I don't believe this solution handles exceptions as well as Ruby's approach. Ultimately, I think people decide that approach is not worth taking in Python. The small theoretical difference becomes a large practical one.
Python's list comprehensions are very cool, but Ruby's blocks are far more generally useful.
Regarding seeing an open method without a close method isn't Pythonic, that's a poor excuse for avoiding some very useful resource management techniques.
And apparently others disagree with it being unpythonic, PEP 343 -- Anonymous Block Redux and Generator Enhancements.
That link is even more relevant because it provides the means to do cool things with code blocks in a clean fashion.
Here's a quick example that's not so pretty:
import re
str = "I have 6 eggs in 5 baskets"
class numToWord:
def __call__(self, val):
tempNum = int(val.group())
if tempNum == 5:
return "five"
elif tempNum == 6:
return "six"
else
return val.group()
str = re.sub('\d+', numToWord(), str)
Scratch that, I made the Python more complex than necessary.
This function definition:
def numToWord(val):
tempNum = int(val.group())
if tempNum == 5:
return "five"
elif tempNum == 6:
return "six"
else:
return val.group()
...with this call:
str = re.sub('\d+', numToWord, str)
...will work just as well.
Don't see any need for regexps here, matt. This should do:
>>> str = "I have 6 eggs in 5 baskets."
>>> nummap = {"6" : "six", "5" : "five"}
>>> for k,v in nummap.items(): str = str.replace(k,v)
>>> print str
'I have six eggs in five baskets.'
python's 2.5 new With statement
will provide a clean syntax for making sure that after a block has been executed the resources are properly taken care of.
(in the case of the comments close something that has been opened)
I don't know how old the comments are here, but the situation doesn't seem to have changed, so:
Ruby's blocks are very ugly; a special syntax for something that should be simple and implicit. For example, Lua:
function f(x) x(2) end
function g()
a = 1
f(function(y)
a=a+y;
end)
print(a)
end
... prints 3, as expected, with no odd syntax: you can pass in many of these around as you want, just like passing around any other function (can you even do that in Ruby?)
Python can almost do this:
def f(x): x(2)
def g():
a = { "x": 1 }
def z(y): a["x"] += y
f(z)
print a["x"]
Python has access to the variables in the surrounding scope, but due to syntactic bugs, there's no way to modify those variables--if you do, it'll helpfully assume it's a local and redeclare it on you; hence the hack of storing it in a dictionary. I really hope this gets fixed (and not in a way that makes me declare all local variables in the inner block).
The "lambda" keyword in Python is just an ugly hack to get around the fact that it's hard to fit anonymous functions into Python's syntax. I almost never use it; I'd prefer real anonymous functions, but declaring a named local function is bearable. It's just the name resolution mess that makes it so ugly.
> seeing an open method without a close method isn't Pythonic.
I'm definitely glad whoever implemented the "with" keyword isn't listening to this guy, who seems to have defined "pythonic" as "cumbersome and brittle"...
(Why are comments here in a tiny column? Horrible page layout.)
Glenn,
I feel you misunderstand Ruby's blocks. Ruby's blocks _are_ a special case of a more general lambda syntax, but Ruby does still have the ordinary lambda syntax.
Here is your code using lambdas (not blocks) (in ruby 1.9.1):
def f(x) x.(2) end
def g()
a = 1
f(->(y) { a=a+y } )
puts(a)
end
The output for above is '3' as in your example.
Post a Comment
<< Home