A Ship at Port

Closures in Ruby and Python

2005-08-16T17:08:00.004-05:00

Edit: This post has come up in a recent (2008-04-27) post on reddit. For an account of where the term, closure, came from look here.

While reading the the Django/RoR comparison by Sam over at magpiebrain the "Ruby has closures, Python doesn't" argument was mentioned. I admit to not being really familiar with the concept, but I believe I understand what it is. The thing is, I don't see what's important about it.

A link was given in the post to an interview with Yukihiro Matsumoto where "Matz" is asked about what a closure is and how it's beneficial. Here's the most relevant portion of the interview:

Bill Venners: OK, but what is the benefit of having the context? The distinction that makes Ruby's closure a real closure is that it captures the context, the local variables and so on. What benefit do I get from having the context in addition to the code that I don't get by just being able to pass a chunk of code around as an object?

Yukihiro Matsumoto: Actually, to tell the truth, the first reason is to respect the history of Lisp. Lisp provided real closures, and I wanted to follow that.

Bill Venners: One difference I can see is that data is actually shared between the closure objects and the method. I imagine I could always pass any needed context data into a regular, non-closure, block as parameters, but then the block would just have a copy of the context, not the real thing. It's not sharing the context. Sharing is what's going on in a closure that's different from a plain old function object.

Yukihiro Matsumoto: Yes, and that sharing allows you to do some interesting code demos, but I think it's not that useful in the daily lives of programmers. It doesn't matter that much. The plain copy, like it's done in Java's inner classes for example, works in most cases. But in Ruby closures, I wanted to respect the Lisp culture.

I respect cool implementations of higher-order concepts as much as the next guy, but I can't help but feel cheated by that response. I've seen some bickering between Python and Ruby clerics that try to demonstrate why closures are important and how Python doesn't have them; nothing I saw gave a practical example.

Anyone have any?

Edit: I may have found a practical example, but I'm not sure what's so special about it.

Here's a wiki entry about the benefits of closures in Ruby and here are the Pythonized versions that work the same way. This is just a special case of closures, I'm still looking for something completely different.

Too sexy

2007-04-10T08:57:00.000-05:00

In response to this post.

From the post:
"That’s a pity really, because even though [Paul Graham] says eventually Microsoft may encounter problems (who would have thunk?), [he] effectively concedes that Microsoft is going to make a lot more money in future. And some people are going to get rich competing with, and taking a chunk out of Microsoft’s revenues - although apparently not Graham’s start-ups because it’s unfashionable to even try."

Microsoft's flagship product is the Windows OS, but supposedly they don't profit from its sales [Edit: this is incorrect, it's actually Windows and Office that keep MS from major losses]. They keep developing new versions of Windows in order to maintain their share of the market - those customers will then pay exorbitant prices for other products (e.g., MS Office) which sustain the company.

The problem is that the OS market has been commoditized and Microsoft Windows is an inferior product. Sun open sourcing Solaris is an indicator of OS commoditization and is Windows OS objectively better than other OSes? Assuming Microsoft continues down the path of producing OSes that barely meet status quo, it's only a matter of time before they lose their grip on the desktop market. Of course, maybe they'll surprise us all and build an awesome Minix-based OS.

If MS loses the desktop wars, they will end up where they started, writing third-party applications for Apple. And there's always web application development.

It's not only that web applications are sexy, it's that the OS wars require a large amount of resources (people and/or money) to win. How many Windows users choose Wordperfect over Word? How many Apple users choose Powerpoint over Keynote? It seems pretty futile for a third-party developer, especially a startup, to attempt to beat first-party applications.

Closure on closures

2006-10-01T10:59:00.000-05:00

In a previous post, closures were discussed without much being said. That was over a year ago, since then I've come across a few posts that clarify.

Here's a description of the concept by comparison to objects:

Objects are data with methods attached, closures are functions with data attached.

(Sorry, I couldn't find the source for this quote, so this is a paraphrase from memory.)

An explanation of where the term closure comes from by Max Hailpern, co-author of Concrete Abstractions - and why they didn't use it in their book:

The reason [it wasn't included] is that "closure" only makes sense in a particular historical context, where procedures had previously been left "open", that is with free variables not associated with any particular binding. This was the case in various pre-Scheme Lisps, and lead to what was known as the "funarg problem," short for "functional argument", though it also was manifested when procedures were used in other first-class ways than as arguments, for example, as return values, where it was confusingly called the "upward funarg problem" (by contrast to the "downward funarg problem," where the arg was genuinely an arg). The "funarg problem" is what logicians had been calling "capture of free variables," which occurs in the lambda calculus if you do plain substitution, without renaming, in place of proper beta-reduction.

So anyhow, an evolutionary milestone in the development of the Lisp family of languages was the realizations that procedures should be "closed", that is, converted into a form where all the variables are bound rather than free. (The way this is normally done, as others have written in this thread, is by associating the procedure text, which still has free variables, with a binding environment that provides the bindings.)

Because this was such a big deal in terms of the languages' evolution, Lisp hackers took to using the word "closure" rather than just "procedure" to emphasize that they were talking about this great new lexically scoped kind of procedure.

As with so many words, it stuck, and people continued using it long after the distinction it was making ceased to be relevant. (When was the last time you saw a procedure that hadn't been closed?)

(Link to source.)

For a mathematical perspective see Ted Dziuba's post on closures as a recurrence relation.

Return of the White Spots

2006-03-30T09:45:00.000-05:00

Having purchased my first Mac two years ago, I was enticed by processor speed improvements to become a member of Apple's public beta program by purchasing one of the first MacBook Pros.

There have been various posts about MacBook Pro noise problems. My machine has exhibited this problem, but I've worked around it by fiddling with the lid. Annoying and occasionally headache inducing, but at least I can pretend all is well if I play loud music.

Or so I thought... My last Mac, a Powerbook (Al), was unlucky enough to be afflicted with the notorious white spots. This morning I noticed a little spot on my screen - "no big deal, just some dirt or something," I thought. Nope, it is a little tiny white spot. It's located on the right hand side, about 0.75 inches from the end and a little under halfway up the screen.

Oh Apple, why do you raise our expectations only to disappoint?

Decision tree building and graphing in Python

2005-11-27T10:58:00.000-05:00

Spent part of yesterday throwing together an implementation of the ID3 algorithm for building decision trees. It uses pydot for creating graphs, so the trees can be outputted in dot format (ascii, for display in Graphviz) or as an image.

The id3 module can be downloaded here. The implementation isn't bullet-proof, but should work fine with good data sets.

Example usage and output:

>>> from id3 import *
>>> dtree = DecisionTree('recycling_bin',
... [{'dept':'EE', 'size':'large', 'recycling_bin':'no'},
... {'dept':'CS', 'size':'medium', 'recycling_bin':'yes'},
... {'dept':'EE', 'size':'small', 'recycling_bin':'yes'},
... {'dept':'CS', 'size':'large', 'recycling_bin':'no'},
... {'dept':'EE', 'size':'small', 'recycling_bin':'yes'},
... {'dept':'CS', 'size':'medium', 'recycling_bin':'yes'}])
>>>
>>> dtree.graph.to_string()
'digraph G {\n"no";\n"yes";\n"yes";\n"size";\n"size" -> "no" [label=large];\n"size" -> "yes" [label=small];\n"size" -> "yes" [label=medium];\n}\n'

Semantic analysis with Python

2005-10-18T19:35:00.000-05:00

I came across Reverend ("a general purpose Bayesian classifier") today after reading Peter Bengtsson's post about it. Haven't tried it yet, but it looks great.

Still looking for a good PLSA, or even LSA implementation, preferrably with Python support.

Anyone know of one?

"Java, my hero"

2005-10-04T18:35:00.000-05:00

I now understand why so many developers on MS Windows cling to Java so tightly, it's their saviour from the hellish experience that is Microsoft Visual C++.

Developing cross-platform applications on a Mac or a Linux machine is not that difficult. Getting those same applications to build on a MS Windows platform is not that difficult. The problem is, what if someone needs to do development work of cross-platform software in the MSVC++ environment? It is possible, but the amount of work required is immense. The quasi-solution of switching to a different compiler/IDE is an option now, but what about 5+ years ago?

Java to the rescue! Well, sorta, application development in Java has many of its own gotchas and "write once, run anywhere" is malarky; but, for developers using MS Windows, Java development frees them from MSVC++ and (this is just a bonus) lets them build applications that are multi-platform. To sweeten the deal, Java has development tools and build processes that are standard on all major platforms. With more marketing than necessary, Sun was able to transform the animosity towards MSVC++ and MFC into migration to Java and AWT/Swing.

As Qt and GCC on Windows gain visibility, I wonder if we'll see some Java application developers downgrading (or is that upgrading?).

Copyright laws and fair business transactions

2005-09-04T14:08:00.000-05:00

Yesterday I bought a few CDs from Best Buy, but once I arrived home I realized that I was missing something. One of the CDs I purchased (or should I say decorative jewel case) was missing its, er, CD. The CD in question was No Doubt's Rocksteady, the jewel case was completely wrapped and the little sticky thing on top of the case was intact, but the case was empty.

I returned to Best Buy to get the CD I paid for and discover that they don't have any other copies. "OK," I thought, "not a big deal. Just give me my money back and I'll run down to Borders and get my CD there." I was expecting the Best Buy associate would have a problem with my proposed solution and I was right. At first they weren't sure what to do, but ended up offering me store credit. According to the Best Buy employee, the store is unable to provide me with a refund because it's a violation of copyright law. IANAL, but I have a good feeling that's probably not true. Hesitantly, I accepted their offer, only because I was already running late to a friend's house.

I can understand the reasoning behind a no refunds policy, but I still think it's unfair to an unlucky customer. All I did was give them money for a CD, and they didn't fulfill their side of the deal. I'm not in the loan business; I bought a CD, not a gift card.

No CD? OK, then give me my money back.
Guess I'll just have to start opening any merchandise I purchase before leaving the store...

Google Talk is Live

2005-08-23T19:25:00.000-05:00

Google has launched Google Talk.
It's a Jabber service, and gmail users can use their email address and password to login to talk.google.com.

What's next?

Pythonic file I/O?

2005-08-09T20:50:00.000-05:00

A coworker of mine has recently been using Python on and off for basic administrative tasks and ran into a problem this week with modifying the contents of a file without rewriting the entire file. He wasn't trying to modify the file size, just change a number of bytes that were somewhere in the file. I had forgotten about mmap, but is that the best Python module for file modification?

It has two main limitations: 1) No way to insert into or delete from a random location in the file. 2) A mmap object maps a file in part or in total, but the mapping must start at the beginning of the file. This is a problem when modifying large files which can't be loaded entirely into memory, a solution would be to allow the mmap to begin a file mapping at an arbitrary position.

I understand there are some technical details to make this work. Completely rewriting a file out may still be needed in some cases; but, all of this should be implemented in a slick module and hidden from the developer. It's just not very pythonic.

Python Rocks

2005-08-05T22:37:00.000-05:00

I was thinking of just getting to sleep after my last post, but started fiddling with a little Python module to handle parsing words from a source into a vocabulary. Well, from the time from my last post, minus a quick browsing of Planet Python and email checking, I have a basic module written that will accept text input, trash all special characters (e.g., (){}!?), use the stemmer, and drop the output in a list.

List comprehensions are great.

Stemming, information filtering

2005-08-05T21:31:00.000-05:00

I've started work on an information filtering system to assist in returning only relevant documents. I'll be using my own implementation of probabilistic latent semantic analysis (PLSA) at first. A couple ideas of how to extend it have already entered my head, and a buddy of mine has started playing around with a probabilistic model based off least squares.

I'm doing all of this in Python and already found a decent word stemmer algorithm that's been written in Python. Not much to it, but it seems to work well. The next step is building a vocabulary; I've found a few with some Google searches, but I'm going to go ahead and write a Python module to ingest text files, clean out punctuation and digits, stem the remaining words, and write the result to a master vocabulary.

For your browsing pleasure

2005-08-03T15:57:00.000-05:00

I came across these two posts recently: The Multiple Self and A Spooky Mind Hack.

They're good reading. If you start The Multiple Self and get tired of it before finishing, go read the second post.

Overall, very good stuff, though I do think there are a few inaccuracies in the definition of the "emotional net". The gist of it is right on.

Ahhh, Django...

2005-08-01T10:42:00.000-05:00

I've been testing out Django this past week, mostly finished the the first two tutorials. There's a lot to become familiar with, but I'm amazed at how powerful the framework is. Though I haven't done much with Rails other than research it, I'm glad that the community is shaping up to allow for exchange of ideas and friendly competition.

This is also a good thing for Python, as I think there were a decent number of Pythoners wishing to do some web application work in Python but ended up using Java (*shudder*) or Rails. Not to discount the work that many put in on templating engines and basic frameworks for Python, but it's nice to use one framework that's a complete package.

dynamictyping.com is coming along, I've just been sidetracked of late, getting into AI solutions for information retrieval.

Probabilistic latent semantic analysis is the topic of the day.

The end is near...

2005-07-26T17:53:00.000-05:00

Not the end of the world, but of this weblog. I'm going to be moving over to a new site, dynamictyping.com, as soon as I find a good host. I'm going to test out Django and have a proper setup. Feel free to send in recommendations for a good hosting company in the Northern VA area.

Planet Intelligence

2005-06-25T07:56:00.000-05:00

I'd like to see if there's any interest in a planet site for weblogs that are geared towards machine intelligence, AI, neuroscience, etc. A site like this would provide a way to keep updated about what other's are doing and encourage the exchange of ideas.

In the mean time, check out Yaroslav's site for links to other related weblogs.

EDIT: Found a group weblog, neurodudes.

The BBS

2005-06-15T08:13:00.000-05:00

Looking forward to watching BBS: The Documentary.

Though I was only a whelp during the end times of the BBS, I have fond memories of playing Barren Realms Elite, downloading shareware games, and chatting with sysops. If anyone from the BizZ BuzZ, R/C Connection, or other boards that ran during those years in northern VA reads this, please post a comment or shoot me an email.

The 703 BBS List.

Technorati

2005-06-14T20:52:00.000-05:00

I have delved even further into the world of weblogging with my new technorati profile. I'm interested to see how long and how much effort it takes for an individual blog to get linked into the saturated blogosphere.

More on Intel processors, Macs, and the future of Linux

2005-06-10T17:40:00.000-05:00

This post is in response to an opinion piece by Steven J. Vaughan-Nichols in eWeek. This is Vaughan-Nichols second editorial on the topic and he's trying to defend his position that the Linux desktop is in danger from Macs with Intel inside.

Some quotes from the piece, followed by a response:
_
"One of the most common themes I've been hearing is that Apple will not—will not, I tell you!—be letting its precious operating system ship on anything except its own branded hardware.

Yes, that appears to be what Apple is planning, but I don't think it'll be able to pull it off.

The Intel-based PC architecture is an open one. How is Apple going to keep people from installing Mac OS on non-Mac PCs? Proprietary firmware? A BIOS, a la Phoenix Technologies' TrustedCore, which can enforce DRM (digital rights management)? Pick a method and I foresee hackers cheerfully breaking it."

Whatever mechanism Apple uses to require Apple hardware can be reverse engineered. This is a fairly obvious concern to Apple, I think they just might have a solution ready to go. One method that would be hard to emulate could use a small chip embedded on all Apple motherboards that the Mac OS would query during boot up. No response from the chip, no Mac OS. And in case someone patched the query checker, run md5sum on the binary to verify it's unmodified.

It's entirely possible Steve Jobs has decided that it's time for the Macintosh to become an OS and Apple chooses a weaker method of hardware authentication. I hope not, a principle of the Macintosh design is tight hardware and software integration - that's part of what makes it such a great user experience.
_
"I've also had people tell me that they don't think Apple will have as much trouble with device drivers as Linux has, either because Apple will use only a small subset of all the possible equipment you can stick on a PC, or because Apple will be able to get better deals from hardware vendors.

There's no question that Linux still has a long, long way to go with device drivers. It's just that Macs have even farther. Macs support only a very limited range of hardware.

In the past, if the equipment wasn't built by Apple, or had its making overseen by Apple, the odds were the device wouldn't work on a Mac. Linux still has big driver problems. Mac OS on Intel will have enormous, but conquerable, driver problems.

On Intel designs, Apple will have to contend with a much wider variety of external equipment. From Webcams to scanners to, heck, even mice, Apple will have to deal with tens of thousands of new devices. Even if "Mactels" turn out to be hermetically sealed boxes, Apple and friends are going to be spending a lot of time working on device drivers. And, lest we forget, historically, Apple hasn't gotten along well with equipment vendors."

First off, since when is Linux device driver support poor? I can plug any old mouse into a machine running a desktop Linux OS, and it works just fine. For that matter, I can connect many digital cameras, printers,
mobile phones, PDAs, video cards, sound cards, monitors, digital I/O boards and they all work. Oh, and in all but the obvious cases (video cards, sound cards) Mac OS has the same level of support. Obviously, MS Windows has the best device driver support since most hardware manufacturers consider it to be the most important platform due to the install base.

On Intel designs, Apple will have to contend with the same variety of external equipment as they do now.
Using Intel processors and chipsets has nothing to do with, for example, Apple choosing to only provide drivers for iSight and no other webcams. It's technically possible to use third party webcams on a Mac, but Apple doesn't provide it.
_
"I've also been hearing from a lot of people who insist that it doesn't matter what Apple does, since Linux will survive no matter what the companies do. Would someone please send these people in their basements the memo from a few years ago that Linux is a business operating system and not just a hobby?"

Linux has support from commercial interests, but it didn't start out that way. The term Linux, as it used in your piece, refers to Linux-based operating systems. There are both commercial and non-commercial Linux distributions, therefore Linux is not a business operating system. On top of that, some contributions to Linux and related software are made from hobbyists and many are from developers that started out as hobbyists.
_
"So, to get back to the original point of my column, if you want Linux to play a serious role on the desktop in the years to come, the time is now to get serious about creating an outstanding Linux desktop operating system. Mac OS X is coming to Intel and, unless things change, it's likely to be unchallenged as the best Intel-based desktop operating system around."

I don't understand the importance of Linux dominating the desktop world. Users will choose the best (that they know of) tool for the job.

Future of AI

2005-06-09T07:32:00.000-05:00

Starting grad school this fall for computer science. There are several concentrations available and I was looking at artificial intelligence. It appears to be frustatingly needing innovation, and I think it will be coming soon with the memory-prediction framework.

Is it worthwhile to study traditional AI concepts if we're finally realizing that they're useless for machine intelligence?

Apple Macintosh and Intel

2005-06-07T19:33:00.000-05:00

Plenty of people have weighed in on the implications of Apple switching to Intel processors for their Macintosh computers. I figure I'll add my two cents...

I think there has been some inaccurate prophesizing over what this change means to Linux and other alternative operating systems. Some editorials on osnews.com (here, and here) along with Dvorak's recent post and an eWeek opinion piece all seem to believe that Linux is in trouble.

This is what they're missing, the Mac is not computer hardware and it's not an operating system, it's a complete computer system. Why do you think Microsoft is still developing their "standard" products like MS Office for the Intel-powered Macs? Because they understand that Apple is selling a different product.

To understand why Linux isn't in trouble, we have to do some stereotyping. Who uses Linux anyways?
Anyone that is going to actually use Linux as a desktop OS is either a geek, or has been convinced by a geek that it's the way to go (and said geek provides tech support). This means that Linux users have the ability to manage their OS, directly or indirectly.

Now, what sort of machines does Linux usually run on? Intel/AMD x86 is the most popular, and almost any hodpodge arrangement of a computer will be supported by Linux. So, along comes Apple with their new Intel-based Macs - why would current or future Linux users care? A Macintosh is still a Macintosh, it's a marriage of hardware and software. If you want a Mac, you buy a Mac computer and it comes with the Mac OS. Sure, you will likely be able to install MS Windows or Linux, but again, so what?

The two most important things that Linux has going for it are its price (free as in beer) and killer hardware support.
Mac OS is completely opposite: it ain't free, and it's hardware support sucks. The Mac OS is meant to run on Mac hardware, and that's that. MS Windows emulates being free by being the default OS installed on most mainstream computers and it also has killer hardware support. For those that aren't interested in purchasing an integrated computing solution, the best known choice is MS Windows, the second is Linux.

Many of those editorials did get one thing right, Microsoft is vulnerable until Longhorn is released, now is the time for Linux to gain some users.