When did you first try Lisp seriously, and which Lisp family member was it?
Meaning any member of the Lisp family. As for "when", many of us had multiple encounters with a Lisp before it really stuck. The "stick" date is of most interest.
I still am paid to work most often with C and C++. I am still on my "road to Lisp," which seems to have many detours. In fact, it seems to rarely involve Common Lisp itself, but often involves "Lispish" languages -- Dylan, NewtonScript, Scheme, Lua, Io, and Ruby.
My determination to use continue to move towards Lisp, or at least towards concepts embodied first in Lisp but now appearing in other languages, has been reinforced by several experiences where I was able to use Lisp(ish languages) in real-world projects.
In the summer of 2003 I began attempting to redesign a chunk of ugly C++ library code using Gwydion Dylan and multiple inheritance. The result was an incomplete prototype, but proved to my satisfaction that, although the tools had convenience and speed issues, the language itself was ready to take a serious role in my project.
If you've used C++ a lot, you know how hard it can be to use multiple inheritance. In Dylan it was easy, and the resulting code, which used generic functions and up to four base classes per concrete class, was much smaller and more easily extensible. Dylan's support for object-oriented programming borrows very heavily from CLOS. If you have only used C++, you are missing out on the true potential of object-oriented design for rapid prototyping. Generic functions and multiple inheritance that actually works nicely make object-oriented design and programming in Dylan an absolute joy by comparison. Dylan has not taken the world by storm, but I really hope to use Dylan or Common Lisp/CLOS itself in the future.
In the late 1990s, I developed a survey engine using NewtonScript on Apple's Newton platform. NewtonScript, inspired by Self, is a simple prototype-based language; although it has a Pascal-like syntax, there are very few special cases to learn, and it supports prototype-based programming, first-class functions, and lexical closure. The survey engine worked by running a "script" which was really a separately-compiled package containing frames (associative arrays) holding data, symbols, strings, and function objects. The survey engine both drove the program and provided a survey implementation library. This meant that the survey data structure acted as a kind of domain-specific language which was readable in its own right.
Later, in order to migrate the design to the web, I worked to port my NewtonScript survey engine to Java using Apple's WebObjects. Java did not provide an "eval" or a simple way to use Java itself as an embedded scripting language, so I decided to try using Kawa, a Scheme implementation that compiles to Java byte codes. My team and I completed the port, using Scheme code evaluated on the fly to make run-time branching and tailoring decisions. One of the things that made it work was the concept of the environment for bindings, which could be manipulated from both the Java and Scheme worlds.
I am currently interested in learning to program more idiomatically in Scheme, and am writing some hobby programs for solving Sudoku puzzles in DrScheme, the PLT project's IDE. I am using SLIB in order to have a library that provides a little bit more leverage. I also have an ongoing research interest in trying to figure out how to use small dynamic languages in embedded systems. I'm currently looking into using one or more of the languages Lua, Io, and Guile for this role.
What led you to try Lisp?
My interest in Lisp and its derivations came about most specifically because of my exposure to two Apple-developed languages, Dylan and NewtonScript.
I've been programming since approximately 1976, when I learned BASIC and some assembly language. In college I learned Pascal in classes and taught myself C and Macintosh programming. I've been using the C family (C, C++, Java) for about 15 years now. I've been curious about Lisp since first reading Hoftsadter's Godel, Escher, Bach and coming up to a solution to his MU problem (I was about sixteen then). But Lisp always seemed very esoteric and far-removed from the platforms I was using, like the Radio Shack TRS-80 Model 1 and Commodore 64.
In college, I was taught how to implement algorithms such as the Knight's Tour recursively in Pascal, but then immediately taught how to implement it instead in a non-recursive way, using stacks, in order to avoid blowing up the stack and crashing the program. In other words, I was taught that recursion was a curiosity to be used with extreme caution. Here and there I used it in small hobby projects, drawing fractals and solving puzzles (for example, I wrote a recursive program in C which used a brute-force approach to solve a combination lock puzzle in Myst). But recursion remained largely a curiosity.
In 1993 I took a job writing programs for the Newton. I learned a new model for object-oriented programming based on prototypes. The idea that objects could have copy-on-write semantics and live mostly in ROM was quite exciting, and I realized that it was this design that enabled the Newton; the C++ runtime model just couldn't be used to write applications on such a memory-constrained platform. I was also thrilled to be able to exchange e-mail messages with Walter Smith, the original NewtonScript language implementor at Apple; he was exceedingly generous with his time in explaining how the Newton runtime actually worked. Dynamic language implementations were alien to me, but I came to understand lexical closures, frames, frame maps, interaction with the garbage collector, and the other concepts that allowed me to move from simply knowing how to program in NewtonScript to knowing how best to take advantage of the runtime.
NewtonScript was also the first language I used where the language itself was available during compile time, and the deliverable package was really a frozen data structure in the implementation language. This is an incredibly powerful concept, and has its roots in s-expressions and Lisp. It was a thrill to be able to write a program that wrote a program, and so much more efficient to be able to run and debug code in real time, with a listener; it was even possible to replace functions on the fly, and I began using a functional style, which minimized interactions between functions and let me test pieces of code using the Listener prior to building them into the completed program.
C and C++ tend to punish and discourage a functional style, because it is extremely unwise to return a pointer to something declared locally, and if you allocate an object in a function you immediately introduce the need to manage the object's lifetime and ownership. This can all be done using a whole slew of modern C++ idioms, but because the compiler throws out almost everything you have told it about your code, debugging is excrutiating, and few compilers can keep up with the more esoteric tricks. I have never been able to get much leverage using STL and templates; between worrying about copy constructors, assignment operators, ownership, deletion semantics, and all of the above in the presence of non-local exits via exceptions, along with the pain of coping with crashing compilers and baffling error messages, it just isn't efficient.
I have never used a Lisp Machine, but the Newton programming environment provided me with at least some of the same flexibility and power.
Around the same time, I was also exposed to Apple's Macintosh implementation of Dylan. I loved the development environment, worked with it as an alpha tester, and even contributed a very modest piece of code for the release CD. I was very upset when the project was cancelled; I still want Dylan to be the language I use to write programs for a living. The Macintosh implementation was not really ready for prime time, but the language design represents an outstanding effort. A lot of the best minds in the Lisp community worked on that project, and Apple does not seem to appreciate what it lost when that team was dismantled.
While I was working to learn Dylan, I quickly realized that I did not understand the idioms behind Dylan: for example, I did not understand how pairs worked, and so was writing code that assembled lists backwards; my algorithms were thus guaranteed to be as inefficient as possible. This is also how I first discovered the cultural gulf between Lisp-family programmers. The authors of the tutorial material all came from a Lisp background and it did not occur to them that a programmer from a C background would need these assumptions to be made explicit. I also didn't "get" closures. To try to learn how this all worked I began studying Scheme. A dim light bulb started to come on, but in the real world I had to write tens of thousands of lines of C++.
These days I have a general sense that the C, C++, and Java are useful tools but can not take me where I want to go. It would also be nice to get some work to do that required me to stretch my software engineering skills. As a driver and application developer who has also written DSP code, it seems that so much of what I do is largely plumbing: writing code to pipe data between a DSP and a bus, between a bus and memory, swapping formats, swapping APIs, between kernel memory and user memory, and between user memory and a GUI.
I spend much of my time using C and C++ as portable assembly language, writing simple imperative state-machine code without very much in the way of interesting algorithms, performance, threading, or timing issues. It is dispiriting to realize just how little most of this code actually does, and how much more efficiently it could be represented.
What other languages have you been using most?
The C family (C, C++, Java). As for libraries, I've gotten the most mileage in the last few years out of Apple's IOKit for MacOS X (Darwin) and Trolltech's Qt framework. I'm currently doing embedded systems work in C++ on top of QNX.
How far have you gotten in your study of Lisp?
Hard to answer, I know. Just looking for a rough idea.
At this point I basically grok much of recursive and functional programming in Scheme and object-oriented programming in Common Lisp using CLOS.
That said, I have done very little with macros other than read about them. I have made a few attempts to get going with macros in Dylan, but was not able to get them to work in Gwydion, and also in Scheme, but have always gotten hung up on the differences between whatever text I was working from and the Scheme implementation I was trying to use. The net result is that all my attempts to really learn idiomatic macro programming have been at least as frustrating as my attempts to learn C++ template metaprogramming with incompatible compilers and horrible error messages.
To get beyond this I will probably need to use one of the commercial Lisp IDEs and work my way page-by-page through Paul Graham's On Lisp. But because learning Lisp is largely relegated to my spare time, I have not been able to devote sufficent time to that endeavor. One idea has been to find a full-time job which would require me to master macros. I interviewed for such a job, but it did not pan out; I would try again for a similar opportunity, though.
I have also not done very much with continuations, and would like to master them. My ideal language would somehow merge Apple's Dylan IDE and Dylan's syntax with full macro and continuation support, full multi-threading and coroutine support, CLOS, and an excellent library with some of the Common Lisp "onions" removed. That's all -- is that too much to ask? : )
What do you think of Lisp so far?
Common Lisp feels too big and the implementations not programmer-friendly enough, but I'm aware that a lot of that impression is because I don't really know how to use tools like Emacs. I have taken a couple of running starts at learning Emacs but generally get frustrated. The Lisp IDEs feel alien, although I have used many other IDEs for C, C++, and Java and feel extremely comfortable with them, and I often get hung up on basics. This is a strong factor because I only get to work at this using scraps of available free time, and so want to see some progress right away. I like PLT Scheme's IDE, but Scheme has its own frustrations.
One of the reasons I am specifically interested in languages like Lua and Io is their support for coroutines and other forms of threading. Multi-threading is very important in embedded systems and is becoming very important in application develoment for multi-processor and multi-core systems as well. The Lisps of the future need to provide both lightweight and heavyweight concurrency support.
It has become a cliche but criticize the Common Lisp library for lack of facilities that have become commonplace, like network programming support, but it is a cliche because it is true. Paid programmers need to be able to leverage their tools, and even if you are Jedi programmer you aren't going to impress your boss if you are too busy designing your lightsaber to deliver your program on time. The Reddit story should not be pooh-poohed but should be considered carefully for what it can teach Lisp supporters about Lisp in the real world, because for every high-profile story like that, there are many more that don't get discussed and many more where Lisp is never considered due to similar barriers. That said, I do believe that Common Lisp has a bright future and would love to use it in a real-world project.
If Common Lisp feels too big and Scheme too small, Dylan feels just about right. I find the "inside-out" object model, using generic functions, to be far easier to use in practice than the C++ model. Personally, I would like to see Dylan revived as an excellent multi-paradigm language that is suitable for attracting programmers who have never worked with Scheme or Lisp and who are turned off by parentheses. Dylan should grow first-class continuations, though, to better support that paradigm for web development.
I should mention that I am more or less neutral these days on the subject of parentheses. I have become comfortable with the parenthesized syntax in Scheme and can finally write functions using let, cond, and if at reasonable speed. I understand the advantages that s-expressions confer in enabling macros. But fans of parentheses should consider that McCarthy originally did plan a syntax for Lisp, and that there can be advantages to having a little bit of syntax to help differentiate what the language is actually doing at a given point in the program. I miss NewtonScript's very light-weight syntax for constructing frames and arrays, and like some of Ruby's syntax, such as support for blocks, although Ruby goes overboard with too many special cases. Personally, I think Dylan has it just about right; Google for Bachrach and Playford's paper on "D-Expressions" for more information. Common Lisp fans should recognize that even though the rest of the world still has not caught up to what Lisp was many years ago, it is not the last word in language design.