Terri also recently mentioned my cousins’ visit. One of the small things that happened that still makes me happy is that I got to pass along my first O’Reilly book, sed & awk, to Margaret.

I bought that book in 1998, shortly after I had my Unix Liberation Moment. It was this gestalt where I realized a whle bunch of things at once:

• I already knew more than I thought I did from using a VAX in college
• Unix is elegant, Unix has a philosophy
• Unix people are smart, literate, and have a sense of humor. (Witness commands like “apropos” to find documentation about a certain keyword or “nice” to tell the kernel to run a command without hogging resources) . Unix is a culture.
• Intuitivity is a function of expectation. Before the Moment, I was a pretty committed Mac user, and I firmly believed in much of the Apple human interface guidelines, like making all parts of an application obvious and visible to the user. Unix actually has a very similar philosophy when it comes to not surprising the user. But Apple had made this extra leap that I had taken in without consideration, that the “user” should be an accountant or a musician or a graphic designer or an academic and shouldn’t have to have an idea of how to use a computer. Unix, on the other hand, was created in an era when “user” meant a fellow computer scientist. Unix applications have vastly less development overhead, because the user interface is simple and assumes that the user is going to do at least a little reading before they run anything, and will take any “undo” precautions on their own. In fact, the scripting languages common in the Unix world make things that seem like magic to both laity and Windows programmers not only possible, but fairly easy. (Don’t get me wrong; the idea that one need not be a computer scientist to use a computer has been hugely beneficial to the world; but the downside is while using a computer in a GUI environment is pretty easy, programming a computer in a GUI environment is very, very hard. And at one time, programming was within reach of committed non-programmers).

Two of those scripting languages to come out of the Unix world are sed and awk.

Imagine being able to very expressivly write search-and-replace commands, like “look for three numbers, a dash, three numbers, a dash, and four numbers; oh, and the first three numbers might have parentheses around them instead of a dash”. Sed lets you do things like that, which might be useful if you were, say, trying to extract U.S. phone numbers from a chunk of text.

Awk is sort of like sed, except it’s more conducive to working with more fielded text files. So, say you had an Excel spreadsheet 20,000 rows long, where you wanted to take a free-text address field and break out the state and country into separate columns. You could spend two hours writing a buggy VB script which worked on 80% of the rows and spend the rest of the day manually fixing the last 4,000 rows. Or you could dump it to a .csv file, and write a 3 or 4 line awk script in about 10 minutes which worked for all the rows.

So, as you can tell, I was very happy when Margaret asked about sed & awk, and mentioned that it might be useful for her work. I felt like I was teaching her the secret handshake, handing her the keys to the car, and passing on the secret of fire, all in one.

This entry was posted on Tuesday, July 5th, 2005 at 11:58 am and is filed under geekery, personal. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.