changedmy.name

Approaching Problems

2021-08-15T00:00:00+00:00

I’ve been thinking about post-mortem processes recently, as we’ve been expanding the group of people involved in our post-mortems and we need to make sure they’re all prepared on how to make post-mortems effective and blameless. One of the aspects is making sure we clearly separate identified problems from potential solutions; most people tend to jump directly into ways to prevent the incident from happening again, but that can unintentionally skip better alternative solutions. It can also lead to implementing solutions that don’t actually solve the initial problem, but that’s a topic for another time.

I have a fairly structured approach to thinking about human error, which is where almost every incident comes from if you dig deep enough. I haven’t heard many other people lay things out this way, and I find it a helpful way to consider the broad spectrum of possibilities, so I am writing it down rather than relying on a series of ad-hoc conversations.

First off, we should dismiss the blame approach: X made a mistake, so we should fire X, or at least tell them not to make a mistake again. The so-called “Bad Apple Theory” has been repeatedly disproved; read The Field Guide to Understanding Human Error by Sidney Dekker for detail on this subject. And simply “not making mistakes” is hardly better, even if it is a common approach. Maliciousness aside, the person involved made the mistake because their view of the world told them it was the correct thing to do; otherwise, they wouldn’t have done it! Therefore, we can’t ask people to simply not make mistakes, because they were already only doing things they thought were correct. We have to address the systems instead, then.

When I worked at reddit, we went had three different CEOs during a year. One interesting thing this taught me was how differently people can approach the same problems. I like to categorize these into tooling and process improvements.

Let’s use an illustration. We’re a small company built by a few senior engineers. As the company expands, we hire our first new engineer, a junior. Things are going well until one day the junior engineer pushes a change into production that breaks the site.

“Never hire anyone ever again”, “never hire junior engineers”, and “try to fix the hiring process to only find engineers who don’t make mistakes” are all examples of strategies that are doomed to failure, so let’s put those aside.

Tooling solutions are the ones engineers normally think of first. The strategy here is generally: improve our software so that it’s impossible or difficult to make mistakes. The Japanese have a dedicated term for this, poka-yoke. Here we might have our production change tool output the diff of what’s going to be changed in production, or add automated test coverage, or create alerting that notifies us when we’ve broken production.

Process solutions instead tackle the human aspect by changing how we work, and are more natural to non-engineers. For instance, we can train our new hires on how the systems work, or have everyone telegraph their intended production changes so another team member can verify the change makes sense.

Different organizations will be used to tackling problems in different ways. Maybe we have multiple teams and want each team to only deploy their own software; do we build a system that restricts their access (a tool), or log the changes and give a stern talk to anyone who deploys another team’s stuff (a process)? I find that the approach that works best will depend on the situation, and so this categorization of potential solutions helps ensure that we’re looking at all of our options and can choose the one that’s the fastest, cheapest, or most effective.

GAS

2016-07-15T00:00:00+00:00

A few weeks ago, as my long-delayed senior project was coming to a close, I started thinking about what I should focus on next. The real answer, as Jenee helpfully pointed out, is writing wedding thank you notes, but I’ve also selfishly decided to start again on Growing Up, the introductory book on web operations that I got an introduction and a chapter through several years ago, before putting my attentions elsewhere.

With my head full of the fantasy of becoming a famous author, I bought my first fountain pen and an expensive Japanese notebook.

Since then, I’ve done some writing and a lot more internet research on pens, inks, and paper. I’m aware of this, and yet I keep telling myself that it’s fine, because I’m still finding my perfect setup, and once I do that, I’ll stop buying and just write.

It’s likely you’ve told yourself the same thing in the past. Maybe you’re even doing it right now.

I’ve been reading through Matt Gemmell’s archives this week. I like his writing. I like his writing, and so I get frustrated when his stationery category contains only one article.

How am I suppose to become a better writer if I can’t buy all the same notebooks as him?

Last year, I acquiesced to a long-dormat desire and bought a camera. Me being me, I delved into the field, starting with a full read of the manual before even purchasing the camera. I’ve taken a few thousand photos, and can see an improvement between the first set and the most recent. But more importantly, I can talk the talk.

This is in large part due to spending many hours on r/photography, the primary reddit subforum on photography. r/photography has a problem, which is a fixation on photography gear; most of the posted articles are about gear, which causes the reader population to bias towards gear discussions, which in turn leads to more posts on gear (it’s also much easier to write a news article about the latest gizmo than to create a useful piece on the art of photography). This fuels what the regulars call “Gear Acquisition Syndrome” (or GAS): an unshakeable belief that the most important thing to do next is buy a new piece of gear.

I, of course, have a wishlist full of leases, umbrellas, and other photography equipment, despite not using everything I already own.

When I was a younger programmer, I delighted in learning new programming languages. I defined myself by them; I was a Python programmer, or a Ruby programmer, because that was the important part of a programmer, I thought - what languages you know.

Later I started to learn that it’s much more important whether you can solve someone’s problem than what tools you use to accomplish that.

I think the programmer equivalent of GAS is adding languages to a resume. It may be easier on the wallet than camera lenses, but it shares similar characteristics: occasionally a blocker towards skill imporvement, but far less often than we think.

Why do we fall into the GAS trap? Whether writer or prose or writer of code, I think the answer is the same: it’s an easy excuse. It’s easy to find things to put on our GAS lists, much easier than it is to set aside the money or time to acquire them. This gives us an excuse for our inadequacies, and a reason for us to delay doing any more work - for a work not done is one safe from criticism. It can remain safely enshrined in our heads, the masterpiece that’s unfortunately locked away because of a missing item. When we see the work of a master, we can comfort ourselves - we have several things just as good, just kept away from the world due to circumstances out of our control.

One day, we will be recognized for our greatness, too. Just not today.

Roland TD-15K Review

2015-05-24T00:00:00+00:00

I’ve been thinking about this for a while, so this is a long review in three parts. First, we’ll start off with a brief background on me. This will be followed by a review of all the components of the TD-15K and a conclusion comparing the drumset to the other options you’ll likely be considering.

Background

I grew up the son of a piano teacher, and thus grew up a pianist. When I was in fifth grade, inspired by my particular interest in rhythm, I think, my parents began looking into band, and in sixth I joined my junior high’s beginner band as a keyboard percussionist.

I continued with marching band, concert band, and winter drumline every year until my first year of college, when classes suddenly got hard and band took more of my time than I was willing to give.

Sometime in late junior high, I think, my parents bought me a used CB drumset from a local drum dealership. We put muffling pads on it, but still the noise was loud enough that I would disturb my mom’s lessons, and more importantly, I was embarrassed to play - after all, everyone would hear me as I made constant mistakes! (Being over-confident was never one of my faults.) I finally did get some experience on a set when a good friend and fellow bandie taught me some lessons on the church drumset, as he was starting a band and wanted to lead on guitar. We practiced a bit, played in church a few times, and had a lot of fun, but things quickly fell apart.

Back in college, a beautiful girl asked me to join a small church band, and while that was again short-lived, it rekindled my desire to drum. After a few years and some full-time paychecks, a bit over a year ago I bought a TD-15K so I could practice in my room without annoying roommates. I enjoyed it so much I played a pretty constant 10-15 hours a week for several months, and though I now have slowed to a few hours a week, the girl (now my fiancee) and I are in another band, and I expect to be playing in some way or another for the forseeable future.

The TD-15K

The Drums

The mesh heads you’ll find on the TD-15K are surprisingly realistic feeling. There was no period of adjustment for me with them - I just played them and they felt like drums.

The one thing that occasionally bothers me is that there’s no sensing of the different parts of the head, so the sound at the center of the drum is exactly the same as on the edge. Minor, but noteworthy.

One interesting thing about electronic drumheads is that the pitch and head tightness are no longer linked. This means that if you accept your electronic kit as your primary drumming platform, rather than practice for “a real drumset”, it’s completely valid to tune up the heads quite a bit - you gain extra bounce, but it doesn’t affect the sound. Some may view it as cheating, but hey - the only reason we don’t do that on physical drums more is because of physical limitations, so I see no need to handicap myself. Bam! instant better drummer. :)

The rims don’t interact with the head in the same way an acoustic drum would. Basically, the TD-15K offers two modes - always rimshot, or rimshot when hitting the rim (configurably) hard, and cross-stick otherwise. With a little bit of preference tuning and some practice, it’s not too difficult to be able to consistently do one or the other in the latter mode.

If you play a lot of jazz, you will probably be disappointed in the lack of brush ability; you have to select drum sounds that have brush samples (which means no switching back and forth rapidly), and, as far as I can tell, there’s no way to perform the standard circular swish-swish.

The Cymbals

I wasn’t very worried about the drums on an electronic kit, as those are fairly easy to replicate. The cymbals, however, were a major concern.

They’re… reasonable. They definitely feel different than acoustic cymbals, but after a month or two of playing (mostly with my eyes closed!), they feel natural to me. The swing is fine, but you’re playing on rubber, and rubber bounces very differently than metal.

It’s possible to do swells on the TD-15K’s cymbals, which was a major concern for me. They have a minimum volume before the brain realizes what you’re trying to do, however, so the control isn’t quite up to what I want.

The bell on the ride takes a bit more force to trigger than I expected. You pretty much have to hit it with the trunk of the stick; no quick jazz-like accents with the tip here.

Although the crash and ride are both full-sensor (that is, you can play on any part of them as they naturally rotate), I’ve noticed they seem to pick up sounds much better on the side Roland expects to you play (as oriented by the logo).

The cymbals support choking, which is nice (and something you’re likely to forget to check on other kits). It takes more force than an actual cymbal, since you’re triggering a squeeze sensor, rather than stopping vibrations, but I can do it pretty reliably. In-store display kits may have a bit more trouble here, given the abuse they tend to get.

The hi-hat varies in texture, with the edge being very soft and rubbery, while the inside is thin rubber on top of a plastic base. This provides an option for getting in diddles when necessary, although it’s still not as easy as on a metal cymbal.

The Hi-Hat Pedal

The hi-hat pedal works reasonably well. There is a gradient from open to close (unlike some other kits, where you have only two or three levels), although it can be a bit hard to tell where you’re at without the visual guide of the two cymbals moving.

Unfortunately, the TD-15K provides no way to adjust the clutch of the hi-hat. There is no physical movement to clutch, and while I’d expected a setting buried somewhere in the brain, there is none. There is a fellow online who has modified his pedals to add an additional knob for this; while he has provided circuit diagrams, I think the only people who would undertake that process are those comfortable with eletronics wiring. The other suggestions given online are all along the lines of “just put a book on it while playing double-bass”.

Every now and then my hi-hat gets stuck open, and I have to stomp repeatedly on it to get it to trigger a close. I haven’t determined if this is a failure for which I should return it, a failure inherent in the system, or a result of playing on too squishy of a carpet floor.

One major advantage of the TD-15K’s hi-hat system, as opposed to one that mirrors a physical hi-hat, is that it’s a remote pedal. I didn’t consider this at all when purchasing, but when I started experimenting with non-traditional kit layouts a few months later, I found it extremely helpful. I now play with a centered hi-hat, something that on an acoustic kit would require a remote system to even try out, but with the TD-15K the placement of my hi-hat and hi-hat pedal are completely free from each other.

The Kick Pad

The same thing applies to the kick pad, although it’s not as accentuated. More importantly, having only a small target pad instead of a large bass drum saves a lot of space, allowing you to place other drums in the same space. I’ve found this helpful in lowering my toms to a position that would normally be physically impossible.

There’s not terribly much feel in a kick drum to start with, but the TD-15K feels right. I don’t really have anything more to say about that.

There is enough room for a double-bass pedal. Since the kit doesn’t come with a pedal, I decided I might as well get a double-bass, and I’ve been pretty happy with my GP Percussion DP778TN. I’ve read of some electronic kit drummers getting patches to protect their kickpad, but mine has stood up well to the hard plastic beaters on my pedal.

The Brain

One of the most important parts of an electronic drumset is the brain; it controls how many sensors you can have, how they act, and the entire sound of the kit. They also tend to be expensive, so upgrading the brain later is a major investment.

The TD-15 brain comes with a reasonable set of 50 pre-loaded kits. Many of them are esoteric, designed to show off some feature of the brain, or just to provide special effects. Nonetheless, there are enough choices of sounds to keep you satisfied for a while, and when you’re not you can purchase third-party kit packs for a reasonable price.

Editing kits is fairly easy, at least as easy as you can get without a touchpad or keyboard. However, some things are strangely limiting, like the hi-hat clutch problem mentioned above. It seems Roland has spent more time on their software than most hardware companies, but not as much as I’d like.

There are 1/4” jacks for headphones, L out and R out, and an 1/8” aux in. The headphones have a separate volume control on the side; I’d much rather have that on the large volume wheel on the front, since the non-headphone out generally goes to a soundboard balanced by someone else. There are a few additional ports that you can see from the specs, but I haven’t yet had need of them.

There are a variety of extra features that come baked-in. There’s a metronome, although it’s pretty basic for someone used to a Dr. Beat. There is some sort of instructional software that I haven’t played around with. And there are 50ish loops to practice jamming with, varying from a few bars to about a minute. All of these have a lot of room for improvement, but it’s nice that they’re there.

The Rack

This is my first experience using a rack to mount drums. It’s pretty much as I heard: easier to set up after moving, but harder to get things exactly where you want in the first place.

This rack isn’t particularly flexible. There are a few components that move up or down, or side to side, but the range is limited, and you can’t move mounts from one arm of the rack to another. This is a shame, because it seems optimized for an uncomfortable setup.

Let me digress for a moment onto the topic of drumset arrangement. With all instruments, there is a personal touch, but it’s far more important for a drummer than most musicians; while one piano will feel different than another, the keys are all the same size and in the same locations. This is not the case for drumsets, where two drummers may choose a different set of instruments, tune them very differently, position them in quite different ways, and in general make it annoying for the other to play their set. However, there is a certain setup that all beginners seem to use: a cross-over right-handed set with large spaces between all instruments. My guess is that this is partly due to the lack of precision in a beginner’s drumsticks - to avoid hitting the sticks against each other, they place everything far apart. The TD-15K’s rack is optimized for this setup, and getting it to something more reasonable can take some creativity; I was suffering from back pain for several weeks until I twisted all the components in strange enough ways to make them natural to my legs and arms. However, I was able to do it.

A few months into playing I switched to a centered hi-hat setup, and had to adjust the whole set all over again. Again, with enough coercion, the rack subsided to my demands.

It seems quite sufficiently stable, but is also fairly light and able to be carried with one arm while folded.

Overall

I’ve been quite happy with my purchase. A large part of it is being able to play regularly despite living in a shared apartment, but the sound of the kit (kits!) is also significantly better than my entry-level acoustic.

Aside from being able to practice, it has made it significantly easier to be part of a band. I can pack up the entire set, minus rack and throne, in one plastic moving box. We can practice in a fairly small room without needing earplugs. When we get around to performing, the sound guys will be able to balance from a position more accurate to the audience, and any recording can be done without needing to set up a whole bunch of mics.

It’s really nice having a full drumset that’s only a little less portable than a cajon. :)

I think the TD-15K is an excellent choice for most drummers.

Comparisons

The TD-11KV

The cymbals are horrible. I wouldn’t recommend this to anyone who had ever played an acoustic kit, or will ever play one in the future. It’s like trying to learn how to play piano on a keyboard - the feel is just wrong, and you’re better off with a cheap acoustic.

The TD-15KV

The TD-15KV is the next step up from the TD-15K; it uses the same brain, but has slightly better pads.

You get better pads than the -15K, but I don’t really have any problems with the lesser ones.

The other change is a “real” hi-hat: there are two cymbals stacked on each other, and you mount it to a stand like an acoustic hi-hat. The feel of the TD-15K’s FD-8 is actually pretty good, and if you want to try out a centered hi-hat setup (which you should, because it’s awesome), you’ll need to buy an expensive remote pedal system.

In short, the benefits are minimal, and not worth the price jump.

The TD-30K

If you really want an upgrade, you should bump to the next family in Roland’s line, the TD-30K. The primary benefit is a better brain, which unfortunately I haven’t had too much of a chance to play with. The price jump is significant, however, and I’m happy enough with my TD-15K I don’t see myself getting an upgrade unless my drumming becomes a part of my career. Roland has a series of videos where they’ve taken professional drummers (including Omar Hakim, an idol of mine) into the studio to play a TD-30K, and they all say it feels incredibly natural. Worth looking into if you have cash to burn.

Other Manufacturers

Yamaha, Alesis, and 2box all make e-kits I’ve heard recommended. You’ll have to see what’s available near you.

Acoustic Kit with Triggers

Don’t do this unless what you really want is an acoustic kit that’s occasionally recorded. You’ll lose the portability, compact size, acoustic flexibility, controlled sound, and metronome/etc. features of an electronic kit, while gaining very little.

Software Engineering in Systems Administration

2014-05-17T00:00:00+00:00

I am a Web Operations Engineer. While this encompasses a broad spectrum of duties, it boils down to ensuring ifixit.com spends as little time in failure states as possible. Or, to be more colloquial:

Ensure shit works.

While there are obviously technical problems abounding in such a technical context, my interest lies more in the human factors of failure - an interest that leads to a rather perverse desire to read books about airplane crashes and nuclear powerplant meltdowns.

In university, I switched from a computer science degree program to software engineering. At the time, this was driven by a desire to skip low-level hardware courses, but I grew to appreciate my choice for different reasons. Computer science, I tell incoming students and their parents, is a field focused on trying to make computers work better; software engineering wants to make computer scientists work better.

Software engineering is the application of engineering principles to the mathematically-derived field of computer science. It turns out that one of the big beefs engineers have with mathematicians is that the latter tend to ignore real-world uncertainties.

People are uncertainties.

People are all different. People get tired. People misunderstand instructions. People have imperfect memories. People are distracted by an upcoming birth, an impending divorce, or even just the football game on later tonight.

People make mistakes, and mistakes lead to failure.

Most sysadmins tend to approach problems from a very technical perspective - a customer reports that they can’t upload images, investigation shows users were no longer able to successfully upload images because the image processing job was throwing errors because there was a divide-by-zero error because the height was zero because a variable name was mistyped. Fix the typo, image uploads resume, on to the next task.

I, however, am interested in something further, the real root causes of problems. Why did the developer not notice this typo? Why did no one in code review or quality assurance notice it, either? Why didn’t any of our tools help them see it before production, and why didn’t any of our tools help me see it once it got there?

The problem stemmed from a human mistake - a simple misstep on the keyboard. Recognizing that humans make mistakes all the time, and building tools to deal with it, ah, now that’s software engineering.

In my university, software engineering was seen as a path for future managers - you trade a few technical courses for ones that just talk about people. “Bah! Computer-science-lite!” they say.

But I contend that anyone making computer programs for people to use must understand how those people use computers, or doom themselves to failure. And after all, aren’t humans just the most complex machines on the planet?

Save a life; study humans.

Sheet Music: Documentation for Musicians

2014-02-09T00:00:00+00:00

Musicians of all sorts are constantly practicing to improve their ability to manifest the music in their heads into the instruments they play. Where there is a large split is in how that music gets into their heads in the first place.

Some people are very good at playing by ear - they hear a song, know where those notes lay on their instrument, and can reproduce it fairly easily. This gives them a lot of freedom, for if they’re familiar with a song, they can play it without much practice or preparation, which is great at parties!

On the other end of the spectrum are the excellent sight-readers, who can play any piece from sheet music. A good sight-reader can play a song whether or not they’ve heard it before; before radios and recorded music became commonplace, this was often how music was spread. A reliance upon sheet music is both an advantage and disadvantage, for while a sight-reader without any music is infinitely less capable than their fellow ear-trained musician, they are limited in the music they play only by the size of their library, rather than the extent of their memory.

In this way, sheet music is documentation, just as much as software docs - it frees up your memory from storing how to play (use) a song (a tool) that you haven’t touched in months or years, and also helps introduce a new musician (user) to it.

Sheet music has an additional benefit when learning new songs. Frequently, sheet music will have additional notations specifying fingering and sticking; while a musician learning by ear can certainly learn how to make the same sounds, they may find out much later that what they thought was so complicated is really quite simple when playing it the way the author intended. Sound like any software experiences you’ve had?

Of course, most professional musicians will strive to be good at both sight-reading and playing by ear. Although there’s no direct parallel to software for playing by ear, it’s important to keep in mind that we have two parts to take care of as well - intuitive design that works well without training, and good documentation to fall back on for advanced usage.

The Myth of the Informed Consumer

2013-11-01T00:00:00+00:00

One of the primary problems of unregulated capitalism (that is, of the libertarian variety[1]) is the assumption of a perfectly-informed consumer. We do not need regulatory laws, we say, because the market will regulate itself. If a merchant sells shoddy products, he will soon find himself out of business from a lack of customers.

While the assumption an unhappy customer will advertise their experience to their friends is a reasonble one, we tend to equate it with a similar, yet distinct, hypothesis: the number of unhappy customers and their friends will grow so rapidly as to soon form the majority of all potential customers.

What often happens in this global economy is the successful existence of a crappy company due primarily to its wide audience and small throughput. Many of its customers may leave dissatisfied, but it’s of no concern to the company

there are plenty more to replace them.

The tool we’ve created to combat this is the online marketplace. Not only does Amazon display user ratings of products, but the people selling them. (Ebay, Etsy, etc. perform similar roles.) We’ve even incorporated this into local (as opposed to global) business, through Yelp. Now when we buy, we buy with confidence we’re getting the best deal.

This system has its flaws, of course. There is the ever-present danger of falsified ratings. Users may make a mistake, or be unqualified to rate an item’s quality. And, one of my personal worst worries, all users may rate a product highly only to find it doesn’t live up to its expected lifetime - but by then, ratings are useless, as thousands have already been purchased, and it has been phased out of production in favor of a newer design.

But there is an even greater problem, one that is far more systemic. As E. F. Schumacher points out early in Small is Beautiful,

Economics deals with goods and services from the point of view of the market, where willing buyer meets willing seller. The buyer is essentially a bargain hunter; he is not concerned with the origin of the goods or the conditions under which they have been produced. His sole concern is to obtain the best value for his money.

Here we run into the problem of the uninformed consumer yet again. If I am interested in not supporting slave labor, how am I to execute this wish in my daily life? I can’t ask many of the merchants I interact with, and those I can have no better clue than I, for they are almost as removed. We have no clear picture available to us of a product’s life prior to the shelf, and who involved in that chain would like to illuminate the matter?

We do see companies provide some information, but it’s certain the market advantage is a considered part of the decision. When Chipotle advertises its open-range pork, what information don’t we know? When Apple advertises its products’ EPEAT Gold status, how many consumers realize they’re purchasing devices with planned obsolescence that fuel wars in central Africa?

At iFixit, we’re friends with the folks at Patagonia. Recently a Patagonia employee came to tell us about their quest for transparency in their product lines. While it’s absolutely awesome they’re doing this, they faced enormous challenges, resulting in much more control over the supply chain than they perhaps would’ve liked. And even with Patagonia, we have to place trust not only in the company’s desire to tell the truth, but their ability to see the whole situation and to correclty judge the reports of all their suppliers.

In short, while we’ve created a good enough system for judging products on a purely-economic basis, we’re absolutely ill-equipped to make purchasing decicions based on any broader criteria.

I believe technology can provide a solution. Just as Amazon and Yelp have helped small businesses to compete with the mega-corporations on values of quality and service, so is there an answer to this supra-economic information scarcity. After all, technology has been responsible for many great forward leaps in information exchange, from the printing press to the World Wide Web.

I don’t know what this solution is. I don’t know what it will look like. I only know that it will come, and it will empower our values to change the world we live in.

[1]: The reader may recall that I self-identify as a libertarian.

Discovering Secrets

2013-10-26T00:00:00+00:00

I use LastPass as my password manager of choice. One of the features available to LastPass users is password sharing, and one of the features of password sharing is the ability to choose whether the sharee will be able to view the password.

Despite the strong language in the sharing dialog (“password will remain a secret” vs. “password will be obtainable”), the LastPass help article (which I’m sure most users will never read) notes at the very bottom that

Savvy end users could potentially access the password if they capture it using advanced techniques during the login process

Savvy end users, eh? I wondered just what was required to expose a shared, “secret” password, and decided to experiment.

Setup

First, I created a new LastPass account. This is free, so that was no problem.

Secondly, in my normal LastPass account, I created a new credential, entitled “testing shared passwords”, with a password of “sharedpassword”. I shared this credential with my test account, with the default visibility option (password will remain a secret) set.

Attempt 1: Bypass Edit View

Viewing the credential through the LastPass website didn’t provide very much:

so I logged in to the Chrome plugin, which provides a much more useful “edit” view:

Ok, well, let’s right-click on the password box and click “Inspect Element”, and see what we find.

Mmhmm, ok, and what might the value of that box be?

You can also get that value by navigating to “Properties” in the right-side box on the Elements tab:

Result: success.

Attempt 2: Bypass Auto-fill View

Since example.com is not an actual site with user accounts, I redirected example.com to localhost in my /etc/hosts, then created an index.html with only the contents <input type="password" id="password" /> and started up a local webserver with sudo python -m SimpleHTTPServer 80. This gave me a simple page, with nothing but the LastPass-filled field:

This proved to be vulnerable in exactly the same way as the plugin edit page.

Remember that in a real attack, the site would be directly accessible; the attacker does not have to create a fake version of the site and host it locally.

Result: success.

Attempt 3: Falsifying the Host

While we’ve shown the client is insecure, what can we do on the server-side?

There are a myriad of ways to create a simple web application; I decided to use Flask.

For this attack, the attacker does need to redirect the target credential’s site in /etc/hosts:

127.0.0.1 example.com

I then created a simple Flask application that would pretend to be example.com, but log any POSTs to the console:

#!/usr/bin/env python

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def index():
   if request.method == 'GET':
      return '''
<form method=post action='/'>
   <input type="password" id="password" name="password" />
   <input type=submit>
</form>
      '''
   else:
      for (key, value) in request.form.items():
         print '%s: %s' % (key, value)
      return ''

if __name__ == '__main__':
   app.run(port=80, debug=True)

When run and submitted to, we indeed see the password:

[$]> # root privileges are required to open ports 1024 and below.
[$]> sudo ./example_com.py
 * Running on http://127.0.0.1:80/
 * Restarting with reloader
127.0.0.1 - - [26/Oct/2013 13:59:02] "GET / HTTP/1.1" 200 -
password: sharedpassword
127.0.0.1 - - [26/Oct/2013 13:59:37] "POST / HTTP/1.1" 200 -

Result: success.

Attempt 4: Sniffing the Traffic

And of course, if the connection is not encrypted, packet-analysis tools like Wireshark can easily pull form data out of a request:

Result: success.

Conclusions

LastPass’s secret shared passwords are nothing of the sort; not only are they not a feature, the illusion of secrecy is enough to make me label them as an anti-feature.

LastPass’s fault lies not in a failure to properly secure a system, but rather in implementing a system impossible to secure. They could force TLS connections to both encrypt the network traffic and provide a reasonable assurance the destination server is actually owned by the proper authority, and the LastPass vault could be adjusted to not display the password at all (like the website). But at the end of the day passwords are meant to be entered into websites - and once the user receives that HTML, they can do anything they want to it, including pulling out passwords.

On Long-View Giving

2013-09-21T00:00:00+00:00

GiveWell is the lesser-known cousin of Charity Navigator - an organization created for rating charities. GiveWell distinguishes itself with in-depth research. But this comes at a price - they can evaluate fewer charities than Charity Navigator. And in fact, they recommend only three, and focus their research efforts on those.

GiveWell is founded on utilitarian principles similar to those espoused by Peter Singer, and as such, one of the four evaluated criteria is cost-effectiveness. As a utilitarian myself, I agree with the points GiveWell makes about preferring not only organizations that are more efficient, but those who make a greater impact per dollar.

There is one very important disagreement I have with them, however, and that is the matter of education.

GiveWell strongly encourages donors to only give to charities doing life-saving work - not only does saving lives fit nicely with utilitarian principles, but it’s fairly easy to measure.

Education, on the other hand, is a long-term ploy - while we can see test scores change, the true benefits of an education won’t play out for years to come.

And it’s not just a matter of basic education on the spread of disease. Teaching at-risk women entrepeneurship helps keep them out of prostitution, giving them enough financial and emotional stability to send their children to school. Teaching those children leads to a growing population of native doctors, engineers, and teachers who can build up a nation’s bodies, infrastructure, and minds.

Even helping education initiatives in the First World can be seen as an effort to help the Third. How many of us blessed to live in more fortunate circumstances can say we would be concerned with the affairs of others so far away without our education, or able to provide the money and skills we do without that same schooling?

I am particularly fond of promoting computer programming in education. There is a huge mismatch between the education in computer science and that of other subjects. And who do you think created GiveWell, Charity Navigator, the external and internal websites of all these charities? Programmers, programmers multiplying their influence to affect millions of donors, and billions of aid recipients.

We must continue to invest in high-return causes with immediate benefits. But we cannot forget that education is the key to the future.

Don't Waste Programming

2013-08-25T00:00:00+00:00

One of my coworkers recently created a distributed queueing system. There are about one billion of these already. I see this sort of thing all the time, and it frustrates me to no end, because there’s no point to it.

Yes, yes, I’m familiar with the arguments:

Oh, I didn’t know about any of those. (Then spend thirty seconds researching first.)
I just wanted something that was simple. (Right. Because you’ll never want any of those features, and all of the hard work solving various logical, portability, security, etc. problems is just extra stuff you won’t run into.)
I wanted to learn [trendy language]. (If that language is only suitable for creating things that already exist, why is it worth your time to learn it?)
I thought I should learn how a [type of system] works. (Then buckle down and read some code, because the way it works in your overly-simplified academic excercise is not how people in the real world do it.)

The most irritating part of it all is the wasted potential. As programmers, we can create almost anything we can imagine! And you use that ability to do… nothing of value?

The world we live in is an imperfect one. There are problems, big and small, everywhere you look. Some of those problems can be solved with software - software you can write! If we want to make our world a better place, we need to take advantage of the infinitely cheap reproduction of software and spend our time tackling new things, not rehashing the same old stuff.

Don’t know any problems you can work on? If you’re really desperate, I’ll give you a list of small projects I haven’t had time to complete, but you should really just spend some time talking to people. Everyone’s got stuff in their life that annoys them, and most people are all-too-glad to talk about it. I’d bet Ice-T could give you at least 50 problems well-designed software could help with.

We’ve got an incredible gift. Don’t waste it.

We're Almost There with Well Repair

2013-08-11T00:00:00+00:00

I recently ran across Integrated Community Development International (ICDI), one of many charities working to bring clean water to rural Africa. One thing sets ICDI apart - a focus on well maintenance. ICDI claims 60% of wells in the Central African Republic are non-functional. As others have noted, it’s far more attractive for donors to give to building a new well, or school, or something else very concrete than to provide funds to help ensure the previously-provided infrastructure continues to serve the community well.

So I am wishing ICDI much luck in popularizing the the idea of long-term giving; a shift in donors’ perspectives is absolutely crucial to fixing the way foreign aid is currently working. But at the same time, I think they miss a key idea.

ICDI funds maintenance teams to take care of the wells. These teams rove around the countryside visiting well locations, and the villages pay a monthly fee for their services. The fee is 8 USD a month, which may not seem like much to us, but is more significant when over sixty percent of the country’s population lives on less than $1.25 a day. ICDI themselves admit it takes time for the villages to figure out how to fund this fee.

What’s missing here is an investment in Africans, not just Africa. One of the fundamental ideas of Distributism is that of empowerment, that it is both economically advantageous and morally responsible to give people ownership of their own means of production. We shouldn’t be trapping these villages into dead-end support contracts they’re forced to pay to keep their clean water - we should be encouraging the right to repair by teaching them to repair their own wells.

Of course, not all villages will be able to spare someone to learn the ins and outs of well construction. But if we help one villager start a well repair business that services his neighboring towns, that’s one new business birthed and several less wells requiring constant funding from foreigners.

It’s time we not only see the heart-breaking conditions of the world’s poor, but the potential inside of them just looking for an opportunity to break out.

The Importance of Programming in the Modern Age

2013-06-21T00:00:00+00:00

I found myself musing once again over my motivation for programming - I’ll save you the reading and tell you that it’s not the problem solving, but the potential for creative expression that gets me. Programmers are the wizards of the modern age, creating anything they can imagine out of absolutely nothing. And this lead me to a realization:

The ability to program is the great democratization.

It used to be that land-owners had control over society. Then, as our world shifted away from an agricultural focus, the very rich joined the power club. But now, with a little bit of learning, anyone can have the control over their own life they’ve always wanted.

When most people use a computer, they have to put up with the poor software they encounter. They may think to themselves, “Gee, it’d be great if I could do X”, but then they sigh and continue on with their lives.

As a programmer, I am exempt from this. When I get frustrated at a piece of software, or see a need not being filled, I am no longer powerless - I can change it, with the power of programming.

And it’s not just your personal life. Hacktivism is a growing trend. And if the lawyers for Anonymous protestors get their way, the power to influence politics will shift - from the owners of the media to the programmers who can replicate themselves a thousand times over.

It’s true that some have forseen this and amassed large groups of programmers under their command. But even here we are not without hope. As video games have become a larger industry than movies, an opposing force of small, independent game developers has risen up to meet it. These are studios of only a few people - often, two - creating the type of games they’ve been frustrated to see the hundred-person studios avoid. And many times, these have proven wildly successful, proving the individual can still take the path to control.

If you find this subject interesting, read up on Distributism, a social, political, and economic theory that advocates giving individuals control over their own lives, something very much in line with the indie game movement.

Taking Another Look at the NSA Surveillance

2013-06-16T00:00:00+00:00

As others have pointed out, it seems a bit silly for Americans to be outraged at the data the NSA has been collecting on them while we willingly give every detail of our lives to corporations. Commentors have responded that the distinction is important, because the government can censor us, while Facebook can merely prevent us from expressing ourselves on Facebook. And how important is that, really?

We already know Facebook’s news feed isn’t a complete picture - there’s just too much information (and too much of it is particularly uninteresting) for the news feed to be a simple most-recent-first list of things your friends post. We don’t know exactly how Facebook decides what to feature; should we?

A recurring theme in cautionary fiction has always been warnings about censorship. In V for Vendetta, a personal favorite, the fascist government attempts to use its control over the media to convince its citizens to discredit and forget about V. And consider this - if some lone man promised to expose governmental corruption and bring down the system, with your help, how hesitant would you be to join him if you found nothing supporting him on Facebook, the Twitterverse was silent on the matter, and your Google searches found nothing positive?

A democracy will fail without informed voters. Voters can only be fully informed if others have the freedom to report their circumstances and opinions. And we cannot have freedom of speech without anonymity, which is exactly what pervasive recording like that done by Facebook, Google, and, yes, the NSA, strives against.

It’s true that we unwittingly give over hordes of information to corporations every day. But the similarities between this data collection and what the NSA does should not be used to excuse the NSA - they should be a sign that our outrage should be extended to privacy-lax corporations and, most importantly, ourselves.

Why Python Encourages Maintainable Systems

2013-04-28T00:00:00+00:00

I love Python. I don’t think it’s the right answer for everything, but I do act as its advocate for many new projects. Since most of my programming has been spent working with legacy systems, I am primarily concerned with maintainability, and Python is built on two ideas that directly benefit that cause: explicitness and a culture of documentation.

Explicitness

If you run import this in a Python console, you’ll get the Zen of Python - a series of guiding principles for the language. First is “beautiful is better than ugly”, and immediately following is my favorite:

Explicit is better than implicit.

Explicitness is a core part of Python; it’s been conciously taken into account when designing the language.

I’ve written previously on Python’s modules; let’s just say that it’s very explicit and gives me a hard-on, and we’ll leave it at that.

While Python is not statically-typed, it is strongly-typed. A string is a string, even if it happens to have numbers in it, so if you want to treat it as an integer, you must first explicitly convert it as such using int(). This helps prevent bugs that surface in weakly-typed languages such as C and PHP.

Perhaps the most obvious example of Python’s obsession with explicitness is self - the most hated part of Python other than using indentation to determine program structure. In Java, a seemingly benign assignment

foo = 'bar'

could either be assigning to a local variable or an instance variable. Worse, the action of this code can change completely by altering other code around it!

Python requires the instance to be explicitly listed as a parameter to class methods, and any alterations to instance variables must go through that variable. Non-Pythonistas think this is crazy. I think it has very good reasons for existing and we’re the only sane ones. Sigh.

A Culture of Documentation

The Python community, as a whole, loves to write documentation. In fact, they love to write good, nay, great documentation, just as much as Rubyists write unit tests for even the simplest programs. It’s just a part of the culture. But how did it end up this way - by accident? Indeed not, as the language itself encourages documentation through a number of means.

At a meta level, while much discussion happens on the mailing lists, important decisions about Python end up as PEPs. PEPs have several purposes, one of which is to document the arguments for and against various proposals, and the rationale used to choose one. When there’s a question of why something is the way it is, there’s a PEP with the corresponding answer.

At a level closer to implementation, Python uses a construct called docstrings. These are comments that are recognized by the Python parser as documentation for a module, class, or function, and bundled as part of the code they describe. Rather than having to dig through a complex webpage of documentation, a developer wishing to receive some information on a particular object need only pass it to the help() function to receive the documentation specified in its docstring. Since this is incredibly handy and doesn’t require any special tools, developers are encouraged to incorporate them into their own code, whichs leads to happier programmers down the road.

Conclusions

Should you rewrite your project in Python? Almost certainly not. But if the maintainability of a new project years in the future is of concern to you, please take a look at Python and consider what it can do for you.

On Being a Volunteer Sysadmin

2013-04-27T00:00:00+00:00

It’s kinda hard to do.

The actual process isn’t difficult (you just do work, like you would normally), but finding out how to contribute is much more of a PITA than it should be. Most notably, it’s more difficult than it is for programmers.

The open-source software movement has grown over the last 30 years to be continually easier to join, at least from a technical perspective. For simple changes, you can submit a set of changes to someone else’s code, in a manner that gives them (and any other interested parties) a notification and an easy way to review the changeset, without using anything more than a browser (hooray for Github!). That’s awesome!

Getting your code up onto a shared platform is insanely easy, too. Github allows a free account to have infinite open-source repositories; this, combined with the lack of oversight (you just put in a name, hit create, and push your repo) has lead to an increasing democratizing of open-source: instead of only sharing a few select projects that the author feels are high-quality or useful to many others, coders are increasingly sharing everything they make - and finding more people want to use it than they ever suspected!

But as a sysadmin, life is not so cheery. Yes, you’re a sysadmin, so life is never as cheery, but issue at hand is that of contribution. For instance, take a look at Skud’s current project, Growstuff.

The Growstuff repository has had people hacking away at it for a couple months. Non-core members have been contributing using the magical Github fork button. Things are great!

But then take a look at, oh, this wiki page on their shared development environment. It’s filled with apologies to the programmers for things that haven’t been done yet, a list of half-filled instructions for manually creating a user account, and a plea for someone to spend just a little time making this all better. But how would I do so? The server is a one-of-a-kind environment - I can’t fork it and submit a pull request after I’ve done some setup. To make the changes, I need to have root access on a shared box - something you want to be very careful about handing out.

And thus, projects start to pick up programmers, and suffer from a lack of good system administration. Such is the world.

At iFixit, we use Amazon’s web services to host all of our content. One of the primary differences between cloud services and a standard VPS-based solution is that any of your machines may disappear at any given time - poof! This is generally cited as a disadvantage, but I prefer to see it in a more positive light - accepting this as a fact of the system forces you to avoid hand-crafted servers and embrace automation.

A server that is configured entirely using a configuration management tool is one that can be forked. It may seem like more work than to just set it up by hand, but as is often the case with open-source, you’ll often find many people wanting to make contributions, but stumped at how to go about doing so. It can even be useful internally; as we’ve been moving our server configuration out of a proprietary system into a git repository, developers have been empowered to make pull requests for changes they want, but I haven’t had time to get to. Even if you distrust developers with root access, this system works - you only need give them elevated privileges on the testing server they spin up, and any changes still need to be pulled in to the canonical branch after they’ve been proposed, which gives you a chance to review them and make suggestions.

So here’s the point of all this: you need to open-source the server configuration for your open-source project. Yeah, you really do. The upfront investment isn’t as bad as you think, and the payoff will be worth it. There’s an army of sysadmins lurking in the shadows, waiting to be someone’s super-hero. Open up and let us share - you might be surprised at how well it works.

On the Freedom of Data

2013-03-08T00:00:00+00:00

This post is part of Mothball March, where I dust off old posts I never quite finished. This is a more-final version of something I originally wrote in December 2012.

In the 1980s, a young computer programmer named Richard Stallman started the Free Software Foundation with the goal of promoting “free software”, software that granted its users a number of rights, including the right to view the source code, determine how the program works, and use that method in any other software. RMS realized that without these rights, we didn’t truly own any of our software or, more importantly, our data - if the company manufacturing, say, your word processor went out of business, you could be stuck with a number of important business documents that you could no longer read. Over the last 30 years, we’ve made significant progress in convincing the technical community of this, and inroads are happening to non-technical folks as well. However, we now face a new fight.

Previously having the freedom to deconstruct a piece of software (that is, to obtain the algorithm used to read data) was all that was needed, since you always had access to your own data. Now, with cloud-based services, that is often not the case.

Take Facebook as an example. Many people upload most, if not all, the photos they take directly to Facebook - and that’s where they continue to live. If Facebook decides to, say, start charging you (or any of your friends) $20 to view your photos, you can be outraged, you can throw a fit - but you’ll have to either pay the fee or give up those memories. This is a fairly far-fetched idea (although not as much as you might think), but any number of other issues could happen - Facebook could suddenly go out of business, or suffer catastrophic data loss, or even just slowly fade into obscurity without ever providing an easy way for you to retrieve your data.

In today’s networked world, free data is just as important as free software, if not more.

A previous (great!) company I worked for, XYZ Homework, provided an online homework service for customers of its books. The system we used was built on an open-source system, but if you wanted to switch off, you’d lose students’ accounts, progress on problems, the problems themselves (both you-created and XYZ-provided).

As a contrast, Github’s primary website is not open-source, but I have absolutely no concerns about it. Much of it is inherently open and distributable (the decentralized nature of git means that everyone working on a project has a full copy of it, including history), but those parts that aren’t, they provide access to with a well-documented and freely-available API. If you’re concerned about losing access to, say, your bug reports, you can easily back them up on a regular basis, and a number of people have developed open-source adapters that will convert between Github’s issue tracking system and popular alternatives.

If you’re a developer, please provide a way for your users to export their data into a competing service; if lock-in is the only thing keeping them with you, you’re doomed already.

And since all of us are users of something-or-the-other, it’s our responsiblity to give data freedom a place in the purchasing conversation - and, if necessary, to make businesses pay attention by refusing to use their services until they free our data.

Considering Ethics in Employment

2013-03-03T00:00:00+00:00

I’ve been reading some Peter Singer recently, particularly his writings on poverty. The very-short conclusion to his argument is that those of us living in developed nations are morally obligated to give to charities to supported those who need help most, in as high a quantity as we can give without being hurting our lifestyle (or our dependents’) in any significant way.

As an example, you could use your salary to purchase a nice Mercedes-Benz, or instead drive a similarly-sized Toyota and give the extra money you would have spent to Doctors Without Borders. Since this doesn’t have a drastic effect on your life (your car still transports you reliably from one place to another, is reasonably safe, etc.) and the benefit you can provide is so great (World Vision claims they can provide enough food and medicine to a child to help them survive for their first five years for only 19 USD), you are doing wrong if you do not follow this path. There are additional complexities to Singer’s argument, but I’ll let you read them yourself.

Through Code.org, I recently found SocialCoding4Good, a site that helps developers and sysadmins connect to projects that need their help. This is great, and I encourage you to sign up, or use some of Code.org’s other resources to find a place to help out.

However, I’d like to present a challenge: is this enough?

Computers are now used in every area of life, and correspondingly programmers work in a wide variety of industries. We need people working on airplanes and traffic lights and shipping logistics. There is a definite focus in Silicon Valley, however, on companies producing things that, well, aren’t really that important on a global scale. The question, then, is this: is it ok to donate a few hours of your time while working for yet another social media company when you could instead be saving lives 8 hours a day?

Easy-Peasy User Testing

2013-02-17T00:00:00+00:00

Steam’s recent Linux celebration sale brought a number of new games into my life. They are, on the whole, fun little games, well-deserving of the $2.50 or so I paid for them. There are a surprising number, however, that do something very important very incorrectly:

Tell me what to do.

It’s such a simple thing, but it can ruin my opinion of your game in 30 seconds. Some games I launch, start the first level, and… well, sit there, trying to figure out what’s supposed to happen.

Honestly, guys, if people hacking out a game in 48 hours can do it, so can you. And you really should do so before making your game available for purchase.

The real problem, though, is something deeper. This issue just makes it evident the developers did absolutely no gameplay testing on people not already familiar with the game. This is critical, even if we developers usually forget it (which is why we’ve nutured a society full of people frustrated at technology). And it’s not even that hard.

Ok, I’ll help you out. If you’re a mobile developer, just tap the shoulder of the guy in front of you at the Starbucks line and say something like

Excuse me. In exchange for me paying for your coffee, would you play this game I’m making while we stand in line?

Not. Difficult.

While mobile developers may have it slightly easier, it’s not terribly hard to do similar things for traditional applications; there are people everywhere, many of them already friends with you, and most willing to provide a few minutes of help in exchange for a beer, a sandwich, or even just karma.

Test your damn product.

Ubiquitous Programming Problems

2012-11-30T00:00:00+00:00

I am very much a supporter of the open-source movement. An essay on the moral obligation to open-source your software has been rattling around in my head for the past few weeks, and while I was mulling this over in the shower this morning, I realized something: most of my programmer colleagues don’t open-source their work because they don’t have any work to open-source.

This might sound strange - a programmer doesn’t have any programs? But many people only ever write code for their employer, which takes licensing out of their hands. The question, then, is: why don’t they write anything outside of work?

Whether it’s good for a programmer to write code outside of work is a heavily-debated subject, focusing on building programming chops vs. becoming a well-rounded individual. I’m not concerned with intentional choices though - rather than asking “Why don’t you program in your spare time? It’s good for you!” I’m asking “How can you not program in your spare time?”.

When I first got introduced to programming, my enthusiasm quickly died off. I made a program that spits out the numbers one to ten - great, and so what? But web development, oh, was that different! With some basic HTML lessons I could make my own web pages - something only gods could do! Suddenly my universe expanded, providing options I never considered possible previously. Much of my joy of programming comes from a continuation of this feeling - there are now so many problems in my life that I have the power to fix. Even if I choose not to, it makes me cope better with buggy or unfriendly software I use - if I complain about it, I’ll have to fix it or make a replacement, so sometimes I choose to just tone down the grumbles. :)

Many of these problems are very small, which makes them great for an hour’s hacking. For instance, I use development builds of my music player and was having troubles with it sometimes not reporting the music I played to the statistics-gathering website Last.fm. I could keep refreshing my profile page to see if tracks were added, but that required a browser window, flash running, waiting for the page to reload, and just seemed too heavy a solution. So, I spent a little time pulling tracks out of the RSS feed Last.fm helpfully provides and made lastfm-tail. It’s quite simple, but solves the problem exactly (I use it nearly every day), and since I had no reason not to, I made it publically available, both as source on Github and easily installable as a Ruby gem.

My life is filled with problems like this, and I can’t help but see the programming solutions to them. Why doesn’t everyone?

I’m also interested in the general consideration of this trait: is it good, bad, or insignificant when evaluating a programmer? There are companies that publically state they favor candidates with open-source involvement - is this one of the reasons?

Using Gnuplot for Great Justice!

2012-10-31T00:00:00+00:00

This post was originally published on It Broke And iFixit.

We here at iFixit are big fans of Graphite, but sometimes I want to plot data that hasn’t made its way into Graphite yet. For instance, today I wanted to see how our CSRF failures have changed over time.

Since we log every failure, we can pull out instances pretty easily with grep:

[$]> grep CSRF php_debug.log | head -n 5
Oct  1 01:16:39 10.250.0.0 php: ◼|>>> [0.0.0.0] Unknown user, invalid CSRF.  Found: klFWI Expected: TdWnO Array
Oct  1 01:16:39 10.250.0.0 php: ◼|>>> [0.0.0.0] Unknown user, invalid CSRF.  Found: klFWI Expected: TdWnO Array
Oct  1 01:16:40 10.250.0.0 php: ◼|>>> [0.0.0.0] Unknown user, invalid CSRF.  Found: b1B35 Expected: 5PDl2 Array
Oct  1 01:16:40 10.250.0.0 php: ◼|>>> [0.0.0.0] Unknown user, invalid CSRF.  Found: b1B35 Expected: 5PDl2 Array
Oct  1 01:16:42 10.250.0.0 php: ◼|>>> [0.0.0.0] Unknown user, invalid CSRF.  Found: MTXff Expected: CQgPQ Array

[$]> grep CSRF php_debug.log > csrf_failures.log

Grouping and counting these entries by date is easy with awk and sort:

[$]> awk '{print $1, $2}' csrf_failures.log | uniq -c
 148275 Oct 1
 120430 Oct 2
Oct 3
Oct 4
Oct 5
Oct 6
Oct 7
Oct 8
Oct 9
Oct 10
Oct 11
Oct 12
Oct 13
Oct 14
Oct 15
Oct 16
Oct 17
Oct 18
Oct 19
Oct 20
Oct 21
Oct 22
Oct 23
Oct 24
Oct 25
Oct 26
Oct 27
Oct 28
Oct 29
Oct 30
Oct 31

By the way, if you’re still scared of awk, go read Why you should learn just a little Awk - it gives you the basics you need to do really useful things without requiring you to learn all the intricacies of the tool.

Now, that’s an interesting set of data, but it’s a bit hard to visualize. Gnuplot is the standard solution for this problem, but its syntax is… a bit difficult to wrangle. Thankfully, Christian Wolf created eplot, a tool that takes stdin and munges it to be gnuplot-consumable. For terminal use, you’ll want to apply this patch:

--- eplot.orig  2012-10-12 17:07:35.000000000 -0700
+++ eplot       2012-10-12 17:09:06.000000000 -0700
@@ -377,6 +377,7 @@
                # ---- print the options
                com="echo '\n"+getStyleString+@oc["MiscOptions"]
                com=com+"set multiplot;\n" if doMultiPlot
+               com=com+"set terminal dumb;\n"
                com=com+"plot "+@oc["Range"]+comString+"\n'| gnuplot -persist"
                printAndRun(com)
                # ---- convert to PDF

Now, run our previous command through eplot and… tada!

[$]> awk '{print $1, $2}' csrf_failures.log | uniq -c | eplot 


  160000 ++---------+----------+-----------+----------+----------+---------++
         *          +          +  "/tmp/eplot20121031-4728-zc5s8k-0" ****** +
  140000 +*                                                                ++
         |*                                                                 |
         | *                                                                |
  120000 ++*                                                               ++
         |  *                                                               |
  100000 ++ *                                                              ++
         |   *                         ******                               |
         |   *****   ********      ****      *                              |
   80000 ++       ***        ******           *                            ++
         |                                     *                            |
   60000 ++                                    *                           ++
         |                                     *                            |
         |                                      *                           |
   40000 ++                                     *                          ++
         |                                      *                           |
   20000 ++                                     *                *         ++
         |                                       *              * *         |
         +          +          +           +     ***********  ** +* ********+
       0 ++---------+----------+-----------+-----*----+-----**---+-*-------+*
         0          5          10          15         20         25         30

Life Goals and Personal Finances

2012-10-07T00:00:00+00:00

Today I went to a talk at Silicon Valley Code Camp that was about managing your time effectively by using life goals. I’m summarizing horribly, but the basic idea was that you’d give yourself a goal in each of 7 areas (I think it was based on The Seven Habits of Highly Effective People) - a goal you’d like to achieve in 10 years. Then, you try to figure out what your plans should be for the next 5 years, such that you’ll be on path to achieving your 10-year goals. This planning keeps going down until you end up with daily goals, and then those are the things that you do. The great thing about this is that you can directly track why whatever you’re doing is helping you achieve your life goals (or why it isn’t!).

There’s an accounting system known as double-entry bookkeeping, in which every subtraction from an account has a corresponding addition to another account. While the motivation for this appears to have been error-detection, I find it more interesting from a money-tracking perspective - that is, seeing what your spending is going towards. For instance, money spent on a kitchen remodel is also “deposited” into the value of the house.

It seems these two concepts have great potential when joined together. Rather than track money and time separately, track how your spending (of both) contributes to your life goals (or doesn’t!). This particularly shines when addressing situations where either money or time can be spent to accomplish a goal - repaving the path to your front porch, obtaining a woven basket for storage, getting vegetables for your dinner.

Does any tool do this? Not that I know of. Time to get hacking.

Simple Apache Access Log Analysis

2012-08-16T00:00:00+00:00

This post was originally published on It Broke And iFixit.

Although there exist excellent tools for doing full log analysis, sometimes you just want to quickly pull out a little information. I recently found myself wanting to parse a section of our Apache access logs and find IPs consistently hitting the same URLs over and over. This lead to a line of awk, sort, uniq and grep chained together with pipes; since that didn’t handle variations very well, it’s now a line of awk, sort, uniq and grep chained together with pipes and wrapped in a little bit of Ruby. :)

Here is the Github gist.

Are Low Test Scores Our Enemy?

2012-07-01T00:00:00+00:00

I watched “Waiting for Superman” today. I had been impatiently waiting for an opportunity to do so since I found out about it several weeks ago; I think education is extremely important in life and feel the public schooling system here in the United States has in many ways discouraged students from becoming further educated. A documentary that critiques this system, then, filled me with anticipation.

As the film progressed, there were a few things that niggled at me; as is usually the case for me, I only began to realize what they were after spending some time reading more on the subject after the film.

When the film introduced the No Child Left Behind Act, I expected the initial praise to lead to disappointment at the negative aspects of increased standardized testing, an emotion that never came.

One of the major problems Michelle Rhee’s more-pay-for-better-teachers campaign introduced was possible cheating on the standardized tests; even if you don’t believe the allegations, it’s easy to see how such a situation could occur.

But even disregarding corruption in the schooling system, we have to ask ourselves - are we measuring the goal we want to achieve? The film touts raised test scores as if that’s what we want - children who are really, really good at taking tests. I don’t know about you, but I want my children to be intelligent, funny, fulfilled people throughout their lives, things that go so much further than our simple STAR tests.

When I first saw Ken Robinson’s famous TED talk on creativity, it resonated deeply with me. Ken has talked, both there and elsewhere, about a different perspective on intelligence, one that values art as highly as mathematics.

Successful entrepreneurship is based upon bringing new ideas to the marketplace. Inventors succeed when they come up with wildly novel creations. Famous mathematicians, while thanked for their contributions, are often overshadowed by great writers, poets, and actors.

I’m very fortunate to have the parents I do. While they encouraged me to do well in school, they realized that school alone could not make me a fulfilled person and spent time teaching me about music, nature, and religion. I remember talking to a friend of mine in 1st grade who, while we were covering addition and subtraction in class, was receiving education in multiplication and division from his parents. I was confused - why bother learning that now when you’ll get to it later and have so many other interesting things to do?

I’m also incredibly lucky to have found employment that is more of a passion than a job. In our high schools, kids are told to focus on the subjects that universities tend to favor for entrance exams - math, science, English. Go down the halls of my high school and ask teens what they want to do with their lives - those who don’t answer with a shrug will tell you they’ll probably go work in the oil fields or at a hair salon, not because they’d like to, but because they see no better choice. If we snuff out kids’ dreams by the time they get to high school, how do we expect to motivate them by telling them that college will help them achieve their dreams?

Our schools need to promote learning, not merely learning of a few choice subjects we deem the most important. Kids need to be exposed to all sorts of things they’d otherwise never touch - how do you expect someone to become a master potter if they never take a pottery class? And most importantly, we need to recognize that diversity is better for us all - math may keep our bodies alive, but music is for our souls.

I For One Welcome Our New Robot Overlords

2012-06-21T00:00:00+00:00

Programmers and system administrators love control. We like to micro-manage eensy-bitsy details so they’re just how we want them, even if the defaults would be perfectly within a normal person’s parameters.

As a Linux sysadmin, I find myself particularly this way when it comes to my servers. I hand-configure all the individual pieces of software and lovingly place them together. This gives me an incredibly deep knowledge of the system that’s useful for debugging, but it has its issues.

As your fleet of servers grows, it becomes increasingly more difficult to run shell commands and edit files across all machines. Tools can make this better, but it’s still imperfect. And in times of high stress (e.g. the website is down), humans make even more mistakes.

But that’s why we have robots. Robots make no mistakes.

Which isn’t really true. Robots make all sorts of mistakes, because you gave them imperfect programming. But we can test their behavior during a low-stress time, when we’re at our best. Then, when the shit’s hit the fan, we can trust the robot to stay cool.

Of course, I’m not talking about actual robots - I’m talking about automating system administration tasks. Yes, those very tasks that we so love to do ourselves. But in addition to us being mistake-prone, we’re not easily replicable, while our programs can be copied and run simultaneously on hundreds on machines.

I know you think you’re awesome (you are!), but look at the facts. Look, and weep for your humanity.

Robots	You
Fast	Cuddly?
Unflappable
Replicable
Don’t require sleep
Metal-hard abs

On the Importance of Documentation

2012-05-11T00:00:00+00:00

It’s your first day at Boppin’ Burgers.

After changing into your uniform and shaking hands with your fellow employees, you’re sent off by Bill, your manager, to go do some work. “Don’t worry,”, he says. “Here’s an order we just finished. Just make ‘em like this.”.

You look at the burger: slightly toasted bun, pattie slightly off-center, cheese dripping over the edge. You’re practically hopping with the excitement of creating these delicious meals for so many people.

At first, things go smoothly - a few orders are placed and you slap burgers onto buns. But then a customer complains that their burger didn’t have pickles on it. Pickles? I didn’t know we were supposed to put pickles on a #2. “Oh, we didn’t put any on that one we gave you at the beginning because the customer requested no pickles.” explains Carl, the coworker you’ve been told to ask any questions of.

Periodically, you get a request for chopped (rather than sliced) onions. Dutifully you grab a knife and chop the onion slices into small pieces before depositing them onto the burger. After doing this several times, Carl notices and stops you. “We get that request often enough that we keep a jar of pre-chopped onions over here.” he says, grabbing the jar from a cupboard. You stealthily dispose of the bowl you had been putting extra chopped onion in.

Four months later, Diana joins the Boppin’ Burgers band. When she asks for your feedback on her first burger, you nod approvingly and slide the pattie off-center a tad. “Why do we need to do that?” Diana asks. “I don’t know.” you respond, shrugging. “It’s just how we do things around here.”

All of these problems could be fixed by perfect communication - if you were live-blogging every detail of your work and Carl was reading it, he would’ve pointed you towards the onion jar the first time you needed it. However, most companies are not staffed by Borg drones, which is why we created Dozuki - Carl can spend a little bit of paid time writing out all the steps of creating a burger, and then new employees need only follow the step-by-step directions.

But internal documentation is just as vitally important for software companies as it is for those in the foodservice or manufacturing industries. The primary problem with using live code as documentation is that no live code is an ideal example - it won’t use every option that exists, and it’ll have special-case behavior that only makes sense in its specific context. More importantly, these things are usually not self-evident - you don’t know that you need to be asking questions, or what questions to be asking.

Just as an example burger created for a customer is inferior to burger-making instructions, learning how to use a library by looking at its use in an application will lead to incorrect assumptions and imperfect knowledge that documentation could prevent. Application examples illustrate a partial set of options, often without adequate descriptions of the reasoning for those options or the behavior of their interactions with each other; good documentation shows off all options, one at a time, with detailed reasoning for their use.

Yes, documentation requires upkeep, but so does code - if you don’t want to do any maintenance, don’t change anything. In most environments, the work put into writing documentation will be vastly less than what you gain - why do you think software vendors have Frequently Asked Questions lists?

If you’re looking to reduce training overhead, eliminate bugs, improve code cohesion, avoid reinventing the wheel - give documentation a try. You just might get addicted.

Open-Source as an Integral Part of Development

2012-05-09T00:00:00+00:00

Every time I start working on a new piece of code, my workflow is something like this:

Come up with a name. If none within 30 seconds, choose an unused one from the Wikipedia list of Cats characters.
Create directory in ~/Documents and cd into it.
tmux new -s project-name
git init
Create a new tmux window.
vim

I create a git repository before I even open my text editor or a REPL. As soon as I write down what I want to do in a README or hack out a script that spits something onto the screen without (too many) errors, I make my first commit. The first break, it goes up on Github. I am a programmer, and this is how I program.

I’ll upload it to Github after I clean it up a bit. One of my personal goals is to find every person who says this and badger them until they release their code.

Your code is ugly? I don’t care - mine’s uglier. I know the dirty secret all programmers share - code never emerges fully-formed, glistening in the sunlight as a shining example of the modern mind. No, the very first thing you write probably doesn’t even work. Oops, forgot semi-colon. Neither does the second

you forgot to define that variable. But gradually a form begins to take shape, crafted by your relentless fingers. This is where your brilliant ideas begin to appear - a chin here, an arm there. This is what I want to see - this fascinating growth, this inspiring genesis.

Stay away! you shout. It’s not finished! you protest.

But I am no mere outsider, no window-shopper casually glancing in. I am a craftsman, a brother - I recognize the unrefined beauty within and celebrate what is to come.

And if the inspiration leaves as quickly as it came, I may pull off the dusty sheet and continue your work someday. Perhaps I never even met you - this shop is long-abandoned, with only the works left to tell of its owner. But as I work, crafting upon your base, there is a connection - I may not have met you, but I know you.

What Can the iPad Teach Us About Software Engineering?

2012-03-30T00:00:00+00:00

Software engineering is interesting. You see, first computer science sprouted out of math as a way to solve a bunch of problems that were previously computationally infeasible. As we stretched the limits of computation more and more, though, we started to run into problems with these large systems we had created. So, reluctantly, the mathematicians turned to those smug know-it-alls, the engineers.

But software is inherently a completely different ballgame. That it’s all virtual has a number of implications; the one I want to talk about today is maintenance.

See, most traditional engineering projects go through a predictable (and sensible) lifespan - they’re designed, they’re built, they’re used, they’re retired. We’ve tried doing this with software, but the problem is that, due to its unique nature, we can continue modifying it after its built, oftentimes while people are still using it! Can you imagine the foundation of your office building being replaced while you’re still in it? Frightening! But since we have the capability to do this with software, we must - if our relentless desire to craft a better product doesn’t do it, the features and fixes our managers and clients request will.

This shift to flexible long-term thinking requires a fundamental shift in how we view construction. Let’s look at a real-world parallel for a moment.

The iPad 3

There’s been a slight bit of drama over Apple’s latest tablet; while we at iFixit aren’t too thrilled about the over-reliance on glue, some are arguing it leads to a better product for consumers.

Fresh out of the assembly line, this is true. Not only does the lack of fastening tabs make a slightly thinner product, but it is likely easier for Apple to produce, theoretically leading to lower prices.

But this is an extremely short-term view. When the battery starts losing life, you’ll be hard-pressed to replace it. Wanted to pass on your old device to your kid? Sorry, bud, but this is a consumable device.

Software is Not Consumable

Software lasts a very, very long time, and some poor sod is going to have to maintain it. Not only is it discourteous to program with a consumable-based mental model, it’s bad business sense.

Rich Hickey, in his talk Simple Made Easy, presented a graph of development speed on a project when following the “easiest” method vs. designing for simplicity.

Now, this graph is completely made up, but it represents his experience, and it seems not uncommon for others to agree with it.

Where does this leave us? We have to take our engineering lessons with a bit of skepticism - will this actually work for us?

And for chrissake, don’t glue your programs together.

Beautifully Indirect Interview Questions

2012-03-03T00:00:00+00:00

As I’m graduating soon, I’ve been interviewing with a number of companies. One of them had a wonderful way of getting to know my opinion on testing without asking me directly. After all, if asked “Do you think testing is important?”, pretty much everyone will say yes. In practice, though, we see that most code ends up with a distinct lack of tests.

In my case, I can see the benefits of testing, but the codebases I’ve worked with have had, at best, extremely sparse tests on the unit level. Several have been at the point where tests are rather difficult to write, since any functions that do exist are hundreds of lines long and do many, many things all mixed together.

I’m a pragmatic person - I write unit tests when they save me time (for instance, in a log parser with a number of different variations on the format). Most of the time, “good enough” testing can be done much quicker by hand, and any important issues are caught by higher-level Selenium tests.

Now, having full test coverage gives you a rather nice benefit - you can deploy at any time with confidence. This benefit, however, disappears when your test coverage dips below 100% - then you still need to go around testing things to make sure shit ain’t broke. I’ve never had the opportunity to work on a project like that, so I’m not test-driven; the reason I stress documentation so much is because I’ve used both projects with excellent documentation (Django) and those with absolutely none (*cough*).

But I digress.

The Questions

You’re a consultant who has just come in to a company. How do you get a rough feel for their code quality?

I mentioned first that I’d look through their VCS logs, mostly because I really like VCSs, but also because you can get a good rough feel for how many bugfixes vs. hotfixes vs. features are happening. After that, though, I’d pick a function on the site and try to follow the code that handles that action. Since this is generally how I’ve worked (“X is broken; fix it.”), it’s understandable that this is what I’m most interested in.

As a sidenote, this is also why I like the explicitness of, say, Django’s URL mapping - I go through the list of regexes until I find one that matches the URL, then I look at the function that’s listed. This is beautiful when you don’t know where to look in a large project and you don’t want to waste time guessing and searching.

Of course, the answer the interviewer was looking for was that I’d look for test coverage.

You said you’ve been using Product X; what would you improve about it?

Product X has no tests. The answer is to add tests. I didn’t even know it didn’t have tests (I don’t expect an open-source project to have them), so I talked instead about something that’s more important to the sysadmin part of me.

I was looking at lastfm-tail on your GitHub.

We talked a little about it, and I said I was pretty satisfied; it’s likely I won’t make any more changes.

This short little script has no tests. I don’t plan on writing any tests for it, since it’s far easier to just run it, and since I use it on a daily basis, I’m likely to find any bugs that I’d think to test for.

And The Lesson For Today Is…

It wasn’t until after the interviews that I noticed what these questions were driving at. This is excellent.

There are a lot of good things to do as a programmer. Unfortunately, you can’t do them all, so you have to prioritize them. Similarly, a company needs to have priorities. One of the reasons for an interview process is to try and see how well the two match up.

Notice that these questions don’t have right answers. I mean, there are answers that you’re looking for, but a candidate will just see them as being a matter of opinion, which they’re more likely to give freely than in a situation where they feel you’re looking for a particular thing.

The Programmer's UI

2012-01-27T00:00:00+00:00

I keep finding The Design of Everyday Things more and more useful, both in understanding why things are the way they are and in helping me improve my own designs. If you don’t have a copy of it, go get one. I’ll wait.

Ok, now that you’re back, I’d like to share how it helped me process the reasons for the existence of one of our users’ most dreaded enemies:

The Programmer’s UI

You’ve seen it. The interface only its designer could love.

Programmers are notorious for creating UIs like this - busy with buttons, lavished with links, filled with forms. But why is this? Why do we create such god-awful things that make our users curse us (and computers in general) to the deepest level of hell?

I think that it, very simply, has to do with one of the steps we take while using a system - determining which actions are available to us.

There are over 30 choices for a user in that screen above - thirty. Say it takes half a second to determine if an option matches the action you wish to take - you could easily spend 15 seconds just figuring out which button to press. The programmer who designed those buttons, however, already knows all about them, so it only takes him half a second - and then he wonders why anyone has difficulty! After all, every option is right there, at your fingertips!

As the arrogant souls that we are, programmers have a tendency to look down on their users. Why can’t they just do the right thing? Why are they doing that!? Why can’t I only have smart people use my software?

To help combat this, I find it useful to ask a fellow programmer for feedback - someone I trust and respect, but who isn’t familiar with whatever I’m working on. Then, if they tell me the UI sucks, I’m forced to admit it and make it better.

Of course, if you want to continue creating horrific UIs, I won’t stop you. It’s job security for me…

The Importance of Usability in Regards to Fire Fighting

2012-01-05T00:00:00+00:00

For Christmas, my parents thoughtfully presented me with a copy of Web Operations, a book in which a number of well-known people in, well, the web operations field write about… web operations.

Anyways, one of the chapters contains an essay by Richard Cook entitled “How Complex Systems Fail”, with commentary from John Allspaw. The thing that struck me the most out of those points was the comment that humans under stress make decicions that are often suboptimal; this struck home for me in relation to a recent experience.

An Anecdote

As part of my Christmas vacation, I stayed at my grandparents’ house for a few days. They’ve lived there for some time, so the house has a few issues - like poor plumbing.

Before taking a shower one morning, I used the toilet in, uh, a manner that would strain their poor pipes. However, flushing seemed to do the trick at first glance, so I continued on with my morning regimen. Some time later, I noticed that not everything had been successfully evacuated from the toilet bowl, so I did what any normal person would do - flushed it.

As the tank failed to drain and the water levels rapidly rose, panic rose within me.

My first thought was to hold down the seat cover. Thankfully, I realized rather quickly that this was a silly idea that wouldn’t really work. :)

Dismissing that, the only other option I saw was to plunge the toilet. Unfortunately, by the time I found a plunger, water had spilled onto the bathroom floor and made a mess of things.

Later, my grandpa asked me why I hadn’t just turned off the water. My answer:

It didn’t even occur to me.

And Here Comes Usability

The Design of Everyday Things is a wonderful book on general, everyday usability. In it, Donald Norman explains his idea of the process a user goes through to perform an action, termed “The Seven Stages of Action”.

One of the stages is translating an intention into an action that can actually be performed; this becomes difficult if the user is not aware of all the available actions or is presented with so many that they cannot easily choose the one that maps correctly to their intention. The former was my issue with the overflowing toilet - the “right” action was one that I never even considered, so how was I to be expected to choose it?

Now, I don’t mean to excuse myself by blaming the poor UI design of the bathroom, but I think it serves as a simple, but useful, example.

Think about your own software - when faced with an emergency, will a user be able to easily identify the options available to them (and then choose the correct one)? Or are they left floundering in a sea of choices, trying every reasonable option until one works?

Often we neglect polishing our internal tools - if it doesn’t affect the customer, it’s not particularly important. But I say to you that it does affect our customers, for it increases our mean time to repair in states of emergency.

Dear Princess Celestia

Make a list of all the tools you use when firefighting.
Improve their usability.
???
Profit.

IDEs are Uncohesive

2011-12-11T00:00:00+00:00

I dislike IDEs. I enjoy being able to swap out pieces of my development toolchain, bit by bit, without having to wait for someone to write a new plugin. I am a grumpy Unix programmer.

This Fall quarter, I took a class about software design. Yes, yes, the course title is “Individual Software Development”, but that’s not at all what it was about. We discussed CRC cards, Design Patterns, Design-by-Contract, and all sorts of ways to make your programs better.

One of these was cohesion.

It’s a generally accepted idea in software engineering that your programs should be cohesive; that is, each part should do one thing and those different parts should all work nicely together.

You might recognize this as one of the most well-known parts of the Unix Philosophy:

Write programs that do one thing and do it well.

IDEs are completely in violation of this principle - they do everything a programmer might want an application to do, and the same issues arise as with a poorly-cohesive method.

Remember that convenient method that calculates a value and then prints it out? It’s a little bit of a pain now that you want the value, but don’t want to send it to stdout just yet.

Remember that nifty way to manage versioning of files inside your editor? It’s a little bit of a pain now that you want to use Fossil instead of CVS.

It frankly astounds me that programmers would apply a lesson to one part of their professional life without applying it to another. In fact, many of them will vehemently defend both positions.

Save your future self - make all your tools tightly cohesive.

The Relationship Between Static Typing and Dependency Resolution

2011-12-03T00:00:00+00:00

They seem similar, to me.

In a statically-typed language, you specify that whoever’s using your code better give you an AbstractBullshitFactoryFactory or else there’ll be hell to pay. Similarly, a packager for a package manager with dependency resolution will tell the package manager that dammit, you need cups installed or else things just won’t work.

Contrast this to a dynamically-typed language (where you’re responsible for whatever you pass in; if it doesn’t work, that’s your own damn fault) and a package manager that lets you install whatever package you want, whether or not it relies upon things you don’t have.

Parallels can be drawn between Java’s interfaces and package manager’s provides clause.

Now, the big difference for me is that my package manager has a --nodeps option, while I’m not aware of any statically-typed languages that let me, as a programmer using a library, override type constraints. Also, automatic downloading of dependencies makes for significantly less work for me, while static type checking does no such thing.

Why Programmers are Egotistical

2011-11-21T00:00:00+00:00

I’m reading through The Design of Everyday Things right now. There’s a small section in the second chapter about learned helplessness and taught helplessness, where the basic idea presented is that when people fail repeatedly on a seemingly easy task, they come to believe that it’s impossible (for them) and just stop trying.

This violently reminded me of a number of incidents I’ve seen while a student in a CS department - students try something, it doesn’t do what they expect, and they give up in frustration (“I can’t do this!”). Those students tend to believe that successful programmers don’t encounter these issues, which is laughable to any professional programmer. We very rarely do things correctly the first time, but we just keep going.

Now, why do we keep going? I believe it’s because we’re all egotistical bastards. Well, egotistical somethings, at least.

I’ve heard a number of non-CS friends describe us as arrogant. I notice that my CS courses tend towards a higher ratio of narcissism than GEs. Hell, Larry Wall told us that one of the three great virtues of a programmer is hubris.

I think this is what keeps us going in the face of difficulties - we believe we’re smart enough to do anything, despite any evidence to the contrary.

So…?

What implications does this have on CS education?

My school has traditionally had a 50% fail rate in our introductory series (101, 102, 103), with 101 being taught in C. To address this, they’ve recently prepended an additional course that uses ActionScript, Racket or Java to teach basic CS concepts in various domains (games, music, mobile, etc.). This isn’t a particularly innovative idea, but it is a good one, and I think it helps with this problem.

By using an easier introductory course, students will (hopefully!) become convinced that they can, in fact, program, and this knowledge should help them persevere when faced with C, a language that is simple but not particularly intuitive to those new to a low-level of computer abstraction.

Beyond this… I don’t know. The rockstar programmer movement is doing a good job for intermediate programmers, so I guess we should invite DHH to give a talk to all our first-year students, or plaster the halls with the awesome puppy. ;)

Git's Staging Index is for the Real World

2011-10-01T00:00:00+00:00

I love git. One thing I love most about it is the staging index:

With most other version control systems, there’s 2 places to store data: your working copy (the folders/files that you’re currently using) and the datastore (where the version control decides how to pack and store your changes). In Git there’s a third option: the staging area (or index). It’s basically a loading dock where you get to determine what changes get shipped away.

This is a unique idea to git. There are two primary arguments I’ve heard against it:

It’s needlessly complicated.
You should never be checking in a state you haven’t tested.

While it’s true that you now have an additional step, I hardly think it’s a needless one; this complaint seems to come mostly from Subversion users who are trying to alias svn commit to git add . && git commit && git pull && git push.

The second is quite a valid argument. It’s not generally a good idea to pull only bits and pieces of what you’ve tested into your commit; it’s the same result as when you forget to add a file that you created (and your changes rely upon). I feel it’s quite necessary to allow this, though, because

I live in the real world.

I know, it’s crazy, but sometimes I have to deal with things that theoretically should never happen if we were completely following Foobar Programming, but the state of the world right now is that it’s broken and I have to fix it and our customers are calling and my boss is asking me why it isn’t done yet and I just don’t feel this is the right time for a major refactoring of the system.

Here’s a real example: at work, we have a daemon system we wrote to process queue requests. On our development machine, the workers source their code from one checkout of the code. This means that any developer wanting to test changes in the queue-processing system has to make changes to that one directory. This gets to be a bit annoying when multiple people have to do this simultaneously.

Now, yes, this needs to be changed. But I have 60 other tickets on me at the moment, and I can’t get them all done right now, so I have to deal with the situation for a bit longer. And if I was using Subversion, I’d have to remove this other person’s changes to this file, save the file, commit it, then revert my removal, whereas right now all I have to do is run git add -p.

BTW, I can actually test my changes separate from theirs, now: I just have to run git stash --keep-index, run my tests, git stash pop and continue on my way.

Here’s another reason I commonly use git add -p: differing test and production environments. There are a number of debug constants we’ve got set to true in development, but really, really don’t want to have that way in production. There are also changes that are necessary simply because the development environment isn’t set up completely the same as production. Should they be the same? Sure, but that will require me to rebuild our entire development infrastructure, and that ain’t happening by Tuesday.

tldr: staging index helps me get shit done.

Rate-Unlimited Internet

2011-05-13T00:00:00+00:00

In recent months, our house has had blessedly little contact with Charter, our ISP. We did get contacted in February, however, because I was attempting to regain data that I had lost in a tragic sysadminning accident - and apparently they aren’t happy about us downloading a terabyte and a half in a month.

You see, Charter (like most other American ISPs) has traffic caps. Now, I have no problem with monthly caps; it just seems to me that they should come without artificial rate limitations. I mean, whether I download 250 GB in 2 days or 30, what does it matter to them?

Oh, that’s right. Because they want me to pay more money.

Architects and Computer Science and Sealing Wax

2011-04-03T00:00:00+00:00

Any student in a Computer Science-y program has undoubtably heard their goal described as “a degree in computers”. Now, this doesn’t bother me as much as many others, but it is still a gross oversimplication. That is not my topic today, though.

The more sophisticated outsider will believe that our degree is in programming. And who can blame them? We certainly spend enough time doing it, yet we are still not quite at the truth. It may help to look at a similar, yet different, field.

Architecture students spend a vast amount of their time modelling, whether using computers or physical materials. An observer might think that this is what they went to school to learn. But it is actually much more - they learn how to design buildings, and merely use models to describe what they’ve created in their minds.

For us it is much the same. We design applications to perform some task, and use programming to transform our thoughts into something a bit more tangible (albeit still not physical). We spend much time programming because we design many things that need to be programmed. We take a few courses on the actual art of programming so that we can spend more time designing and less time getting the computer to see our design the same way as we do, just as architecture majors receive instruction in AutoCAD, not to become modellers, but to get more time to create ideas.

That is, after all, what we really deal in - ideas. Ideas so crazy that there’s no possible way to manifest them in the physical world.

We are working towards a degree in professional dreaming.

On the Inconsistency of the OS X Focus Model

2011-03-24T00:00:00+00:00

I recently switched jobs. My new work computer is a MacBook, so I am once again working primarily on OS X, something I haven’t done in a few years. Now, don’t get me wrong - I haven’t gone and installed Arch Linux on it yet, so I obviously find it decent enough to spend hours of my life with. However, there are a few things that are just incredibly irritating about the OS - and the focus model is one of them.

My standard working setup is to use two desktops (spaces) for most work. The first has a terminal on the right for editing and a Firefox window on the left for testing.

The second desktop has a Firefox window (or perhaps several, tiled) available for querying of… well, whatever.

Here is a common scenario for me:

Hack hack hack in terminal.
Switch to second desktop to look something up.
Switch back to first desktop and start typing.

Unfortunately, this doesn’t work very well in OS X - my keystrokes are instead sent to Desktop 1’s Firefox window!

You see, in OS X, focus is determined by application, rather than by window. While this doesn’t make very much sense to me (in my mind, the last thing focused on a desktop should be focused once again when I switch back to it), I suppose it does make a certain amount of sense to, shall we say, “normal” people. I’m hardly representative of their primary target audience, particularly as a tiling window manager user.

The real problem I have with the focus model, however, is that it isn’t consistent. If I want to open a new Firefox window on a blank desktop, I have to:

Switch to the desktop.
Focus Firefox (cmd+tab, Spotlight, Quicksilver, dock, etc.).
cmd+n

If focus is truly defined by application, I should be able to have Firefox previously focused (say, by browsing the web), then switch to the new desktop and open a new window, skipping step 2 entirely. However, focus switches to Finder.

And no, it’s not just something special about Finder; in my first example, switching to the second desktop automatically focuses Firefox because there isn’t a Terminal window open on that desktop. If I cmd+tab, however, I get focus switched to Terminal, even if there are other non-Firefox windows on that desktop. Beh.

Why Python's Modules Are Badass

2011-02-16T00:00:00+00:00

I like learning new languages. While I may not agree with all of the designers’ decisions, every non-esoteric language provides things to learn from.

One of my favorite things about Python is its approach to modules and namespaces.

Creating a module is easy

In the simplist cases, nothing special is needed. Want to use a class Foo defined in awesome.py? Just from awesome import Foo and you can utilize Foo in your code.

Adding directories doesn’t complicate your life much; create an empty file named __init__.py in a directory and you’ve got a module. For instance, given this directory structure

my_program.py
lib/__init__.py
lib/awesome.py

and building upon the previous definitions, my_program.py can use the statement from lib.awesome import Foo to make our Foo class available.

Explicitness leads to maintainability

Here’s a common situation in older PHP code:

require('user.php');
require('authenticate.php');
require('util.php');

if (logged_in()) { // Where is logged_in() defined?

Code navigation features or search tools can alleviate the irritation this causes, but it’s a solved problem! In Python, you’re likely to see something like

from user import User
from authenticate import logged_in, login
from util import time_since_login

if logged_in(): // logged_in is defined in authenticate.py

‘As’ promotes flexibility

Python allows you to import modules under different names. Say you’re developing some application that uses the cpickle module.

import cpickle

data = cpickle.load('file.pickle')

In an effort to make your application more “enterprisey”, your boss orders it moved over to the JVM, using Jython. But alas! cpython is a module written in C, and therefore not available on Jython. Do not fear - one line changed switches your program over to the pure-Python pickle module:

import pickle as cpickle

data = cpickle.load('file.pickle')

Notes and technicalities

If you want to clobber your namespace with everything defined in a module, you can use the from foo import * syntax. This is commonly done when working with GUI toolkits, but you’re liable to get lynched if you use this syntax more than once in a file.

Any code in __init__.py is run when a module is imported. I’ve seen this used to provide a succinct API for users while retaining code separation for the developers, like so:

# __init__.py
from foo import Foo
from bar import Bar

The way a language like Lua would change the name of an imported “module” is to change the assignment operation:

pickle = require('cpickle')

You can do something similar in Python, too, but it’s more direct to just use as.

Also, Python being Python, you can modify the globals() and locals() dictionaries if you really want to (and occasionally there’s reason to). So, things can easily be introduced into your namespace through other methods, although they usually aren’t.

Why I Don't Use Text Messaging

2011-01-17T00:00:00+00:00

That’s right - I am a 21-year old Software Engineering major who doesn’t use text messaging. Not only that, but it’s disabled entirely on my phone, so I don’t receive texts either.

Why, you ask? Having anticipated your potential question and bated you into asking it, I have kindly provided answers below.

Your whole feces lights up when she’s in the room

I have a flip-phone. You know, those things that were pretty standard back in 2007, when I got it? Typing letters on a numeric keypad is, well, painful. Even touchscreen virtual keyboards aren’t that great. I’m holding out for a phone with a qwerty hardware keyboard.

Do one thing and do it well

Phones make and receive phone calls. Computers browse the web, play games, take pictures, play music, and find directions. I have a phone - you have a portable computer. Recognize that, although they share superficial similarities, they are quite different devices.

Alright, poop coming out

In this day and age, people tend to be much more connected to each other than previously. Sure, it leads to increased dorama among teens. Yes, there are people who break up via text message. What really bothers me, though, is that people tend to not ever really be anywhere.

Just take a stroll around town, particularly a college town like SLO. 8 people at dinner, 3 with their heads in their phones. 30 students in a class, 12 browsing Facebook.

As easily-distracted as I am, I need as much help as I can get paying attention. One of my goals is to fill everyone with the assurance that if they’re talking to me, I’m listening; texting will only be yet another thing for me to fight.

Because that leads to fully-formed ideas

As a method of communication becomes easier, the overall quality of messages sent tends to drop. If someone writes you a letter (like, on paper, and sent through the non-electronic mailing system), they probably took some time to do so. Emails are a lot less formal, but still often have sentences and paragraphs. By the time we progress down through mailing lists and forums to Twitter, Facebook statuses, instant messaging, text messaging, and IRC, the mean quality of ideas being communicated is low enough to despair anyone concerned with the future of the human race.

Now, don’t get me wrong - I have a Google Talk account, and you can frequently find me on the Freenode and Rizon IRC networks. It’s just that the latter, in particular, fulfills my need for brain-dead thought-vomiting - no need to share any of that with people with whom I can have actual conversations.

Now, am I just a grumpy prematurely-old man? Probably.

Where I Talk About Myself

2011-01-04T00:00:00+00:00

It’s well-known that putting people to work in the right area can make all the difference between your star employee and the one who burns the building down. I seem to be more particular than most, so in addition to working on my discipline to do what I’d rather not, I thought I’d give any future managers a quick overview of my inclinations.

I specialize in generality. That is to say, I love knowing little bits of a lot of things. Whenever I find an interesting library or application, I add it to an extensive list of online bookmarks for easy recovery when needed. A friend of mine was starting a project that needed to quickly transmit large application states across the network; the web programmers in the group wanted to use JSON, while the C programmers preferred a rickety homebrewed binary format. When I was asked, I first suggested Google’s protocol buffers project, then, when that proved to be too complicated for their needs, BSON, which met both the simplicity requirements of the first group and the performance requirements of the second. It ended with me quite pleased with myself.

I have a particular dislike of most issue trackers, so I’ve tried many and can provide pros and cons of each. I can find you an OS, a webserver, a programming language, a framework, a templating library, a graphical toolkit, or damn near anything you need. Now that I’m writing this, it seems rather strange, since I was quite certain I hated requirements-gathering. But what I’ve described, my dear reader, is exactly that, and the fact that I’m addressing programmers and sysadmins rather than “customers” gives me little solace.

I believe that those who love their work in this field do so for one (or some combination) of three reasons. The first is the mental stimulation gained from solving puzzles; the second, the wonder of creating anything you can imagine, out of nothing. While these are perfectly fine motivations, I am primarily interested in the third: the joy of solving a problem for someone.

For this, I thrive in support roles. You’re tired of copying and pasting emails into your issue tracker? Here are three options that will create bug reports when you send emails to them, and I can hack up a Python script to do that with our current software, if you prefer. Frustrated with the poor copy-editing in your team’s documentation? I’ll have the CI tool run a few scripts to check for commonmistakes and email the team the offending member’s name in case of failure.

(As a sidenote, this puts me in a strange position, halfway between programmer and system administrator. I haven’t quite figured out what job title I should be looking for, so if you know, please share!) Editor’s note: the position is generally referred to as DevOps.

This also manifests strangely in that I don’t really like to work on a project from scratch - if I could be paid to work solely on refactoring (making things more pleasant for my fellow programmers, but producing no visible change to those further out), I would be a happy man. I suppose this is that drive combined with a bit of OCD and a desire to see things done The Right Way.

In the end, I suppose I’m just an inexperienced 21 year old, blabbering about how much he knows while still wet behind the ears. It is my sincere hope, however, that you have come to understand me a bit more, for better or for worse. Honesty is, after all, the best policy.

Dammit, my eyes!

2010-11-28T00:00:00+00:00

You’ll want to be going here.

Ruby for a long-term project? Eh...

2010-11-12T00:00:00+00:00

The other night, my housemate and I were discussing language choices for a project, and he mentioned Ruby. I surprised him by stating,

I wouldn’t let you start a team project in Ruby.

People who have talked with me might find this a bit strange. I love Ruby, use it in quite a few of my projects, and will tell anyone who listens about how Ruby takes the best parts of several different languages and combines them into something that’s just fun.

But while it’s an amazing language to write one-off scripts that I don’t expect anyone other than me to hack on, there are some serious flaws that prevent me from recommending it for anything meant to live for a while.

Matz doesn’t understand version numbers

Seriously. People shouldn’t be stuck on 1.8 because trying to run things on 1.9 breaks them. Upgrading from 1.9.0 to 1.9.2 shouldn’t break the primary build tool. Redmine shouldn’t require very specific versions of Ruby and Rails to work. We have this thing called Semantic Versioning, y’know.

The gem ecosystem is becoming CPAN

Five different modules that all do what you want, but in an incredibly buggy and not quite correct way? And they haven’t been updated in 5 years? Yeah…

Ruby == Rails ?

It’s damned difficult to find a good Ruby RSS feed that doesn’t spend (at least) half its time talking about Rails. I use Ruby. I don’t use Rails. This seems incomprehensible to some people.

Rails is a buzzword

Rails has done great things. I may disagree with it philosophically on some things, but no one can deny that it has brought a new focus on programmer-friendly frameworks for the web.

However, it became very hyped and buzzwordy, which normally is followed by a rush of not-particularly-skilled programmers wanting to Do Cool Stuff Now. You then end up with a particularly bottom-heavy triangle of people in the community, usually with a proportionate amount and quality of code. That’s what prompted Zed’s famous rant.

Explicitness is not the name of the game

Okay, this is more of a reason to use Python than a reason not to use Ruby. But in a Python project, if someone uses a function or class, I can take a look at where it was imported from (unless they did a from foo import *, which is almost always a Bad Thing), then just follow the path directly to a file, since module names are based directly on directory and file names. Ruby provides more flexibility in that you can define any module in any file, and even reopen modules to monkeypatch things in. While this additional flexibility could be useful in certain circumstances, it generally means I have to spend more time searching around for the defining code. It also means you can’t do cool stuff like iPython’s foo??, which shows the code for foo. Between that and docstrings, you’re set to explore unknown code while in the REPL.

Similar deal with lack of keyword arguments.

Now, could these things be fixed? Theoretically yes, but I think that you have to have the right mindset to start off with to avoid a huge uphill battle. And really, if Matz hasn’t made a commitment to version stability by now, then it’s indicative of a lack of care in that department, which doesn’t bode well for the future.

Ruby is my 10-year old son - he’s a blast to play cops and robbers with and take caving, but I’m sure as hell not putting him in charge of the household finances.

A Software Maintenance Course

2010-11-11T00:00:00+00:00

I love my department. I really do. I hear complaints from students elsewhere about having mostly theory-only classes, feeling like they’re receiving a Java certification, not touching on functional languages, not doing any team projects, not learning about version control, and I can proudly say that we do not have those issues.

Now, it’s not perfect, certainly. But, I have much hope that it will continue to improve.

This year, a new course has been added in front of 101, denoted by the course number 123. The actual implementation differs depending on the professor, but the important thing is that students are now being introduced to programming through tools that make it easy to make cool stuff, rather than looping through C arrays (woo!).

I was one of many people who whined and complained about the way things were, and am now convinced that maybe, just maybe, someone will listen to me once again if I leave my shadowy lurking place.

So, here it is:

We need a course for undergraduate students that teaches software maintenance.

Sounds like a blast, doesn’t it?

Most software development is maintenance.

I believe this. Anecdotally, my (rather limited) job experience has been primarily modifying code other people wrote, rather than making shiny new systems. Some studies have been done to support this, too; the ones in the link there are all rather old, though, so I’d be interested in seeing some new information.

Anyways, while the university should not be merely training students to be nice employment-ready drones, it should be giving them the tools to succeed in the rest of their career, and maintenance looks to be a significant portion of their future.

Working with someone else’s old code helps you write better new code.

I believe this, too (oh, I see! You write down the points that you believe!). People generally say that reading code makes you a better programmer (references forthcoming), and it sure as hell helps if it’s not your own code. ‘Cause then it’s like, cheating, y’know. You don’t get better at writing by only reading your own books.

But yes, having to modify code that is foreign to you teaches you what things are important to code comprehension, and what aren’t. In the words of Zed Shaw, you should avoid coding like an asshole.

I find that good people tend to write good, compasionate code, and assholes write total asshole code. Their code beats you over the head with their nerd-cock. Everything is clever. There’s no consistent style, or if there is it’s the style of someone who doesn’t care if you can read it. It’s all about them, not you, and that makes them an asshole.

…

You see, assholes don’t care whether other people can read their code. What they love is if you have to struggle to read their pieces of dense idiotic shit. To them, punctuation, style, indentation, spacing, or comments are just not necessary. You have to be able to read it the way they wrote it and fuck you if you can’t. They’re going to be different whether that helps you or not because goddammit, that’s what they fucking want to do and fuck you if you complain.

In school, the majority of our projects are 10-week one-offs. You get an assignment, hack on it for a bit, then turn it in and toss it in the trash right when the whole thing is starting to fall apart.

A few quarters ago I worked on a project for my AI course. It was in C++. I had never programmed in C++ before, nor did I receive any instruction other than “oh, it’s like C, but with classes” from the “C++ expert” on our team. Now, don’t get me wrong - they’re great guys. But this project was showing just how badly it needed to be re-architected and re-written because none of us knew what the hell we were doing, but we turned it in and went on with our merry little lives.

When people don’t have to clean up messes, they don’t learn to avoid making them in the first place. A friend of mine is on a team that had a heated debate about choice of implementation language. While the new and upcoming language some members wanted to use is, well, new and upcoming, it’s lacking in tools, documentation, and most importantly, well-developed libraries. I’ve used libraries that someone hacked out in a weekend and stuck up on GitHub - and ended up having to maintain them myself or remove the use of them in my project. But these people don’t know better, and don’t really care, either. So, inflict pain upon them and perhaps they’ll learn to be a bit more compassionate.

So, how do we do it?

An excellent question, my friends, and one I had no answer to when I started writing this page.

A little Googling led me to a paper describing a course at the University of Western Ontario, apparently given in late 1999 or early 2000. Unfortunately, most anything else is either as old as me and much too long to read, talking about a class without a practical project, or behind irritating paywalls. Pah.

Basically, I see our course being something like the SE Capstone series, albeit not as long or tied up in requirements documents and nonsense. After students are in teams (either of their own choosing or the professor’s), present them with a number of potential projects, both things that companies want done and needy open-source projects. Let the teams choose on what they wish to work and let them loose.

The thing that bothers me most about this design is that it tends towards large modifications. In my (again, very limited) experience, most of my work has been to do a little this and a little that, jumping around the codebase like caffeinated double dutch champion. The majority of open-source projects are like that, too; newcomers take ownership of something from the issue tracker and hack it out, then move on to the next.

This sort of approach to coursework seems anatagonistic to grading, however, and I’ve been assured that our classes need to be graded, not merely educational. So, I suppose we stick with Approach #1, unless any of you have a genius idea.

So that’s it, my grand plan. If this finds its way to you (I’ll assume it did since you’re reading this), leave me a comment below with any thoughts you have, or, if you feel like it, come find me on campus somewhere and chat for a bit. I love chatting about this sort of thing.

Fixing Rake in Ruby 1.9.2

2010-10-25T00:00:00+00:00

When a task in Rake fails, Rake is supposed to simply tell me it failed and then tell me to use the --trace option if I want to know more. However, tonight it was acting as though I always called it with --trace:

┌─[pearson@Bragi] - [~/temp] - [Sun Oct 24, 11:41]
└─[$]> cat Rakefile 
task :foo do
	fail
end
┌─[pearson@Bragi] - [~/temp] - [Sun Oct 24, 11:43]
└─[$]> /usr/bin/rake foo
(in /home/pearson/temp)
rake aborted!

/home/pearson/temp/Rakefile:2:in `block in <top (required)>'
/usr/lib/ruby/1.9.1/rake.rb:634:in `call'
/usr/lib/ruby/1.9.1/rake.rb:634:in `block in execute'
/usr/lib/ruby/1.9.1/rake.rb:629:in `each'
/usr/lib/ruby/1.9.1/rake.rb:629:in `execute'
/usr/lib/ruby/1.9.1/rake.rb:595:in `block in invoke_with_call_chain'
/usr/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
/usr/lib/ruby/1.9.1/rake.rb:588:in `invoke_with_call_chain'
/usr/lib/ruby/1.9.1/rake.rb:581:in `invoke'
/usr/lib/ruby/1.9.1/rake.rb:2041:in `invoke_task'
/usr/lib/ruby/1.9.1/rake.rb:2019:in `block (2 levels) in top_level'
/usr/lib/ruby/1.9.1/rake.rb:2019:in `each'
/usr/lib/ruby/1.9.1/rake.rb:2019:in `block in top_level'
/usr/lib/ruby/1.9.1/rake.rb:2058:in `standard_exception_handling'
/usr/lib/ruby/1.9.1/rake.rb:2013:in `top_level'
/usr/lib/ruby/1.9.1/rake.rb:1992:in `run'
/usr/bin/rake:31:in `<main>'

I tried doing the reverse of the make-Rake-always-trace trick, but that didn’t seem to have any effect.

I thought about digging through Rake’s source to see what was happening, but that was far more work than I wanted to do, and could do at this point in time.

So, instead, I had a full write-up to StackOverflow, and moments from posting, noticed that the include path was using 1.9.1, even though I had Ruby 1.9.2 installed. Then I remembered that Ruby 1.9.2 did some weird stuff with gems.

Eventually, I tried re-installing rake, from gem, rather than the version included in 1.9.2. This lead me to a different error:

┌─[pearson@Bragi] - [~/temp] - [Sun Oct 24, 11:58]
└─[$]> rake foo
/usr/lib/ruby/1.9.1/rubygems.rb:340:in `bin_path': can't find executable rake for rake-0.8.7 (Gem::Exception)
	from /usr/bin/rake:19:in `<main>'

and then the terrifying advice from J. Random Blogger:

Apparently 1.9.2 comes with a version of rake internally, but is unable to find it for some reason relating to the rake.gemspec file. Remove the file to fix this issue.

And you know what? It fixed it. I kept a backup just in case, but damn, that’s sketchy. Ruby on my system is starting to feel like the CPAN mess I have to deal with at work - it’s gotten to the point that anytime I touch something vaguely related to Perl in yum, our software breaks, and I spend several hours in quality time with the CPAN shell downgrading and reinstalling packages.

Erratic Programmers

2010-08-24T00:00:00+00:00

I am an erratic programmer. I flit around from project to project, never staying long, even if I end up contributing for a long time overall.

This is why I love GitHub, git, and Ruby.

When I have an idea for a project, I can’t be arsed to set up Subversion; git init is all I need to start keeping revisions. After I’ve done a bit of hacking, I need to be able to throw it up somewhere more safe than my computer while minutes away from sleep. For GitHub, that looks something like this:

Go to GitHub.com.
Click “New Repository”.
Enter some half-arsed name and click “Create Repository”.
Copy a line and paste it into my terminal.
git push origin master
sudo shutdown -hP now

I don’t have time to deal with Sourceforge’s approval process. I don’t need to worry about Google Code’s project cap. Seriously, 25 projects? I’m up to 59 on GitHub right now, with my 2nd year’s anniversary around the corner.

I expect my projects to be worked on for a few days, abandoned for a few months, worked on for a day, abandoned for a month, and so on. GitHub understands this. I see a potential improvement to someone’s project, and I just click “Fork”, “Edit” (oh, simple web-based editing, you are awesome), “Commit”, and send a Pull Request. Whenever the original author gets around to logging in and viewing his messages, he just has to go to his Fork Queue and pull in my change, easy-peasy. If either of these processes took more than 3 minutes, we wouldn’t do them.

Let me repeat that for you.

If either creating and submitting a patch for a project or merging in that patch takes more than 3 minutes, the erratic programmer won’t do it. Do not overestimate our attention span.

This is why easily-modifiable code is important. You hacked up a little script to do something cool, without extensive testing, a spec, or any of those other things that generally come with “real software”. I go looking for something that does X, find your script, and love it, except for the fact that it doesn’t also do Y. Golly gee, though, you wrote it in Ruby, so I can monkey-patch the class to add Y functionality while still allowing people to install your original gem.

I believe there was an ending somewhere, but it’s been replaced by some thoughts about AI programmers. Send me a pull request if you find it.

The Quality of the PHP Source

2010-08-23T00:00:00+00:00

I ran across this gem in the PHP source today:

/* MD% crypt implementation using the windows CryptoApi */

It worries me that a commit where a 5 is replaced with % made it through, even if it is only in a comment.

The Importance of Limiting Functions to One Thing

2010-08-20T00:00:00+00:00

One of the reasons I love dynamically-typed languages is that they tend to allow a programmer to use old unmaintained code in ways the original author didn’t anticipate. This is especially important if you’re trying to interface with a library that the user will likely already have installed previously, so you’d rather not vendorize it (include it in your source directly).

Using a dynamically-typed language doesn’t automagically preclude you from this problem, however; there are still a number anti-patterns that will lead to curses called down upon your name some unspecified time in the future.

I recently came across some code in the following form:

<div id="foo"><?php echo thingies_table('foo') ?></div>
<div id="bar"><?php echo thingies_table('bar') ?></div>
<div id="baz"><?php echo thingies_table('baz') ?></div>

[snip]

function thingies_table(parameter) {
	[get a list of thingies from the database]
	[do some stuff to the list]
	
	return build_table(thingies);
}

Now, this isn’t nearly as bad as the more common version:

<div id="foo"><?php thingies_table('foo') ?></div>
<div id="bar"><?php thingies_table('bar') ?></div>
<div id="baz"><?php thingies_table('baz') ?></div>

[snip]

function thingies_table(parameter) {
	[get a list of thingies from the database]
	[do some stuff to the list]
	
	print build_table(thingies);
}

but it’s still not good.

You see, I wanted to combine the results of several thingies_tables into one, something like

<div id="foo-bar-baz">
	<?php
	$foo = thingies_table('foo');
	$bar = thingies_table('bar');
	$baz = thingies_table('baz');
	print build_table(array_merge($foo, $bar, $baz));
	?>
</div>

This, however, isn’t possible with build_table inside thingies_table, which is why it should be pulled out, either into another function or the <div> code itself.

Be considerate to your successor and limit each function to only doing one thing.

Differing Usage of Python and Ruby

2010-08-03T00:00:00+00:00

I love using Ruby for quick one-off scripts that don’t even deserve to be placed in a file. For instance, I did something rather stupid the other night (deleted all the .specification files of my installed RubyGems), and, while my gems all still worked fine, gem has no idea they existed, which is a Bad Thing.

Anyways, I wanted to reinstall all of my gems:

┌─[pearson@Bragi] - [~] - [Tue Aug 03, 01:42]
└─[$]> ls /usr/lib/ruby/gems/1.9.1/gems 
addressable-2.1.2            do_sqlite3-0.10.2   linecache19-0.5.11  ruby_core_source-0.1.4
ansi-1.2.2                   erasmus-0.0.3       liquid-2.1.2        ruby-debug19-0.11.6
api_cache-0.2.0              extlib-0.9.15       maruku-0.6.0        ruby-debug-base19-0.11.23
archive-tar-minitar-0.5.2    facets-2.8.4        mechanize-1.0.0     rubyforge-2.0.4
bluecloth-2.0.7              fast-stemmer-1.0.0  mg-0.0.8            rubygems-update-1.3.7
bson-1.0.4                   gemcutter-0.6.1     mime-types-1.16     ruby-prof-0.7.3
builder-2.1.2                gherkin-2.1.5       mongo-0.18.3        ruby-prof-0.8.2
cheat-1.3.0                  gherkin-2.2.0       mongo-1.0.6         showoff-0.2.4
classifier-1.3.3             gist-1.2.1          mongo_ext-0.18.3    sinatra-1.0
columnize-0.3.1              git-1.2.5           mongo_ext-0.19.3    stemmer-1.0.1
compass-0.10.3               gli-1.1.1           net-scp-1.0.2       syntax-1.0.0
contest-0.1.2                haml-3.0.15         net-ssh-2.0.23      term-ansicolor-1.0.5
crack-0.1.8                  hoe-2.6.1           nokogiri-1.4.3.1    test-unit-2.1.1
cucumber-0.8.5               httparty-0.6.1      octopi-0.2.8        thor-0.14.0
curb-0.7.7.1                 hub-0.0.0           open4-1.0.1         thoughtbot-shoulda-2.11.1
data_objects-0.10.2          jekyll-0.6.2        polyglot-0.3.1      ticgit-0.3.6
diff-lcs-1.1.2               jeweler-1.4.0       rack-1.2.1          ticgit-2010.02.08
directory_watcher-1.3.2      json-1.4.3          rake-0.8.7          treetop-1.4.8
dm-aggregates-0.10.2         json_pure-1.2.0     rdiscount-1.6.5     trollop-1.16.2
dm-aggregates-1.0.0          json_pure-1.4.3     rdoc-2.5.9          turn-0.7.0
dm-core-0.10.2               light_mongo-0.4.0   RedCloth-4.2.3      xml-simple-1.0.12
dm-core-1.0.0                limgur-2.0.0        repl-1.0.0          yard-0.5.8
dm-mongo-adapter-0.2.0.pre3  linecache-0.43      rest-client-1.6.0

While I probably could have done this with sed, Ruby’s a bit more familiar to me, and allows me to play around in irb to make sure I’ve got things right.

sudo gem install $(ruby -e 'Dir.entries("/usr/lib/ruby/gems/1.9.1/gems").map {|d| d[/[A-z-]+/]}.map {|d| d.chop if !d.nil?}.each {|d| puts d}')
sudo gem install do_sqlite3 linecache19 open4 ruby-debug19 ruby-debug-base19

and that was it. Awesome.

The same thing in Python would’ve been something like

import os
import re
files = os.listdir("/usr/lib/ruby/gems/1.9.1/gems")
files = [re.match(r"([A-z-]+)", file).group()[:-1] for file in files]
for file in files:
	print file

or, in one-line style:

python -c 'import os; import re; for file in os.listdir("/usr/lib/ruby/gems/1.9.1/gems"): print re.match(r"([A-z-]+)", file).group()[:-1]'

…except for the fact that I apparently can’t put for statements on one line if they’re after anything else.

Anyways, I prefer Python (over Ruby) for any program that has me working with another programmer, either directly or through a (small) library. You see, Python has wonderful things like docstrings and help(), as well as a community that values documentation so much they actually write it. Write documentation? Willingly? Crazy, I know, but it really spoils you.

With Ruby, you often end up with some API documentation thrown up, and that’s about it. Even the Ruby core docs (which are pretty good) have nothing on the Python equivalent. And hell, the Python module for regular expressions not only has a rather long section detailing the meaning of various characters in regexes, but has a link to a full-fledged regex guide. This kind of documentation fanaticism trickles down to the one-man libraries in a way that makes me not cringe when having to learn how to call someone else’s code in a way that it does what I want.

So yeah, when it comes to writing things by myself, I love Ruby. But sometimes, the values I share with the Python community makes it worth putting up with crap like len.

HTML Templating in Ruby

2010-07-29T00:00:00+00:00

(or “Why I moved a project to Python”)

I have a relatively simple project that requires me to scrape off a few paragraphs from a website, then print ‘em out in a nice HTML page. Well, right now it’s not so nice:

which is why I decided to make an actual full-fledged HTML page, with <body> tags and everything!

Since I didn’t really want to output HTML directly from Ruby, I went looking for a nice, simple HTML templating system.

First up was mustache. It was a nice and simple Mustache.render IO.read("funi.html"), :entries => entries until I remembered that Mustache doesn’t allow any logic in templates, which meant that I couldn’t iterate over my entries in a loop and do whatever to them. Out.

Second was haml. I got my template all coverted over, and then plugged in a line similar to what I was using in another project: puts haml :index, :entries => entries. NameError? Huh?

So, it appears that the Sinatra folks added a haml method. That’s fine, I can just call haml programmatically, and… what the hell is this? How do I freakin’ call haml?

You see, all of the haml documentation covers how to write templates in haml, with none of it describing how to actually render the damn things. Any docs I could find on that were Rails-specific, which is utterly unhelpful for a script like this. I spent a while digging through the haml source before giving up in frustration. Seriously, guys. No.

Ah, liquid, my Smarty-like friend. Put the template back to HTML and throw in a puts Liquid::Template.parse(IO.read "funi.html").render('entries' => entries), and we’re good to go.

Liquid error: undefined method 'to_liquid' for #<Nokogiri::XML::NodeSet:0x9baf0c0>? But I’m not printing out anything from a NodeSet; I throw it into a loop, and make the appropriate calls to get a string for you to digest. Oh, that’s right - now I remember why I gave up on you on another project.

You see, liquid bills itself as being a “secure” templating system, which means that it’s fucking annoying to use. You end up with shit like this:

# a bunch of bullshit liquid requires
module Nokogiri
	module XML
		class NodeSet
			def to_liquid
				self
			end
		end
		class Element
			def to_liquid
				self
			end
		end
	end
end

just so you can use freakin’ NodeSets in your template. I’m not taking any of that.

Well, let’s see if the ruby toolbox has anything else interesting. Hmm, there’s erector:

require 'erector'

class Hello < Erector::Widget
	def content
		html do
			head do
				title "Hello"
			end
			body do
				text "Hello, "
				b @target, :class => 'big'
				text "!"
			end
		end
	end
end

Hello.new(:target => 'world').to_html

Lol no.

So yeah, I got fed up with this nonsense and ported over to Python so that I could use Jinja, which is a shame, because I really like Nokogiri. But hey, do what works, right?

(As a side note, one of the reasons I like Nokogiri is that it has actual documentation, rather than just API docs slapped up, like most Ruby projects. The Python community tends to treat documentation very seriously (while Rubyists feel the same vigor towards unit tests), which makes it rather difficult to use anything that’s not well-documented after having programmed in Python for a bit. That was one of my main problems with Android (and Rails, a few years ago), and it’s gotten to the point that I almost automatically dismiss any framework or library that doesn’t have pages of well-written documentation.)

Moving to vim

2010-07-20T00:00:00+00:00

I really want to move from Komodo to vim. Sure, I love Komodo, but sometimes it can just be freakin’ annoying.

So, here’s a list of things I want vim to be able to do for me to switch:

Display line numbers
Split screen horizontally or vertically, and move files between the two
Show a tree-based overview of a directory (preferably with the ability to open files from there). Showing and hiding this quickly is important.
Use the indentation already present in a file
Use the line-endings already present in a file
Show me when indentation is mismatched, or allow me to turn on whitespace characters easily.
Code folds are created automatically based on syntax/indentation/braces/whatever.
I can complete words that I’ve typed previously.
Tab completion.
Tooltips or similar remind me of function parameters.
I’m presented with valid choices for css rules.
Autocompletion of braces, parens, end blocks, etc.

Updates will be made to this page as I find answers.

How I Define Program Size

2010-07-01T00:00:00+00:00

A small program fits entirely in your head.

A medium program will not, but you can hold the overall structure and one piece in memory. When you need to work on something other than your current section, you know enough about the program on the whole that you know where to look, and can pick up that piece fairly easily.

A large program is like several medium programs mushed together - you know what’s happening overall in your section and specifically what’s happening in your subsection, or you’re some sort of overall architect and are aware of both the individual sections and how they work together, but have no idea about the specifics.

The Myth of Knowing Everything

2010-06-04T00:00:00+00:00

I occasionally hear people say, essentially, that all programming languages are the same. For instance, I’m taking a course from a difficult (but good) professor this next fall quarter. I’m a little worried, as the class is traditionally taught in C++, and I’m a scripting language guy. I’ll express this to someone and hear

Ah, but it doesn’t matter. Once you know one programming language, they’re pretty much all the same, right?

No, they aren’t.

Now, sure, if you take a C hacker and make him write Lisp, he’ll probably get the job done. But it’ll look like C, not Lisp, and that is a Bad Thing.

More importantly, take the Lisper and force him to use C, and he’ll be lost without all of the powerful features that he’s used to using all of the time (well, I suppose the C hacker would find the lack of pointers annoying, too).

You see the same thing with any change in paradigm: object-oriented vs. procedural, functional vs. imperative, dynamically-typed vs. statically-typed. If you’ve ever been on a project where someone is learning the implementation language, it’s painfully obvious which code is theirs, because they just approach problems differently.

Good programmers often recommend learning different programming languages, not merely for the practicality, but for the different mindsets gained from each. Trust them, and stop dismissing my fears. Fear can be a good thing.

Scala First Look - Leaky Abstractions

2010-06-04T00:00:00+00:00

I just received a copy of Programming in Scala and am working my way through it (slowly). There were a few things that struck me as odd to start with, but the authors explain that they are a result of everything being extensible (in the sense that standard operators are just predefined methods). Then, I came to non-scalar datatypes.

val array = Array("zero", "one", "two")
println(array(0))

val list = List("zero", "one", "two")
println(list(0))

val tuple = ("zero", "one", "two")
println(tuple._1)

Wait, what?

Apparently someone else had the same reaction as me, as the section about tuples ends with an explanatory infobox:

You may be wondering why you can’t access the elements of a tuple like the elements of a list, for example, with “pair(0)”. The reason is that a list’s apply method always returns the same type, but each element of a tuple may be a different type: _1 can have one result type, _2 another, and so on. These _N numbers are one-based, instead of zero-based, because starting with 1 is a tradition set by other languages with statically typed tuples, such as Haskell and ML.

Truth be told, I didn’t even notice the 1-based indexing until I read this, since I was so distracted by the incredible break in syntax conventions, and I really dislike 1-based indexes.

This is one of the worst leaky abstractions I’ve seen in a while. The underlying system doesn’t allow the expected behavior, so I as a Scala programmer am thrown off and confused by this seemingly unexplainable change. This is exactly the kind of thing that Matz is referring to when he states that Ruby follows the Principle of Least Astonishment.

Postfix in a Functional Language

2010-05-21T00:00:00+00:00

One of my issues with Scheme was that I found myself having to think ahead and write backwards. For instance, given these (fairly simple) English instructions:

Multiply 5 times 5, and 3 times 2, then add them together, and subtract from 5.

you’re likely to end up with code something like this:

(- 5
  (+ (* 3 2)
     (* 5 5)))

Backwards, backwards, backwards!

In postfix, however, you get this:

(((5 5 *)
  (3 2 *)
  +)
  5 -)

or, if you assume only two arguments for each operator,

5 *
2 *
+
-

which reads more like standard procedural code:

a = 5 * 5
b = 3 * 2
c = a + b
return 5 - c

(or so). Easier for me to grok.

It seems that there are very few programming languages that use postfix other than those used primarily for calulators, which is a shame.

Don't Ask Me Commands

2010-05-14T00:00:00+00:00

I’ve noticed that some team leads (who are often placed in such a position due to their technical prowess, rather than any leadership ability) have a tendency to ask questions when they really want to tell me to do something. For instance:

So, are we going to have the FooBar done by Monday?

My response to that question will be based upon my current set of priorities, and very well may be “No.”. Now, if you said

Make sure FooBar is done by Monday.

then I will adjust my schedule accordingly.

From a programming perspective, questions are effect-free. If you call, say, piano.tuned? (or piano.getTuned()), no instance variables in piano should be modified!

So, if/when you lead a team, no matter how small, please grow enough balls to tell someone when you want something.

If Women

2010-05-10T00:00:00+00:00

One of my housemates was discussing some class material he was assigned to read that claimed that, among other things, women are better at multitasking. I was thinking about how men tend to think very procedurally, which is also how we tend to write programs. So then, if early programmers were mostly women, would we be having as many issues with concurrency as we are now?

I’d be interested to know if there’s any sort of correlation between gender and ability to easily grok different concurrency models.

Are You Trying to Write a Novel Without Reading Classics?

2010-04-28T00:00:00+00:00

While writing some thoughts of mine about open-source software (for which I broke my 10-minute rule and haven’t finished for a few days >_>) I realized that there are two primary reasons I think open-source programmers tend to be better than the average coder:

They write more code.
They read more code.

While I’ve heard lots of contention over the first point, I don’t recall reading much of anything about the second. The topic just came up in my LUG’s irc channel, though, and I got two people to agree with me (and none dissenting, so I must assume I’m correct!).

@xiong_chiamiov realizing you suck is one of the most important parts of learning how to code, imo

@xiong_chiamiov which is why I strongly recommend /reading/ other people’s code on a regular basis

I often think that I’ve come up with a pretty cool solution, until I see what other people have done. I saw this with my “discovery” of the model-view paradigm, with my second irc bot, and repeatedly in my Programming Languages course (damn you Austin!).

Oh, but yes, back to what I mentioned in the title - programmers who don’t make a habit of reading other people’s code are like authors who don’t read others’ books.

So, whether it be a short story or an epic novel, find yourself some printouts and get reading.

Open-Source vs. Opened-Source

2010-04-22T00:00:00+00:00

Increasingly, companies are starting to realize the benefits of open-source software. While you effectively lose the ability to sell the software itself, you get free fantastic testers, who will even sometimes contribute patches to fix issues that they discover. Any large OSS project has at least one person who has put tremendous effort into writing helpful documentation, too, which tends to be sorely lacking in proprietary systems.

I’ve noticed some companies attempt to tap into this without fulling understanding that open-source is not merely an issue of licensing, but a viewpoint, almost a development methodology in itself.

In most OSS projects, there is a leader (or a core team) who has to approve every change to the code. Aside from the benefits of inherent code review, this means that a sufficiently strong-willed leader can set strict rules (for instance, all changes to the API must be accompanied by a corresponding change in the documentation) that really, honest-to-goodness, must be followed for your code to make its way into the official version.

Developers who contribute to projects in their spare time tend to code a lot (and read lots of code from people much better than them!), so you also tend to get higher-quality code on average in OSS than from run-of-the-mill “software engineers”.

Anyways, in the past few years I’ve seen more projects “open-sourced”, with companies hoping to get the community to fix all the problems created by their crappy programmers.

As much as I love Komodo Edit, it is a perfect example of this. Although ActiveState tries to encourage community participation, every single open bug (and there are a lot!) is owned by a foo@activestate.com. This is not the sign of a healthy community, and OpenKomodo has been around since 2007.

Take a look at something like Django or Rails, though:

Each of those different colors represents a different person making commits, with the size being directly related to the number of lines changed. This, my friends, is working open-source.

Roving Bands of Programmers

2010-04-13T00:00:00+00:00

Right now, one of my housemates is watching a National Geographic special on YouTube for one of his classes (yeah, yeah, terrible copyright infringement on the part of his professor). They were talking about small hunter-gatherer groups, which somehow reminded me of small Agile teams.

“Why don’t they band in large groups and decide things in committee?” I snickered.

Then, the narrator mentioned how the groups moved wherever food was available, and I had an amazing thought - what if we had roving bands of programmers?

Imagine this: a team of highly-skilled programmers fend off boredom by changing targets every week or so. Their tight communication and above-average skill, combined with broad experience and a fresh look at things, could resolve some of those long-standing tickets every project has. Most open-source projects tend to have only one main developer, so the arrival of these roguish hackers could be a real energy shot.

Then, without a warning, they disappear into the night, as quickly as they came…

Tree Traversal of Wikipedia

2010-04-09T00:00:00+00:00

I often hear computer-sciencey nerds taking informal polls to see if more of their friends browse Wikipedia using breadth-first or depth-first traversal strategies. You should all be aware by now of how these trees are generated (below, xkcd #214).

Anyways, the difference between those two is simply whether new tabs are opened next to the current tab, or on the far right.

Much more interesting, to me, is the difference between depth-first (also called preorder) and postorder traversal, which boils down to whether or not the root (the article you are currently reading) is visited (completely read) before its children (the links you open as you read). Proponents of depth-first ridicule the postorderers’ skittish attention, while the latter retort back that they prefer to actually understand what they’re reading.

Now, since I use the amazing navigation popups, I can get an idea of what something is without having to leave the article I’m reading. gg.

Thoughts on Team Size

2010-04-04T00:00:00+00:00

Working on a team is a fact of reality for most people, programmers not excluded. Now, there is a lot to say about teams, but I’m really just interested in spouting off a few thoughts about the team’s size, at the moment.

Teams of five to eight seem to work well, in my (rather limited) experience. Someone will emerge as the leader, someone will keep them on track, and the rest of us end up fitting together with our own specialities and peculiarities. There may be some troubles between some of the more conflicting personalities, but you’re likely to have a mediator to help resolve the situation peacefully.

Pairs, however, suck, or rather, have high potential to suck. If you don’t choose your partner carefully, then you may be missing a personality trait that is essential for your success.

Even more importantly, everything becomes personal. With a larger team, having the team leader tell you to get something done isn’t so bad, because it’s “the team” telling you. With just one other person, though, it comes out as “you aren’t pulling your fair share of the load”, even if that’s not what was intended.

Working by yourself, of course, avoids this issue, because it is personal.

Also, as I like to tell people, it’s hard to miscommunicate with yourself.

vi Necessities

2010-03-24T00:00:00+00:00

If you do any sysadminning at all, you need to know the very basics of vi, as it’s about the only editor (ed being the other) that’s almost guarenteed to be installed on any UNIX machine you come across. It also happens to be ingrained into the UNIX way, as evidenced by applications from visudo to NetHack. :)

Evoke vi from the commandline with an optional filename after it: vi 2010-03-24-vi-necessities.md.

Navigate through the file using your familiar arrow keys.

When you reach a point in which you wish to either insert or delete text, press i to enter “Insert mode”, type as normal, then press Esc to return to “Normal mode”. You may or may not be able to use the arrow keys while in insert mode.

When you are finished editing the file, type :wq and hit enter to write the changes to the file and exit vi. If you wish to exit without saving changes, use :q! instead.

This is the minimum required.

Small git Repos are Ok!

2010-03-20T00:00:00+00:00

I love git. Every project I start has begins with a git init, and most of them end up on GitHub, which is why I have so many freakin’ repos there. Well, that’s not actually all of the reason, although it is a big one. The other two are that I have a tendency to use git for things it’s not really designed for, and that I have create new repositories willy-nilly. It’s actually the latter that I want to talk about today.

I often come across people who group lots of unrelated files into one repository, either because that’s how they happen to be on their filesystem, or they have an adverse reaction to small repositories. I have a feeling that this is due to prior usage of subversion and the difficulty of creating projects on certain project-hosting sites, although I can’t be sure. Perhaps due to the fact that I was introduced to mercurial and git before subversion plays a part in my differing view.

I am a proud Arch Linux user, and as part of that I maintain a few fairly unimportant packages for the Arch User Repository (AUR). Since I always like being able to see just what changed in pkgbuilds, I version all of mine with git and host them on GitHub. All of them are single-file repositories. I prefer to keep them separate, though, for a few reasons:

If someone wants to take over one of my packages, they can do so without getting all of the other crap.
I can generate nice changelogs.

While the latter can still be done on a per-file basis, pkgbuilds can contain other files (usually patches), and I’d rather not back myself into a corner unneccesarily.

Summary: more of this, less of this, and don’t criticize me for being a hypocrite.

Why I Love Komodo Edit

2010-03-18T00:00:00+00:00

Ever since I moved to Linux (winter break of my freshman year, so January 2008 or so), my editor of choice has been Komodo Edit. Although it has a certain amount of popularity among Pythonistas, it is not nearly as popular as several other editors, so I thought I’d discuss my reasons for using it. I’ve also been considering moving to something else, but haven’t been satisfied with other choices yet, so this is a bit of a fishing trip (you can find contact info on me over at, of all places, the contact page).

The winter break of 2009 (breaks are a great time to get things done), I took the plunge and started learning basic vi keybindings. Komodo supports this natively, and they are too well ingrained into my fingers for me to move away from them. In fact, I use Vimperator on all of my computers as well to keep the same thought process for my two most-used applications, and I’d love for everything I use to share this common trait. So, for good or for bad, I must have vi-like operation.

Komodo also has the right amount of stuff, but not too much. As seen below, my window is generally fairly minimalistic.

I have line numbers (can’t live without ‘em), tabs so I can visually see what files I have open at any given time, some other minor information, and some menu buttons, which I find useful for accessing commands that I don’t use frequently enough to memorize the shortcuts to.

I can also split the screen vertically or horizontally, and move files easily between the two sides, which is incredibly handy on my widescreen laptop. Since I am using a tiling window manager, I can simulate this to an extent, but it’s still a nice feature.

Ctrl+Shift+P unhides my project listing. For small projects, standard terminal navigation is sufficient, but as the size and number of directories increases, so does the value of being able to easily visualize it. I prefer to keep this out of the way when I’m writing, so the ability to show and hide it quickly is a must.

While I can, of course, set preferences for file encoding, indentation, and line endings (and can change those on a per-language basis), Komodo will respect those in the files that I edit. This is incredibly necessary when working on different projects, each with their own rules. Any mixing of indentation will give me green underlines (similar to those in Word indicating incorrect grammar), and a ctrl+shift+8 will show me the whitespace characters so that I can fix them. The same lines appear for differing line endings; ctrl+shift+7 displays them, and one of the menu options makes them consistent. Both of these are things I’ve consistently had to fix on a recent group project, where all the other members used either gEdit, Eclipse, or vim.

There is always code folding available at the places I want it, and it clearly shows a fold without being distracting. Even more of a plus is that Komodo restores the state of the folds in a document when I reopen it, so that I don’t have to refold all of the code I’m not interested in at the moment all the time.

It was trivial to use a dark color theme for Komodo, although changing some elements of the interface was a bit more difficult. Nonetheless, it is nothing like the annoyance that is Eclipse.

I can use tab to complete words that I’ve been typing in the document (without any special modification), or to complete tooltip options based on the languages Komodo has them for. Having to use something other than tab is unacceptable, since I’ve been trained by my editor, my shell, and my irc client to do so.

Along these same lines, I’m auto-indented in when opening a block, and closing keywords are either added automatically or pop up for completion.

What do I not like about Komodo? It’s an X application, so, while it still has functionality to edit files over FTP/SFTP/SCP (and even add them to projects), it cannot be run in a screen session, detached, and reattached at another location. Being based upon Gecko, it can also be a bit of a memory hog, although I still avoid using swap with both it and Firefox (and a few other things) running on one gig of RAM.

Hello, World!

2010-03-17T00:00:00+00:00

Since the beginnings of programming, so have all projects begun. We wouldn’t want to stray away from tradition now, would we?

As is common in the profession, I have neglected my own website for quite a long time, favoring instead more interesting projects. Although I finally created a design and basic skeleton of a site a few months ago, and even got it running on one of our virtual machines, I still hadn’t created enough of something to use it quite yet.

I also began to realize that what I was doing was overkill. I started thinking about some of the pluggable Django blog apps, but even those are far more than I need. Enter Jekyll.

So, my hope is that now I will actually write out things on something a bit more permanent than my mind. We shall see…

@xiong_chiamiov	realizing you suck is one of the most important parts of learning how to code, imo
@xiong_chiamiov	which is why I strongly recommend /reading/ other people’s code on a regular basis