Process is not a four-letter word

As a software engineer, I’ve worked with a lot of different development processes over the years, and I’m a pretty big fan of a number of them. Also, as a Getting Things Done (GTD) fan and advocate, I have a pretty rigorous process I’ve imposed on myself for doing my daily work. However, I constantly hear my colleagues decry “process” as an unnecessary burden, and look to eliminate it whenever possible. I think this attitude is somewhat mistaken at best, and actively harmful at worst.

Let me start, though, by being clear what I mean by “process”. A process is the means by which something gets done. Using that definition, if you get anything done at all, there was some process by which it happened. It could be a good process, a bad process, or even just plain trial and error. It’s still the way it got done.

So, process clearly isn’t actually the enemy here, since we all like to get things done. Really, the problem my colleagues are reacting to is having a bad process with too many unnecessary, laborious, or complicated steps. This is definitely a problem, but not the worst one you could have.

The worst is having no process at all. Of course, that’s not actually possible, so really what this means is that your process is made up as you go along, on the spur of the moment, each time you try to accomplish something. It may feel good not be restricted from tackling the job in whatever way occurs to you, but you’re very unlikely to take the optimal path from concept to completion. Instead, it’s very likely that pieces will be missed along the way, thrown back in haphazardly, not completed in time, or in the wrong order, and stakeholders will be unaware of what’s going on, or when to expect results.

The worst part of this ad-hoc process is that you’re doomed to repeat the same chaos every time you do it again. Since everything was made up on the spur of the moment, you’ll most likely wind up doing it a completely different way next time. Any lessons learned about what worked and what didn’t (even if you’re able to recall enough of what you did to try to analyze it), won’t be applicable the next time.

The essence of process improvement is repeatability. If you can reliably repeat the same sequence of steps each time, you now have a chance to: 1) clearly understand what you did, and 2) deliberately make changes the next time.

So, one step better than total chaos is what most people think of when you say “process”: a clunky, cumbersome, set of steps which create a lot of unnecessary busywork for those trying to accomplish the goal. I say this is still better than no process at all because it is repeatable. That means, you can tweak the process each time you repeat it to make it better.

The very best processes, of course, are those which only include exactly as many steps as are necessary to ensure the right work is done, done well, and all the right people know what’s going on at each step along the way. The problem is that it’s not always possible to predict what steps those are, and whether those will continue to be the right steps in the future (e.g., as the team grows, or the problem gets more complex).

That leads you to the real goal: to create a process which includes a means of improving the process itself. There are many examples of such processes (SCRUM, Kanban, etc.) when they are practiced well. In those cases, the built-in self-correcting nature of the process allows the details to change to adapt to new circumstances.

Of course, this ideal implies a few failure modes. One is to adopt a “we’ve always done it this way” sort of attitude which fails to allow for refinement, growth, or adaptation. Then, no matter how good or bad the process is at any given moment, you’ll never get better. And, as times change, the process will suit the current situation worse and worse.

A second failure mode is to change lots of thing with each repetition. It is very hard to successfully and consistently change a habit: even when you’re only trying to make one small change. This becomes nearly impossible when you pile on too many things at once. And, even if you managed to do it, it’s very difficult to judge which changes had what outcomes. Better to keep things simple and tackle one change at a time.

✧✧✧

As I said to start, a “process” is the means by which something gets done, and any time something gets done, there was a process by which that happened. Instead of treating “process” as a four-letter-word, I think we should embrace it. Whatever process you’re following for whatever goal you have, start by striving for repeatability, and then—bit by bit—keep making it better.

Organizing Members of a Class

In all the coding books I’ve read, I don’t recall any of them specifically talk about organizing the different members of a class.  However, in all the code I’ve read, the lack of any standard or order to how members are organized has been the single biggest obstacle to feeling at home with a new codebase.

The reason this matters so much has to do with our brain’s limited ability to keep track of a lot of things at once.  If we can only keep a few things in our head at a time, then it becomes hugely important for the readers of our code that we provide a hierarchy which doesn’t demand they understand more than a half-dozen or so sections at a time.  Moreover, it’s super important that these be immediately recognizable, and that they be in a consistent order so that they can be easily found.

Using Dividers to Create Hierarchy

The way I create an immediately recognizable sense of order is to create visual markers which clearly mark each section of a file. So, for example, let’s consider just the bare framework of a module in Python which contains two classes:

from some_package import a_module


############################################################

KNIGHTS_SAY = "Ni"
EXPECT_SPANISH_INQUISITION = False


############################################################

class TheMainObject(object):

    def __init__(self):
        # do constructor stuff

    # Properties #################################

    @property
    def alpha(self):
        return self._alpha

    @alpha.setter
    def alpha(self, value):
        self._alpha = value

    # Public Methods #############################

    def calculate(self):
        # calculate a value here

    # Private Methods ############################

    def _private_calculation(self):
        # do some private calculation

############################################################

class SomeHelperObject(object):

    # etc, ...

The first section (lines 1–3) contains all the imports. One of my favorite things about Python is that this gives you a complete listing of external dependencies, and an exact listing of where they are all coming from. Someone reading module can, in an instant, scan this very first section to see everything that’s coming in from the outside.

Next, I put in a full-width marker line with no label. As this indicates the top level of your hierarchy, it should be the full line width your team allows. That way, it’s immediately clear that you’re moving from one top-level part of the hierarchy to another when you see one of these lines.

The next section contains a few constants. These are clearly to be used across all the major sections of this module, and possibly even outside it.

Next, we have the class initialization section for the first class (lines 11–16). This is where the actual class declaration lives, along with any constructors. The semantic meaning of this section, while not explicitly labeled, it pretty clear: we’re setting up the class here.

Next, we have a subdivider labeled “Properties” followed by all of this class’s property declarations. If we had multiple properties here, they would each be listed in alphabetical order (more on this later). It’s important to note that this is a sub-divider, and is therefore shorter than the primary dividers since it represents the second level of the hierarchy. It’s important that it be enough shorter (15–20%) than the primary dividers that it be easy to see the difference even at a quick glance.

Next are the public methods, followed by private methods. As additional members of the second layer of the hierarchy, they use the same dividers. This pretty much covered your bases for most languages.

Sequencing and Sorting

With the visual hierarchy in place, then next thing I consider is the ordering of each element of the hierarchy. Since code is read far more than it is modified, my general approach here revolves around attempting to make the code as easy to read as possible. To that end, it’s important to keep the various levels of the hierarchy in as predictable and clear an order as possible.

For the top level of the hierarchy, we’re talking about sections for defining classes, constants, imports, and (depending upon your language) free-standing functions. In many languages, you’re required to list imports first, plus, that ordering has the advantage of making it clear what this code depends upon, so it’s useful to keep them up front. Next, I list the constants which are going to be used throughout this file. Next comes the primary class in the file (if there is one). Finally, come the sections for helper classes. In most cases, the part of the file you can readily see without scrolling down contains the imports, constants, and the start of the primary class section. So, in a single glance, the reader of the code can easily see what is the primary purpose of this file and what its dependencies are.

In the second level of the hierarchy, at least for a class, we’re mostly looking at various groupings of methods. I arrange these sections in order from most public to least. The reasoning is that a reader of the code is most likely to want to use the class, and then to inherit from it, and finally to actually change its implementation. This ordering addresses those various needs in order.

The third (and final) layer of the hierarchy consists of the members themselves. In 99.99% of cases, I simply sort these in alphabetical order. It is extremely tempting to try to order these in some “logical” order, but I have found that almost always leads to chaos. The reason is pretty simple. At first, the “logic” is pretty clear. The original author knows what the rational is, and puts things in that order. However, when the next author comes along, perhaps the logic of the ordering isn’t super clear. Maybe the next author is a little lazy. Perhaps they need to add a member which doesn’t fit into the logical scheme the other members follow. Inevitably, though, members start to be added wherever it “feels” like they seem to fit: or merely at the end of the list. Sooner or later, the only organizational scheme a new author can discern is: random.

Alphabetizing the list of members solves all these problems. It’s obvious from very brief inspection what the scheme is. That makes it obvious where to find an existing member, and where new ones should be added as there is only one correct place for each member. Moreover, it helps keep code diffs very easy to read and understand as there is a lot less movement of code from one change to another. Finally, it makes it extremely easy to tell whether a certain member is present or not (e.g., whether a certain method has been overridden).

✧✧✧

I have worked in at least a dozen different languages from PowerPC Assembly to C++ to Python to Go. In each one, I apply the principles of establishing visual hierarchy and creating an objectively correct ordering of elements in the file to ensure that the ultimate goal of making the code obvious and readable are satisfied. It doesn’t matter what the language is, whether it’s Object-Oriented, or what other conventions the language encourages. I find applying this framework makes my code immediately recognizable and earns praise from other developers for being among the most orderly and rational code they’ve ever seen.

✧✧✧

Want to see these techniques in practice? Check out some of my code at github.com/andrewminer.

What I’ve learned in 20 years with my wife

In a little over a year, my wife and I will have been married for 20 years. I was 21 years old when we got married, and 17 years old when we started dating. So, it’s been more than half of our lives that we’ve been together. And, unlike many couples we’ve known, we’re even more happy together now than we’ve ever been. I’ve been reflecting on why that is, and there are a number of things I attribute it to.

Respect

We both have a deep and reverent respect for one another. Ask anyone who knows us: we are two very different people. Instead of making this more difficult for us, though, I find that we often hold one another’s differing abilities in awe. To a large degree, I think that’s possible because we both have a great deal of respect for ourselves, and we’re able to view our partner’s superior talents with delight and admiration, rather than feeling threatened by them.

Likewise, we each have very different struggles. When one of us feels the need to improve ourselves in some way, the other is invariably sympathetic, honest in their feedback, and kind in that moment of vulnerability. And, when that person makes progress, the other is there to cheer them on, and applaud their success.

Listening

It’s terribly clichéd to say that the essence of a long-lived, successful relationship is communication, but that doesn’t mean it’s not true. What I always find dissatisfying about such advice, though, is that it’s not nearly specific enough. What isn’t said that that the most important part of communication is for each person to really, actively, listen when the other person is talking. This is most especially true when the other person is sharing an uncomfortable truth, either about themselves, or about the relationship.

If I’m the person doing the listening, that means that I am actively trying to understand my wife’s point of view before formulating one of my own. Most importantly, I keep quiet while she’s talking. At intervals, I might echo back what I think I’ve heard, and ask for her to correct anything I’ve misunderstood. I might ask clarifying questions where I’m uncertain. I’ll often ask her to elaborate on certain points I think may be connected.

Most especially, there are a number of things I do not do while I’m the person listening. First, I do not deny the truth or validity of her opinions and emotions. I might question facts, but if there’s any disagreement, we immediately turn to some source of actual evidence, or just let that point go. We both try to avoid the desire to be the one who was “right” and “won” the point. I also try not to jump in and “solve” the problem until she’s had her full say. Even then, I try (not always successfully), to ask her what sort of help (if any) she actually would like, before offering suggestions.

And, to be clear, we both play both roles here. There are plenty of times (more often, in fact), when I’m the one bothered by something, and I need someone to listen while I work through it aloud. In such times, my wife is invariably a patient and insightful listener.

Teamwork

Since the start of our relationship, we have faced the world as a united team. In fact, when we first got engaged at 18 and 20, we faced a fair amount of skepticism by standing together, and deciding how we jointly wanted to respond. Countless other examples have followed: when I got a new job out of state, deciding whether to have kids, and many others. No important decision is made without talking it over together first, and we value each other’s advice and opinion so well that we will often consult each other on relatively minor issues too.

Part of that teamwork, though, is trust in the other person to follow through as a member of the team. This applies both to what we say we will do (e.g., I look after our investments, while she looks after our health care), and what we say we won’t do (e.g., spend large sums of money, have relationships outside our marriage). This isn’t blind trust either: after all this time, we both have given and received ample evidence in big and small things that the other person is worthy of being trusted.

Deliberate Time Together

We are both people who get very engaged in our work, projects, and hobbies. We have careers, a house to manage, a son to raise, and more than enough to keep every hour of the day overflowing with things to do. However, pretty much since we both started working full time, we have deliberately maintained a weekly date. We’ve usually made it some kind joint lesson or another (e.g., flute, clarinet, ballroom dance, singing), or an activity we both enjoy (cooking, dining out). We generally change the activity at least once a year, either as circumstances change, or just to keep it from getting dull.

One of the most important aspects of these dates is the time surrounding the activity where we can talk. It has proven essential for us that there’s enough time that you get past the “How’s your day been?” level of questions and down to the longer-term, difficult, or uncomfortable conversations. I remember that a lot of these happened around when I came out. Over the past couple weeks, these have been about our son’s transition into a simultaneous High School / College program where he’s struggling a bit. These are the kinds of conversations which cannot happen in the ordinary rush of day-to-day life, and we have found it essential to deliberately carve out space to make sure they can happen.

Deliberate Time Apart

Just as we deliberately plan time together, we also deliberately give each other space to be individuals. Not all activities, friends, and hobbies need to be shared. Again, this seems obvious, but I also see plenty of couples where one person feels left out or hurt when their partner wants to go do something on their own.

As an example, when our son was a toddler, and my wife was the stay-at-home mom, there came a point where I could tell that she was very much in need of some time off. So, I planned a trip which had her drive away, alone, in her car with a box of envelopes with numbers on them. This lead her through a scavenger hunt to pick up some of her favorite treats on the way to a weekend stay in a lonely B&B on the coast of Washington state. She had nothing to do, no obligations to anyone, and everything planned out for her in advance. A few months later, after I’d just gotten a new MINI Cooper and finished a stressful project at work, she arranged a similar trip for me high up in the back roads of the Cascade mountains with a new book. We were each thrilled with our own trips, but would not have enjoyed the other person’s.

As another example, I enjoy playing video games (something I share with my son), but they make her motion sick. She loves singing in a chorus, but that’s just not my thing. Every once in a rare while, we’ll share in those activities, but mostly we’re very supportive of the other person spending some time alone with their hobby.

Physical Intimacy

Yes, sex is definitely important in our relationship. However, there’s more to it than that. Anyone who has seen us together will attest that we are constantly touching each other in small, loving ways which have nothing to do with sex. Hugs, kisses, holding hands, snuggling in our big armchair to watch a movie, or even just a light caress as we walk past one another are constant and daily occurrences. This is a constant reinforcement of the sense of closeness and intimacy we share with our favorite person.

As for actual sex, we apply the principles above. We have different levels of interest at different times, we each react physically to different things, and we each are different in desire to try new things. We negotiate those differences by respecting each other’s limits (while still asking for what we each want), actively listening to what the other person likes, and being deliberate in setting aside time to make sure that part of our relationship doesn’t get buried in the daily grind.

✧✧✧

I’m positive there are a bunch of things I’m missing, but I feel like these are the most important ones. Of course, these are the things I think have made the most difference in our relationship with our particular personalities. I suspect, though, that most couples will find these principles apply to them as well.

Getting the most from your log file

Think back on the last time you urgently needed to figure out what a live, production piece of software was doing. It’s very likely that nearly the first thing you did was to pull up it’s log. It’s also very likely that the majority of the content was either useless, unintelligible, or flat-out misleading.

The reason most logs fall into this state is that they are a piece-meal accumulation of statements which were thrown in over time. This is a problem because the only possible reason to waste the computer’s resources producing them is to be as correct, intelligible, and useful as possible. In short, log files should be explicitly written for consumption by people.

As with anything else, clear writing requires the author to carefully select the content, and to create it with the reader in mind. In the case of log files, we know other engineers (or scripts they write) are going to read them, and they are going to want to know how the server has been behaving. Moreover, no one reads log files unless something is going wrong. Therefore, log files should be written to make it as easy as possible to debug why a server isn’t behaving as it should. To that end, here are a few specific guidelines I’ve found most useful:

Make it easy to track down errors

It should go without saying, but it is imperative to make it as easy as possible to identify where – in the code – an error occurred. In Java this may be printing out the stack trace; in C/C++/Objective-C, this may be using the __FILE__ and __LINE__ macros. Most languages have some similar feature. Along these same lines, try to avoid having logging statements which look too much alike. This makes it easy to misinterpret the log, and completely identical statements make it impossible to know exactly where an error occurred.

Log all inputs and outputs

The fastest way to track down the reasons for strange output is to know whether the input was strange, too. This is especially true for a server (i.e. a malformed request), but it applies to other programs as well, provided you have a sufficiently broad view of “inputs” and “outputs”.

I consider an input/output to be any data which enters (or leaves) the program’s runtime context. This includes: user input (mouse/keyboard events and the like), files read/written to disk, data sent/received on a socket, or requests/results from a database query. I recommend at least logging something every time such an input or output occurs, and if reasonable, the actual content itself.

Logging inputs and outputs provides a number of major benefits. First, you know exactly what the system was asked to do. This allows you to immediately know whether a problem is in your program or outside of it. Second, it gives you a pretty clear indication of what will reproduce a problem (if there is one). Finally, it allows you to report errors to the owners of the “other” system in case you received an unexpected input.

Record how long it takes

Errors are frequently just that a system is taking too long to respond. Being able to quantify the time required for each request is an extremely valuable way to isolate the problem. However, it is not sufficient to simply record how long your system took. You must also record timings for each request you make outside of your application’s runtime context. That way, you’ll know right away if your program appears slow because it’s waiting for something else: whether it’s a web service call or reading a file from disk.

Don’t log the expected case

If something is supposed to happen, you generally don’t need to know about in the log (unless it’s one of the things mentioned above). Bear in mind that no one is interested in reading a program’s log unless the program is misbehaving. At that point, the person wants to read as little of the log as possible. Including a lot of “business as usual” statements in the log only makes the process more difficult.

Use log levels to indicate action required

Since a log file’s purpose is to assist in figuring out why a program isn’t running correctly, it’s extremely valuable to use log levels to indicate how “badly” things are going. I recommend thinking of the action required as a guide to what log level to use:

  • FATAL: page someone immediately
  • ERROR: send an email immediately
  • WARN: send in a daily report
  • INFO: include for later debugging purposes

Depending upon your logging system, the various levels often have different names, but these should map to whatever you use. Even if you don’t actually have an automated system to page people based upon your log file, it’s extremely clarifying to think about each level in these terms.

Make it easy to parse

Finally, bear in mind that most uses of a log file involve searching or parsing them using automated tools. Be sure to include standard fields in each line so that these tools can easily extract information from them. Here are some examples of what fields you may want to include:

  • time (down to milliseconds and time zone)
  • the number of the line in the log statement (to ensure each line is unique)
  • the current thread
  • the current user ID / IP address / other client-related identifier
  • the log level
  • the file/class

There’s a lot more to be said on logging, but if every log I’ve read implemented these few principles, my life would have been a lot easier. If there’s some piece of wisdom that’s too good to be missed, feel free to add it in the comments!

The Right Size for a Method

One occasionally hears of some programming zealot who swears up and down that methods should be kept to n lines or less. The actual value may vary, but the arbitrary nature of the claim remains. However, there is a kernel of truth there, and it has to do with preserving unit economy.

We all know that the longer a method is, the more we have to keep in our minds to understand it. There are likely to be more local variables, more conditional statements, more exceptions caught and thrown, and more side-effects of all those lines of code. Furthermore, the problem grows faster and faster as there are more lines of code since they all potentially can have an impact on one another. Keeping methods short has a disproportionately large benefit.

Of course, claiming that there’s some absolute “correct” number is clearly nonsensical. The same number of lines in C, Lisp, Java, Assembler, or Ruby will accomplish radically different things, and even how one legitimately counts lines will change dramatically. What does not change, though, is the need for the reader (and author) of the code to understand it as a whole. To this end, one should strive to keep the number of discrete tasks a method accomplishes to within the range of what people generally can remember at once: between one and six.

Each task within a method may have several lines of code of its own; how many tends to vary widely. Consider the process of reading a series of rows from a database. There may be a task to establish a database connection, another to create the query, another to read the values, and perhaps one more to close everything down. Each of these may be composed of anywhere from one to many lines of code.

Tasks may even have subtasks. Consider the example of building a login dialog. At some point, there is likely to be some code which creates a variety of controls, and places them on the screen (e.g. an image control for the company logo, a text field to capture the user name, etc). In the method which does this, one may consider the process of creating the components a single task which has a number of subtasks: one for each component.

In both cases, the important consideration is how organizing the method into tasks and subtasks helps preserve unit economy. By creating tasks which have strong cohesion (i.e. you can name what that group of code does) and loose coupling (i.e. you can actually separate that group of lines from the others), you give the reader ready-made abstractions within your method. In the first example, the reader can readily tell that there’s a section for setting up the connection, and be able to mentally file that away as one unit without the need to remember each line of code in it. In the latter example, the reader can categorize the series of UI element creation subtasks as a single “build all the UI components” task, and again be able to abstract the entire thing away under a single unit. Even if there are a dozen or more individual components, it still can be considered a single task, that is, a single mental unit.

This ability to create abstractions within a single method is why there is no absolute “right” size for a method. Since grouping like things into tasks and subtasks preserves the reader’s (and author’s) unit economy, it is quite possible to have a method which is very long in absolute terms, and still quite comprehensible. It also implies that a fairly short method can be “too long” if it fails to provide this kind of mental structure. The proper length will always be determined by the amount of units (tasks) which one has to keep in mind, and the complexity of how those tasks are interrelated.

Response to “Diversity and Justice”

A day or so after publishing Diversity and Justice, I received the following comment from a female acquaintance:

I appreciate you offering this patch! Some code comments from a more senior developer in the area:


Good start with “there are traits which matter and traits which don’t”! Your error is that you’re assuming the only variables that matter are the ones in your Engineering function, but you’re using classes that are defined globally. Banana-gorilla-jungle stuff. 


That’s why you’ve got the common misconception “since ‘female’ doesn’t directly come into play in ‘Engineer’, an object descended from both ‘female’ and ‘Engineer’ is the same as an object descended from both ‘male’ and ‘Engineer’.” In fact, because female is a trait that matters globally, anyone in the female Engineer group has had to pass through a filtering function “stand_up_to_assumption_of_incompetence” that requires that engineer be confident in their knowledge, or patient, or stubborn, or lucky, or sociopathic, or possessing some other extra trait; most of which are favorable for Engineering prowess.

If you have two pools of engineers and one is pre-screened, it makes logical sense to exhaust the pre-screened pool before turning to the random pool!

Also, most people who work in this field use a fork of the Justice framework that comes with a bunch of open source tools specifically for working with these filtering functions; like Feminism or Anti-Racism.

Again, thanks for weighing in on the project!

To be honest, my first reaction was to feel annoyed and patronized. I was tempted to ignore the comment and just move along with my day. But I couldn’t, in good conscious, let it go so easily.

First, while my first reaction was to view this as a “negative” comment, I’m not 100% sure that was the intention. I’m perfectly willing to accept a well-reasoned and friendly rebuttal to my ideas. In fact, that’s one of the reasons I publish things publicly, instead of just keeping them to myself. However, I’m a lot less interested in debating a point with someone who is already hostile. But, knowing the person who posted the comment (I attended her wedding!), and being good friends with her husband, I am much more willing to give her the benefit of the doubt as to her intentions.

Second, if you look past the tone of the message to its actual argument, I think there’s validity to her point. That is, any woman who is still in the software field after any length of time has had to put up with a certain amount of bullshit which gives her a certain talent and toughness above the average candidate. That being the case, you could consider female candidates even more attractive than the average male candidate. I think there’s some truth there, and I was even considering various similar ideas when I wrote the original article. But, to keep things concise, I decided to exclude them.

However, the reason I ultimately decided to respond with a new post is because I think this comment actually demonstrates the third part of the trap, as I outlined in my original post:

Finally, it’s a trap for everyone trying to correct the problem. By framing the situation as one group oppressing the other, it creates a powerful “us vs. them” mentality which is hard to overcome. People in the minority group are tempted to identify all people in the majority group as oppressors. Allies in the majority group feel unfairly accused, and are less likely to want to remain allies. Progress in correcting the situation is impeded to no one’s benefit.

Andrew Miner

Whether it was the commenter’s intention or not, my reaction to the comment was exactly what I described: an ally who was being unfairly attacked. To put it more bluntly, I felt I was effectively being told: “Get out of here; you don’t know what you’re talking about.” with a little pat on the head for at least trying not to be a sexist asshole.

✧✧✧

While I wish the comment were more unambiguously friendly in tone, I appreciate the time my acquaintance took to write it, and I think she makes a good point. I would prefer, though, if we all were to follow Lori Lakin Hutcherson’s admirable example when she (a black woman) responded to her friend (a white man) who was struggling with the idea of “white privilege”:

I truly thank you for wanting to understand what you are having a hard time understanding… because I realized many of my friends—especially the white ones—have no idea what I’ve experienced/dealt with…

Lori Lakin Hutcherson

Lori embraced her friend’s attempt to figure things out and be an ally. She was willing to accept his professed good intention (despite his ignorance), and help him understand her own point of view in a friendly, sincere, and very powerfully honest way. I think we should all aspire to follow her example.

Diversity and Justice

I have been thinking about diversity a good deal lately. In fact, at work, I just attended the first meeting of a group to support and promote women in the workplace. I fully support the effort, but it got me thinking… what is the deeper issue here?

At my current employer, we’re trying to build airplanes. That requires considerable diversity: most especially in talent and experience. We need aerodynamicists, structural analysts, inlet designers, electrical engineers, program managers… on and on. We also need people at all levels of experience in their professions. But do we need people with blue eyes? Honestly… does eye color matter when building an airplane? Couldn’t we build a plane without anyone who has blue eyes? What about people with freckles? Do we actually need people to have uteruses? What about dark skin? Do we really need to have a man on staff who’s attracted to other men?

For those first few categories, I can hear you thinking… “Well, I guess not… but, what difference does it make?” But, by the end of that list, I would hope that you were starting to get offended. I think this is the really important point. While it’s likely that you could build an airplane without people having a certain eye color, complexion, gender, race, or sexuality, we should still feel outraged to exclude people based on those traits.

I think the deeper issue here is one of justice. When trying to assemble a group of people for whatever purpose, there are traits which matter and traits which don’t: based upon the purpose at hand. It is manifestly unjust, therefore, to exclude someone based on inessential traits: most especially when they do have the traits necessary for the endeavor.

I think focusing on diversity of race, gender, etc. instead of justice is a trap which bites in three ways.

First, it’s a trap for the company. Imagine a field where 80% of its members are men, and 20% women. Assuming an even distribution of talent, if you were trying to hire the top 10 people in the field, you’d get 8 men, and 2 women. However, if you are obliged to hire 50/50, you’d get 5 men and 2 women from the group of the 10 best people in the field, and then have to hire 3 more women who were not in the top 10. This is clearly not the best outcome for the company.

Second, it’s a trap for the people being hired. First, there are the 3 men who were excluded. It’s unfair, and we definitely sympathize with them. However, it’s the women who actually face the nastier part of the trap. First, the women who were hired won’t know whether they were hired for their talent, or to fill a quota. This is demoralizing all on its own. Even worse, no one else in the company will know either. One can easily imagine the nasty bias and mistreatment which will ensue from people who were already skeptical. With this combination of factors, a reasonable person would feel extremely unwelcome and move along rather quickly.

Finally, it’s a trap for everyone trying to correct the problem. My framing the situation as one group oppressing the other, it creates a powerful “us vs. them” mentality which is hard to overcome. People in the minority group are tempted to identify all people in the majority group as oppressors. Allies in the majority group feel unfairly accused, and are less likely to want to remain allies. Progress in correcting the situation is impeded to no one’s benefit.

I think the better approach is to focus on eliminating injustice: for any group which experiences it, and in whatever form it takes.

This approach requires and supports a great many of the actions one would take with a pro-diversity agenda. Unbalanced representation of women speakers in a conference you’re planning? It is just and pro-diversity to look a little harder to find more qualified speakers who happen to be female. Engineering staff looking a little too white? It is both just and pro-diversity to set up a recruiting trip to universities whose color balance isn’t so pale to hire talented newcomers who might face prejudice elsewhere. Notice though, that it is only just in that you’re selecting people who are qualified through their skills, talents, and accomplishments: not because of some inessential trait. The difference is that you’re putting in the extra effort required to avoid and/or unwind unjust biases against those people to promote justice towards them.

Paradoxically, I think this approach is also more inclusive. Promoting justice is a cause which anyone can support, and for any other person. Instead of creating innumerable “us vs. them” pairings, holding justice as the primary creates only one: those in favor of justice vs those who are not.

Using Exceptions Robustly

Before Object-Oriented programming (OOP), error conditions were commonly reported via a return code, an OS signal, or even by setting a global variable. One of the most useful notions introduced by OOP is that of an ‘exception’ because they drastically reduce the mental load of handling error cases.

In the old style of error handling, any time a function was called which could result in an error, the programmer had to remember to check whether an error occurred. If he forgot, the program would probably fail in some mysterious way at some point later down the line. There was no mechanism built into the language to aid the programmer in managing all the possible error conditions. This often meant that error handling was forgotten.

The situation is much improved with Exceptions, primarily because they offer a fool-proof way to ensure that error handling code is invoked (even if it is code for reporting an unhandled exception). This both makes it unnecessary to remember to check for errors, and it increases the cohesion of such code (i.e. it can be gathered into “catch” blocks instead of mixed in with the logic of the function). Both of these help preserve the unit economy of the author and reader of the code.

Unfortunately, despite being such a useful innovation, exceptions are often abused. We’ve all seen situations where one must catch three different exceptions and do the same thing for each. We’ve all seen situations where only a single exception is thrown no matter what goes wrong, and it doesn’t tell us anything about the problem. Both ends of the spectrum reflect a failure to use exceptions with the end user of the code in mind.

When throwing an exception, one should always keep two questions in mind: “Who is going to catch this?” and “What will they want to do with it?”. With this in mind, here are a number of best practices I’ve seen:

Each library should have a superclass for its exceptions.

Very frequently, users of a library aren’t going to be interested in what specific problem occured within the library; all they’re going to want to know is that the library either did or didn’t do its job. In the latter case, they will want the process of error handling to be as simple as possible. Having all exceptions in the library inherit from the same superclass makes that much easier.

Create a new subclass for each distinct outcome.

Most often, exception subclasses are created for each distinct problem which can arise. This makes a lot of sense to the author, but it usually doesn’t match what the user of the code needs. Instead of creating an exception subclass for each problem, create one for each possible solution. This may mean having exceptions to represent: permanent errors, temporary errors, errors in input, etc. Try to consider what possible users of the component will want to do with the exception, not what the problem originally was.

Remember that exceptions can hold data.

In most languages, exceptions are full-fledged classes, and your subclasses can extend them like any other parent class. This means that you can add your own data to them. Whether it is an error code for the specific problem, the name of the resource which was missing, or a localization key for the error message, including specific data in the exception object itself often is an invaluable means for communicating data which would otherwise be inaccessible from the ‘catch’ block where the exception is handled.

Exceptions should be self-describing in logs.

In most applications, when an exception is finally caught (i.e., not to be re-thrown or wrapped in another exception), it should be logged. The output produced should be as descriptive as possible, including:

  • a plain-English description of what happened
  • the state of any relevant variables in play
  • a full stack trace of where the error occurred

The Secret History of Women in Coding

For years now, I’ve been following the thread of stories about the overwhelming imbalance of white men in the software industry. Over and over, I’ve seen the questions: “How did this start? Where do we fix it?”. This is the first article I’ve read which makes a serious, well-researched effort to actually answer the question. If, like me, you’ve been concerned about this subject, I highly recommend spending some quality time reading it over.

I’ll take a stab at summarizing some of the high points, but I urge you to go read the full article yourself.

In the early days of computing, the hard part was the hardware. So, that was an area which was pretty strictly segregated along sexist lines. However, writing the software was seen as less challenging, and therefore the process of identifying programmers was mostly done using purely objective measures: usually a battery of tests which determined an applicant’s abilities to solve logic problems. In these tests, men and women tended to score equally well, and so the earliest programmers included a great many women.

In fact, sexism crept in here as well… to favor more women programmers. Since the software part was seen (erroneously) as being largely secretarial in nature (it definitely isn’t), women were often favored for such positions. It wasn’t uncommon to see these massive, room-sized machines tended to by all-female teams of programmers.

This persisted right up to around the personal computer revolution sparked by Apple and IBM in the late 70’s and early 80’s. Before this point, computers were exclusively found at large companies and universities. All students or job applicants hoping to work with them were expected to come in with no experience, and to be taught along the way. This all changed when most families could afford to have a computer in the home.

With computers being commonplace in the home, the gender biases of the typical American family started to play a massive influence over who would eventually become programmers. Young boys would be encouraged to play with the new machines, while girls would be steered toward more typically feminine pursuits. By the late 80’s and early 90’s, this lead to college computer science departments starting to see a huge influx of freshmen who already knew a fair bit about computers: most of them men.

According to the article, this is what started the feedback loop which drove women out of computer science. Overwhelmingly, the students who seemed the most precocious were male, and the professors started to offer preferential treatment to those students. Not surprisingly, this creates a very hostile environment for students starting out the program without any computer experience: largely women.

In the predictable 4–5 years later, the same situation started to play itself out in industry. Programming jobs were increasingly seen as the province of men, and those women talented and brave enough to break into the industry faced an increasingly uphill battle as the balance of men to women continued to shift. Combined with programming increasingly being seen as a challenging intellectual endeavor, the latent sexism already present lead to women being increasingly ignored, trivialized, and passed over. Over the next 10 years, you start to see the alarming figures of 80%–90% men in technical positions at the largest, top-tier companies in the software industry.

What to do?

Even better than merely recounting this dismal history, the article actually talks about some places which are successfully counteracting this trend. In particular, I was impressed with Carnegie Mellon’s approach (which has resulted in 40% of students in the CS department being female). Recognizing the difficulty of new students entering the program without prior programming experience, they’ve started offering different classes to incoming freshmen based upon whether they have prior programming experience. By the time these students have gotten through the first half of the degree, they’ve pretty much all equalled out based upon their own natural talent.

The article talks about a number of other positive steps being taken as well, but not without pointing out that many other problems remain unsolved.

✧✧✧

I do not accept guilt by association. So, despite the villains of this piece being male, I do not personally feel guilty to share some coincidental characteristics with them. Instead, I see a group of people who have been treated unfairly for far too long, and I find myself in the position of being able to say something against it. This is not about hating men, nor wanting to coddle women. It’s about hating injustice, and wanting to put a stop to it in whatever form, no matter how similar the perpetrator, or how different the victim.

I am proud to look back to all the women mentioned in this article (and the much larger number who weren’t) as fellow engineers who made my career possible. I appreciate them for it, and I’m glad to pass along word of their accomplishments.

One of my parents died

I just found out that one of my parents died last night.

Fortunately, it wasn’t my father. He’s been the one solid backdrop of my life from the moment I was born. For a long stretch between when he divorced when I was 5 and remarried when I was 12, he was the only solid thing in my life. He’s a man of simple virtues, deeply held. He instilled in me, from my earliest memories, a deep and life-long value of honesty, hard work, humor, and family. To this day, I marvel at how he raised two challenging young boys on his own, while dealing with the pain of a messy divorce. He means a great deal to me, and I’d be a wreck right now if I’d just lost him. I’m getting choked up right now even thinking about the possibility. Fortunately, it wasn’t my father.

Fortunately, it wasn’t my mother. I only met her at 11 years old when she and my father started dating. I’m not sure I would have had the courage to jump into a family with two almost feral boys and with a father working two jobs to keep it all together, but she did. She instilled in me a love of culture: fine music, theater, fine dining, cultured manners. She also turned me from a bright, but indifferent student to someone who excelled in school and graduated with multiple honors. On my 18th birthday, we went down to the city hall and adopted each other, and she’s been my mother ever since. I’d be devastated if I’d just lost her. Fortunately, it wasn’t my mother.

It was my father’s first wife, my natural mother. She walked out on our family when I was 5. She was already seeing another man who was heavily into drugs and an alcoholic. Soon after, they married and moved away. My memories of the rare occasions when my brother and I would go stay with them are not pleasant. He was abusive to my mother, and while not abusive to us, still terrifying. She continued the path of drugs and alcoholism as I became an adult, and started my own family. Right around the time my own son was born, I broke off my relationship with her forever.

That was over 15 years ago. I haven’t seen her, or talked to her since. The little bits and pieces I hear though my cousins have made it clear that nothing had changed with her. And, last night, her long-abused body finally gave up.

Am I sad? No, not really. That may seem callous now, but I did my grieving for her as a 5-year-old boy. And, how did I grieve. I didn’t understand who she was, or why she was gone, but my mother—half of my universe of trusted people—was gone. I wished and longed for my parents to be reunited. No matter the shouting and arguments. Eventually, as I grew older and started to understand more, that feeling settled into anger. Finally, as an adult facing those same choices, my feelings changed into disgust.

So, to me, my natural mother died over 30 years ago. I cried. I mourned. And I finished along time ago. Now, I don’t have anything left for the person who didn’t want to be my mother all those years ago.

✧✧✧

My real parents are those people who choose to love me. They are the people who gave of their own character to shape mine, and to set me on the best course in life that they could possibly manage. They are the people who have walked the tightrope with me of growing up and striking out on my own. They’ve been with me as I’ve built my career, and as I’ve grown my own family. I literally have tears streaming down my face as I write this: the depth of feeling I have for these two people is so overwhelming.

I regret that my experience with my natural mother has made it difficult to say in person to my real parents what they so richly deserve:

I love you both much more deeply than I could ever express in person, and much more than these written words covey. Thank you for choosing to be my parents.