Month: January 2019

What I learned from Edward Tufte

Andrew MinerLeave a comment

I recently was able to attend Edward Tufte’s seminar on presentations and data graphic design.  This blog post covers the essential elements I took away from the lecture.

On Space vs. Time

When one has a great deal of content to convey to an audience, it cannot be blurted out all at once and in the same spot: it must be spread out over either space or time (or both).  A slide deck spreads out content in time, using the same space over and over. A document or web page shows everything at once, spread out in space. Documents play to human being’s strengths, while slide decks play to their weaknesses.

Humans have a natural ability to visually consume a complex field of data by instantly shifting modes from high-level scanning to detailed inspection and back again.  This makes it possible—and even quite easy—for a person to scan through a long document, identify sections of interest, and dive into that piece for a closer look. The same is true for presenting many hundreds or thousands of data points in a graph; the viewer can rapidly scan the overall structure of the data and zoom in to particular interesting details.  With information arranged with spatial adjacency, it is easy for people to compare and contrast, scan and examine, and learn most efficiently.  This leads to a very high throughput of data transfer from the author to the audience.

Humans, on the other hand, have a limited ability to precisely remember detailed data for any length of time.  They also have a limited attention span: particularly when presented with data which is either confusing or boring to them.  This makes it very difficult for a person to hold the context required to compare data presented sequentially over a series of slides.  The needs of a slide presentation (i.e., a limited space per slide which must be legible at a distance) means that the content is broken into tiny chunks and widely distributed over time as the presenter talks over each point: often re-iterating the content on each slide slowly (relative to the speed of reading).  It is impossible for any individual listener to speed up or slow down the presentation to suit their needs, or to scan ahead to answer a question, or to skip back to revisit an unclear point. This leads to a very low throughput of relevant data transfer from the author to the audience.

It is highly preferable, therefore, that information displays maximize “spatial adjacency” of material with a visually dense presentation with varying levels of headers, “data paragraphs”, and whitespace to allow viewers to readily identify and select from large blocks of content at a single glance.

On Giving Presentations

For presentations of virtually any scale, it is far better to provide a narrative document (i.e., like this one) instead of a slide deck.  The document should be from 2–6 pages long, and include all the information to be discussed integrated into a single flow. Tables of numbers, charts, graphs, pictures, etc. should all be integrated with the narrative description of the subject matter.  In all cases, references should be included to source materials, primary sources, etc. The document should be written to be a permanent record of what was discussed, and therefore should be complete and self-contained.

For the actual presentation, the meeting should begin by handing out copies of the actual document to each attendee.  This is followed by a study hall session long enough for everyone to carefully read the document and make note of any questions, thoughts, or disagreements.  Once everyone is ready, the remainder of the meeting is not spent re-hashing what was just read, but instead is spent in discussion those questions, thoughts, and disagreements each person noted while reading.

The primary advantages of this style of meeting are:

  1. People can read through the document at their own pace, and to serve their own needs.  Sections irrelevant to a certain person can be skimmed, while sections of intense interest can be lingered over carefully.
  2. People can easily jump ahead to see if a question is answered later, or skip back for extra clarity on a point they may have misunderstood.
  3. People read much more quickly and with much greater throughput than can be presented aloud, so meetings can often be shorter.
  4. The document serves as a permanent record of what was presented which everyone can take away with them to refresh their memories later.

On Judging an Information Display

“The purpose of information display is to assist people in reasoning about the content.”

Edward Tufte

When judging an information display (i.e., charts, graphs, tables, etc.), people judge both the quality of the data and the reliability of the presenter.  To establish both, apply these six principles both when making and consuming an information display.

show comparisons, contrasts, and differences

The information display should be deliberately designed to make it easy to compare various data sets or points within each data set.  The author should be thoroughly conversant with the data, and deliberately highlight those points of contrast which are most surprising, interesting, or useful.

show causality, mechanism, explanation, and systematic structure

Information displays should endeavor to show how certain data sets were the cause of other data sets.  In charts, for example, one can use labeled arrows to not only show the direction of causality, but also to describe the mechanism or process by which it happened.  On graphs, this can be a block of text describing some causal connection with an arrow pointing to where this is shown in the data.

show multivariate data (i.e., 3 or more variables)

The real world is complex, and includes a lot of interconnections between different data sets.  Information displays should attempt to draw in as many of these various data sets as possible to show the interconnections between them (see: Minard).

completely integrate words, numbers, images, diagrams, etc.

When helping someone understand a data set, it is very unhelpful to segregate data based upon its source, format, or media.  Instead, pull all sources of data into the single information display so that they can be compared side-by-side with the other data relevant to the story.  Data labels and other text should be integrated into the data display whenever possible instead of being relegated to sidebars, legends, or other documents.

document the display thoroughly

The reader should be left with no questions about what it is they are seeing or where it came from.  This often requires extensive textual, even narrative, explanations included within and along side the information display. A title, the author’s name, units for all numbers, and links to source data are a minimum.  One may also find it helpful to include a paragraph explaining the principle features of the data set, interesting comparisons to make, or surprising results.

presentations stand or fall based on the quality, relevance, and integrity of the content

Showing the content in the clearest and most accessible fashion should be the only purpose of an information display.  Design for the sake of design should be avoided at all costs. Any extra line, letter, or decoration should be eliminated if it doesn’t serve to help the reader understand the data better.  The data will tell the story better without confusing or distracting embellishment.

✧✧✧

Naturally, this only covers the most essentialized version of what Tufte presents over the course of his full-day lecture. Along with the lecture, you receive his four published works on data display:

I found the lecture both extremely informative, and productive. I came home bursting with ideas on how to improve the data displays of the various projects I was working on, and I’ve been able to put his precepts to good use on a number of occasions since. I highly recommend attending if he comes to a city near you.

Atomic Habits by James Clear

Andrew MinerLeave a comment

I recently picked up the audiobook version of “Atomic Habits” by James Clear. While I’ve only just started my second listen through, I already think it will become one of the most influential books I’ve read: right behind “Getting Things Done” by David Allen, and “The Fountainhead” by Ayn Rand.

The basic premiss of the book is that while goals are great for setting a direction, they are really lousy as a means to achieving anything. Instead, success comes from changing your daily habits—sometimes in very tiny ways—so that they accumulate, inevitably, and almost without effort, into success. This is accomplished by dissecting the life-cycle of a habit, and taking specific actions for each stage to ensure a new habit sticks. The same applies to habits you’d like to break: just apply the opposite actions for each stage to break the habit.

What impresses me the most about this book is its specificity. A lot of self-help books do a great job of laying out some interesting ideas or principles, but then fail to help the reader make the jump to practicing what is written. Not here. Every chapter starts with some motivating anecdote, then describes the principle involved, and then works through several different ways to put things into practice. Each chapter includes various kinds of mental exercises, checklists, and specific actions to take.

Another thing I like, is that the author fully understands how challenging it is to jump in at the deep end of creating some complex new habit (or breaking a very familiar one). He talks through various ways to simplify the process of easing into the new habit so that it doesn’t require tremendous willpower to accomplish it. Just a slow process of continuous improvement from very easy steps to more complex ones.

It’s not a long book at all, just 5½ hours in the audiobook version, and it’s caught my brain on fire with future possibilities. I highly recommend giving it a read.

GTD: Mastering Capture

Andrew MinerLeave a comment

I started using the Getting Things Done (GTD) method for staying organized almost 10 years ago now. Since then, I’ve learned a lot. This is part of a series describing where I’ve gotten to with my own GTD practice.

✧✧✧

The first step in the GTD workflow is capture. This means writing down anything and everything you run across which may have value at some point in the future. These days, this is almost 100% electronic. In order of frequency, I capture by:

  • adding an entry to my Things app (on iOS or MacOS)
  • sending an email to myself
  • telling Siri on my iPhone: “Hey Siri, remind me to…”
  • asking my wife to email me a reminder
  • putting a physical object in a conspicuous place (e.g., my inbox)

Given by job, I spend most of my time in front of a computer, and any time I’m not actually in front of a computer, I pretty much always have my phone on me. Since I use Things as my “trusted system”, it’s often most useful to just use the capture tool built in to that app (whether on the computer or on the phone).

If, for some reason, I’m not sitting at a computer with Things on it, I’ll just email myself and process it later.

If I’m driving, or otherwise not able to type, I’ll tell Siri to remind me, and use Things’ integration with the Reminders feature to automatically sync.

My wife also uses GTD, so we very often will discuss things with one another and then send one another an email to capture the request.

And, finally, if the thing to do actually involves a physical object, I’ll use the object itself as the means of capture. For paper mail, I have a wooden box on my desk. This also works for items to repair, and other small items which need my attention. If it’s something I need to take into work, for example, I’ll just make sure to put it next to where I put on my shoes and coat.

✧✧✧

While that pretty well covers what I do for my capture step, it’s worth noting that I find it absolutely essential to separate the capture step from process step. If mix these up, it creates a real impediment to capturing effectively and/or processing effectively.

The processing step is really the most thought-intensive part of GTD, and often requires a decent amount of time which you don’t always have in the moment you need to capture something. For example, if I’m in a meeting and hear something I need to follow up on, I need to make the shortest and quickest note possible so that I can return my attention to the person speaking. I literally don’t have time to think through all the processing steps.

By keeping the two separate, I can record some very tiny number of words in my inbox, and return my attention to what’s going on. I avoid both missing out on capturing altogether as well as entering something half-baked into my trusted system… which would, of course, become less trustworthy as a result!

Learning and Knowledge Retention

joewildingLeave a comment

This is a guest post by Joe Wilding, the CTO and Co-Founder of Boom Supersonic.

✧✧✧

I was asked by someone recently: “How do you know so much about your field?”  My short answer was, “I read a lot.” To that he replied: “Yeah, but how do you retain all of that knowledge?” I didn’t have a crisp answer at the time.  But, as I have thought about that question since, I have come to realize I have developed a pattern over the years which has allowed me to retain much of the knowledge I have read.

While I have to admit this method requires additional effort, I’m personally convinced it’s required for long-term retention.

The Method

The method consists of two elements. The first is ensuring that you deeply understand the content when you first read the material. The second element is making the content sticky by refreshing your memory of it in a deliberate recurring process.

Deeply learning the topic

There are many ways to fully understand a topic when first exposed to it. If the topic is simple enough, the act of reading it, watching a video, or hearing it explained may be sufficient. For more complex topics, other tactics may be required. For me, they all come down to forming some sort of a model of the concept that makes sense to me. This model can be mental, or something that you actually sketch or turn into a diagram. Good books or other sources will do this for you, but not always.

I prefer to understand how the concept works based on the fundamental governing principles, whether that be physics, math, psychology, economics, etc. If math is involved, I do not gloss over the formulas. I pay attention to the inputs, the units, and the exponents on each variable. I try to deduce why each variable is there, and why others are not. I try to get a feel for how the answer would change based on different values of the inputs. If it is a topic i really want to understand I will enter the formula into a spreadsheet, plot it, and watch how the results change with different inputs. This “live feedback” method can increase your understanding tremendously and very quickly.

I also tend to formulate an understanding of the topic such that I can explain it to someone else. Often, I will literally do that: either out of necessity, or because I am typically surrounded by others who love to learn. It is very powerful to express a concept in your own words and to be prepared to answer questions or explain the parts that are not obvious.

Making it stick

A very unfortunate drawback of the human brain is that the knowledge it contains tends to fade over time. This is particularly true for concepts that are learned and then not accessed again before it is evolved into long-term memory. This means that all of the time and effort you put into learning a new topic could be lost if you don’t take action to make it stick.

This is less of an issue if the learned topic is something you will be using frequently for an extended period of time (such as in your daily job). However, much of what I read is a little more obscure, or something I will need only on infrequent occasions. To ensure this knowledge is not lost, I employ a method I read about many years ago called the Super-Memo Model, developed by the Polish researcher Piotr Woźniak.

The following graph shows how this works:

The graph shows that as a new topic is learned, but then not used again, the brain starts to lose that information on a decaying curve called the “curve of forgetting”. Nearly all of the knowledge on a topic can be lost in a few months. For example, try to remember something you might have heard on the news or read in a paper from a few months ago.  If you didn’t have a direct connection to it, you probably can’t.

The happy side of this story is that if you refresh your memory of the topic, not only do you quickly get back to the 100% status, the rate of decay on the curve also decreases. If you can remember to do this three or four times, the decay curve becomes very flat and the information will be accessible nearly forever.

At this point you might be saying: “Great, I have to read everything four times if I really am going to learn it?!” Not at all. If you fully understand the topic the first time—in the way I talk about above—the refresh effort can be very quick. You just have to have a method of making the information quickly available and developing the discipline to actually go back and review it.

My favorite method for easy future accessibility is to take notes when I am first learning the topic. I summarize the key concepts, keep the sketch or diagram if there is one, hang onto the spreadsheet, and list all the references on where the knowledge came from. After that, it is just going back and rereading it a few times in the future. You could schedule these in your calendar, but I usually just keep the notes file on my desktop and then go back and reread it from time to time until I feel like it is fully committed to long term memory. At that point, I usually file it away in a folder for future reference.

There are other methods for reviewing material, including: reading other sources on the same topic, teaching it to others, or using the knowledge on a recurring basis. It doesn’t really matter what the method is. It is just important to refresh your memory routinely until you’ve really mastered it..

I’ll admit that this method requires effort, and I certainly don’t use it for everything I read. But if I have a topic that I really want to master long term, I have found that this method works every time.

Improving your estimates

Andrew MinerLeave a comment

Estimating most projects is necessarily an imprecise exercise. The goal of this post is to share some tools I’ve learned to remove those sources of error. Not all of those tools will apply to every project, though, so use this more as a reminder of things to consider when estimating, rather than a strict checklist of things you must do for every project. As always, you are the expert doing the estimating, so it is up to your own best judgement.

Break things into small pieces

When estimating, error is generally reduced by dividing tasks into more and smaller pieces of work. As the tasks get smaller, several beneficial things result:

  • Smaller tasks are generally better understood, and it is easier to compare the task to one of known duration (e.g., some prior piece of work).
  • The error on a smaller task is generally smaller than the error on a small task. That is, if you’re off by 50% on an 8 hour task, you’re off by 4 hours. If you’re off by 50% on an 8 day task, you’re off by 4 days.
  • You’re more likely to forget to account for some part of work in a longer task than a shorter one.

As a general rule, it’s a good idea to break a project down into tasks of less than 2 days duration, but your project may be different. Pick a standard which makes sense for the size of project and level of accuracy you need.

Count what can be counted

When estimating a large project, it is often the case that it is made up of many similar parts. Perhaps it’s an activity which is repeated a number of times, or perhaps there’s some symmetry to the overall structure of the thing being created. Whichever way, try to figure out if there’s something you already know which is countable, and then try to work out how much time each one requires. You may even be able to time yourself doing one of those repeated items so your estimate is that much more accurate.

Establish a range

When estimating individual tasks (i.e., those which can’t be further subdivided), it is often beneficial to start out by figuring out the range of possible durations. Start by asking yourself: “If everything went perfectly, what is the shortest time I could imagine this taking?” Then, turn it around: “If everything went completely pear-shaped, what shortest duration I’d be willing to bet my life on?” This gives you a best/worse-case scenario. Now, with all the ways it could go wrong in mind, make a guess about how long you really think it will take.

Get a second opinion

It’s often helpful to get multiple people to estimate the same project, but you can lose a lot of the value in doing so if the different people influence each other prematurely. To avoid that, consider using planning poker. With this technique, each estimator comes up with their own estimate without revealing it to the others. Then, once everyone is finished, they all compare estimates.

Naturally, there are going to be some differences from one person to the next. When these are small, taking an average of all the estimates is fine. However, when the differences are large, it’s often a sign that there’s some disagreement about the scope of the project, what work is required to complete it, or the risks involved in doing so. At this point, it’s good for everyone to talk about how they arrived at their own estimates, and then do another round of private estimates. The tendency is for the numbers to converge pretty rapidly with only a few rounds.

Perform a reality check

Oftentimes, one is asked to estimate a project which is at least similar to a project one has already completed. However, when coming up with a quick estimate, it’s easy to just trust to one’s intuition about how long things will take rather than really examining specific knowledge of particular past projects to see what you can learn. Here’s a set of questions you can ask yourself to try to dredge up that knowledge:

  • The last time you did this, how long was it from when you started to when you actually moved on to another project?
  • What is the riskiest part of this project? What is the worst-case scenario for how long that might take?
  • The last time you did this, what parts took longer than expected?
  • The last time you did this, what did you forget to include in your estimate?
  • How many times have you done this before? How much “learning time” will you need this time around?
  • Do already you have all the tools you need to start? Do you already know how to use them all?

There are loads of other questions you might ask yourself along these lines, and the really good ones will be those which force you to remember why that similar project you’re thinking of was harder / took longer / was more expensive than you expected it to be.

Create an estimation checklist

If you are planning to do a lot of estimating, it can be immensely helpful to cultivate an estimation checklist. This is a list of all the “parts” of the projects you’ve done before. Naturally, this will vary considerably from one kind of project to the next, and not every item in the checklist will apply to every new project, but they can be immensely valuable in helping you not forget things. In my personal experience, I’ve seen more projects be late from the things which were never in the plan, than from things which took longer than expected.

✧✧✧

Estimation is super hard, and there’s really no getting around that. You’re always going to have some error bars around your estimates, and, depending upon the part of the project you’re estimating, perhaps some considerably large ones. Fortunately, a lot of people have been thinking about this for a long while, and there are a lot tricks you can use, and a lot of books on the subject you can read, if you’d like to get better. Here’s one I found particularly useful which describes a lot of what I’ve just talked about, and more:


Software Estimation: Demystifying the Black Art

Meditation for Practical Skeptics

Andrew MinerLeave a comment

I have been meditating on and off since I was a teenager. When I mention it to others, though, they’re often somewhat surprised since I don’t really seem like “the type”. I’m not into eastern religions, energy fields, chakras, or any other form of mysticism. In fact, I’m a very practical, secular, engineer. To me, meditation is a purely practical, secular activity which helps me gain more control over my body and mind.

It’s worth saying what I actually mean by meditation, as there are a lot of different things which fit under that heading. I specifically mean the activity of sitting quietly while keeping one’s thoughts focused on a single idea. Most often, this is just following the rhythm of one’s own breathing, but lots of things could be a good focus.

This is a lot harder than it sounds, though, as you will find a constant stream of distractions trying to draw your attention away. This could be an itch, being a bit hungry, thinking about what you’re going to do next, or worrying about something on your mind. It might even be just some weird random thoughts which keep popping into your head. Whatever it is, it’s surprisingly difficult to sit for any length of time without thinking about anything in particular.

In fact, the distractions are an expected (and even essential) part of meditation. The essential skill is to be able to retain some part of your mind which isn’t distracted, and use that to notice that you’ve become distracted. Then, stop briefly to merely notice the distraction, and allow your attention to glide back to the focus of your meditation. With practice, this gentle re-direction of thought becomes very fast and natural. Once you get really proficient at it, it becomes possible to keep your focus for longer and longer between distractions.

This ability to focus is, for me, one of the two primary benefits of meditating. The more I practice this skill, the more I can quickly and gently turn my attention away from something I don’t want to think about at that moment, and back to something I do want to think about.

The most common time this comes up is when I’m trying to get some work done. This skill allows me to redirect my attention back to my task at hand whenever I get distracted. This allows me to more readily slip into that “flow” state where complex work becomes easy (e.g., writing some difficult bit of code, or a blog post). Or, if its a day where my brain seems all over the place, it at least gives me a tool to try to corral it back where it belongs.

The second time this ability to redirect my thoughts becomes most useful, though, is when I’m feeling stressed about something. My stress often takes the form of endlessly circling around a difficult issue which is outside of my control. I will keep coming back to the issue and imagining worse and worse outcomes until I’m at the breaking point of anxiety and tension. I find that the skills built by meditation allow me to acknowledge the stressful thought, and dismiss it by replacing it with a more productive one.

The second primary benefit I get from meditating is the mental reset it offers. Each morning, just after I’ve finished washing up and getting dressed for the day, I sit down for ten minutes and meditate. For me, this creates a sense of absolute calm: mentally and emotionally. In fact, I’d say it goes further than that: it creates an anchor. No matter how much my day gets stormy and tries to toss me about, meditating helps me carry over some of that sense of calm, cheerfulness, and rationality underneath.

✧✧✧

If you’re interested in trying meditation (and I highly recommend it), I recommend downloading the Headspace app. It comes with a free set of guided, 10-minute, meditations lead by an expert. The meditations are completely free of any mystical elements, woozy music, or gimmicks. Just a gentle voice walking you through the exercise. You can stick with the free material as long as you want, or sign up for a subscription (I have) to get access to other exercises. I’ve enjoyed it quite a lot, and I’ve gotten a lot of benefit from it.

Being Approachable: Even as an Engineer

Andrew MinerLeave a comment

I work with a lot of engineers, and we aren’t always the most approachable bunch.  Between the abstruse jargon of our fields, the deep concentration our jobs require, and the occasional individual who lacks practice socializing (ehem), we can be rather prickly.  I freely admit, as a younger person, I was particularly unapproachable: even compared with other engineers.  However, over time, I’ve learned a lot.  In fact, someone even called me “outgoing” a few days ago.  Given my history, I found it so odd an experience that I started reflecting on what’s changed.

Body Language

The first thing which comes to mind, and probably the simplest, is a conscious use of body language.  When someone approaches me at work, I have learned to deliberately take my hands away from the computer, and square up my shoulders to face the person.  This has the effect of turning me completely away from my computer, and turning my eyes and face directly at the person talking to me.  This unambiguously shows my visitor that they have my complete attention.

The second piece of welcoming body language I’ve learned is to smile and give a warm greeting.  Perhaps its a little obvious, but enough other people I work with don’t do this that it clearly isn’t.  By smiling, I not only make myself seem more approachable and cheerful, but it actually does make me more cheerful.  Even if I’ve been dealing with something stressful or frustrating when the visitor approaches me, smiling almost forces a reset of my mental state so that I’m ready to be welcoming and helpful.

The last bit of body language I’ve changed is to adjust my position so that we share a comfortable distance and eye-line.  If my visitor is standing, I’ll stand.  If my visitor takes a seat, I’ll stay seated.  Either way, I’ll shift so that we have approximately the same posture.  Of course, I don’t pop out of my seat… that would just be weird.  But if a conversation seems likely to be extended, I’ll make both of us more comfortable by matching up with the posture of my visitor.

Active Listening

Another skill I’ve been working on for a long while is active listening.  Briefly, this is when you try to quiet your own thoughts when listening to another person, and deeply absorb what they’re saying.  Then, when they’re finished, acknowledge what they’ve said before adding in your own ideas.  This especially applies when brainstorming or debating with another person.

I think we all find it much, much more natural to be half-way listening to what someone else is saying, and half-way preparing our own response in our head.  However, resisting the urge to do this makes it much more likely that I’ll actually hear and understand what the other person has to say, and that, in turn, will make it much more possible for me to have a sincere and honest conversation with that person.  Over time, as people come to expect this from me, they feel much more comfortable approaching me with whatever question, request, or even disagreements.

Offering to Help

Another major change in my own behavior is that I’ve come to persistently offer to help other people I’m working with.  Even when my time or ability to help is very limited, I try to offer what I can.  Occasionally, this will lead me to over-commit myself (although using GTD helps a lot with knowing my limitations), but often the person either doesn’t need my help (but is appreciative of the offer), or only needs something fairly simple.  Either way, when people come to trust that I’ll always be ready to offer what help I can, they come to feel much more comfortable approaching me with a question, or request.  Even when I can’t always offer much help, they know I’ll freely offer what I can.

Being Optimistic & Action-Oriented

When I think back on all the people I’ve most enjoyed working with, this, above all, is the trait they all shared. By this, I don’t mean being some kind of ridiculous Pollyanna. Instead, think this attitude is contained in the question: “So, what are our options?” When faced with some kind of difficulty, whether its some difficult technical problem, or a difficult people problem, this question re-orients the discussion away from the uncertainty, stress, anxiety, guilt, and/or sadness the situation creates, and toward a positive future where things are better. Also, by asking for options, you start a positive process of working together, instead of attempting to assign blame for the difficulty.

✧✧✧

I expect in writing this, I’ve left out a whole bunch of things I’ve learned to do better over the years. I’m equally certain that I’ve left out a bunch of things I haven’t figured out yet. Got some of your own? Leave them in the comments below.