Delightful Irony
Docker Fail
One of my favorite truisms in software development is the following:
“When two things aren’t the same, then they’re different.”
I don’t care how hard you work to make development and production environments the same, I don’t care how portable you think your programming language is across operating systems, and I especially don’t care about claims made on a product’s marketing material. Case in point:
[Docker] makes for efficient, lightweight, self-contained systems and guarantees that software will always run the same, regardless of where it’s deployed.
No doubt the Docker team has worked incredibly hard to make the above statement true, but nothing in life is guaranteed with 100% certainty. Last week I discovered that the cron utility will not execute any crontab or script that has more than one hard link pointed at it (don’t ask me why not, it just doesn’t). And due to the way Docker’s overlay filesystem works, the operating system can report a regular file as having multiple links. At least it does when the container is running under Kubernetes. But not always! When running with docker-compose on my Mac the crontab only reported one hard link, and thus cron worked great.
Took me nearly a day of fiddling to determine why the container worked great locally but failed in Kubernetes. Argh.
Adventures
Yesterday the team spent most of the day debugging an urgent production issue. The root cause: a seemingly innocuous change made to a custom storage library eight months ago. Lesson learned: changes may have unintended side-effects.
And today I’m back to decompiling Java bytecode from a production server. The original source is long gone. Good times.
Slimming Down
Yesterday I migrated a bunch of microservices from the baseline Node 6 Docker image to an Alpine Linux version that I custom-built to be as small as possible. Average image size went from 800MB to 80MB across the 6 services, for a total savings of over 4GB per deployment.
Not a bad way to spend a day.
Busy, Busy, Dreadfully Busy
Any engineer worth her salt must learn to prioritize well. The next couple weeks are crunch time at the office, so my posts here will be spotty at best. My hope is that once things settle down I can dive into some meatier topics. But for now, superficiality rules.
Share In The Blame
In 2007 I spent most of the summer in the desert flight testing a sensor system in the back of a C-130 (incidentally, our test aircraft was used a few years later in the filming of the opening scene of The Dark Knight Rises). During my off-hours there wasn’t much to do in town, so I listened to quite a bit of music, including this particular album:
The message of this song is one I believe applies to many areas of life, software development not the least. As I said yesterday, in any situation where a mistake is made, no matter who committed the error, it’s a worthwhile effort for everyone involved to consider what they could have done differently. The way I figure, to admit absolutely no responsibility is to relinquish the one bit of control I do have over a situation, namely myself. And that’s a power I’m unwilling to give up.
It can be scary to live in a state of constant self-evaluation, but I find it freeing. It’s the mark of deep-seeded insecurity when one is unable to admit his own faults.
User Error Isn’t
It happens to every developer at some point. You spend a bunch of time trying to determine the source of an urgent bug, only to find out the problem was caused by action or inaction of the user. It’s easy in that moment to be frustrated, but may I suggest a more circumspect path? It behooves a thoughtful developer in these moments to consider if the source of the user error was caused by poor design. Perhaps the button wasn’t in an obvious place, or the side effects of the command were not obvious, or the name of the feature didn’t describe the outcome. In many (even most) cases, user error is preventable with good design.
Yes it’s difficult, and perhaps impossible in some situations. But it is the high calling of the software engineer to design products in a way that a user almost subconsciously knows how to operate without error. Viewing design with this attitude saves a lot of frustration.
It might seem this is a counter argument to Not Even Mostly, but actually they go hand in hand. A user must be carefully listened to and considered, but sometimes they must also be guided into the features they really need, versus what they say they want. Do that hard work up front, and you’re more likely to have a system that minimizes the potential for mistakes.
Also, go read The Design Of Everyday Things.
Covfefe
Even the smartest and most experienced developers make bone-headed mistakes, the kind that require hours of painful debugging only to find a comma was misplaced, or a one should have been a zero. While a good amount of effort should go into avoiding these errors, the energy required to prevent them completely would be astronomical. A better approach is to balance prevention and detection, hence the importance of good peer review and solid automated testing.
Winging It
I keep a running list of topics, but today I’m ignoring it and shooting from the hip. A stream of consciousness post, if you will.
Yesterday evening I added a couple new features to an Ionic app. I’m really appreciating the improvements brought by Angular 2; it’s made the architecture quite a bit cleaner than before, and adding new capabilities is quick and painless. I still have a ways to go before I fully grok observables, but I’m getting there.
Was talking with a musician friend of mine on Sunday about the difficulty of making meaningful progress on a project without having at least 2 continuous hours of focused time to work on it. Half hours here and there don’t cut it. That’s an important truth both for us nerds who try to do side work in our spare time. It’s also an important reminder for non-nerds who feel the need to interrupt the nerds in their charge.
Speaking of interruptions, nearly 6 hours passed since I wrote that past sentence due to an urgent production bug. Who knew when I woke up this morning that by noon I’d be decompiling Java classes from a production server. Yay!
Numbers Do Lie
The Internet is a funny place. Yesterday was the first weekday in some time that I didn’t write a post, yet I had the most page views in a single day since I created this thing. Apparently someone (or some-bot) in Canada decided to read every single post I’ve ever written. A roughly similar thing happened a few weeks ago, when a post I spent quite a bit of time on got only a handful of views, while the filler I wrote the next day got three times as much traffic. I’m not complaining, just observing that it can be difficult to predict how consumers will respond to a product.
I think there’s a lesson there for aspiring software developers (and really creators of all types). Don’t get too hung up on numbers or popularity. Don’t think less of yourself just because your app didn’t blow up like Facebook or Instagram (those guys will tell you that was luck as much as it was hard work). Simply do the best work you can.
When I decided to revive this blog, I wanted to post daily for two reasons. I figured it would draw more traffic to have more content, but more importantly, my writing muscles had atrophied and needed regular exercise. Ultimately this whole thing is for my own enrichment, but if it helps others, great.