Jud – Page 35 – Making Of Things

The Devil Is In The Details

By Jud | June 16, 2017 | Comments 0 Comment

Software development is not for the faint of heart.

The SQL DELIMITER statement is particular to the mysql client tool, so you can’t use a standard dump file to bootstrap initialize a new database instance using mysqld –bootstrap. Eight hours later I figured out a workaround, namely exporting stored procedures directly from the mysql.proc table.
A single stray character can entirely break a critical piece of system functionality; finding and removing it required a late night from multiple team members.

And that was just yesterday.

Back in college I co-authored a paper entitled “A Probabilistic View of Certain Weighted Fibonacci Sums” (available here). Looking back on it, I wonder if that particular combination of words had ever been used together before. Contrast that to a sentence like “Could you go to the store and pick up some milk?” Certainly that combination of words has been used many times.

All this was brought to mind after I uttered the following this morning:

“For the purposes of our system, both syntactic and semantic errors should cause input to be rejected as malformed.”

Do you think that sentence has ever been uttered before? What about your average everyday statement? How might we go about analyzing such a thing? It’s these questions that will kick off my first significant blog series, “The Tyranny Of The Finite”, which I hope to start next week.

Until then, today’s challenge is to go out and use a sentence that is unlikely to have ever been said before.

Delightful Irony

By Jud | June 13, 2017 | Comments 0 Comment

Docker Fail

By Jud | June 12, 2017 | Comments 0 Comment

One of my favorite truisms in software development is the following:

“When two things aren’t the same, then they’re different.”

I don’t care how hard you work to make development and production environments the same, I don’t care how portable you think your programming language is across operating systems, and I especially don’t care about claims made on a product’s marketing material. Case in point:

[Docker] makes for efficient, lightweight, self-contained systems and guarantees that software will always run the same, regardless of where it’s deployed.

No doubt the Docker team has worked incredibly hard to make the above statement true, but nothing in life is guaranteed with 100% certainty. Last week I discovered that the cron utility will not execute any crontab or script that has more than one hard link pointed at it (don’t ask me why not, it just doesn’t). And due to the way Docker’s overlay filesystem works, the operating system can report a regular file as having multiple links. At least it does when the container is running under Kubernetes. But not always! When running with docker-compose on my Mac the crontab only reported one hard link, and thus cron worked great.

Took me nearly a day of fiddling to determine why the container worked great locally but failed in Kubernetes. Argh.

Adventures

By Jud | June 9, 2017 | Comments 1 comment

Yesterday the team spent most of the day debugging an urgent production issue. The root cause: a seemingly innocuous change made to a custom storage library eight months ago. Lesson learned: changes may have unintended side-effects.

And today I’m back to decompiling Java bytecode from a production server. The original source is long gone. Good times.

Slimming Down

By Jud | June 6, 2017 | Comments 0 Comment

Yesterday I migrated a bunch of microservices from the baseline Node 6 Docker image to an Alpine Linux version that I custom-built to be as small as possible. Average image size went from 800MB to 80MB across the 6 services, for a total savings of over 4GB per deployment.

Not a bad way to spend a day.

Busy, Busy, Dreadfully Busy

By Jud | June 5, 2017 | Comments 0 Comment

Any engineer worth her salt must learn to prioritize well. The next couple weeks are crunch time at the office, so my posts here will be spotty at best. My hope is that once things settle down I can dive into some meatier topics. But for now, superficiality rules.

Share In The Blame

By Jud | June 2, 2017 | Comments 0 Comment

In 2007 I spent most of the summer in the desert flight testing a sensor system in the back of a C-130 (incidentally, our test aircraft was used a few years later in the filming of the opening scene of The Dark Knight Rises). During my off-hours there wasn’t much to do in town, so I listened to quite a bit of music, including this particular album:

The message of this song is one I believe applies to many areas of life, software development not the least. As I said yesterday, in any situation where a mistake is made, no matter who committed the error, it’s a worthwhile effort for everyone involved to consider what they could have done differently. The way I figure, to admit absolutely no responsibility is to relinquish the one bit of control I do have over a situation, namely myself. And that’s a power I’m unwilling to give up.

It can be scary to live in a state of constant self-evaluation, but I find it freeing. It’s the mark of deep-seeded insecurity when one is unable to admit his own faults.

User Error Isn’t

By Jud | June 1, 2017 | Comments 0 Comment

It happens to every developer at some point. You spend a bunch of time trying to determine the source of an urgent bug, only to find out the problem was caused by action or inaction of the user. It’s easy in that moment to be frustrated, but may I suggest a more circumspect path? It behooves a thoughtful developer in these moments to consider if the source of the user error was caused by poor design. Perhaps the button wasn’t in an obvious place, or the side effects of the command were not obvious, or the name of the feature didn’t describe the outcome. In many (even most) cases, user error is preventable with good design.

Yes it’s difficult, and perhaps impossible in some situations. But it is the high calling of the software engineer to design products in a way that a user almost subconsciously knows how to operate without error. Viewing design with this attitude saves a lot of frustration.

It might seem this is a counter argument to Not Even Mostly, but actually they go hand in hand. A user must be carefully listened to and considered, but sometimes they must also be guided into the features they really need, versus what they say they want. Do that hard work up front, and you’re more likely to have a system that minimizes the potential for mistakes.

Also, go read The Design Of Everyday Things.

Covfefe

By Jud | May 31, 2017 | Comments 0 Comment

Even the smartest and most experienced developers make bone-headed mistakes, the kind that require hours of painful debugging only to find a comma was misplaced, or a one should have been a zero. While a good amount of effort should go into avoiding these errors, the energy required to prevent them completely would be astronomical. A better approach is to balance prevention and detection, hence the importance of good peer review and solid automated testing.

Author: Jud