June 2017 – Making Of Things

Do The Right Thing

By Jud | June 30, 2017 | Comments 0 Comment

I love it when a programming language behaves in an unsurprising and helpful way. This is Python in a nutshell.

Consider the following situation I came across yesterday. I needed to pair off two lists. Naturally, Python has a built-in function for this:

x = [1, 2, 3]
y = ['a', 'b', 'c']
zip(x, y)
# [(1, 'a'), (2, 'b'), (3, 'c')]

But what happens if the lists are different lengths (as they were in my case)? Python does the logical thing, and stops with the shorter list:

x = [1, 2, 3]
y = ['a', 'b', 'c', 'd', 'e', 'f']
zip(x, y)
# [(1, 'a'), (2, 'b'), (3, 'c')]

This is exactly the behavior I needed, no special case handling required. Woot!

One of the hallmark characteristics of a nerd is the ability to enjoy a wide variety of things without worrying about how popular they might be. On that note, I listed to this song yesterday and unashamedly enjoyed every minute of it. Wish I’d been there.

Gimme That Foot Gun

By Jud | June 22, 2017 | Comments 0 Comment

The past couple of days I wrote about the dangers of providing too much functionality. In a fit of cognitive dissonance I now want to contradict myself and demand dangerous power when it suits my needs.

I’ve been working the past week to get a particularly gnarly application running in a set of docker containers. There are over a dozen services, plus a Rabbit queue and a database. Many of the services do not handle database connection failures in a robust way. During my testing I wanted a simple way to ensure they waited a bit before trying to connect, as the database container needs a minute or so to seed itself and get ready for connections.

Unfortunately,docker-compose does not have any form of manual startup delay feature. This is by design, as the Docker team (rightly) argues that having services intolerant to connection failure is a bad thing. However, it’s frustrating to not have the power to do the wrong thing in the short term.

Then again, it turns out it wasn’t too tough to augment my compose file with a depends_on clause that includes a health check, which is a more reliable solution anyways.

High-Powered Foot Assault Rifle

By Jud | June 21, 2017 | Comments 0 Comment

Giving a user too much flexibility can be a dangerous thing. Better to do the hard work of separating actual requirements from “desirements” (as an old government customer of mine used to say).

Yesterday’s lesson: the Perl HTML::Tidy library doesn’t handle certain UTF-8 encoded characters properly. In particular, it truncates a non-breaking space (code c2 a0) into a bare control byte (c2), which at best gives a mangled character, and at worst an invalid encoding sequence. Pass that result to another utility, and you can expect failure.

The above incident was particularly difficult to debug because the app in question does not keep edit history. I was reminded how valuable it can be to track changes over time, not just in source code, but about the data on which an application operates as well. Made me think that git might not be a half-bad back-end storage mechanism in many situations. Anyone have experience with that?

Foot Gun Locked And Loaded

By Jud | June 19, 2017 | Comments 1 comment

I’ve spent more time than I want to admit in the past several days digging into a Java+Tomcat web application. If I never see another XML file in my life it’ll be too soon.

Java isn’t a bad language by any means, but it does often bring some baggage. It seems to offer more than the average number of opportunities for developers to run off the rails. And since it’s a typical first language in many degree programs, pretty much everyone knows a bit of it, in particular poor-quality developers who never bother to learn anything else and are happy to bang out haphazard enterprise software 9-5 for the rest of their lives.

To be clear, I’m not saying all Java developers are below average. I’ve known a number of counter-examples myself. I’m only saying that many below average developers end up focusing on Java. And that’s unfortunately resulted in a metric crapton of poorly written Java apps.

The Devil Is In The Details

By Jud | June 16, 2017 | Comments 0 Comment

Software development is not for the faint of heart.

The SQL DELIMITER statement is particular to the mysql client tool, so you can’t use a standard dump file to bootstrap initialize a new database instance using mysqld –bootstrap. Eight hours later I figured out a workaround, namely exporting stored procedures directly from the mysql.proc table.
A single stray character can entirely break a critical piece of system functionality; finding and removing it required a late night from multiple team members.

And that was just yesterday.

Balance Of Probability

By Jud | June 14, 2017 | Comments 0 Comment

Back in college I co-authored a paper entitled “A Probabilistic View of Certain Weighted Fibonacci Sums” (available here). Looking back on it, I wonder if that particular combination of words had ever been used together before. Contrast that to a sentence like “Could you go to the store and pick up some milk?” Certainly that combination of words has been used many times.

All this was brought to mind after I uttered the following this morning:

“For the purposes of our system, both syntactic and semantic errors should cause input to be rejected as malformed.”

Do you think that sentence has ever been uttered before? What about your average everyday statement? How might we go about analyzing such a thing? It’s these questions that will kick off my first significant blog series, “The Tyranny Of The Finite”, which I hope to start next week.

Until then, today’s challenge is to go out and use a sentence that is unlikely to have ever been said before.

Delightful Irony

By Jud | June 13, 2017 | Comments 0 Comment

Docker Fail

By Jud | June 12, 2017 | Comments 0 Comment

One of my favorite truisms in software development is the following:

“When two things aren’t the same, then they’re different.”

I don’t care how hard you work to make development and production environments the same, I don’t care how portable you think your programming language is across operating systems, and I especially don’t care about claims made on a product’s marketing material. Case in point:

[Docker] makes for efficient, lightweight, self-contained systems and guarantees that software will always run the same, regardless of where it’s deployed.

No doubt the Docker team has worked incredibly hard to make the above statement true, but nothing in life is guaranteed with 100% certainty. Last week I discovered that the cron utility will not execute any crontab or script that has more than one hard link pointed at it (don’t ask me why not, it just doesn’t). And due to the way Docker’s overlay filesystem works, the operating system can report a regular file as having multiple links. At least it does when the container is running under Kubernetes. But not always! When running with docker-compose on my Mac the crontab only reported one hard link, and thus cron worked great.

Took me nearly a day of fiddling to determine why the container worked great locally but failed in Kubernetes. Argh.

Adventures

By Jud | June 9, 2017 | Comments 1 comment

Yesterday the team spent most of the day debugging an urgent production issue. The root cause: a seemingly innocuous change made to a custom storage library eight months ago. Lesson learned: changes may have unintended side-effects.

And today I’m back to decompiling Java bytecode from a production server. The original source is long gone. Good times.

Month: June 2017