Tyranny Of The Urgent

Tyranny Of The Urgent

By | | Comments 0 Comment

A wide variety of tasks face the developer each day. Some can be accomplished quickly, like responding to emails, and some need much time, such as implementing a new product feature. Some require little thought, like cranking out a refactoring across several components, and some take a lot of mental energy. Some have immediate near-term deadlines, and others are not due for some time.

It’s particularly easy to focus only on the tasks that must be done soon (so-called “urgent” work) but the responsible developer must fight this tendency. Urgent and important are not synonyms.

Ain’t No User Here

Ain’t No User Here

By | | Comments 0 Comment

Here’s a new one. I was debugging a problem with a web server that hadn’t been used in a while that was allowing users to log in, but failing to perform a number of other functions (gotta love an API that happily returns a 200 OK even though it’s clearly not working because there’s no data in the response). No one could think of anything that had changed, but clearly something had.

After some spelunking through logs, I found some DB permissions errors. Obviously some DB calls were working, so that was odd, as I knew credentials were only specified in one place.

Another hour of research later, I discovered that MySQL views run as the user who created them, not the user who queries them. This actually makes sense for use cases where you want to give controlled read access to parts of the DB to a less-privileged user. Logically then, if that user is deleted, the view fails to run. And in this situation that’s exactly what happened.

There’s a number of lessons to be learned from this situation:

  1. A user probably should fail to delete if it owns other objects in the database. Otherwise unexpected side-effects occur.
  2. Accurate error messaging is essential to debugging. When trying to run the view, I got a simple “user does not have permission” error, which told me nothing about the underlying problem, which meant another hour or two of research. This goes for APIs especially. Please know when to use a 4XX vs. 5XX error especially, and even better learn the subtle differences between 502, 503, and 504, and when each should be used.
  3. Don’t try to diagnose problems on Friday afternoons before holiday weekends. You’re asking for trouble.
Do The Right Thing

Do The Right Thing

By | | Comments 0 Comment

I love it when a programming language behaves in an unsurprising and helpful way. This is Python in a nutshell.

Consider the following situation I came across yesterday. I needed to pair off two lists. Naturally, Python has a built-in function for this:

x = [1, 2, 3]
y = ['a', 'b', 'c']
zip(x, y)
# [(1, 'a'), (2, 'b'), (3, 'c')]

But what happens if the lists are different lengths (as they were in my case)? Python does the logical thing, and stops with the shorter list:

x = [1, 2, 3]
y = ['a', 'b', 'c', 'd', 'e', 'f']
zip(x, y)
# [(1, 'a'), (2, 'b'), (3, 'c')]

This is exactly the behavior I needed, no special case handling required. Woot!

It’s All Geek To Me

It’s All Geek To Me

By | | Comments 0 Comment

One of the hallmark characteristics of a nerd is the ability to enjoy a wide variety of things without worrying about how popular they might be. On that note, I listed to this song yesterday and unashamedly enjoyed every minute of it. Wish I’d been there.

Gimme That Foot Gun

Gimme That Foot Gun

By | | Comments 0 Comment

The past couple of days I wrote about the dangers of providing too much functionality. In a fit of cognitive dissonance I now want to contradict myself and demand dangerous power when it suits my needs.

I’ve been working the past week to get a particularly gnarly application running in a set of docker containers. There are over a dozen services, plus a Rabbit queue and a database. Many of the services do not handle database connection failures in a robust way. During my testing I wanted a simple way to ensure they waited a bit before trying to connect, as the database container needs a minute or so to seed itself and get ready for connections.

Unfortunately,docker-compose does not have any form of manual startup delay feature. This is by design, as the Docker team (rightly) argues that having services intolerant to connection failure is a bad thing. However, it’s frustrating to not have the power to do the wrong thing in the short term.

Then again, it turns out it wasn’t too tough to augment my compose file with a depends_on clause that includes a health check, which is a more reliable solution anyways.

High-Powered Foot Assault Rifle

High-Powered Foot Assault Rifle

By | | Comments 0 Comment

Giving a user too much flexibility can be a dangerous thing. Better to do the hard work of separating actual requirements from “desirements” (as an old government customer of mine used to say).

Yesterday’s lesson: the Perl HTML::Tidy library doesn’t handle certain UTF-8 encoded characters properly. In particular, it truncates a non-breaking space (code c2 a0) into a bare control byte (c2), which at best gives a mangled character, and at worst an invalid encoding sequence. Pass that result to another utility, and you can expect failure.

The above incident was particularly difficult to debug because the app in question does not keep edit history. I was reminded how valuable it can be to track changes over time, not just in source code, but about the data on which an application operates as well. Made me think that git might not be a half-bad back-end storage mechanism in many situations. Anyone have experience with that?

Foot Gun Locked And Loaded

Foot Gun Locked And Loaded

By | | Comments 1 comment

I’ve spent more time than I want to admit in the past several days digging into a Java+Tomcat web application. If I never see another XML file in my life it’ll be too soon.

Java isn’t a bad language by any means, but it does often bring some baggage. It seems to offer more than the average number of opportunities for developers to run off the rails. And since it’s a typical first language in many degree programs, pretty much everyone knows a bit of it, in particular poor-quality developers who never bother to learn anything else and are happy to bang out haphazard enterprise software 9-5 for the rest of their lives.

To be clear, I’m not saying all Java developers are below average. I’ve known a number of counter-examples myself. I’m only saying that many below average developers end up focusing on Java. And that’s unfortunately resulted in a metric crapton of poorly written Java apps.

The Devil Is In The Details

The Devil Is In The Details

By | | Comments 0 Comment

Software development is not for the faint of heart.

  • The SQL DELIMITER  statement is particular to the mysql client tool, so you can’t use a standard dump file to bootstrap initialize a new database instance using mysqld –bootstrap. Eight hours later I figured out a workaround, namely exporting stored procedures directly from the mysql.proc  table.
  • A single stray character can entirely break a critical piece of system functionality; finding and removing it required a late night from multiple team members.

And that was just yesterday.

Balance Of Probability

Balance Of Probability

By | | Comments 0 Comment

Back in college I co-authored a paper entitled “A Probabilistic View of Certain Weighted Fibonacci Sums” (available here). Looking back on it, I wonder if that particular combination of words had ever been used together before. Contrast that to a sentence like “Could you go to the store and pick up some milk?” Certainly that combination of words has been used many times.

All this was brought to mind after I uttered the following this morning:

“For the purposes of our system, both syntactic and semantic errors should cause input to be rejected as malformed.”

Do you think that sentence has ever been uttered before? What about your average everyday statement? How might we go about analyzing such a thing? It’s these questions that will kick off my first significant blog series, “The Tyranny Of The Finite”, which I hope to start next week.

Until then, today’s challenge is to go out and use a sentence that is unlikely to have ever been said before.