Tag: Dive Deep

A Tale As Old As 2001

A Tale As Old As 2001

By | | Comments 0 Comment

For the next week or two I’m going to go back through my old drafts and finish them up. That means the stories are at least a year or two old. For this one, I’m curious if Edge finally changed the behavior. Anyone want to try it out?

When you’re debugging a pernicious issue, there’s no greater feeling than Google search auto-completing your first couple search terms and matching a page that describes your problem to a T. The challenge of course is figuring out those magic couple of words.

The team was recently trying to figure out an IE11-only problem (ugh) where our authentication mechanism was failing, but only for a subset of customers, with no obvious commonality. The server would return a Set-Cookie header, but the browser completely ignored it. WTF, Microsoft!

We’d spent an entire day trying to come up with a solution, until finally stumbling into the root cause: underscores in the subdomain. Chrome and Firefox are cool with them, but IE silently refuses to store cookies when they’re present. The details are a fascinating combination of unexpected side effects from a bug fix, misinterpreted web standards, and lingering backwards compatibility. This post captures the story nicely.

My product manager had never been thrilled with the way we’d been handling domain names. While I couldn’t have anticipated our design would lead to this misadventure (and a simple s/_/-/ solved the problem), I probably should have given his critique a closer listen.

Way Way Back

Way Way Back

By | | Comments 3 comments

I’ve had the pleasure of working with a large variety of technologies over the course of my career. Yesterday I was working on an interface to an old government database, without the aid of documentation, natch. After a few hours I was able to extract data from the system, but I was unable to decode it. Google is a great tool (I couldn’t get 15 minutes into my day without it), but if you don’t know what to search for you can’t find anything. Thankfully some careful guesses led me to the Wikipedia entry for EBCDIC, a character encoding developed in the mid-60s.

“Extended Binary Coded Decimal Interchange Code (EBCDIC) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. EBCDIC descended from the code used with punched cards and the corresponding six bit binary-coded decimal code used with most of IBM’s computer peripherals of the late 1950s and early 1960s.”

It’s a fun thing in this industry to say you’ve worked with 50-year-old technology, but only once you’ve figured it out. Surprisingly there’s some modern tooling to interface with this particular encoding (yay Python!), so once I knew what I was dealing with it was straightforward enough. Maybe even easier than what it used to be, if this anecdote is to be believed:

EBCDIC: An alleged character set used on IBM dinosaurs. It exists in at least six mutually incompatible versions, all featuring such delights as non-contiguous letter sequences and the absence of several ASCII punctuation characters fairly important for modern computer languages (exactly which characters are absent varies according to which version of EBCDIC you’re looking at). IBM adapted EBCDIC from punched card code in the early 1960s and promulgated it as a customer-control tactic, spurning the already established ASCII standard. Today, IBM claims to be an open-systems company, but IBM’s own description of the EBCDIC variants and how to convert between them is still internally classified top-secret, burn-before-reading. Hackers blanch at the very name of EBCDIC and consider it a manifestation of purest evil.

Lions And Tigers And Permutations, Oh My!

Lions And Tigers And Permutations, Oh My!

By | | Comments 1 comment

This blog is turning into a bit of a “debugging adventure of the day”, which wasn’t the original intent, but whatever. I’ll get around to meatier posts someday.

This morning’s adventure was debugging a pretty serious issue with a web application where selections would not save. Initially it only manifest itself on Windows machines, so we thought Microsoft was to blame. Oddly, though, Chrome failed to work but Edge was fine. Weird.

Wanting to check a few other variables, we grabbed another co-worker’s Windows laptop, and it worked fine in Chrome. WTF!?! Nearly identical hardware, same OS, same browser, and same browser version, but one machine worked, and the other didn’t. Cleared caches, rebooted, etc. Same results.

But did you catch the word “nearly” there? Turns out there was a difference between the two laptops. One had a touchscreen, and the other didn’t. Fast-forward through an hour of slogging through a bunch of hacked together JavaScript code to support drag-and-drop on a variety of devices, and sure enough, the “non-mobile yet touchscreen enabled” case had its own special snowflake handler that wasn’t working correctly in Chrome.

Why you ask? Well, in some browsers, touch events are delayed by 300ms (this is so things like double-tap and dragging can be detected properly). This causes the mouse click event to go first, and when clicking on a button that submits the form, the form gets processed before the touch event can fire that needs to run and configure the inputs to the form submission. If you touch the button instead of clicking with the trackpad (a condition that only makes sense on a touch-enabled laptop vs a tablet), it works okay.

That means this situation only manifest itself on a touchscreen-enabled non-mobile device, in Chrome, when the user clicks the submit button with the trackpad vs touching the button. Can’t hardly blame QA for not finding that one.

The Devil Is In The Details

The Devil Is In The Details

By | | Comments 0 Comment

Software development is not for the faint of heart.

  • The SQL DELIMITER  statement is particular to the mysql client tool, so you can’t use a standard dump file to bootstrap initialize a new database instance using mysqld –bootstrap. Eight hours later I figured out a workaround, namely exporting stored procedures directly from the mysql.proc  table.
  • A single stray character can entirely break a critical piece of system functionality; finding and removing it required a late night from multiple team members.

And that was just yesterday.

Balance Of Probability

Balance Of Probability

By | | Comments 0 Comment

Back in college I co-authored a paper entitled “A Probabilistic View of Certain Weighted Fibonacci Sums” (available here). Looking back on it, I wonder if that particular combination of words had ever been used together before. Contrast that to a sentence like “Could you go to the store and pick up some milk?” Certainly that combination of words has been used many times.

All this was brought to mind after I uttered the following this morning:

“For the purposes of our system, both syntactic and semantic errors should cause input to be rejected as malformed.”

Do you think that sentence has ever been uttered before? What about your average everyday statement? How might we go about analyzing such a thing? It’s these questions that will kick off my first significant blog series, “The Tyranny Of The Finite”, which I hope to start next week.

Until then, today’s challenge is to go out and use a sentence that is unlikely to have ever been said before.

Docker Fail

Docker Fail

By | | Comments 0 Comment

One of my favorite truisms in software development is the following:

“When two things aren’t the same, then they’re different.”

I don’t care how hard you work to make development and production environments the same, I don’t care how portable you think your programming language is across operating systems, and I especially don’t care about claims made on a product’s marketing material. Case in point:

[Docker] makes for efficient, lightweight, self-contained systems and guarantees that software will always run the same, regardless of where it’s deployed.

No doubt the Docker team has worked incredibly hard to make the above statement true, but nothing in life is guaranteed with 100% certainty. Last week I discovered that the cron utility will not execute any crontab or script that has more than one hard link pointed at it (don’t ask me why not, it just doesn’t). And due to the way Docker’s overlay filesystem works, the operating system can report a regular file as having multiple links. At least it does when the container is running under Kubernetes. But not always! When running with docker-compose on my Mac the crontab only reported one hard link, and thus cron worked great.

Took me nearly a day of fiddling to determine why the container worked great locally but failed in Kubernetes. Argh.

Adventures

Adventures

By | | Comments 1 comment

Yesterday the team spent most of the day debugging an urgent production issue. The root cause: a seemingly innocuous change made to a custom storage library eight months ago. Lesson learned: changes may have unintended side-effects.

And today I’m back to decompiling Java bytecode from a production server. The original source is long gone. Good times.

Winging It

Winging It

By | | Comments 0 Comment

I keep a running list of topics, but today I’m ignoring it and shooting from the hip. A stream of consciousness post, if you will.

Yesterday evening I added a couple new features to an Ionic app. I’m really appreciating the improvements brought by Angular 2; it’s made the architecture quite a bit cleaner than before, and adding new capabilities is quick and painless. I still have a ways to go before I fully grok observables, but I’m getting there.

Was talking with a musician friend of mine on Sunday about the difficulty of making meaningful progress on a project without having at least 2 continuous hours of focused time to work on it. Half hours here and there don’t cut it. That’s an important truth both for us nerds who try to do side work in our spare time. It’s also an important reminder for non-nerds who feel the need to interrupt the nerds in their charge.

Speaking of interruptions, nearly 6 hours passed since I wrote that past sentence due to an urgent production bug. Who knew when I woke up this morning that by noon I’d be decompiling Java classes from a production server. Yay!

Numbers Do Lie

Numbers Do Lie

By | | Comments 0 Comment

The Internet is a funny place. Yesterday was the first weekday in some time that I didn’t write a post, yet I had the most page views in a single day since I created this thing. Apparently someone (or some-bot) in Canada decided to read every single post I’ve ever written. A roughly similar thing happened a few weeks ago, when a post I spent quite a bit of time on got only a handful of views, while the filler I wrote the next day got three times as much traffic. I’m not complaining, just observing that it can be difficult to predict how consumers will respond to a product.

I think there’s a lesson there for aspiring software developers (and really creators of all types). Don’t get too hung up on numbers or popularity. Don’t think less of yourself just because your app didn’t blow up like Facebook or Instagram (those guys will tell you that was luck as much as it was hard work). Simply do the best work you can.

When I decided to revive this blog, I wanted to post daily for two reasons. I figured it would draw more traffic to have more content, but more importantly, my writing muscles had atrophied and needed regular exercise. Ultimately this whole thing is for my own enrichment, but if it helps others, great.

Like Eating Your Vegetables, But Better

Like Eating Your Vegetables, But Better

By | | Comments 1 comment

Good software engineers read. And not just technical blogs or books on coding, but on all manner of subjects. Politics, mathematics, philosophy, science, history, and theology have much to contribute to the work of professional software development. But even more important to cultivating a healthy creative mind is the reading of fiction.

Maybe it’s just me, but I feel that a rich diet of fiction has fallen out of favor in our culture, especially in contrast to the consumption of technical information. That’s unfortunate, for stories have a unique power to shape the mind. For me, classic science fiction in particular has inspired me to consider how technology can be of benefit to humanity, and warned me of ways it might be a danger. No O’Reilly book is going to tell you that.

While I’m on the topic, can I say how much it bothers me that we categorize books as “fiction” and “non-fiction”? Those are horrible labels. Novels and other stories can communicate truth in a myriad of ways, and it’s quite possible for non-fiction to be either mistaken or outright false. If you tell me “I only want to read things that are real, not fake” I will absolutely make fun of you. Consider yourself warned.