Ramblings of a SQL Hillbilly

Or, the mad ravings of a data warehouse forklift operator

T-SQL Tuesday: Assumptions

For this month’s T-SQL Tuesday, Dev Nambi has chosen the topic of assumptions. He assumed I would have a big assumption at work to talk about. Good assumption!

Probably the biggest assumption we deal with in my unit at work is Things Are The Way They Are For A Reason, And Therefore We Should Continue To Do Them That Way (hereafter abbreviated Things Are And Should Be.) I love talking about this assumption because I have a long and storied history with it. Well, I have a storied history with it, anyway. And length of time is all in the eye of the beholder. I know, because I used to be able to count to approximately a million in fifteen minutes, and now I can spend fifteen minutes ordering breakfast and not notice. But I digress (which surprises nobody!)

When I began working as a Data Warehouse Forklift Operator, I entered a culture that had deliberately taken a “Produce First, Understand Later” approach. If it could be templatized it was, and if it couldn’t be templatized, it was maintained by one of the more savvy and experienced developers. We were in the process of moving from an ETL tool that did everything for you (badly) and to a tool that only did some things for you (SSIS). The old tool transformed data in DB2, and now we wanted to transform the data on SQL Server. For the first several months I worked on code, I assumed that Things Are And Should Be. After all, I was new to the data warehousing world, and we had experienced Architects and Designers who Knew What They Were Doing, right?

Our ETL Process

Fast forward about six months. I had discovered many cases in which the way we did things was suboptimal, circuituous, and even Not Quite What We Wanted. I began to suspect that my Big Assumption was perhaps not too helpful. I spent a lot of time waiting on code that worked, but took its sweet time with the large quantities of data it was processing. Some of the architecture of what I was working with seemed overly complicated. Bits and pieces were clumsy. But, I liked what I did and I was still new, after all, so what did I know?

After a year or so of running the Data Forklift, I began to wonder why the code I was modifying was written like it was. After all, surely I could do my job better if I knew what exactly I was trying to accomplish. So, I began asking questions – and found that, more often than I was comfortable with, the answer was I Don’t Know, We’ve Just Always Done It That Way. So I began to trace execution paths and transformations through the code. Many post-it notes and spreadsheets were sacrificed to the study of the code in order to understand. There were flashes of inspiration in the shower, Aha! moments while driving home, and many an evening’s sleep interrupted as I determined the answer to another question of Why Would You Do It That Way? While I was left scratching my head, I also determined that I would pursue the path of the architect, understanding the process so deeply that I could see my way clear to improvements.

It’s now been three years, and I now have a half dozen junior developers depending on me to teach them… well, pretty much everything about what we do, why, and how! Err… And how. Anyway, I find them asking many of the same questions, and I’ve had to resist the urge to give them the same old answers. It’s easy to default back to Things Are And Should Be. After all, much of the code they’re now questioning is code that I’ve modified from the way we used to do things! However, I believe now that the best thing I can do for the team is to head Things Are And Should Be off at the pass. If I can pass on all I know to them, then we’ll all be better. A developer who questions nothing will never become an asset to the team – and my team has too much potential to settle for mediocrity.