Michael Descy

The Excel gods have not been smiling on me this evening. I can’t get two different worksheets merged the way I want to, which is without requiring my users having to do anything other than click “Refresh all Queries.” It isn’t working. Poor data quality seems to be the culprit.

Did you know you that, in Windows 10’s File Explorer, you can easily copy a file’s full path to the clipboard? You right-click it while the Shift key is down, and select the “Copy as path” item from the context menu, which otherwise isn’t there. 🤯

I think that widescreen 16:9 presentation slides look worse than 3:2 ones. Text gets too wide to read comfortably. Some people divide their slides into two areas side-by-side, like they are two slides in one, which also isn’t good. Those are too busy, I think.

The once and future lawnmowing man

My monthly “don’t have a lawn that embarrasses me” budget has increased dramatically over the past few years. It has gone from a good buy (about $30/week) to eye-wateringly expensive (a little over $100/week). Part of the increase is that we changed lawn services from a company that did a poor job to one that did a good job. I have tried to shop around for a cheaper, but still good, service, but haven’t found one yet. My wife doesn’t want us to revert back to the old, not-so-good lawn service we used to use. Even if we did, I’m sure we would get a much higher rate than we used to, because prices have increased all around.

Since this summer, I have been bandying about the idea of mowing my own lawn again (it’s been 10 years since I last did, when my mower broke). This week I decided that, by next spring, I will buy a good electric mower and make time to mow my own lawn each week. Despite having to buy a mower, which could cost $700-$800, I wills till save hundreds of dollars per month for a relatively small amount of my time.

It’s true that I won’t be able do as good a job as the service does with edging and weeding, but I don’t care. I will also have to figure out how to buy and use herbicide, which is the main secret to a non-embarrassing lawn in my area; I think I can do it. It will feel good to know that I am saving money. I also think it will feel good—most weeks, at least—to get a little more sunshine and exercise each week.

It is stupefying to me that Apple’s new low-end iPad supports the first-generation Apple Pencil and requires a special adapter to pair and charge it. Who thought that would be a good idea?

📚 Greenlights

I read Greenlights by Matthew McConaughey last night. I picked it because it was the first nonfiction book I found in Libby that I thought would be fun to read on a quiet night. I first heard about the book from an interview McConaughey did with Bill Maher where I initially thought Maher didn’t like him. I am a fan of McConaughey’s acting and his positive attitude, though i am not always sure if he really has a coherent philosophy. (This book didn’t help in that regard.)

The book contains some entertaining stories, including his stint in Australia, but my overall impression is that McConaughey led a charmed life, thanks in large part to being handsome and lucky in his youth, and rich and famous in his middle age. His book emits, elides, or spins into positive any negative experiences he had in his life, which is his point about “red” and “yellow lights” becoming “greenlights”, but it does leave a shallower impression of who he is and what he has learned from life.

I started using https://hckrnews.com to browse Hacker News in a less redundant way. It collects the stories that reach the front page of Hacker News and groups them by day. Each story shows up once and doesn’t move around, so it takes a lot less time to browse.

I finally changed over my personal email address to use iCloud. That will save me $20/year on my Hover.com bill and give me an excuse to use my iCloud email account, which has been checked fruitlessly by Apple Mail for many years.

Blah

I stopped writing blog posts a couple months ago for no reason whatsoever. It has been very hard to get back into the habit of posting something regularly.

I have not lost interest in blogging. Something stranger has happened: I have lost interest in myself. I don’t completely know why.

I am not overly depressed or anything—just maxed out. My general feeling about the past couple months has been that I have put in more effort, and reaped fewer rewards for that effort, than I have in years.

I have some light reading tonight: Polars User Guide. I basically hate Pandas’s API and hope to use something more nicely designed, and a heck of a lot faster, instead.

📺 I finished watching Slow Horses Season 1 tonight. I loved it and can’t wait for Season 2. I’m wondering if I would like the books that the show is based on. I also wonder if it will get more than two seasons.

I’m uninstalling Visual Studio from my work computer. It feels weird to do it, but I haven’t used it in years. Visual Studio Code killed it for me.

Is there no good and fast way to create an ETL job in Python?

I love using PETL to read, transform, and validate datasets. Its API is simple and straightforward to use, and its documentation, while not perfect, is really good. Unfortunately, PETL’s data processing—especially loading data into SQL Server—is very slow—to the point of being completely unusable for anything but the smallest datasets.

Odo promises to be much faster at data loads, and seems dead-simple to use, but I cannot get it to work. It turns out that the package hasn’t been updated in five years, which is why it will not work with the modern Python version I am using. I just wasted an hour on it and will probably give it up completely.

I can’t find any other ETL tools that will do the data load for me. I have to rely on the SQL Server Import/Export Wizard, which usually works and is fast but is a one-shot thing because I don’t have SQL Server Integration Services at my disposal.

I suppose I will either have to fork Odo, get it working, and figure out how to get my fork of Odo into my project, or I will have to write my own import routine using SQL Alchemy. Both seem difficult to me and not worth the trouble. I’m not the database administrator on my project—I’m supposed to be the data analyst!

Chess is just poker now

The chess cheating scandal du jour is puzzling and fascinating. Matteo Wong’s article in The Atlantic does not really unpack the issue, but instead provide some depth into how computers have changed the game over the past 25 years:

What once seemed magical became calculable; where one could rely on intuition came to require rigorous memorization and training with a machine. Chess, once poetic and philosophical, was acquiring elements of a spelling bee: a battle of preparation, a measure of hours invested.

It is interesting, though understandable from a technological standpoint, that the concern in the 1990s was that a person might help the computer engine cheat. Today, conversely, the concern is that a computer engine (combined with some spycraft tech, it must be said) might help person cheat.

Long-running data feeds are a real thing after all

Over the weekend I wrote a SQL Server stored procedure and some user-defined functions it relies on that cleaned up data and inserted it into a new table. It was the sort of thing that took a short time to write, but an awfully long time to execute and to troubleshoot. It gave me an opportunity to learn more about using SQL Server’s “Include Actual Execution Plan” and “Include Live Query Statistics” features, which helped point me in the direction of creating an index that supposedly sped the whole process up by 30%.

The first version of my load script ran for 9.5 hours, and then bombed because it tried to insert a NULL value into the primary key of a table. Zero rows were inserted. I revised it to create a second version, which eventually got killed by the server (it timed out or over-burdened the database server, I guess) before it completed after over ten hours. Zero rows were inserted. The final version ran yesterday and overnight last night for over 16 hours, which makes it the longest-running data feed I have ever written by over 10 hours. This run was successful. Only 37,000 rows were inserted, but to get the data I had to comb over 45 million records, which is why it took so long.

As an auditor, I have not always believed it when IT would tell me a data feed took all weekend to run. This experience helped me understand than it can happen, especially when it is an ad-hoc job.

🎵 Today’s listen: Autofiction by (The London) Suede. It’s really good! They went “punk” after 30 years. 😀

I wrote a SQL script that queried, cleaned, and loaded data into a table. It failed after 9 hours of runtime because of a NULL value in the source data that I had forgotten about. 🤦‍♂️

I am still learning new T-SQL operators. Today I discovered CROSS APPLY, which is for joining to a table-valued function (with parameters pulled from the other tables).

🎵 Today’s listen: Asphalt Meadows by Death Cab for Cutie. I’m excited to hear new music from one of my favorite bands from the aughts.

Tableau is a very powerful data visualization tool. The more I use it, though, the weirder it is to me from a UI standpoint.  I find it both easier and harder to use that it’s nearest competitor, Microsoft Power BI.

I coded and tested an update to one of my iOS apps tonight. It has been too long since I touched Xcode. I have been working more evening hours this year than ever before—mainly because I am on dad duty during daytime hours—and don’t get to work on my hobbies as much as I used to.

My son lost his first tooth today! It happened while he was at school, eating an apple. His teacher wrapped it up for us and sent it home with a note.

iOS 16 is very nice! I am enjoying the new Lock Screens and how they tie to the system Focus feature.

My idea to eat healthier on my vacation was poorly conceived. 😅

Whitefield, New Hampshire, band concert. My father-in-law is playing a song he arranged and my kids are dancing and running around the town green.