My data analysis bot

Today I spent the majority of my work day creating a massive Excel template. Tomorrow I will code the data analysis bot that will fill that template with data tables and command logs that document the analytical procedures. Over time, these two days worth of work are going to save me many, many hours.

I have always automated the data analysis part of my job, but never enough of it for my tastes. Although I am not a programmer by trade, I think like one. One of the creeds of a programmer is “don’t repeat yourself.” Sadly, at work, I repeat manual data analysis procedures all the time. Certain parts of my job resisted automation for years. Lots of work I do is ad-hoc and may never be repeated, so it rarely makes sense to automate it fully or to generalize the approach for other data files and projects. Other work I do, like pulling samples, happens all the time. I get fed up when I am performing it the same way, and no better than, how I did it years ago.

I put some though into it earlier this year and discovered that documenting the work, rather than performing it, was the biggest time-waster in my day. Documenting my work in work papers has always taken a very long time. While necessary, it is the least productive time I spend on an analysis project. (In audit, work papers are critical. They support our observations and can be followed, step-by-step, to reperform our work.)

This year, I finally figured out how to create a bot that would not only run a “standard analysis” on a certain type of dataset (that is the easy part for me), but would also document the work in an Excel work paper (that is the hard part). It is not perfect yet, but it has saved me a lot of time over the past six months. I plan to keep working on it, mostly by building new work paper templates and standard analyses, next year and into the future, so that it keeps payment me dividends.

I am presenting my InsurTech slides to a new audience on Wednesday after about a month away from the material. I have to start rehearsing soon. Hopefully it will all come back to me quickly once I look at the slides again.

Why do I even bother running a home file server?

Over the past year, my TrueNAS Core server has been bugging me every few months about one of my boot failing. In this case, the boot drives are simply two USB sticks, run in a mirrored configuration. If one fails, the other one handles the load. (The reason I use a USB stick for a boot drive is that my server has an internal USB port for that very purpose, and no other place for the boot drive to go.)

This week, after yet another USB stick failed, I tried to resolve the problem by buying new USB sticks and installing TrueNAS onto them. The OS installs took hours and the server would not reboot. Eventually I discovered that I could buy an SSD drive with the form factor of a USB stick, which is what I have wanted for years. It fits inside my server like a USB stick, but contains a fast, durable SSD drive instead of slow, fragile flash memory. I installed TrueNAS onto it, which seemed to take seconds rather than hours, but I could not boot from it. I spent an hour swapping USB drives and trying to boot the server until it would not boot TrueNAS from any USB drive I had. It was a disaster.

I gave up and installed Ubuntu Server onto the new SSD drive. I knew that Ubuntu supports ZFS now and could import my existing data pool. Luckily, it boots like a champ. After the install, I found some instructions to help me set up ZFS, Samba, and Minio, and—after editing file permissions—everything is set. I didn’t lose any data during the OS switch, but I did mess up my Arm backups when I tried to move them into a new Minio storage location. Luckily, I wasn’t depending on those backups for anything, because my important data is in the cloud and I have backups on Backblaze B2 as well.

I use the NAS daily for media sharing and backups. It takes very little maintenance except once in a while when it becomes a headache and money sink. I sometime wonder if I should go back to having an external hard drive instead, now that 8 TB external hard drives are pretty cheap. The problem with that approach is that external hard drives get hot and die, which happened to me so many times that I bought a NAS.

I will miss running TrueNAS (which used to be called FreeNAS) because I have run it for ten years and used to be really into its FreeBSD underpinnings. Now, all the servers I rely on run either Ubuntu Server or Debian, and I will just have to deal with administering them via SSH rather than via a web page.

I shared the article I wrote to one of my mentors today. He said he loved it and that he would help me get it published in an industry journal. Things are looking up!

My first adjustable desk ended up non-adjustable

I received a motorized sit/stand desk today for my kids to use in the family room. I assembled the whole thing this afternoon, which was not too hard but took about an hour. Once it was done, I discovered that the left side would not lift. It was a frustrating way to end up. I emailed product support and hope to get some kind of answer. I’m afraid that, no matter what, I’m going to have to disassemble the whole thing to fix it. That will be a pain.

Some days I feel like a hero just because I got dinner on the table and my kids’ homework completed.

My wife and I had parent-teacher conferences with my daughter’s teachers. My daughter is doing great (which we already knew) and her teachers love her. I am very proud of her.

My main (or only) contribution to gift-giving for my kids this holiday season

My main (or only) contribution to gift-giving for my kids this holiday season was a biggie: I bought them a multi-function color laser printer. (Don’t worry, my wife bought them a ton of other presents that are a lot more fun.)

I got the idea to buy a color printer because my son’s preschool prints everything in color, and the administrators there said that color printing is phenomenal for kids’ worksheets. Beyond filling out preschool worksheet, my son loves to take photos and to draw. Once he gets a newer iPad I’m sure he’ll draw more electronically and will love the ability to print out his creations in full color. My daughter will likely be printing out book covers. She likes to write and illustrate story books, and a color printer will allow her to do that with even more panache. She also makes lots of presentation slide decks for school, and I thought she might need to print them out to turn them in. It turns out that she submits everything to her teachers electronically, so that use case probably won’t pan out.

I didn’t intend to get a big, multifunction printer, though. I had wanted to buy a color printer, which was much cheaper. Due to the global supply shortage, I had to buy what was available, which meant ponying up for the next model up in the lineup, which includes a scanner, too. It cost a lot of money, and I’m well aware of how much toner and paper are going to cost me each year. I think will be worth it, though, to have a mini print-shop running in the family room from now on. My kids are both very excited to have it. I can’t wait to see what they do with it.

Feeling dispirited

I had a couple dispiriting meetings toward the end of the week that killed the enthusiasm I had for what I have been working on over the past few months. First, one of my co-presenters said he thought that the lack of audience questions during our webinar indicated that everyone in the audience was just playing on their phones while we talked, waiting to collect their CPE credits. Then, later in the week, one of the partners I report to on technology initiatives seemed unenthusiastic about the automation bot I created, which took basically all year to figure out how to do.

I have started to doubt myself again. I am wondering if the presentations I give and the white papers I write are too basic to be useful. Even worse, I am wondering again if my tech skills are not the best fit for my company.

Even outside of work I have felt my creativity drop. I have struggled to think up new ideas to write about or to muster the energy to work on my writing until the ideas start to flow again. I want to start coding a lot more again but have been too tired to get into it almost every night.

What I want is inspiration and energy. What I probably need is a diet, exercise, and more sleep.

A Hanukkah COVID exposure scare

In the middle of our Hanukkah celebration tonight my wife got a text that informed us that our son had been exposed to a classmate with COVID a couple days ago. My brother-in-law’s family left immediately because my son and his kids are unvaccinated. (They are all too young.) I’m not concerned that my son is sick (pre-symptomatic) because the rest of us as vaccinated.

My son’s entire preschool class must stay home all next week, due to the exposure rules. My wife and I will have to figure out how to make that work.

Tonight I fell asleep in my chair in front of my whole family. I have officially become my dad. 😅

Success

Today, after almost two months of not having any usable time work on it, I finished coding the automation project that has been bedeviling me for half the year. I got over the technical hurdles months ago; all I needed was eight hours uninterrupted, so I could rewrite my code and test it. It took me a long, long time to find that time, but I found it today. It felt great to finally get it working, and even better to remove about half of the code as I optimized it. I am relieved that it is done. I will present it to one of our partners tomorrow, and plan to turn my attention to other things after that.

Final revision

Today I made my final revision to the article I wrote for my company newsletter. It is an article about Big Data, which is a term that is both readily understandable and has no agreed-upon definition, which I intentionally did not bother to define in the opening paragraph. I was asked to define it anyway, right up front, and to cite it from a source. I did so and ended up with a real head-scratcher of a second sentence. I hated it so much that I ran it by my mentor. He confirmed that the definition I inserted was awful, and he sent me over a paragraph that helped me see a new way to open the essay. I ended up re-writing my first paragraph entirely, and came up with something that I am more proud of.

I am starting to miss coding in Swift every day. I must get back to it.

Hanukkah is a great holiday, but does it have to be eight nights in a row? It’s night three and I am exhausted. 😀

I finished the article I was writing last night just in time for Thanksgiving. I am relieved and hope I don’t have to revise it hastily next week after I turn it in.

Trying to land the plane

Right now I am trying to finish the article I am writing for my company newsletter. At turns I love it and I hate it. I don’t think it is my best work. I don’t even think I can give my best work right.

The article resembles an introduction to the (huge) white paper I am working on far more than a standalone article. This was my intent, but because of that, it lacks a proper ending.

Tonight, I am trying to wrap it up. I have been trying, little by little, for the past two weeks. I feel like I’m piloting a plane, circling and circling, and don’t know how to land.

I don’t know how to end it, and even if I did, I fear that it is longer than my company wants it to be (by a lot). The good news, if I could call it that, is that it is due on Monday. That means that it will be done when it has to be done, no matter what, because I don’t have a choice.

My day was actually really good. My mom and I had a nice time together on the long car ride from her house to mine. We did not hit much traffic, either. We all had a big family dinner this evening, too, and it was great. I’m surprised and grateful at how chill everything has been.

I am driving to Connecticut tonight to pick up my mom and bring her back tomorrow for Thanksgiving. I am hoping that I left late enough to avoid a lot of the nasty rush hour traffic.

My car cat

The oddest thing happened to me today. My neighbor texted me this morning saying that she heard a cat crying and thought it was stuck in my car, which was parked in my driveway. I went out to look—and looked pretty thoroughly—but saw and heard nothing of a cat. I thought my neighbor must have been mistaken.

An hour later my neighbor texted me again saying the cat was still in there, but stopped crying when I went outside to look. I went out again dutifully, looked again but far more quietly for the cat, and still found no trace of it.

An hour after that, my neighbor texted me again and said that the cat was crying louder and louder, and that she wanted to call animal control to get it out of my car. She asked me if that was OK with me, and I said yes. I went out a third time, and still did not observe in any way a cat in my car.

Finally, about 10 minutes after that, while I was in my house, I opened the window and then for the first time I heard a cat crying over and over and over again. I went out and heard the cat and the sound was coming from the engine block of my car. I still couldn’t see a cat though and did not know what to do.

Fortunately, a police officer and the animal control officer in my town came over and, after about half an hour of trying, were able to pluck out a tiny, tiny kitten that had m crawled beneath my car’s engine and lodged itself in there. Fortunately everything is all right with my car and nothing bad happened to the cat. They took it away in a cardboard box I gave them, and presumably brought it to a shelter.

My wife and I went grocery shopping together (without the kids), for the first time since my daughter was born 9 years ago. We went to a huge Italian market we had never been to before, bought a bunch of stuff we didn’t plan to, and had a really great time together. It’s funny that these little things can be so wonderful.

This is my handiwork for the night. Most of the kids’ Hanukkah presents are wrapped. 😮‍💨

Writing to move forward…in my career

After a week away from it, I finally sat down and wrote a little bit more of my article for my company’s newsletter. I still have about a week and half to complete it, and I am most of the way there.

My topic has drifted a bit from what I intended to write. I started with the intention of writing about how to execute a data call project. (A data call project is when a government organization asks private companies to send it data, which it then uses for analysis and reporting.) What I actually wrote so far is an introduction to how the insurance industry (which is the industry I worked in) extracts value from Big Data, and a call to action stating that insurance regulators should also leverage Big Data to help them regulate the industry. The whole thing will eventually serve as an introductory chapter to a much larger white paper (or is it a short book?) that I plan to complete over the course of next year.

It’s all very dry stuff, especially to those outside my field. It is important for me, though, to start writing down the things I know about at work. I think that getting published in industry journals, starting humbly with my own company’s periodic newsletter, will help me improve my profile in the industry, and is the best way for me to advance in my career and to make a difference. I know how to write clearly, which is a skill I have been undervaluing for many years. Now I plan to rely on that strength to move forward.

A rebuilding year

A real “rebuilding year” is when a sports team is broken down and reassembled, basically from scratch, after—and as a response to—a dreadful season. 2020 was a dreadful season for, well, everyone on earth due to the COVID-19 pandemic. It certainly was for me. I experienced some deep trauma back in 2018 when my father died, which I thought went away after a few months, but never really did. Watching my father die (of Alzheimers and old age) completely messed me up. The emerging pandemic in early 2020 let the lingering feelings I had from that time metastasize into full-blown depression and anxiety. Work and home life were both stressful to me all the time. My relationships with everybody suffered. I was a wreck. So was everyone else at the time, which blinded me to how much of a mess I was. At some point, my eyes slowly opened to what was really going on with me, mental-health wise. My problem was that once I realized that my brain wasn’t working right (as in, rationally), I could not trust any of my thoughts, even the ones that could lead me to getting better. It took me months to figure out how to get help, even though, in hindsight, it wasn’t really that hard. By the end of 2020, I was well on the way to recovering from my mental health problems, but I hadn’t rebuilt my life yet.

2021 has been a rebuilding year for me, pretty much from its first day. I made it a point to reach out to people. At work, I set up one-on-one meetings with everybody in my department, and asked for coaching sessions from people higher than me in the organization. At home, I made efforts to listen and to be more present in my relationships with my family members. Internally, I tried to be less hard on myself. I gave up some activities that were productive but stressful. I decided to write a blog entry every day. I allowed myself to waste time on mindless fun, like just listening to music or playing video games, after the kids were in bed (or supposed to be in bed, at least). I had thought that I would get the COVID vaccine and society would get back to normal. Of course, that hasn’t quite happened yet, but I am optimistic that next year will be the time when masking and social distancing pretty much go away. (I’m aware that a lot of people don’t wear masks when they go out, but my wife and kids wear masks at school for hours and hours each day, so masking and social distancing are very much going strong for my family.)

Over the course of this rebuilding year, I have re-learned what I love to do, which I had forgotten over the past few years: Break complex things down to make them understandable; ask big, deep (or maybe dumb) questions; and to communicate ideas, especially in writing.

I am both doing less than I used to and doing more. I am programming less and writing more. I made it a point to improve my presentation skills, and I did that. I taught myself how to make better slides, even though it was very challenging at first, and how to rehearse and revise a talk until it is brisk and concise. I have given …five or more presentations at work this year, on department-level and company-level Webex calls. I created a slide deck for our managing partner to present to the board of directors. I am getting more well known for my technical skills and presentation skills at work, too, which makes me feel great about my job for the first time in years. I also made it a point to write an article for my company’s newsletter, and I am about 85% done with that now, and plan to wrap it up next week. Lastly, I learned to type in the Colemak-DH layout on an unusual, ortholinear keyboard (still really slowly, I admit), and that proved to me that my brain was flexible enough to do it. I am literally rewiring my brain.

By the end of the year, I will not be rebuilt, but will keep rebuilding. That is the most important thing I learned this year.

I don’t journal anymore, and I think…”what am I hiding from myself?”