The qualifying exam will be given September 16. While I don't expect it to be as brutal as the one I took at Cornell, it's something to be taken rather seriously.
It makes for a potentially redeeming weekend. Right after finishing the exam, I'll be heading down to Berryman to run the Mark Twain 100 the next day. That's the only running race I've ever dropped out of. As with leaving Cornell, there were very good reasons to pull out, but it still feels like unfinished business.
So, in the space of 48 hours, I can assert that I really do intend to finish a PhD and assert that the DNF at Mark Twain was the fluke it appeared to be. Granted, I'm the only person in the world who looks at at either of those decisions to quit as a failure (well, maybe my Cornell adviser saw the first that way), but there is something very satisfying about going back and doing something right when the first try didn't work out too well.
Sunday, June 5, 2016
Friday, June 3, 2016
Overtime
So, what's so important that I don't have time to write blog posts? Well, work, family, health for starters. Astute readers will note I didn't mention school; I haven't had time for that either. Wise readers will note that I listed those first three in the wrong order. I would agree, but I won't pretend that's been my reality lately.
This isn't unexpected, though it's coming a few weeks earlier than usual. Work usually shifts into overtime sometime in June and really heats up in July and August. The reason for this is that the primary initiative my group supports goes live every year on September 1. If that was the end of it, I'd be fine with it because everything in St. Louis heats up in July and August. Writing code in an air conditioned office isn't a bad way to pass the time.
Unfortunately, that's not it. September is usually pretty normal, but October and November, the two months when it really is nice outside, get busy again as the aforementioned initiative comes to a close and we get hit with a lot of support calls from folks frantically trying to get stuff done before the deadline.
I've tried to inject a little sanity into the process. I'm finding this a bit easier in my new group. We're smaller and our part of the system, while it has it's challenges, generally gets fewer changes from one year to the next. We're putting out some big efforts right now, but there's every reason to believe that we will be finished with development in the next few weeks.
That's a good thing because, for reasons that are unknown to me, the powers that be are planning on turning this thing on in early August rather than waiting for September. That's compressed our schedule a bit, but not tragically so.
This is what I signed up for. I had the good fortune (though that's not how I viewed it at the time) of working on a very high-stakes, deadline-driven project before I even got through undergrad. I realized that this is just how the industry works. I could either embrace it, resent it, or find some other line of work.
I chose the latter option, leaving engineering to become a bike racer. I learned that even "dream" jobs like being a pro athlete have some pretty serious negatives. But, I also learned that if you can get past that, it's possible to derive tremendous satisfaction from almost any job as long as it's something you have a passion for and are willing to put out some effort.
And, it's been mighty useful for this whole grad school thing to be able to look at the effort required and realize that I know how to put that out. So, I'm not complaining. I just have a lot to do.
This isn't unexpected, though it's coming a few weeks earlier than usual. Work usually shifts into overtime sometime in June and really heats up in July and August. The reason for this is that the primary initiative my group supports goes live every year on September 1. If that was the end of it, I'd be fine with it because everything in St. Louis heats up in July and August. Writing code in an air conditioned office isn't a bad way to pass the time.
Unfortunately, that's not it. September is usually pretty normal, but October and November, the two months when it really is nice outside, get busy again as the aforementioned initiative comes to a close and we get hit with a lot of support calls from folks frantically trying to get stuff done before the deadline.
I've tried to inject a little sanity into the process. I'm finding this a bit easier in my new group. We're smaller and our part of the system, while it has it's challenges, generally gets fewer changes from one year to the next. We're putting out some big efforts right now, but there's every reason to believe that we will be finished with development in the next few weeks.
That's a good thing because, for reasons that are unknown to me, the powers that be are planning on turning this thing on in early August rather than waiting for September. That's compressed our schedule a bit, but not tragically so.
This is what I signed up for. I had the good fortune (though that's not how I viewed it at the time) of working on a very high-stakes, deadline-driven project before I even got through undergrad. I realized that this is just how the industry works. I could either embrace it, resent it, or find some other line of work.
I chose the latter option, leaving engineering to become a bike racer. I learned that even "dream" jobs like being a pro athlete have some pretty serious negatives. But, I also learned that if you can get past that, it's possible to derive tremendous satisfaction from almost any job as long as it's something you have a passion for and are willing to put out some effort.
And, it's been mighty useful for this whole grad school thing to be able to look at the effort required and realize that I know how to put that out. So, I'm not complaining. I just have a lot to do.
Monday, May 30, 2016
Memorial Day
As the son of immigrants who were not in the military, I don't have any direct connection to anyone who lost their life while fighting for the United States. Frankly, I don't have any real connection with fallen soldiers from anywhere else, either. I'm a typical mix of Irish, Scotch, and English. They have been hacking at each other for centuries, so I'm sure a few of my ancestors died in battle, but I know nothing of them. My Grandfather had a brother killed in WWI; that's the most recent casualty that comes to mind.
That's why I was glad we spent a good chunk of today at the Elks. Kate is a member of BPOE#9, here in St. Louis. Aside from being a good time (the Elks make no apologies for mixing socializing with their mission), it's nice to be involved with an organization that supports veterans and their families. A number of the folks there are veterans and several have children in active duty. I made a point of indicating my appreciation. I'm very much against war, but I surely do understand that, if it comes to that, you really want your folks doing the fighting knowing you are behind them.
That's why I was glad we spent a good chunk of today at the Elks. Kate is a member of BPOE#9, here in St. Louis. Aside from being a good time (the Elks make no apologies for mixing socializing with their mission), it's nice to be involved with an organization that supports veterans and their families. A number of the folks there are veterans and several have children in active duty. I made a point of indicating my appreciation. I'm very much against war, but I surely do understand that, if it comes to that, you really want your folks doing the fighting knowing you are behind them.
Sunday, May 29, 2016
Back at it
If you've surmised from the missed posts in this blog and entries in the study log that I'm taking a breather, you're mostly right. I say mostly because we're back to working overtime at work (which is normal for this time of year). Still, it's time to get back on it. Tomorrow will be the last easy day. After that, I'm planning on the following schedule to prepare for the Q:
Monday: 1 hour working on the CISS Paper and 1 hour on Algebra
Tuesday: 1 hour working on the CISS Paper and 1 hour on Calculus
Wednesday: 1 hour working on the CISS Paper and 1 hour Real Analysis
Thursday: 2 hours on Statistics
Friday: 1 hour working on the CISS Paper 1 hour Statistics
Saturday: 1 hour addressing whatever didn't go well in the preceding week.
Sunday: Off
I'm going to be on vacation from June 13-22, which is actually a good opportunity to get in additional study. Kate and Yaya don't exactly roll out of bed at the crack of dawn.
Monday: 1 hour working on the CISS Paper and 1 hour on Algebra
Tuesday: 1 hour working on the CISS Paper and 1 hour on Calculus
Wednesday: 1 hour working on the CISS Paper and 1 hour Real Analysis
Thursday: 2 hours on Statistics
Friday: 1 hour working on the CISS Paper 1 hour Statistics
Saturday: 1 hour addressing whatever didn't go well in the preceding week.
Sunday: Off
I'm going to be on vacation from June 13-22, which is actually a good opportunity to get in additional study. Kate and Yaya don't exactly roll out of bed at the crack of dawn.
Thursday, May 26, 2016
Traffic cop
Well, here's one way to kill traffic: act like a cop. Ever since I called out the Russians for potentially stealing my ideas, my hits on this blog have gone in the tank. And, almost none from Russia in the last few days. I think that's pretty damn funny.
Wednesday, May 25, 2016
Big Data Geography
I met with Walt Maguire, Big Data Chief Field Technologist for HP Enterprise (that's a truly great job title) today. Sharp guy, as you would expect. Lots of good technical insights, as you would also expect. I could write about that, but most of it is already on HP's website in the form of various white papers. One quasi-technical observation that I thought was both funny and instructive was the geographical differences in the approach to technology adoption. This one probably isn't on HP's website, though I'm sure he doesn't mind me sharing it.
On the West Coast, particularly Silicon Valley, companies just want the ideas and the enabling technology. They'll write the actual implementations themselves. In a few cases, this makes sense. Facebook and Google really are breaking new ground in terms of data volume, so no off-the-shelf solution is going to work for them. But, he sees it more as a pervasive culture thing than necessity. They just think that they'll do a better job. And, since they do pay the highest IT salaries, they're often right.
On the East Coast, what they want is certification and compliance. They aren't so concerned about whether it's the latest and greatest. They want to know that it will work and that it won't get them in trouble with regulators.
Midwestern companies tend to be later adopters; looking for both proven technologies and the big price breaks that come by being a version behind. In short, they're looking for how to get the job done without putting a lot of capital at risk.
He didn't mention the South. Having several industry friends in Texas and having just worked with a consulting group from Atlanta on another project, I could speculate that they're biased towards open source platforms since they are also trying to control costs, but are less risk averse because they generally have pretty strong technical talent available to fill in the gaps.
He wasn't advocating one philosophy over any of the others; just noting that there are many ways to get this stuff done and the best solution for one organization isn't necessarily the best for another. Of course, anybody who knows anything about business already knows that, but putting it in geographical terms was an interesting way to frame it.
On the West Coast, particularly Silicon Valley, companies just want the ideas and the enabling technology. They'll write the actual implementations themselves. In a few cases, this makes sense. Facebook and Google really are breaking new ground in terms of data volume, so no off-the-shelf solution is going to work for them. But, he sees it more as a pervasive culture thing than necessity. They just think that they'll do a better job. And, since they do pay the highest IT salaries, they're often right.
On the East Coast, what they want is certification and compliance. They aren't so concerned about whether it's the latest and greatest. They want to know that it will work and that it won't get them in trouble with regulators.
Midwestern companies tend to be later adopters; looking for both proven technologies and the big price breaks that come by being a version behind. In short, they're looking for how to get the job done without putting a lot of capital at risk.
He didn't mention the South. Having several industry friends in Texas and having just worked with a consulting group from Atlanta on another project, I could speculate that they're biased towards open source platforms since they are also trying to control costs, but are less risk averse because they generally have pretty strong technical talent available to fill in the gaps.
He wasn't advocating one philosophy over any of the others; just noting that there are many ways to get this stuff done and the best solution for one organization isn't necessarily the best for another. Of course, anybody who knows anything about business already knows that, but putting it in geographical terms was an interesting way to frame it.
Tuesday, May 24, 2016
What's impossible is what's required
We get fruit delivered at work twice a week. This morning, I took an apple and noted that wasn't a particularly good apple. Not terrible, just not great. I'm not sure how long apples have been around. Certainly, they'be been cultivated for several thousand years. One would guess they existed wild for a lot longer than that. The presence of fresh apples in the northern hemisphere in May is an extremely recent development. So, I didn't bother complaining about the less than great apple to those who supply the fruit.
Technology users (myself included) are generally less understanding. New features and higher levels of performance are not only expected, they are required. Just because it's impossible doesn't take it off the table because impossible is a temporary state.
Two years ago we put together an analytic cube for a group at work that holds 50 billion fact rows tied to about 300 dimension attributes. When we first turned it on, queries were coming back in around 20 minutes. That seemed like a pretty big improvement over the 2-3 weeks they had been spending to get the same information out of the data warehouse, but the users wanted performance comparable to other cubes they used. The fact that this one was 20 times larger didn't impress them.
Through some very aggressive tuning, we were able to get most queries coming back in just a few minutes. This month, we gave a demo of their data on a true scalable platform (Hadoop/Impala/atScale) and showed that we can, in fact, return their results in seconds; there's just the little matter of paying for the hardware. And, if they really decide money is no object, we also showed how adding Vertica into the mix gave them another order of magnitude in speed.
It will be interesting to see how this unfolds. I'm pretty sure they'll go for for the top of the line solution. There's already talk of initiatives with trillion-row data sets. And, when those come about, they'll want those answers just as fast.
Technology users (myself included) are generally less understanding. New features and higher levels of performance are not only expected, they are required. Just because it's impossible doesn't take it off the table because impossible is a temporary state.
Two years ago we put together an analytic cube for a group at work that holds 50 billion fact rows tied to about 300 dimension attributes. When we first turned it on, queries were coming back in around 20 minutes. That seemed like a pretty big improvement over the 2-3 weeks they had been spending to get the same information out of the data warehouse, but the users wanted performance comparable to other cubes they used. The fact that this one was 20 times larger didn't impress them.
Through some very aggressive tuning, we were able to get most queries coming back in just a few minutes. This month, we gave a demo of their data on a true scalable platform (Hadoop/Impala/atScale) and showed that we can, in fact, return their results in seconds; there's just the little matter of paying for the hardware. And, if they really decide money is no object, we also showed how adding Vertica into the mix gave them another order of magnitude in speed.
It will be interesting to see how this unfolds. I'm pretty sure they'll go for for the top of the line solution. There's already talk of initiatives with trillion-row data sets. And, when those come about, they'll want those answers just as fast.
Subscribe to:
Posts (Atom)