JTDS & Grails

Recently I had the chance to do some tuning on a system that was built using Grails (formerly Groovy on Rails) and the short and hopefully accurate enough scoop for our purposes is that includes an ORM layer – a data abstraction layer. ORM’s excel at rapid development/time to market scenarios, and they can also be used to deliver platform independence. The downside – to use SQL people anyway – is that the abstraction can cause pain as it does vanilla level generation of SQL statements.

I haven’t coded in Grails. Looking at the code though, I see some of the attraction. Once you point an object at either a table or view you just apply a where clause. No SQL statements, no low level SQL details. I’m  not saying I’m changing my stance on hand coded and well implemented data access, just that I see that it generates the layer I’d typically code by hand (though perhaps not as well in a few cases!).

The first bit of strangeness is that it requires that each row have a unique id. Easy enough for a table, not always possible for a view, so the answer had been to use ROWNUMBER in the views to provide a key. Performance with RowNumber varies. In some cases it was fine, in some cases it would tip a query from 100 milliseconds to over four seconds, going from really nice seeks to pretty horrible scans (removing RowNumber would take us back to a nice plan). We fixed some of these with the trick of using ‘select 0’ in the over clause, some we replaced with checksum, and some were ok as they were. As I understand it the purpose of the keys is much like foreign keys in our world, used to tie various objects together. We tried using NewID, but hitting ‘back’ caused pain when we got an entirely new set of keys each time. Moving to stored proc calls is possible, but seems to not work well with the objects – and I’ll admit to not knowing nearly enough about what is possible here.

It can also be chatty, lots of inexpensive calls to the database to verify security, though I’m not sure if that’s a feature/limitation, or a side affect of how it’s implemented. Something to watch for.

It was using the JTDS driver, here’s one link I found. From what I could find the default setting is to send all parameters as Unicode, which in a lot of cases forced us into scans instead of getting seeks. It also builds a prepared plan (sp_prepexec) for each statement, which also adds some overhead. Both can be adjusted with settings in the connection string. Just moving back to ANSI was a very nice performance gain.

Is there a real productivity gain? Hard for me to tell, but the ‘complexity’ that it hides was basically moved to views, so that one view equaled one object. The resulting queries against the views were trivial, and the occasional update seemed fine as well. The big gain is that as you add or remove columns you don’t have to modify any data access code (you might have to change your views of course).

I’m not sold on it, but we’ve been able to wring good performance with a relatively few tweaks once we dug into it. If you use it just keep an eye on it as it gets built, don’t wait until too late to assess performance (which is true for any data centric project).

PASS Update #53 (Bylaws)

This week we posted some pending changes to the PASS bylaws. We try to do a review at least once a year and it’s a hard process. Hard because we have to agree on the change we want first, and then we have to figure out how to state it cleanly because this is a legal document that binds us. It’s surprisingly hard to translate ideas sometimes into simple and effective language.

I hope you’ll stop and go read them, the whole document. It’s not that long, some of it is boilerplate, and it describes our legal obligations and limits.

Aside from some over due clean up there are some important changes:

  • Revises the officer selection process to be simpler (removes the old Officer nomcom)
  • Clarifies the term limits (which were almost impossible to follow before)
  • Puts in place a rule limiting the number of representatives on the Board from any one company.

Over the years there has been an implied progression from VP Marketing to VP Finance to President, with the theory being that you need the experience in all the roles to be an effective President. There’s some truth to that, but combined with the idea of having a ‘slate’ it set up a system where those selected as leaders largely selected the next set, and they had no motivation to maintain relationships within the Board. I think this change will be a good step towards fixing the problem.

Limiting the number of representatives per employer is to avoid the equivalent of a hostile takeover, and to just plain avoid the appearance of any undue influence from any one one employer. It was hard to write, because the definition of ‘employed’ can vary greatly. Is a full time contractor any employee? Someone who works for a global subsidiary? I think this is a good change.

There’s a lot that is not in the bylaws, and is (or should be) in working process documents. For example, most of the election is a “process”, and that is both by design and a correct way to proceed. No public organization should need to pass a bylaw change to amend day to day working details. Good stuff to think about.

And just in case you haven’t read the bylaws, here’s something you may not know. The Board of Directors alone can change the bylaws. We can make changes by publishing them for comment for 30 days and then passing a vote. We can change them again at any point using the same process. I’m not saying that to hint at evil intent, it’s a side affect of our poorly defined membership process. At some point I hope we can get back to being able to solidly and clearly define our members, and once there, I think we should make one final pass to get the bylaws set, and then publish one final change that would require a vote of the members to pass future changes.

Governance is important, and it’s not glamorous or fun. Take an interest, and let us know what you think.

A Plug for Wirefly

Wirefly is a website devoted to purchasing cell phones and plans. I found it when I was researching my Droid X, and you know how it is – should I trust a web site to deliver my phone? At the time the difference in cost was substantial, $99 via Wirefly which included 2 day Fedex, or $299 in the store. Decided to try it, worked fine, very smooth.

I didn’t think much more about it until a friend was discussing his plans to upgrade his whole family to new phones and the prices he has found so far. I mentioned Wirefly in passing, he checked it out and decided to go with it, ended up saving a couple hundred dollars on the package.

Anyway, if you decide to purchase a new phone it’s at least worth a look, if you can live without the immediate gratification of buying it in the store!

What Should PASS Be? I Challenge You

PASS isn’t what it should be. I hear that a lot, and in many ways I agree with you. We’re finally growing and evolving, but we’re still far from what I think most of you expect from a true professional association.

But.

I’m not sure you or I have realistic expectations. So I want to challenge you. Draw an image of what you want PASS to be in 3 years and share it on your blog (or post a comment here if you don’t have a blog). Imagine we just hired you to be CEO of PASS and you were going to “fix” things, what would you do? What’s your vision for providing benefits to chapters or members, or for growing membership, or for global growth, or whatever areas you think are badly served right now?

Maybe I just don’t have the vision – I’m limited by my own biases and experiences – but I’d really like for PASS to be what you want it to be. An organization that serves you, excites you, makes you proud to be part of it, proud to support it, and willing to challenge it if it steps off track.

Maybe it’s a paragraph, maybe it’s a thousand words, but I hope you’ll write something. We’ve got several hundred bloggers in the SQL space, and a whole lot of members. What you write may not change the world, but maybe it will.

Notes from the February 2011 MagicPASS Meeting

I attended the second meeting of MagicPASS (no web site yet) on Feb 16 at Stetson University in Celebration, FL. Great facility – classroom style seating, dual projectors and screens, couldn’t ask for better. The only downside to it so far is that it’s $200 per meeting for the space, quite a chunk of overhead to add to a chapter. Parking is limited but good enough for a chapter meeting. I’d love to see a SQLSaturday held there, serve the south side of Orlando one year, but parking would definitely be a problem.

We started the meeting early at 5 pm with an informal class/presentation by me on using SQL Profiler. We borrowed the idea from the Dallas group, and I’ll be doing two more of these at the March and April meetings respectively. I think we had at least 10 show up for the early session, and I think ended up with about 22 for the main meeting.

Kendal Van Dyke (President!) did a nice meeting kick off, lead into a good discussion of SQLRally. We asked who had not heard about it (some), and who wasn’t interested in attending. The latter was a good question, we had one attendee whose primary focus isn’t SQL, but the remainder of the ‘not interested’ were more on the order of challenges – couldn’t afford it, couldn’t take time off from work, etc. One person was more interested in the depth that the pre-con seminars interested rather than the 2 day event, which is good, and by design! It was nice to slow down and talk about it, though I worried we were taking time away from the main agenda. Seemed to work out ok. The lesson here is that it’s not enough to just say ‘SQLRally’, turn it into a conversation. Contrast it with SQLSaturday and the PASS Summit. Talk about conference style learning versus all day learning on a single topic.

Kendal also had a nice networking segment, people drew names out of a box to find someone to chat with while we took a break for pizza. Then a nice plug for the meeting sponsors, and on to the feature presentation by Bradley Ball. He did a nice presentation on compression, did a good job of explaining compression results without getting mired in the details, yet managed to show a good bit of detail – if that makes sense! We chatted afterward about trying to figure out what should or shouldn’t be included, a standard question for all presenters.

A few of us migrated to the Celebration Tavern for some post meeting chat. Not right next door, but maybe a 4-5 minute walk, with the parking lot in between meeting and bar so it’s fairly convenient. A good meeting, good location, good people to network with – couldn’t ask for more!

Note: Splitting the original oPASS group into two groups was (and is) a risk, but so far the results seem encouraging. Travel time is definitely a consideration for most attendees.

Who Is This Super Hero?

We recently switched from Brighthouse to DirectTV and of the first things we did was to find the kids channels to see what we gained or lost. I flagged a few to record for use on the proverbial rainy day, and one of the ones I selected was Hong Kong Phooey, a cartoon that ran from 1974 to 1976 with the main character voice by Scatman Crothers.

For some reason (including some obvious encouragement from me) my not quite four year old daughter has been quite taken by it, and wants to watch an episode each night. When it’s time to start she’ll call for me to sit with her through the intro that she sings along with, then she says “now go away Daddy!” and watches the rest of the show like the princess she already is.

Below is the intro song which I think might make you smile, and if you end up singing it to yourself later in the day then my work here is done!

Working on the Railroad

I work with a client that has an office about 25 yards from a train track and recently the railroad company decided it was time to do some maintenance on a large length of the track including that piece by the office. That ‘maintenance’ consisted of replacing all the ties under the rails. If you’ve ever handled a railroad tie it’s heavy, probably 100 plus pounds, and it’s treated with creosote, not the kindest stuff to get smell or get on your skin.

So picture how you might go about replacing all those ties, approximately 3000 per mile. Would it add to the problem if I told you that the rails are welded together forming one big long rail? How long would it take?

We got to watch this over the course of about a week. They have maybe 10 different machines, all mounted on wheels that ride on the track and they had at least two sets of machines working in parallel. The process went something like this:

  1. Remove spikes
  2. Remove 4 out of every 5 ties
  3. Drop replacement ties out to the side
  4. Insert new tie under the rails (rails never taken up)
  5. Sweep away loose gravel
  6. Stage a new nailer plate on each tie on each side
  7. Lift up the rail and side the nailer plate between rail and tie
  8. Nail the rail in place
  9. Lift up the rail, insert forklift type arms into soil on each side of tie and compact the soil so that tie is at perfect height and is level
  10. Verify rail spacing and level
  11. Clean up

It’s one guy per machine driving, maybe 5 more doing the odd cleanup or staging job. The machine that inserts the new ties (which we nick named “the stuffer”) is pretty amazing. Big arm slides out, claw grabs the ties, just pushes it under the track. Takes about 30 seconds. It’s the first picture below.

Great engineering, great process. If you ever the get chance to watch it’s worth it. We were able to watch from up close on the street as well as from the roof of the first floor, spending 5 minutes here and there to see how it was progressing. I grabbed a handful of the old spikes which get discarded (recycled probably) and they were in pretty good shape, mildly rusted. I’m planning to clean one up and I’ll post a before and after photo in a follow up post!

2011-02-07_08-59-48_546

2011-02-07_12-09-04_782

2011-02-07_10-09-38_429

2011-02-07_12-09-20_968

Book Review: Packing for Mars

Packing for Mars by Mary Roach ($15 @ Amazon) is a ton of fairly interesting trivia about the space program, space travel, and the challenges of trying to go to Mars. She talks about doing tests on Earth to figure out how bad body odor would be if someone didn’t bathe for an extended period (30 days!) and that after about 5 days the body stops producing skin oil because it’s oily enough. Stories about volunteers on bed rest for a month to monitor muscle atrophy and volunteers subsisting only on milk shakes during a test period. She chases down a few urban legends related to space, some very strange including one about trying to prevent Enos the monkey from orbital solo sex.

There’s at least a chapter devoted to sex in space, and another on bodily functions. The latter goes into almost painfully graphic detail about the challenges of weightlessness. For example, follow this link to see a picture of the fecal bag that has a finger slot to aid, ahem, disconnection. Imagine using this while seated in the equivalent of an airline seat and your neighbor looking out the window, and then you get to add in a package of disinfectant that you rub into the contents.

In the end you realize that space travel isn’t simple and isn’t glamorous at the human level. Even if we have the technical means to make the trip, getting people there in a way that keeps them physically and emotionally healthy is not going to be easy. Going to Mars is far different than going to the Moon. The moon trip might be grueling, but it’s only four days one day, far different than a 2 year trip to Mars.

It’s a fun read, but only if you’re interested in space and or trivia.

Corporate Culture

One of the things I enjoy about work is seeing the various cultures that evolve. Some companies are rigid, some are fun, some very loosely organized, etc. It’s almost magic how it happens, and it’s rarely all done on purpose. Each manager has a style, each employee has a work rhythm, and they get combined in interesting ways. Sometimes functional, sometimes not!

I like to look for the things that go beyond a single manager, things that are likely to live on if they get promoted or move to another company. For example, a current client has a thing where every Friday an IT manager breaks in doughnuts. The managers rotate the job, and then it’s 10 minutes of sugar treats in the break room on Friday morning. Every once in a while a manager forgets and then the other managers cheerfully chastise them, and then someone gets deputized to go get the doughnuts. It’s a little thing, but it’s fun, and it gets people in the room for a few minutes that might otherwise not see each other during the week.

I’m looking for other examples of interesting patterns, preferably positive ones. Is there something you’ve seen that spanned departments or teams that added richness to the culture. Something you smile about now, or tell people about with pride?

Work Goals for 2011-Part 2

It’s been a few weeks since I wrote my early thoughts on my goals for this year, and you can tell it’s a struggle when it takes me to the second week of February to get them finished. Why such a struggle? I think it’s because I’ve lived in nearly a constant state of change and growth for the past four years. Now it’s time to do something I’ve done several times before, step back, consolidate, and recharge. You might call it a working sabbatical, though that’s stretching the definition of sabbatical to the breaking point. For the next year it means more consulting and contract work, not much travel, less teaching, less volunteering. I expect it to be a largely technical year, spending time looking into areas that I think will be interesting over the next few years, some of it SQL related.

What I’ve really been thinking about is; what will make me feel like I’ve had a good year and what sets me up to be where I want to be in 2012? Here’s my revised list:

Writing

  • Blogging. I like writing, and my friend Steve Jones has challenged me to write better. I’m not sure how to quantify better, but I know when I’m trying harder, so that will be the first test. I’ll try to talk more about this as I go. I’ll strive for every day, but it’s not a goal any longer, I know I can do volume, so it will be about stuff I want to share, or that I need to think about. Blogging takes time, but I believe the time investment continues to pay non direct cash dividends. At the end of the year I’ll call it a win if I’ve blogged at least 3 times a week on average, and if I’ve written at least a dozen serious pieces that end up being done reasonably well.
  • Technical Writing. I haven’t done much of this lately, and I need to do more, so I’m going to try to write something technical once a month, ranging from a tip to an article for one of the community sites.
  • Book. I’ve still got tons of notes on my idea for a first time book for managers, but this is a tough goal as I don’t see time in the first half of the year to work on it. So I’m going to book some days on my calendar for the second half and push. It’s a stretch goal, but one that would be exciting to accomplish.

Speaking

  • Presentations. I’m going to try to do one presentation a month, about 12 for the year. It’s a volume goal, but I’m also going to experiment more, and to work on finding better ways to build presentations. This fits in well with my goal of spending more time with the Florida community.
  • Virtual Presentations. I’m still not comfortable doing these compared to in person, so I want to do at least 6 this year to get more practice. We’ll call that part of the 12 above, but I hope to do a few extra.

PASS

  • Vice President. I spent some time in Dec and Jan thinking about this, and came to the tough decision not to pursue one of the two slots this year. Saying no now effectively means I’ll never get to run for President, which I think would be an interesting experience. Deciding now let’s me focus on things that matter to me and to PASS. I know that’s not a goal exactly.
  • SQLRally
    • Continue to do what I can to make the 2011 event successful and to capture lesson for the 2012 event in the US
    • Drive us through the site selection for 2012 and monitor the launch of the partnership between the selected site team and PASS HQ.
  • SQLSaturday.
    • Push through a series of technology fixes and features that will make things easier for users at all levels, more on this soon, but I’m determined to make it happen in the first half of this year.
    • Coach HQ to take full ownership by the end of the year. Not just coaching new events, but fully engaging with the leaders and the community, helping to grow new locations and all the rest that requires real connections to our members. My goal is that at the end of the year the SQLSaturday portfolio will either get merged into a larger one, or will just go away – that will be up to the next President. Either way, that gives HQ a year with me near by but not actively involved to make sure things are working. Not an easy goal, and not all within my control.
    • Drive the thought leadership, on topics ranging from event size to budget to sponsor satisfaction.
  • Limit my time. I’m doing my best to do the more important things I can do each week in 2 hours. Some weeks will require more, such as the ones with in person meetings, but I’m going to try to use the limit to force me to learn how to change the board into a strategic board that can still get things done.
  • Transparency and Governance. I think we’ve made progress, and I’m hoping the changes recommended by the ERC will help us make more, as will the pending changes to the by-laws.
  • PASS Culture. Here I’m going to write a goal that I don’t know how to achieve yet, but by the of 2011 I’d like to see PASS aligning it’s efforts with the expectations of its members and scoring well in a satisfaction survey.

Mentoring

  • I’ll continue to coach and mentor where I can on an adhoc basis, but I expect most of my efforts to focus on helping people get started blogging. Blogging is far more approachable than speaking and is a great way to feel connected, so it’s a good way to encourage those that want to grow. My only measurable goal is to conduct one get started blogging class with a focus on writing, not on the technical trivia. More on this in the next couple weeks.

Work

  • I’ve already started the task of changing my ‘job’ to consultant from trainer, working with a great client that offers me some interesting challenges. Over the next month or two I’ll be trying to add a second client, giving me two different cultures to work in, and then I’ll supplement that with the occasional smaller project.
  • Most of the teaching I’ll do this year will be in a mini-class format of 1 or 2 or 3 students, something that is more mentoring than pure teaching, and I expect to do at most one of these a month.
  • Get back to being entrepreneurial. I don’t talk much about SQLShare here, both because it’s very much a work in progress and because I try to keep the blog non-commercial. It’s a great place for me to learn lessons about using and applying technology, and ideally it will generate a small profit. But aside from that I still need to scratch the itch to build and try things, but I haven’t decided where just yet. Not sure if that’s trying to build a product or a service or something, but trying things is what drives me, and I have to find a way to do more.

I think I could write a 100 goals, but some of them are subsets of the above, others are just part of how I work, such as continuing to build my LinkedIn network, now at just over 800. The ones I’ve written down are the ones that feel like the matter, and will help me allocate the all too little time I have to spread across all of these efforts.

So the next part is, where do I get the time? Beginning in March I’m planning to ‘work’ four days a week for clients, leaving me one day a week to try to build stuff. That’s a nice bucketing for me, because it’s easy for me to compare what I earned on that fifth day versus what I could have earned working for a client. I don’t want to get overly focused on short term results, but it will be a very visible reminder about the time to money equation. Saturday or Sunday morning will be my writing and speaking prep time, up to four hours as I need it, though hopefully less. PASS time will be an hour on Tuesday or Wednesday evening or Thursday lunch, plus another hour in small chunks replying to email and brain storming.

As I write this it feels doable. This is all stuff I want to do. I can’t control all of it and that’s ok. I’m trying to find the middle ground between setting goals so safe and achievable they don’t matter and setting goals that are more wishful thinking. The hard part is setting limits. I have so many ideas! So many things I want to do and try, and my impulse is to work more, because I can, but it’s not sustainable.

Are these good goals, worthy of me and my efforts? Have I done enough thinking on how I’ll measure success? I’d enjoy hearing any thoughts you have.