Gary Allison's Leadership Blog

Monday, an article was posted on Forbes titled “Why Software Doesn’t Follow Moore’s Law”. This topic has been on my mind some lately as we’ve been working to rollout a new platform with greater scalability, lower maintenance costs, and significantly enhanced capabilities. For engineers that have worked only on the new platform, it is easy to disparage the original platform. Many of them were were not even professional engineers when the original platform was written and they lose sight of how software architectures have changed over the years from the traditional three tier architecture to the lambda architectures of today.

It’s interesting to break this down a bit more. Are engineers smarter today than they were 8-10 years ago? And were those of 10 years ago smarter than those of 20 years past? In my humble estimation no, and one could argue todays graduates know appreciably less about how computers actually work but lets not digress.

What has changed is the way software is constructed. One can argue that modern languages have a major role to play in this, but one can argue also they are an outcome of the changes in architecture. But why has the architectural change occurred?

I believe a significant part of the change in software architecture is attributable to Moore’s law, somewhat in opposition to Mr Maccaba’s article in Forbes this week. Certainly engineers 10 and perhaps 20 years ago understood the advantages of isolation and service oriented architectures. Why didn’t they build systems in this way?

A major reason what that the systems and networks in place really couldn’t support it. Without gigabit networks, fast processors, and super cheap compute, the emphasis was on highly optimized code paths. REST Call? You would have been laughed out of town. Now the opposite is true.

Thus, Moores law has enabled much less efficient architectures to be feasible. Mr Maccaba also asserts this, but missed the point. These new architectures are much more scalable, can be built with fewer dependencies, can be changed independently since they are loosely coupled, and allow previously unobtainable compute goals to be broken down into independent parts and delivered. This is true whether we are talking about large scale web application infrastructure or big data analytics.

By constructing software architectures that take advantage of Moore’s law, we are solving problems that could never be solved before and constructing software systems of higher quality that we can actually deliver in much faster times to market. While certainly not doubling every 18 months, the time to market of new highly scalable solutions is measured in months today instead of years.

At the end of the day, I feel the point Mr Maccaba has missed here is that Moore’s law doesn’t apply to humans. Thus, we leverage Moore’s law to make humans more effective in software engineering through architectures that do deliver significant increases in scalability, quality, and capability.

Coaching & Mentoring for Dummies

I’ve often thought about writing a book about leading software development teams to share some things I’ve learned through the years.  Perhaps I still will, but in a way this blog serve the same purpose.  It seems the market for this particular topic may not be all that large and  to appeal to larger audience would probably water down the content so, I’ll likely just stick to this blog as time permits.

Recently I had the pleasure to work with Marty Brounstein and have been reading his book Coaching and Mentoring For Dummies.  At first you may be put off by the title, but I’m really glad I dove in. In fact, it makes me glad I haven’t tried to write a book.  It’s that good. Many times, I found myself say yes, yes, that’s exactly right.

What I really like about this book is that it takes many of the things you may innately do and codifies them through a set of simple disciplined tools.  Now following this to the letter will not make you a great leader, but there is so much in here that is fundamental to building a great team. Read it. Do it.

The scope of this book is too broad to cover it here in this brief post. Its a cornucopia of leadership 101 as much as coaching. Not surprising since these are so intertwined. Right off the bat in chapter 2, the topic of building commitment and the strategies there I have found to be spot on through my career.  When I see managers manage from positional influence, it makes me nauseated and I instantly lose respect for them. Leadership is exerting personal influence and can only be accomplished through hard-won respect. I blogged about this about 8 years ago and still feel strongly about it today.

If you are a book skimmer, don’t miss Part III, The Fine Art of Mentoring. Some of my most admired mentors and greatest leaders I’ve worked with have perfected the techniques here. I’m still a work in progress, but try to follow these practices, because they really do empower others and help you be a more effective leader. Just one little gem here I’ll call out:

In many situations, such as when mistakes are made or performance efforts don’t go as well as hoped, opportunities exist for employees to learn lessons and correct their course. Using questions to explore what happened and what can be done to make matters work better provides experiences for learning and growth.

Bingo Marty. This approach can turn a very difficult situation where people are already feeling defeated by failure into one of a hopeful feeling for an better future.  Blame is easy, improvement is not.

Next week I embark on a new professional adventure which I am very excited about. I am challenging myself to be a great coach even when the pressure is high and things don’t go as planned. I hope you will find Marty’s work insightful and helpful.

As a leader of a Engineering Team, you may have had your fair share of experience with “Free kittens”. Free kittens are one off products, partner integrations, new platform support, or customizations that really only apply to one customer or are focused in a market that is outside your wheelhouse. These will be presented frequently to you as a fuzzy small requirement in a SOW, contract, or just as likely an email you’re copied on – fuzzy, cute, and often not described in much detail.

Of course, free kittens need some one to take car of them, buy them food, expensive trips to the vet when they are sick, and no matter how cute, theres a lot of crap to clean up. Same is true for the kittens that show up at your office door / inbox, usually delivered by Sales. (To be fair though, they can come from Product Management, Support, and directly from Customers themselves.)

As leaders, it is not usually effective or productive to say, “no thanks, your kitten is kinda ugly”. We are in the business to build and deliver software and services that delight customers which requires us to be adaptable and aggressive in meeting their needs. What is often more productive is to ask questions and educate the kitten custodian about what it will take to live with this kitten for the next 5-10 years (software kittens sometimes, but not always, have a shorter lifespan that real kittens).  In so doing, hopefully a business decision can emerge.

I’ve had more kittens than I can count show up at my door over the years. The latest one that came in very recently was run a one-off instance of our SaaS solution in one of the newer cloud hosted IaaS platform that I won’t name here. Of course we can do it (all things are possible with software) is the answer. However, there is this small matter that no one in this company has ever worked with that particular IaaS cloud provider.

I often find people miss the point on how to provide a service – its not a one time effort. Clearly training is needed for everyone who would touch this new platform: Dev, QA, Support, Services, etc. You need need a smooth repeatable process to dev, build, QA, deploy, secure, monitor, troubleshoot, update, backup, and recover from disasters. Turns out those cute little free kittens are somewhat pricey! Is it worth it for this one customer, or can we amortize this effort over a whole set of customers that we would move to this new platform? Can this large investment strategic? Good questions…

Of course, Sales people are not comp’ed on taking care of the kitten, just on its adoption. Sometimes this is called “happy ears” – they just want to hear yes, whether from you or the prospect. The sales person or product manager won’t have to feed it or clean the litter box, so it doesn’t hurt to inform them of the smells that can occasionally accompany these cute little guys. With the right business conversation that considers both the true value and cost of the kitten, you can drive a decision that works for your SaaS platform.

Just read this post from today and wanted to share as well as pile on: Learn about your managers through the people they manage
by Richard Rosenblatt. On understanding what is really going on in your organization, Richard recommends

So, how do you get a more multi-dimensional perspective on how your direct reports are doing? It’s easy. Spend time with their peers and direct reports.

In practical terms, I recommend using one-on-ones with all your direct reports and augment this with skip level on-on-ones over lunch.  I also have found that round table lunches with non-manager thought leaders in the team are invaluable in gaining feedback and also suggestions for improving the team.

If you feel you don’t have enough time for one-on-ones with your team, I encourage you to read Ben Horowitz’ blog from last August on the topic.

I’ve worked for CEO’s that used skip level one-on-ones and hallway conversations to both keep their finger on pulse on the business but also to help understand what was really working in the organization.  Personally, I never felt threatened by this because of the utmost trust I have in my teams but also because we were talking frequently in our one-on-ones.

This emphasis on transparency builds trust and trust is an essential ingredient for winning teams.

Last year I wrote briefly about the importance of having a great culture in this post, but it deserves a deeper dive.  Early in my executive career I had the great fortune to work with a terrific CEO and mentor. One of the many things I learned from him was the power of culture and the way it can work to juice your team. The power of a competency culture in particular was something that I grew to understand under his mentorship.

In a nutshell, this taxonomy would classify cultures as control, collaboration, competency, and cultivation.  The importance of these comes down to how decisions are made and specifically, how quickly the organization can act. We are all familiar with control cultures – all decisions are made by “the boss” according to a structured authority framework where successively important decisions must be made by higher levels of management. In a control culture, all decisions made by lower levels in the org are subject to being overturned at a higher level of management.  The fall out of this culture is that the people responsible for implementing decisions often never really buy in, resulting in little empowerment and innovation.

In a collaborative culture, decisions are made by consensus.  This requires nearly unanimous agreement and the process for reaching an agreement on complex issues can be lengthy.  The advantage of course is that everyone buys in to the decision and is committed to its execution.  Great buy in, but slow.  Not a compelling recipe for innovation.

A cultivation culture is fairly uncommon in business and is more identified with a religious system, where the organization exists for the focused purpose of growing the individual with a higher purpose in mind. Which brings us to…

It is the competency culture that drives the engine of the most successful tech companies.  This is an empowering culture where authority and responsibility both are pushed down to the lowest levels in the organization.  Decisions are made quickly, by owners.  The owner’s responsibility is to involve stakeholders in the decision making process. Once all input is considered, and  the decision is made, it is everyone’s role to commit to the decision.  It is ok to disagree with the decision, but in this culture you disagree and commit.  This culture depends on you hiring the very best people available, and giving them all the responsibility they can handle, plus a little.

Everywhere I’ve worked since, I’ve tried to build a competency culture. When it works it really is golden.  People thrive and grow and suprise you with innovation.  Teams live and breathe ownership, commitment, and teamwork because they know they personally can make a difference.  I don’t know why I’ve taken so long to write about something so core to my management philosophy but here it is.

For more information on these cultures and how they can determine the fate of your organization, I recommend this document by Dr William Schneider.

Last night, I read a passage in the Steve Jobs Bio recounting a conversation between Steve and Mike Markkula as Steve was contemplating his return to Apple.  The passage was about reinventing a company, and struck me as very poignant as I had just spoken to my team a few weeks ago at our 2012 kickoff about reinvention.  The passage

They spent the rest of the time talking about where Apple should focus in the future. Jobs’s ambition was to build a company that would endure, and he asked Markkula what the formula for that would be. Markkula replied that lasting companies know how to reinvent themselves. Hewlett-Packard had done that repeatedly; it started as an instrument company, then became a calculator company, then a computer company. “Apple has been sidelined by Microsoft in the PC business,” Markkula said. “you’ve got to reinvent the company to do some other thing, like other consumer products or devices. You’ve got to be like a butterfly and have a metamorphosis.” Jobs didn’t say much, but he agreed.

How prophetic. Also evident in the book is that not only did Apple have to reinvent itself, Jobs had to reinvent himself to a degree in order to become an effective CEO.

Reinvention is the key to all success and the path forward. For teams and companies to grow, they must recognize that the things that worked in the past which made them successful, will have to change, be reinvented, to continue that growth.  Since growth = success in our measurement system, then change requires reinvention.  Approaches, processes, technologies, teams, communication – it all must be reinvented all the time.

Five years ago, I wrote a blog about leading through change.  Rereading this now, I feel it is still on target.  At the time I was thinking about change that is sourced from the outside.  Reinvention is sourced internally.  The same strategies apply.

Last week, one of my team members forwarded me a link to this blog by Savio Rodrigues, entitled Why devops is no silver bullet for developers.  It’s a well written blog and Savio makes some good points, namely that environments that the Devops team hopes to build on need to be standardized. He comes so close to hitting some important topics right on the head, and then just misses the mark slightly, IMHO.

Savio nails it when he points out

“One thing I’ve come to understand is that these two groups tend to think differently, even if they are using the same words and nodding in agreement.”

Bingo Savio.  He goes on to say,

“It’s no surprise developers want to adopt tools and processes that allow them to become more efficient in delivering new applications and continuous updates to existing applications. Today, these two tasks are hindered to a degree by the operations teams that are responsible for production environments”

But then, he misses an opportunity to drive the point home and starts a discussion about standards. I agree standards are important, but what needs to be reckoned with are the very different culture, goals, and reward systems between the two disciplines of Engineering/Development and IT/Operations.

How are these teams measured and rewarded? The answers to these questions tell you many things about the team’s culture. A Development team is typically measured and rewarded by amount of innovation, quality of their deliverables, timeliness of delivery, and responsiveness to market.  An IT team is measured and rewarded typically by uptime, stability, security, and control.  (Note rewarded can mean “not punished due to failure” as well as more expected definitions of reward).

All of the above seem like good things! We want uptime, innovation, quality, stability, etc!  Right? I envision one could draw a Venn diagram for the Dev culture and the IT culture and there would be overlap, but there would be just as much outside the intersection.  Innovation is often at odds with stability.  Responsiveness to market can be at odds with uptime, etc.

We’ve had the good fortune of having a few opportunities to implement a new Devops model.  When everyone is rowing together the boat certainly moves faster in the desired direction. But it is difficult. It requires continual investment in the Devops team because at the core, these two very different cultures aren’t going away anytime soon.  Savio sees it too when he says, “This isn’t a technical issue. It’s a cultural issue.” I’d suggest we spend as much time looking at the measurements and rewards as we do thinking about standardizing platforms.

In a prior post, I wrote about the relentless pace at which our Agile development moves, especially as it is teamed with delivery of software as a service (SaaS).  One of the consequences of this speed of constant delivery is that there’s never time to go back and “clean up” any of those important but not urgent tasks that you just didn’t get to in the sprint.  You know the ones I mean – that last unit test you really should write, fixing a low priority bug, UI tweaks, or automating all of the QA tests.

We’ve found that at the core of this challenge is that the team may not all be engaged simultaneously on the same project.  As I work with other Engineering executives, many face the inherent conflict of QA sprints that lag behind development sprints.  On the surface, this seems natural.  After all, there’s nothing to test until it is written?

We know from test driven development practices this doesn’t have to be the case. Even if not following strict test driven development practices, with careful development story planning, the UI aspects or API stubs can be built first in the sprint, so by the time the QA team has spent a day or two planning test scenarios or preparing test data, they can begin automated test development.

We also find that we need for new code development to stop at some point in the sprint. From that point to the end, developers are only fixing bugs found by the QA team in the sprint, completing those final few unit test cases, taking on usability feedback, and otherwise driving to complete.

By taking this approach, the developers and QA engineers stay together, focused on one goal as a team.  That goal is sprint complete of a high quality and finished deliverable.  The result is far fewer lose ends, a higher quality product with less rework, and just as important a team that works together.

On my day off today, I find myself contemplating how little time there is to smell the roses (and write blog posts).  We have so tuned our processes that really there is no downtime.  It turns out that when you add Agile development processes to delivering software as a service (SaaS), what you achieve is a relentless pace of innovation.  That’s good right?

Well, it does have a few consequences that are important to manage.  First is that there is never time to go back and finish that unit test, test automation, or UI tweak you really wanted to do last sprint.  You need to be complete and ready to move on to the next project.  We’ve been working on ways to improve here which will be part of a follow up post (really it will).

This post is about another consequence – we deliver innovation so quickly while also working on the way we deliver that it is easy to lose sight of how far we come.  As I overheard on a flight yesterday, “the days are long and the years are short”.  So true.

To combat this in my team, we try to be as rigorous about our celebrations as we are about our delivery.  We deliver somewhere around 10 times a year (even more with minor updates) and this will only grow as our product line continues to expand.  Still there are certain deliveries that stand out as larger accomplishments and 3 times a year as a team we will take a day and celebrate.

Often these celebrations include a team engineering event – we’ve built tinker toy towers, water balloon launchers, and aqueducts.  We’ve played kickball, laser tag, and cruised the lake on a party barge.  Once a year in Dec/Jan, we will take stock and as a team, share what we feel are our largest accomplishments as we also layout how we want to raise the bar for the next year.

It can be exhausting at times to be a part of the relentless pace of Agile delivery, and at those times when the release is rolled out to clients, maybe the last thing you want to do is plan a celebration. But every time we have one of these events, I am reminded what great people we have on our team and how much they deserve (and need) these times together.

Oh, how I love this post by Joel on The Duct Tape Programmer!  This is such a salient point that applies to so much more than the context here.  “You see, everybody else is too afraid of looking stupid because they just can’t keep enough facts in their head at once to make multiple inheritance, or templates, or COM, or multithreading, or any of that stuff work.”

I just ordered my copy of  Coders At Work.

On a total Tangent, I just completed the transfer of my domain name off of Network Solutions to Dreamhost.  Dreamhost is awesome and dirt cheap.  They have one -click installs of just about everything you’d ever want to run on your website and their registrations are almost free they are so inexpensive.  I’ve been hosting with them for over a year and have had zero issues (other than imap email, but that’s another story).  I 100% recommend Dreamhost.

Lots of discussion lately about measuring productivity has had me spending time I should be sleeping thinking about the same.  I love accountants and finance folks.  I find them very bright and love the way they typically approach any business discussion from the point of logic, but they can be an intractable lot as well.  They’d love to measure software engineering efforts like a consultancy – hours in, output out, utilization metrics pop right out the other side of the equation.  More utilization of the team means more productivity!  Wonderful!  Its so simple and we should have figured this out so long ago.  All that time wasted counting KLOCS and function points….

Of course it doesn’t really work. You can count hours, or days, or whatever to your hearts content but you are only measuring effort.  And, measuring effort of a software development group is an exceptionally tricky (and potentially dangerous) thing.  Its not the effort that matters, but rather the results.  So how do we measure the results?  Ahhh there’s the rub.

What we need to measure is the business value of the stories the team is being asked to build.  For the consultant this is very simple – you are paid by the hour for the consulting performed.  Thus, hours billed X hourly rate = business value.  The business value of a software going into a product is not so easily measured.  But, lets assume this is a solvable problem.  It gets even more interesting in the planning phase when you are making product choices.  For a proposed feature, what is the busniess value?  Now suddenly, this is not a software engineering question at all.

Do you know what this is?

load AX, BX

Then chances are you know how computers really work, and chances are you are, what’s the word, old?  I’m currently working on a presentation for an upcoming conference that describes how Software as a Service, cloud computing, and web services are changing the landscape for education.  The thought occurred to me recently when working on the outline of the presentation that many of today’s computer science students have no idea how computers really work.  Systems level knowledge of programming and operating systems seems to be arcane. Maybe that’s good in a way.

No you can build full-on applications in scripting languages, deploy them in the cloud and not worry about load balancing, firewalls, and nusiances like memory management.  Maybe in 10 years, developers of the future will laugh at us when they recount, ” yea, you used to have to write applications in this complex language called Java, now you just drink it, ha, ha, ha”.  I used to laugh at the PL-1 and FORTRAN programmers.

Maybe this is good.  Now, developers can focus on solving business problems and spend less time on the nuainces of programming.  Or, maybe they actually have less control and are forced into the confines of what is offered in the toolbox of Google, Amazon, Salesforce and the like.  And, since soon they will lack any real systems development skills, maybe they give up all chance of getting outside of those boxes?

This week finds me at Salesforce.com’s Dreamforce show where a number of very interesting developments are coming to light.  In Marc Benioff’s keynote yesterday, he emphasized the role of cloud computing in the future of all application development, throwing jabs at Microsoft all along the way.  Benioff cast the cloud into these sets of services:

• Amazon is the server plumbing and storage – their EC2 elastic computing cloud providing all the virtual servers you need while S3 provides boundless very low cost storage
• Google is the Microsoft Office and Sharepoint alternative, offering shared applications like calendaring, documents, and spreadsheets, with the capability to share all three, plus adwords that can feed into salesforce leads
• Facebook offers the social graph where new applications can leverage and integrate to spread virally
• Salesforce’s force.com is the application layer to develop business apps and tie all the above together on every platform, mobile to all browsers

This staking out of claims on the cloud computescape is a fascinating thought to me.  While I’m not sure that all other other companies would agree with the above positioning, there has never been a more exciting time to be in software development – the cost  of building a truly scalable application that can service tens of thousands of users  is truly in every developer’s reach.  I woke up at 1 am this morning with my mind racing about how the applications I and others could build!  This clearly has implications for global software development, lowering the bar for anyone in any country who has a great idea to build it out with a small team and bootstrap an effort self funded.

It also means that if you were thinking of corporate IT as a long term career, you should think again.  These services are going to consolidate into a few very large providers, at the end of the day, this is good for our industry. It allows the great ideas, the great innovations to be born more quickly, removing unnecessary barriers and hurdles.

Recently, we evaluated several Agile Project Management tools.  Having used XPlanner for years, it was time for a change.  XPlanner is great in some ways – fairly lightweight and easy to use for developers.  It really falls apart though when you start to consider multiple agile teams working together to deliver a release.  It is just really problematic to get a group view of where you are, what progress is being made by the team, and where the hot spots might be.  To accomplish this, you need to drill into the details of every scrum team and study the metrics / charts.  I even went so far as to change the source code to build a dashboard – that’s when we started to approach diminishing returns.

The other shortcoming of XPlanner is the management of the product backlog and release planning.  Yes, you can work around this, but intrinsically, the tool does not support building a backlog and then moving stories into a sprint.  Yes, this can be done, but it is arduous.  The interface also is stuck in Web 1.0 land, making data entry into a form submit after form submit affair.

So then what?  Surveying the market and talking to many of my longtime friends developing software with agile process, we quickly build the short list to replace XPlanner. We looked in detail at Rally, VersionOne, and FogBugz.  Though FogBugz had some very interesting capabilities around predicting the accuracy of estimates, it didn’t really seem to support agile planning methodologies and the scrum process.  Also, though the predictive capabilities are interesting, this really isn’t a huge benefit in my opinion if agile is really used and you know your people.

So, it was down to VersionOne vs Rally.   Both companies did extensive demos for our leadership team and key stakeholders.  Both tools intrinsically are built around the scrum agile process.  Both were priced around the same level with VersionOne being just a little less per seat, per month, but Rally matched and beat this price point in our negotiations.  The huge gaping hole in VersionOne for us was that it really didn’t assist with resource planning at all.  That is, they don’t enable you to enter the amount of available resources in terms of hours, days, etc, and then in the planning cycle show you where you are in using those hours as you take stories from the backlog and add them to the sprint.  Both tools track burndown during the sprint of course, but only Rally lets you know if you are planning too many stories in the sprint.  Even XPlanner supports this, so it is a big miss for VersionOne.  We can only assume they are working to add this capability.

Also, the rollup reporting for an entire release is more powerful and flexible in Rally.  This was a big plus for us.  To be sure, Rally isn’t super sophisticated in resource planning.  It doesn’t allow the individual team members enter their availability and then sum it up for the sprint.  (I would like this feature – I need to add this to the Rally Community.)  Rather, it just allows you to add the total number of hours available for a sprint at the beginning of the planning cycle.  How you figure this out is up to you.  After you add the total number of hours available, it shows you hours remaining as you add stories.

In coming blogs, we’ll talk more about the pros and cons of Rally as a Agile management tool.