Gary’s Tech Thoughts Blog

Oh, how I love this post by Joel on The Duct Tape Programmer!  This is such a salient point that applies to so much more than the context here.  “You see, everybody else is too afraid of looking stupid because they just can’t keep enough facts in their head at once to make multiple inheritance, or templates, or COM, or multithreading, or any of that stuff work.”

I just ordered my copy of  Coders At Work.

On a total Tangent, I just completed the transfer of my domain name off of Network Solutions to Dreamhost.  Dreamhost is awesome and dirt cheap.  They have one -click installs of just about everything you’d ever want to run on your website and their registrations are almost free they are so inexpensive.  I’ve been hosting with them for over a year and have had zero issues (other than imap email, but that’s another story).  I 100% recommend Dreamhost.

Lots of discussion lately about measuring productivity has had me spending time I should be sleeping thinking about the same.  I love accountants and finance folks.  I find them very bright and love the way they typically approach any business discussion from the point of logic, but they can be an intractable lot as well.  They’d love to measure software engineering efforts like a consultancy – hours in, output out, utilization metrics pop right out the other side of the equation.  More utilization of the team means more productivity!  Wonderful!  Its so simple and we should have figured this out so long ago.  All that time wasted counting KLOCS and function points….

Of course it doesn’t really work. You can count hours, or days, or whatever to your hearts content but you are only measuring effort.  And, measuring effort of a software development group is an exceptionally tricky (and potentially dangerous) thing.  Its not the effort that matters, but rather the results.  So how do we measure the results?  Ahhh there’s the rub.

What we need to measure is the business value of the stories the team is being asked to build.  For the consultant this is very simple – you are paid by the hour for the consulting performed.  Thus, hours billed X hourly rate = business value.  The business value of a software going into a product is not so easily measured.  But, lets assume this is a solvable problem.  It gets even more interesting in the planning phase when you are making product choices.  For a proposed feature, what is the busniess value?  Now suddenly, this is not a software engineering question at all.

Do you know what this is?

load AX, BX

Then chances are you know how computers really work, and chances are you are, what’s the word, old?  I’m currently working on a presentation for an upcoming conference that describes how Software as a Service, cloud computing, and web services are changing the landscape for education.  The thought occurred to me recently when working on the outline of the presentation that many of today’s computer science students have no idea how computers really work.  Systems level knowledge of programming and operating systems seems to be arcane. Maybe that’s good in a way.

No you can build full-on applications in scripting languages, deploy them in the cloud and not worry about load balancing, firewalls, and nusiances like memory management.  Maybe in 10 years, developers of the future will laugh at us when they recount, ” yea, you used to have to write applications in this complex language called Java, now you just drink it, ha, ha, ha”.  I used to laugh at the PL-1 and FORTRAN programmers.

Maybe this is good.  Now, developers can focus on solving business problems and spend less time on the nuainces of programming.  Or, maybe they actually have less control and are forced into the confines of what is offered in the toolbox of Google, Amazon, Salesforce and the like.  And, since soon they will lack any real systems development skills, maybe they give up all chance of getting outside of those boxes?

This week finds me at Salesforce.com’s Dreamforce show where a number of very interesting developments are coming to light.  In Marc Benioff’s keynote yesterday, he emphasized the role of cloud computing in the future of all application development, throwing jabs at Microsoft all along the way.  Benioff cast the cloud into these sets of services:

• Amazon is the server plumbing and storage – their EC2 elastic computing cloud providing all the virtual servers you need while S3 provides boundless very low cost storage
• Google is the Microsoft Office and Sharepoint alternative, offering shared applications like calendaring, documents, and spreadsheets, with the capability to share all three, plus adwords that can feed into salesforce leads
• Facebook offers the social graph where new applications can leverage and integrate to spread virally
• Salesforce’s force.com is the application layer to develop business apps and tie all the above together on every platform, mobile to all browsers

This staking out of claims on the cloud computescape is a fascinating thought to me.  While I’m not sure that all other other companies would agree with the above positioning, there has never been a more exciting time to be in software development – the cost  of building a truly scalable application that can service tens of thousands of users  is truly in every developer’s reach.  I woke up at 1 am this morning with my mind racing about how the applications I and others could build!  This clearly has implications for global software development, lowering the bar for anyone in any country who has a great idea to build it out with a small team and bootstrap an effort self funded.

It also means that if you were thinking of corporate IT as a long term career, you should think again.  These services are going to consolidate into a few very large providers, at the end of the day, this is good for our industry. It allows the great ideas, the great innovations to be born more quickly, removing unnecessary barriers and hurdles.

Recently, we evaluated several Agile Project Management tools.  Having used XPlanner for years, it was time for a change.  XPlanner is great in some ways – fairly lightweight and easy to use for developers.  It really falls apart though when you start to consider multiple agile teams working together to deliver a release.  It is just really problematic to get a group view of where you are, what progress is being made by the team, and where the hot spots might be.  To accomplish this, you need to drill into the details of every scrum team and study the metrics / charts.  I even went so far as to change the source code to build a dashboard – that’s when we started to approach diminishing returns.

The other shortcoming of XPlanner is the management of the product backlog and release planning.  Yes, you can work around this, but intrinsically, the tool does not support building a backlog and then moving stories into a sprint.  Yes, this can be done, but it is arduous.  The interface also is stuck in Web 1.0 land, making data entry into a form submit after form submit affair.

So then what?  Surveying the market and talking to many of my longtime friends developing software with agile process, we quickly build the short list to replace XPlanner. We looked in detail at Rally, VersionOne, and FogBugz.  Though FogBugz had some very interesting capabilities around predicting the accuracy of estimates, it didn’t really seem to support agile planning methodologies and the scrum process.  Also, though the predictive capabilities are interesting, this really isn’t a huge benefit in my opinion if agile is really used and you know your people.

So, it was down to VersionOne vs Rally.   Both companies did extensive demos for our leadership team and key stakeholders.  Both tools intrinsically are built around the scrum agile process.  Both were priced around the same level with VersionOne being just a little less per seat, per month, but Rally matched and beat this price point in our negotiations.  The huge gaping hole in VersionOne for us was that it really didn’t assist with resource planning at all.  That is, they don’t enable you to enter the amount of available resources in terms of hours, days, etc, and then in the planning cycle show you where you are in using those hours as you take stories from the backlog and add them to the sprint.  Both tools track burndown during the sprint of course, but only Rally lets you know if you are planning too many stories in the sprint.  Even XPlanner supports this, so it is a big miss for VersionOne.  We can only assume they are working to add this capability.

Also, the rollup reporting for an entire release is more powerful and flexible in Rally.  This was a big plus for us.  To be sure, Rally isn’t super sophisticated in resource planning.  It doesn’t allow the individual team members enter their availability and then sum it up for the sprint.  (I would like this feature – I need to add this to the Rally Community.)  Rather, it just allows you to add the total number of hours available for a sprint at the beginning of the planning cycle.  How you figure this out is up to you.  After you add the total number of hours available, it shows you hours remaining as you add stories.

In coming blogs, we’ll talk more about the pros and cons of Rally as a Agile management tool.

This week Apple unveiled the iPhone 2.0 device and more importantly showed the results of their SDK released just 3 months ago for iPhone. As someone who has been leading teams in building mobile applications off and on over the last 6 years, the results are really impressive. The ease of which very sophisticated applications can be ported to iPhone is astonishing. During the keynote at the WWDC, some interesting games and just amazing medical imaging applications are on display.

More than simply an interesting use of mobile technology, in my view Apple has created the first viable platform to move laptop users to the truly mobile device. They have the device, now with 3G speed, they have the platform based on their desktop OSX, and they have the developer tools and APIs to quickly build the application.

Contrast this with google’s android. Cool concepts, plenty of big budget behind it, but no devices, and with so many vendors in play there is likely to be subtle differences in implementation / compatibility. These are problems that have plagued Windows Mobile and Java 2 Mobile Edition, both of which propose to be a unifying platform. Truth is that you have to build and test for each device you intend to support. This is expensive and inconvenient as you start putting conditionals in the code for devices, screen sizes, etc. And, is it just me, or are the top folks at google starting to look like IBMers?

This is where Apple has it right, in my view – great dev tools (as Microsoft ahs clearly shown is a key part of promoting a new platform), great platform (with innovative location and push notifications), and that gorgeous device. To top it off, they have a terrific market distribution channel with iTunes. Boy, do they have this right. I’m completely impressed.

The Third Sin of Software Projects is failing to understand its all about the people – attracting, retaining, and helping the best software development professionals available to succeed. The people on the project are the key determining factor of success. Let’s look at a few of the key factors in what we need in our people.

Productivity – you’ve probably read that top notch software developers are 2x – 5x more productive that the average developer. I’ve lived it and can attest. It’s not that the best will write 2 – 5 times as much code; it’s that they either can achieve the goal 2x – 5x more quickly, or just simply solve problems that others find unassailable. When I was early in my career I had the good fortune of working with just such a superstar: Brian. While I could hold my own in some of the most difficult projects, I could also see that Brian was simply a brilliant developer. He was simply world class, solving the most challenging issues and making it seem almost effortless. What if you could build a team mostly of this type of person? You could do anything! (FWIW: this also opened my eyes that I needed to focus my career on an area where I felt I could truly excel – leadership.)

Motivation – Software professionals are different than many other professionals in what motivates them. They are motivated by working with other smart people, working on challenging problems and cool technology, and by achieving a level of success personally and technically. (This last one is a complex subject as there are as many meanings of success as there are different types of people in software development). As leaders, we must understand that software professionals are motivated differently, and try to understand what success means to each person in our team.

Teams – In addition to having the most productive and motivated professionals, we need to be able to have them work effectively in a team. Dysfunctional personalities, poor attitudes, or weak contributors will torpedo a team very quickly. Any of these attributes in a team member negates all the positives above. Take a developer who is tremendously productive and knowledgeable, but just has a very negative attitude and they can drag everyone else down, kill cooperation and communication, and single handedly cause a project to fail. Conversely, a team that is made up of top-notch people that are motivated, focused, and check egos at the door can turn out the most innovative solutions imaginable.

Trust - finally, the key essential ingredient in the organization that makes it all work is trust. People within the team must trust each other to deliver, development must trust product mangers, teams must trust their leaders, and all the way through the organization. Very recently, I’ve heard about a situation where a new leader came into a development team and half the team left the company. It’s safe to say that more emphasis on building trust was needed! A subtle implication of trust is that integrity is required for trust to exist. The best companies I’ve worked with have a deep appreciation for all these ingredients, starting with the CEO. Those that don’t will fail.

In this business, we are all subject to schedule pressure which is essentially market pressure. This pressure is often real, coming from executive management, the board, or investors. Despite this, there are some necessary steps in the development process that can seem expedient to skip that in many cases, just shouldn’t be skipped:

• Prototyping the UI and getting feedback and buy in from Marketing and target customers. In the rush to get to market, we can easily skip this one. After all, once the design is done and the developers are implementing, its very expensive to change. Also, when working with new technologies, you don’t always know up front what is possible in the UI.
• Throw away the prototype. Its tempting to keep the hastily constructed prototype and evolve it into the final product. Especially when considering the need to prototype to explore UI options and technologies, this early deliverable is critical but also a mess under the covers. Learn from it and toss it.
• Internationalize the code and externalize strings up front. Its tempting to say “we can do that later”. Later is incredibly expensive. In our globalized world, most commercial software must be internationalized – design it in.
  Continue Reading »

Talking about requirements makes people’s eyes glaze over, so lets ask a more poignant question. What market are we trying to win? This focuses our vision in a much more effective way. The first deadly sin of software projects lies in not asking this question. Requirements management is a fundamental necessity in software development. Changing requirements causes obvious schedule impacts but has more subtle impacts in quality of the product and maintainability of the architecture. But, lets face it, few systems worth building have all their requirements known up front. There is always some element of discovery as the business matures, as early customers give feedback, as new deals are brought to the table.

The deadly mistake that can make your software project, and even the company, fail is a lack of focus on precisely how this product will win in the market. To answer this question, this presumes we can answer these questions of precisely which market are we in, who are our competitors, what are the major technology movements in the market, how will the product be marketed and sold, how should it be priced, and who will buy it. Even if you are the “technology” guy in charge of the software project, you need to know the answers to these questions because it affects in important ways the requirements of the system.   Most companies start out with a pretty firm grip on how they plan to win in the market. What trips them up time and again is not realizing the answers are changing. The deadly sin here is that if the answers to any one of the questions above change over time, they can dramatically affect your requirements and ultimately lead to the failure of your software project (and the company).  These changes can take place very quickly and very subtly in a small project. For example, selling a product into a new channel or to a new type of customer can quickly mushroom into a set of new requirements that can crush your existing commitments. To mitigate this, it is essential that the product management team and development leaders be integrated into the sales processes to gain visibility into these opportunities so they can anticipate new requirements as early as possible. This can be difficult as everyone is excited about the new customer or opportunity, and many times it becomes a “peeling the onion” exercise. At first, it seems that the new customer only needs a small change to the product, and then another, and then another. These changes are driven because they are fundamentally a different customer than has ever purchased before. Recognizing this difference is crucial.   

Another good example I’ve encountered many times is the difference between reseller and direct customers. Resellers are extremely valuable sales channels, but have very different requirements in products and billing than do direct customers. It is important to recognize and understand these differences up front. A reseller may want to embed your product in theirs or rebrand your service / web site. They often want to hide the visible aspects of your product because they want to present only their brand to the customer. These features must be factored in up front as they are very expensive to retrofit.

Finally, the new customer may not necessarily drive changes into your core product, but rather may require you to drive changes into your core processes. For example, a telco customer may have very long cycles between applying updates and may require different service levels that your other customers. Each of these have a cost component associated. The longer upgrade cycles may mean you have to keep alive older branches of source code alive for emergency patching, along with test environments that support that branch. These kinds of patches can be very expensive both to you and the customer. Also, you may have to go through longer or additional test cycles for the releases that go to these customers in an effort to avoid these costs.

In conclusion, asking “What market are we trying to win” means understanding the answer to the question and measuring each sales success against that answer. If the sale represents a fundamental change to this answer, then take the steps to deliberately plan for the product and process changes necessary to deliver on the opportunity.

In our examination of reasons why software projects fail, we start with a few basic assumptions. These seven deadly sins of software development assume that many of the fundamentals are in place: the team is staffed with trained experienced developers, you have a modern source code control system, build processes, test methodologies and automation, bug tracking system, project management systems, etc. In essence, we are assuming at least a CMM Level 2 team – without at least this level of process maturity, there are many other opportunities for the project to fail. This series of short articles focuses on the set of issues that sneak up on experienced software developers and development managers. They are seemingly common issues, but can surface in subtle and surprising ways while teams are heads down implementing a project. We’ll examine some examples along the way as well. Although we may have to change a few names, they will be real examples drawn from experience.

Its been a very busy forth quarter rounding out 2006. Business travel has taken me to South America twice, we’ve been working on closing a number of deals, and December saw the largest number of products all go GA ever in a month at Simdesk. All these are great reasons for not keeping up with the blog, but its time to get back to it.In 2007, I’m planning to write a series of articles on Software Development revolving around common sense knowledge every leader of a project should know. Though I place them in the realm of common sense, I’ve also seen them sink projects over and over again for those that are unaware (or just too busy to notice). Every series like this needs a catchy title, but I’m afraid we’re out of luck. We’re going to call this one “The 7 Deadly Sins of Software Projects”. If you’re in the business, I’ll wager you’ve either seen a few of these or maybe even been guilty yourself. Its easy to do because they can sneak up on you. Stay tuned.

Ok, so your want to startup or install self-directed teams.(see a previous post on change). As with any change, this is a big challenge and there is no guarantee that it will be accepted completely by the team. Some will be uncomfortable with accepting accountability, a strong Scrum master will be needed, and some just don’t like having to fit within a structured process. In our adoption of Scrum, a few things have helped us gain acceptance:

Training We are fortunate to have a certified Scrum Master on board who has trained the whole team. A full day of training was invested in every member of the team. Additional training is needed for Scrum Masters because their execution needs to be effortless for the process to flow.Executive Buy-in and Support All levels of management must support the process. The integrity of Sprint content must be held in tact for the process to work. During the sprint, business needs sometimes require us to trade Sprint tasks for backlog items, but the integrity is maintained if the team signs up to the change.Strong Leads that are Willing to take on Scrum Master role this process of self directed team still requires someone on the team to drive the process. In my experience, a strong and effective lead developer is one of the most rare and valuable commodities in the Software Engineering world.In short, the only mistake we have made with Scrum is not more quickly adopting the process everywhere in the team. At the end of the day, team success is the greatest motivator for this or any other change.

About 18 months ago, we adopted a self-directed process called Scrum in part of our development team – the part that focuses on end user applications.  Most of the team was a little apprehensive, but they are solid folks and they put it in action. 

Recently, we made the move to transition the whole team to Scrum, something I should have done long ago.  Initially, I believed that complex pieces of server-side software had longer life cycles, longer testing cycles, and therefore perhaps the Scrum process wasn’t a good fit.  I was wrong.  What we could have seen are the same benefits on the server-side that we’ve seen on the client.  So, now we are rolling it out team-wide.  So what are some of those benefits?

The most important benefit is the placing of ownership where it needs to be – at the team level and ultimately at the individual level.  Whether they are taking a turn at Scrum Master, or just part of the team, each person owns reporting on their progress, the whole team owns the sprint backlog delivery, and has the power to say no to things not in the Sprint.

Team planning and ownership of the deliverable by the team – each Sprint (we use a month), the team signs up to a number of developer days, test days, and Tech writing days available.  We estimate the line items in the backlog and allocate the available days.  Its all team driven, and as such, the team owns the plan for the Sprint delivery.  In addition, the team demonstrates the deliverable from last Sprint – a powerful way for them to show accomplishments.

Peer-to-peer accountability – another significant benefit.  With Scrum, there is no place to hide from missed deliverables, poor quality, and lack of skill.  Any team member can take any item of the backlog and work on any piece of the system. As such, Scrum breaks down artificial barriers.  And, with daily scrum meetings on what was accomplished and planned for each day, it is pretty easy to tell if someone isn’t pulling their weight.

Forcing the business to define priorities – Scrum forces a booked plan, reconciled to business priorities at the beginning of each Sprint, monthly for us.  It provides a systematic way to pull in all the business inputs (sales, marketing, support, et al) into the planning process – and enforces that this is there only chance to get input into the development cycle until next Sprint. (Not really of course, but it does force the conversation.)

 next Blog…. Success Criteria for Adoption of Scrum

The Second Law of Software Dynamics:  shipping software is a commitment, not than an event.

Customers generally expect once they have adopted your software or service that you are going to continue to develop it, service it, respond to their requests for tweaks, and keep up with competitive forces.  In short, they made a commitment (time, dollars, training, or perhaps a key part of their business) to you, and they are expecting the same.

Yes, this applies to Web 2.0, SAS, and whatever the new paradigm of the day is. It is all just software in different cloth. It may be delivered in a different way, or have a new business model attached, but the commitment remains the same.

(What is the First Law?  Given any non-trivial system, as long as you test software, you continue to find bugs. I need to blog on when is enough, enough someday soon)