Robert on Software Development

Changes

Back in March, I was promoted to Chief Technology Officer. This was a pretty big deal for me, as you can probably imagine, and it greatly expanded the scope of my responsibilities. That’s partly why I’ve not been spending enough time writing for this blog. I’m actually proud of my posts here. I think they have real value, but to be honest, they’re not as focused as I’d like them to be.

So, I think I will narrow the scope of this blog to my experiences as CTO of a software company. This will include development, of course, and maybe even some code. But I also want to share nuggets of wisdom I come across, stories of success and failure, and frustrations as well.

We ship our product on June 7. It’s my first commercial software project. Everything I’ve done previous to this has been in the enterprise, relatively safe in the confines of a large corporation where the users were my coworkers and didn’t spend their own personal money on the stuff I created. This is something entirely different.

I do hope you’ll return regularly, or add me to your RSS reader. I will treat you to stories about cloud computing, rich client applications, raising capital, dealing with constant change, and pig farming.

Posted in Software Development | No Comments »

Hierarchy Melarkey

When my development team got started on our new product, we established a hierarchy based on product ownership. One developer “owned” a particular piece of the product. Bill owned Documentation, Mark owned UI, Victor owned the Financial module, and so on. This had its own advantages, as it let an individual be the expert about a piece of the development effort, which let him be the go-to guy for any questions about how his piece worked.

However, what we quickly realized is that the application became very disjointed and inconsistent. We were trying to work within the expectation of the stakeholders, who assumed that one developer would handle one logical piece of the application. It makes sense at a high level, but the fact is, software development isn’t an assembly line. (Not that it doesn’t try to be at times.)

When we adopted our agile approach, we decided that no single person would be the sole expert on the system. Some people will learn more about different aspects of the application than others, but through information sharing, everyone would have access to the knowledge of others.

Figuring out how to share that information is the hard part.

Posted in Enterprise | No Comments »

Scheduling and Estimation

We are in the throes of development of our product, and while coding marches on, we’re asked on a near-daily basis: when will it be done?

That’s a tricky question, one that has plagued developers and managers for years. There are countless books on how to estimate software development, and it seems everyone has his or her own idea of how it should be done. In my own personal experience, I’ve seen everything from determining the number of lines of code and calculating based on that (part of the CMMI Personal Software Process) to plugging the number of ASP pages in an Excel spreadsheet and ending up with a time based on some arcane calculation.

In my current project, we had to face the specter of determining timelines this week, and without a clear vision of what “complete” means, we have to take things step by step. We had several choices for our estimates.

The first option involved using the traditional time-based estimate. “This will take three hours,” or “I think this will take two days.” This type of estimate is good for determining a stop date for development, and managers like it. However, it can be very inaccurate and put the developers in a position where they have to defend against accusations of failure. Because we’re learning about our vertical market and WPF at the same time, it’s extremely difficult for us to get any accuracy on time and date estimation.

When I was at HP, we used a “level of effort” estimation. The first iteration for the LOE was 150%. In other words, the project’s timeline would be met somewhere between –150% and +150% of the initial estimate. So if we said 10 days, it could be done as late as 25 days (-5 days wasn’t really an option). In the second iteration, we reduced it to 50% LOE, and then finalized on 30%. In a big project, a 30% LOE can be a huge variance. Consider a project with dozens of developers and thousands of hours of work. A 30% LOE on a 14,000 hour project is 18,200 hours. If you’re paying by the hour, ouch.

The second option we looked at was the one we went with: relative sizing. Rather than evaluate a task in terms of how long it would take, we looked at how difficult it was. Each task is given a number based on the Fibonacci sequence (1, 2, 3, 5, 8…). A “1” is very simple. A “2” could be more complicated, but still pretty easy. And so on, until you reach the upper numbers, which imply that the task is extremely difficult and should be further broken down.

We looked at one of our key deliverables and broke it down into a couple dozen tasks. Then we got together as a group and sized each task. Creating a screen might be a 2, while developing its databinding is a 3. We wrote down each task on a sticky note and put it on our board. Each week, we go through the tasks and choose the ones that will be “on deck” for that week, and put the rest in the backlog. The developers then assign themselves tasks, one at a time, based on the difficulty of the task and their comfort level with it.

One of the nice effects of using this technique is that upper management (including myself) can look at the board and get an idea of what’s going on at any given time. Each task’s progress is also tracked in our daily scrum reports, which are logged online. If a developer needs help due to a roadblock, he makes a note in the scrum report and we figure out how to solve it.

At the end of each week, we look at the results of the sprint and decide if our estimation of what we could complete is accurate, and we adjust the following week’s tasks accordingly. This is called establishing velocity. We are able to determine what our work product is from week to week, and make a determination on how long something will take by looking at evidence from previous sprints. If Developer A takes an hour on average to complete a 1, and three hours for a 2, then we can determine that within a 30 hour productive cycle that he can do 30 1s or 10 2s, for example. It’s not supposed to be that precise, but rather give us an idea of what we can expect.

We’re young with this process, but I feel confident about it. I need to be able to respond quickly to removing roadblocks for the developers while also producing my own work output. By viewing the board, I see what is going to be done, and by participating in the board, I set the expectation for my own work.

In my next post, I’ll discuss hierarchies of the individuals involved and how interaction works between them.

Posted in Enterprise | No Comments »

A New Day

Where has the time gone? My last entry was on July 29, 2009. To say that a lot has happened since then would be a grand understatement. The short version is that I left the world of SharePoint consulting and accepted a full-time position at a software company here in Des Moines. As the Director of Product Development, I serve in multiple roles. Product architect. Project manager. Software developer. Tech lead. You name it, I do it. I have a team of eight people, including developers, QA, and documentation.

This job was a dream come true for me and to be honest, accepting it turned my life upside down. I canceled my plans to move to Dallas and start my SharePoint consulting firm. I canceled my SharePoint certification tests. I ceased all work on Flipism. And most surprisingly, I bought a house. I decided to fully invest in this new opportunity, and I accepted all the risks that came along with it.

We are developing the application in C# 3.0 and WPF. I decided to build the application in WPF because I believed it offered us the most flexible platform for future opportunities and growth. It was a risky gambit, partly because nobody on the team knew WPF. I had a limited exposure to it from my days at HP, but I was hardly an expert. But my team pulled it together and we took the plunge into WPF. While I acknowledge that developing the product in WinForms would be “easier,” I don’t think that it would make our product better. WPF has given us many gains, and the architecture lends itself well to our future plans.

I’ve learned so much since I started on August 31. While the company has been around for several years, my team is brand-new, and the product we’re working on is brand-new. We’ve all had to learn how to work together. We come from different corporate cultures and technical backgrounds. I’ve had to draw on the entirety of my experience in order to lead this team, but I love it. Love love love it. I feel like every decision I’ve ever made in my life and career has led to this. It’s hard for me to feel regret about bad choices I made when I was younger because I might not have been led to where I am now. I know that sounds corny, but it’s true.

Obviously I have a lot of hope about the future and the change it brings. And to that point, the focus of this blog has to change as well. Pretty soon I will relaunch this blog to focus on my experiences as a leader in a software company. Hopefully what I share with you here will be of interest, or at least entertainment value.

So tell me: do you want to come along for the ride of your life?

Posted in General | No Comments »

SharePoint Consultants: Why We Are

Yesterday I had lunch with two colleagues. As we chowed down on pulled pork and spiced apples, we talked about what makes something “enterprisey” and the scenarios in which going with the “right option” isn’t always the best option. It was a good lunch.

It seems like in every conversation about work in which I’m involved, the topic of SharePoint arises. Most people hate it and are aghast when I tell them that I work with SharePoint and actually like it. Of course I can understand their frustrations with SharePoint; it can be a real pain to use, especially when you don’t know what you’re doing (and it’s so easy to not know what you’re doing). When things go wrong, they go spectacularly wrong. The API is difficult to code against, the documentation is nearly non-existent, the tools for administration are a joke, and the system requirements are ridiculous. Add all that to the enormous licensing costs and you have to scratch your head while you wonder aloud, “What the heck are you thinking?”

But that’s just it: SharePoint is a challenge, and when it’s done right, when you deliver that piece of the application that SharePoint really excels at, you end up with something that feels about as close to magic as you’re going to get. Take the time to master SharePoint, and you have a tool that sometimes feels like it inherently understands your business processes. Have documents that need approvals? It’s built in. Need to store multiple business documents in a shared location with metadata to separate them? It’s built in. Need to kick off some kind of process when a piece of data changes? It’s in there. You can develop a custom ASP.NET application, host it inside SharePoint, and take advantage of what WSS and MOSS 2007 offer. That’s what we’re doing at my current customer, and each time I add a new piece of functionality to the system, such as a document library, I solve a complex business problem that formerly plagued them. The “wow factor” from that is intense.

As I work with SharePoint more and more, and embrace its quirks more and more, I remind my customers of the need for a dedicated SharePoint person on the team. That’s the thing about SharePoint: you can’t just install it and walk away. You need someone to tune it, optimize it, keep it running, and support the people who use it. It’s like a DBA, but for something a little more specialized.

Find the right person to do this and you’ll find yourself singing SharePoint’s praises. With time, the pain and suffering will all drift into the haze of days gone by.

Posted in Enterprise | No Comments »

Origins

Not too long ago, I was working with Microsoft Excel and had to write some macros. I finished up my work in Excel but left it open, and several hours later, I came back to my computer and the Visual Basic code editor was still open, minimized along with Excel.

Seeing that Visual Basic window minimized to my desktop brought back a wave of emotion and memories. About 14 years ago, I started working at a crappy mortgage services company. I was 20 years old, futureless, prospectless, but not hopeless. I was making $6 an hour as an administrative technician (I created WordPerfect and Lotus documents). I had heard about this thing called Visual Basic, which allowed you to easily create programs for Windows. I had written BASIC code a lot in high school, and I knew a lot about computers, so Visual Basic sounded very appealing to me. It was my chance to actually enter a career, and I was quite passionate about computers and technology.

I went to the mall and bought Visual Basic for Dummies. It was so Dummied that it didn’t show how to do some important things (for example, showing a form; for a while, I used form1.Visible = True and form1.Visible = False to show and hide a form, but I eventually discovered form1.Show and form1.Hide) but I didn’t know any better. My wife and I were quite poor, so we didn’t own a computer. But I was determined to learn Visual Basic and get a job as a computer programmer. I read that book over and over, at least five times, cover to cover. I memorized every little detail, wrote code on paper to practice my style, and dreamed daily about my chance to be a programmer. I would even hide out in the bathroom at my job and read the book, imagining myself writing code for something important.

Someday.

It’s weird to romanticize computer programming, but that’s how big it was to me. After I left the mortgage company in January 1995, I went to work at a utility company in Austin, where I was responsible for creating an Excel workbook that tracked all the materials ordered for building substations (exotic, I know). It was then that I got my first chance to code in Visual Basic, writing macros for this huge spreadsheet.

And here I am, almost 15 years later, realizing the success that I started with that Dummies book so long ago. I could be a pessimist and say I haven’t gotten anywhere, but truthfully, I’ve accomplished all of my career goals. I’ve been promoted all the way to architect, I’ve published books, I’ve led teams, I’ve shipped products, I’ve traveled, I’ve been offered speaking engagements, and best of all, I’ve helped other young people get started in this business.

I’ve had my fair share of failures, too. Internet companies, technologies (OS/2? What was I thinking?), missed opportunities, and I’ve learned from them all.

So now when I look at that little minimized window, I feel appreciative for what I have, and the opportunities I’ve been given. Now, I want to do something truly great.

Posted in General | No Comments »

This Library Has No Card

There’s a term that comes from online games known as grinding. This term is used to describe a repetitive activity intended to accomplish some goal. For example, you might find yourself repeatedly killing a certain type of creature in the hopes that it’ll give you a particular piece of loot. That’s grinding.

The point of bringing that up is to say that I feel like I’ve been grinding a lot on a particular task at work. It seems like a pretty simple task, involving SharePoint document libraries and Word documents, but something about the whole thing just isn’t sitting right. Not yet. I’ve spent a lot of time on my whiteboard, mapping things out. Each time I’ve come up with a solution, I go into SharePoint and figure out why it won’t work. Content query web part? Nope. Document library web part with querystring filter? Nope. Individual libraries for each user group? Nope.

And so on, and on, and on, until the end of the day comes and I throw my hands up in the air in defeat as I trudge out to my Jeep for my mercifully short commute home through downtown Des Moines. Yet I still have the specter of the problem hanging over me, taunting me, urging me to just stay in bed and pull the comforter over my head until SharePoint somehow magically solves itself.

As a grown up, I have figured out that such solutions aren’t really solutions at all, and I do indeed have to face the music, so to speak. The first step to overcoming the demon is to admit that I am the one with the problem: my skills in this particular area aren’t strong enough, so perhaps I can’t figure this one out on my own.

Tomorrow I’m bringing in another set of eyes to work on the problem with me. Hopefully we can collaborate effectively and devise some kind of solution that will solve the problem without taking forever to implement. This guy’s SharePoint knowledge is next to nil, but he’s a smart developer so maybe just talking about the problem will give me the eureka moment.

I’ll let you know.

Posted in Software Development | No Comments »

The Path to Self-Discovery on Rails

Until about two weeks ago, I had no idea why so many developers were going ga-ga over Ruby on Rails. It wasn’t because I had a problem with Rails; in fact, I didn’t have an opinion at all. That’s the beauty of ignorance.

But as I started to delve deeper into SubSonic for my current project, I realized I needed to learn a bit more about the product that SubSonic is somewhat based on, specifically ActiveRecord. I figured, what better way to gain insight into how SubSonic works than to understand the genesis of the whole thing.

My foray into Rails development has been swift and deep. Now I get it: Rails lets you build Web applications very quickly, and it does that by requiring an adherence to a specific architecture that is governed by one person. It makes total sense.

That paragraph could be taken as a bash on Rails, but in fact, it’s not. I really like Rails. I really like that Rails is overseen and designed by one person (well, technically two). I think that’s a strong feature of the framework. In the .NET world, we don’t have that. .NET is meant to be general-purpose. It’s designed to let you create your own frameworks, and it gives you a million ways to do everything right and a million ways to do everything wrong. As I said to a colleague several years back, “.NET’s flexibilty is inflexible.”

Learning Rails is making me a better developer. It’s forcing me to learn new ways of doing things. I didn’t expect the exercise to have such an immediate and valuable impact. If I take nothing else away from this experience, I’ll at least have a much broader understanding of this business.

And MVC, of course.

Posted in Software Development | No Comments »

Fixing Issues with the SharePoint SP2 Upgrade

Yesterday I installed Service Pack 2 for WSS 3.0 and MOSS 2007 on the two servers in our Development farm. I purposely did not follow the guidelines for upgrading SharePoint as I wanted to see what happened when I ran the installer. Since it’s a dev environment, there was no risk.

Long story short, the upgrade on both of my servers failed. Looking at the logs, I saw this:

[SPManager] [ERROR] [6/2/2009 10:44:16 AM]: The system cannot find the path specified.

That’s it. That’s all I had to go on. After some hemming and hawing and tearing my hair out, I figured out what the problem was. It’s not that the installer can’t find a path, it’s that the IIS metabase is pointing to a “dead site,” or a site defined in IIS for which the directory structure doesn’t exist. That fixed the problem on the database server.

On the farm’s front-end server, I got the same error, but I didn’t think that it was a problem with IIS.

My solution was to simply detach and reattach the content database, which is what Microsoft says to do in its best practices document for upgrading SharePoint.

If you choose to not follow the directions and end up with the “file not found” error, try this.

First, execute this command (replace the bracketed items with your details, without brackets of course):

stsadm –o deletecontentdb –url [http://example.com] –databasename [contentdbname] –databaseserver [databaseserver]

Don’t worry about the word “delete” in the command. It doesn’t actually delete your database. It just detaches it.

Once that’s done, reattach the content database via:

stsadm –o addcontentdb –url [http://example.com] –databasename [contentdbname] –databaseserver [databaseserver]

Now run psconfig to tie it all together:

psconfig –cmd upgrade –inplace b2b –wait –force

The configuration completed and everything worked.

My suggestion is to follow Microsoft's SharePoint upgrade best practices document to the letter. This is the best way to head off these kinds of issues. And always make sure you back up your content and configuration databases before you upgrade. Few things are worse than rebuilding a SharePoint farm from scratch.

Posted in Enterprise, Software Development | No Comments »

Making Dynamics Dynamic (Part I)

I’ve been giving a lot of thought to Microsoft Dynamics lately. Microsoft Dynamics encompasses Microsoft’s enterprise software for customer relationship management (CRM) and enterprise resource planning (ERP). Microsoft developed CRM in-house, and bought the ERP applications from various companies so it could compete in the enterprise applications spaces held by companies such as Oracle (which owns Siebel and PeopleSoft) and SAP.

Microsoft Dynamics is made up of five different products: CRM for CRM; AX, GP, NAV, and SL for ERP. Figuring out how the four ERP products differ, and which one to choose, is not immediately clear. Because the products all come from different originating vendors, there’s even some overlap. For example, AX offers a CRM module that overlaps with Dynamics CRM. GP has a ledger built in, but so does AX and SL. And with all these initializations (which stand for the companies that originally developed the apps, such as “SL” for “Solomon”), it’s easy to get the products all confused.

The ERP products are also developed in different programming languages and don’t always fit cohesively into the Microsoft family of products. This makes it quite tough for custom developers like myself to walk into a shop that says they’re using Microsoft Dynamics for ERP. Which product? What customization has already occured? What modules are installed?

While it might sound like I’m being harsh on Microsoft, I’m actually not. I know Microsoft has been working on improving the Dynamics experience the past few years. CRM, for example, is a very polished product and the new version, coming out later this year, runs on SharePoint. I’ve also heard that the Dynamics team is working on new versions of the various products that will be built on .NET, with better Office integration and a lower barrier of entry for developers. As an enterprise IT guy, this pleases me. Microsoft will probably rename the applications once their focus shifts from supporting existing customers to acquiring new ones, so it becomes easier to figure out which application covers supply chain management, for example.

I think the Microsoft Dynamics tools can be expanded beyond their original domain, too. Take CRM, for example. When people talk about CRM, they’re almost always talking about it within the context of sales. A CRM solution tracks and organizes a company’s contacts with its current and prospective customers. But can Dynamics CRM be applied to domains where sales aren’t the objective?

That’s the topic for part II of this series.

Posted in Enterprise | No Comments »