However, what we quickly realized is that the application became very disjointed and inconsistent. We were trying to work within the expectation of the stakeholders, who assumed that one developer would handle one logical piece of the application. It makes sense at a high level, but the fact is, software development isn’t an assembly line. (Not that it doesn’t try to be at times.)

When we adopted our agile approach, we decided that no single person would be the sole expert on the system. Some people will learn more about different aspects of the application than others, but through information sharing, everyone would have access to the knowledge of others.

Figuring out how to share that information is the hard part.

That’s a tricky question, one that has plagued developers and managers for years. There are countless books on how to estimate software development, and it seems everyone has his or her own idea of how it should be done. In my own personal experience, I’ve seen everything from determining the number of lines of code and calculating based on that (part of the CMMI Personal Software Process) to plugging the number of ASP pages in an Excel spreadsheet and ending up with a time based on some arcane calculation.

In my current project, we had to face the specter of determining timelines this week, and without a clear vision of what “complete” means, we have to take things step by step. We had several choices for our estimates.

The first option involved using the traditional time-based estimate. “This will take three hours,” or “I think this will take two days.” This type of estimate is good for determining a stop date for development, and managers like it. However, it can be very inaccurate and put the developers in a position where they have to defend against accusations of failure. Because we’re learning about our vertical market and WPF at the same time, it’s extremely difficult for us to get any accuracy on time and date estimation.

When I was at HP, we used a “level of effort” estimation. The first iteration for the LOE was 150%. In other words, the project’s timeline would be met somewhere between –150% and +150% of the initial estimate. So if we said 10 days, it could be done as late as 25 days (-5 days wasn’t really an option). In the second iteration, we reduced it to 50% LOE, and then finalized on 30%. In a big project, a 30% LOE can be a huge variance. Consider a project with dozens of developers and thousands of hours of work. A 30% LOE on a 14,000 hour project is 18,200 hours. If you’re paying by the hour, ouch.

The second option we looked at was the one we went with: relative sizing. Rather than evaluate a task in terms of how long it would take, we looked at how difficult it was. Each task is given a number based on the Fibonacci sequence (1, 2, 3, 5, 8…). A “1” is very simple. A “2” could be more complicated, but still pretty easy. And so on, until you reach the upper numbers, which imply that the task is extremely difficult and should be further broken down.

We looked at one of our key deliverables and broke it down into a couple dozen tasks. Then we got together as a group and sized each task. Creating a screen might be a 2, while developing its databinding is a 3. We wrote down each task on a sticky note and put it on our board. Each week, we go through the tasks and choose the ones that will be “on deck” for that week, and put the rest in the backlog. The developers then assign themselves tasks, one at a time, based on the difficulty of the task and their comfort level with it.

One of the nice effects of using this technique is that upper management (including myself) can look at the board and get an idea of what’s going on at any given time. Each task’s progress is also tracked in our daily scrum reports, which are logged online. If a developer needs help due to a roadblock, he makes a note in the scrum report and we figure out how to solve it.

At the end of each week, we look at the results of the sprint and decide if our estimation of what we could complete is accurate, and we adjust the following week’s tasks accordingly. This is called establishing velocity. We are able to determine what our work product is from week to week, and make a determination on how long something will take by looking at evidence from previous sprints. If Developer A takes an hour on average to complete a 1, and three hours for a 2, then we can determine that within a 30 hour productive cycle that he can do 30 1s or 10 2s, for example. It’s not supposed to be that precise, but rather give us an idea of what we can expect.

We’re young with this process, but I feel confident about it. I need to be able to respond quickly to removing roadblocks for the developers while also producing my own work output. By viewing the board, I see what is going to be done, and by participating in the board, I set the expectation for my own work.

In my next post, I’ll discuss hierarchies of the individuals involved and how interaction works between them.

It seems like in every conversation about work in which I’m involved, the topic of SharePoint arises. Most people hate it and are aghast when I tell them that I work with SharePoint and actually like it. Of course I can understand their frustrations with SharePoint; it can be a real pain to use, especially when you don’t know what you’re doing (and it’s so easy to not know what you’re doing). When things go wrong, they go spectacularly wrong. The API is difficult to code against, the documentation is nearly non-existent, the tools for administration are a joke, and the system requirements are ridiculous. Add all that to the enormous licensing costs and you have to scratch your head while you wonder aloud, “What the heck are you thinking?”

But that’s just it: SharePoint is a challenge, and when it’s done right, when you deliver that piece of the application that SharePoint really excels at, you end up with something that feels about as close to magic as you’re going to get. Take the time to master SharePoint, and you have a tool that sometimes feels like it inherently understands your business processes. Have documents that need approvals? It’s built in. Need to store multiple business documents in a shared location with metadata to separate them? It’s built in. Need to kick off some kind of process when a piece of data changes? It’s in there. You can develop a custom ASP.NET application, host it inside SharePoint, and take advantage of what WSS and MOSS 2007 offer. That’s what we’re doing at my current customer, and each time I add a new piece of functionality to the system, such as a document library, I solve a complex business problem that formerly plagued them. The “wow factor” from that is intense.

As I work with SharePoint more and more, and embrace its quirks more and more, I remind my customers of the need for a dedicated SharePoint person on the team. That’s the thing about SharePoint: you can’t just install it and walk away. You need someone to tune it, optimize it, keep it running, and support the people who use it. It’s like a DBA, but for something a little more specialized.

Find the right person to do this and you’ll find yourself singing SharePoint’s praises. With time, the pain and suffering will all drift into the haze of days gone by.

Long story short, the upgrade on both of my servers failed. Looking at the logs, I saw this:

[SPManager] [ERROR] [6/2/2009 10:44:16 AM]: The system cannot find the path specified.

That’s it. That’s all I had to go on. After some hemming and hawing and tearing my hair out, I figured out what the problem was. It’s not that the installer can’t find a path, it’s that the IIS metabase is pointing to a “dead site,” or a site defined in IIS for which the directory structure doesn’t exist. That fixed the problem on the database server.

On the farm’s front-end server, I got the same error, but I didn’t think that it was a problem with IIS.

My solution was to simply detach and reattach the content database, which is what Microsoft says to do in its best practices document for upgrading SharePoint.

If you choose to not follow the directions and end up with the “file not found” error, try this.

First, execute this command (replace the bracketed items with your details, without brackets of course):

stsadm –o deletecontentdb –url [http://example.com] –databasename [contentdbname] –databaseserver [databaseserver]

Don’t worry about the word “delete” in the command. It doesn’t actually delete your database. It just detaches it.

Once that’s done, reattach the content database via:

stsadm –o addcontentdb –url [http://example.com] –databasename [contentdbname] –databaseserver [databaseserver]

Now run psconfig to tie it all together:

psconfig –cmd upgrade –inplace b2b –wait –force

The configuration completed and everything worked.

My suggestion is to follow Microsoft's SharePoint upgrade best practices document to the letter. This is the best way to head off these kinds of issues. And always make sure you back up your content and configuration databases before you upgrade. Few things are worse than rebuilding a SharePoint farm from scratch.

Microsoft Dynamics is made up of five different products: CRM for CRM; AX, GP, NAV, and SL for ERP. Figuring out how the four ERP products differ, and which one to choose, is not immediately clear. Because the products all come from different originating vendors, there’s even some overlap. For example, AX offers a CRM module that overlaps with Dynamics CRM. GP has a ledger built in, but so does AX and SL. And with all these initializations (which stand for the companies that originally developed the apps, such as “SL” for “Solomon”), it’s easy to get the products all confused.

The ERP products are also developed in different programming languages and don’t always fit cohesively into the Microsoft family of products. This makes it quite tough for custom developers like myself to walk into a shop that says they’re using Microsoft Dynamics for ERP. Which product? What customization has already occured? What modules are installed?

While it might sound like I’m being harsh on Microsoft, I’m actually not. I know Microsoft has been working on improving the Dynamics experience the past few years. CRM, for example, is a very polished product and the new version, coming out later this year, runs on SharePoint. I’ve also heard that the Dynamics team is working on new versions of the various products that will be built on .NET, with better Office integration and a lower barrier of entry for developers. As an enterprise IT guy, this pleases me. Microsoft will probably rename the applications once their focus shifts from supporting existing customers to acquiring new ones, so it becomes easier to figure out which application covers supply chain management, for example.

I think the Microsoft Dynamics tools can be expanded beyond their original domain, too. Take CRM, for example. When people talk about CRM, they’re almost always talking about it within the context of sales. A CRM solution tracks and organizes a company’s contacts with its current and prospective customers. But can Dynamics CRM be applied to domains where sales aren’t the objective?

That’s the topic for part II of this series.

Now I shall look into my broken crystal ball and share you with my prognostications of SharePoint 2010 and beyond.

STSADM becomes a web application
I don’t know how many holy wars this one will cause, but I expect there will be much rejoicing, especially among those who are new to SharePoint development and administration. STSADM is a powerful (and extensible) tool that requires some, well, getting used to, but once you wield its power, you can do anything.

A web-based version of STSADM would make some of the more basic operations a bit easier, such as restoring a content database or provisioning a site. Silverlight would be pretty great for this. A WPF STSADM would be gorgeous.

SharePoint Central Administration is an ASP.NET 4.0 AJAX application
Let’s admit it: SPCA is clunky. The workflow of setting up a server and applications inside it is, well, clunky. A modern, Web 2.0 version of Central Administration would be a wonderful thing. It’d also be nice if CA could recover from my mistakes, but that means I would have to upgrade.

SharePoint sites are containers
Someday, everything SharePoint hosts for you will exist as a container, a self-contained bit that can be easily copied (or moved) from one SharePoint farm to another. Each of these containers will be object-oriented, extensible via code. Yeah, I can dream. And I dream BIG.

Visual Studio supports SharePoint projects
We sort of have this already, but the way it’s implemented is not de facto from Microsoft and you have to get separate implementations for, say, Web Parts versus SPJobDefinitions. Visual Studio will support a fluent interface for SharePoint application development, and support SharePoint interactive debugging natively. It will be a day of much rejoicing.

I’ll add SharePoint Designer integration into Visual Studio (and thus Team Foundation Server) here as well.

Documentation is plentiful and meaningful
This isn’t meant to make a dig at Microsoft, but let’s be honest: SharePoint developer doco is quite anemic. As new versions of SharePoint are released, the documentation will drastically improve and will serve as a starting point for SharePoint developers. Quick Starts will be included as well.

SharePoint will have a brand new .NET 4.0 object model
This is wishful thinking (due to compatibility) but someday we’ll get an entirely new SharePoint object model built on .NET, possibly 4.0 or later. It’d be awesome to have something like LINQ to SharePoint, or even represent SharePoint functionality in the Entities Framework.

ASP.NET MVC will be available for SharePoint
Someday SharePoint will natively support ASP.NET MVC out of the box. You can do this now, to an extent, but I’d like to see SharePoint make its data available in the model (like lists), and give us a way to build web parts as views, and perhaps expose its object model in an MVC-fluent way.

These are just a few of the things I’d love to see SharePoint offer in future versions. I acknowledge that I’ve barely scratched the surface, but my crystal ball can only do so much!

In a future post, I’ll take a look at some of the more technical aspects of SharePoint development and what I’d like to see in the future.

While we IT developers have been using source code control for what feels like eons, some of us don’t think about using it at home. For a lot of the folks coming from the world of Visual SourceSafe, source control’s primary benefit is sharing files among team members; ditto for the document library check-out function that my SharePoint-using friend pointed out.

But as homegrown software projects grow more and more complex, it becomes even more important to have a source control solution for those projects. Such began my search for a source code management (SCM) tool. I knew up front what my basic features are:

• Version control
• Merging
• Branching
• Visual Studio integration
• Scalability (needs to support more users someday)

That’s not too much to ask, right? A quick Google search for “free version control” turned up Git, Mercurial, and Subversion. (There were other choices, but they were free only for open source projects.) At work, we use SourceGear Fortress, and I’ve been a user of Visual Studio Team Foundation Server for many years, but those products have costs, and for my simple projects at home, I wanted to minimize my cash outlay.

What I found with Git, Mercurial, and Subversion is that they are excellent tools but are way more complicated than what I need. I really want something with the relative simplicity of Visual SourceSafe but the power of something more like Subversion; this product does not exist without a cost. I really like Vault, but man, $249 for a single license?

I have a license for Visual Studio TFS, but it’s way overkill for just one user. I’d need to run it in a Virtual Server environment and that’s a lot more than I need.

Right after I decided to bite the bullet and use Subversion, I went to close the tab containing SourceGear’s website and something caught my eye:

A full-featured 30-day evaluation is available at no charge — and Vault is always free for single users.

Free for single users? Why didn’t I see that sooner?

Thus ends my quest for a SCM tool that is a) free b) easy to use and c) scalable. If I’m someday able to add another developer to my home project, then I can surely shell out the 249 bucks for an extra license. The Visual Studio integration is tight and I’m very used to it from my day job.

Please note: I’m not trying to slam SVN, Git, or Mercurial. Those are awesome SCM tools. But for the single user like myself, they’re overkill.

What personal SCM tool do you use? What do you use it for?

Take design patterns for example. Design patterns are very useful tools for building applications because they draw upon what’s already been done, so you don’t repeat the work of thousands of other developers. Not everyone is a fan of design patterns, so there’s a ton of discussion about their merits and pitfalls all over blogs, discussion forums, and programmer Q&A sites. The thing is, design patterns don’t make your software finished. They don’t drive the business side of software development; that is, a design pattern is not going to make your calculations for you. But nobody is talking about this. Nobody is talking about a design pattern for managing business rules.

It seems like developers and software architects are happy to just assume that the business rules are going to be taken care of in the process of building the software. We like to think about using ORM for the data layer and MVC for the flow control and SharePoint for the document lifecycle. In a lot of cases the business rules are a foregone conclusion; I know that if I’m going to develop an application that calculates bus shipping costs, I’ll get the formulae for those calculations from somewhere and I can worry about how they work when the UI/database/flow control/etc. are done.

My current customer has their business rules all over the place. They’re in stored procedures. They’re in ASP.NET code-behind files. They’re in ASP.NET pages. The rules aren’t documented anywhere. If I need to figure out how the fiscal year is calculated, I either have to ask the domain expert or dig around in several code files. That’s a relatively simple calculation. For some of the more complex rules, such as determining eligibility for reimbursement, it’s nearly impossible for me to figure them out on my own. The prospect of getting all those rules under control while fixing other bugs and adding new features is daunting to say the least. It will easily take years.

In future application development projects, assuming I get to be in on the design phase, I will ensure that the business rules are captured in a way that is meaningful both to the stakeholders and the developers. Windows Workflow is a great way to do this, for example. There are also enterprise tools from Computer Associates and Rational that will keep track of business rules (or rather, requirements).

Managing your business rules is a fundamental piece of IT governance. It’s a very important IT survival skill.

I have been brewing an idea in my head for a couple of months, since early October. The idea has germinated in my mind, growing from high concept ("It’s a site that does x") to more detailed ("It’s a site that does x so you can y") to pretty fleshed out ("It’s a site that brings together z so you can y and get x"). Or something like that.

But like all great ideas, mine sat in my head or served as dinner conversation for my bored friends. When the idea reached critical mass–that is, when I wished the application existed so I could actually use it–I knew I was really on to something. I decided to actually make my idea a real thing.

So I began to plan. And plan. And plan. I acquired hosting. I acquired tools. I set up source control. I set up issue tracking, and features tracking, and installed all the software I needed. I set up a separate development machine. I organized my home office to support my work. Once I got everything set up, I took a step back and admired creation: "Wow," I said. "Look what I did."

Which, of course, means absolutely nothing, because I’m in the business of creating software, and I hadn’t created a single bit of anything.

I don’t have a database design. I don’t have an architecture. Until a week ago, I waffled on ASP.NET webforms versus ASP.NET MVC. I had a host set up for a month before I canceled it in favor of another who offers URL rewriting, so at least I made a sound design decision there. My Visual Studio project has my web site project in it with empty code files. I have nothing.

In my rush to get everything squared away to support the development of my idea, I failed to actually do any development. Sure, it was necessary to get things in order: source control is vital, the database needs to be up and running, and so on. But that configuration is where I directed my initial project passion, and now that it’s complete, I have a false sense of accomplishment. In a way, I almost have to start over. I have to rekindle the passion for my project and direct the efforts into the actual creation.

Fortunately, I was able to realize what was going on before I let my idea enter the world of "I should have worked on that more," like so many other ideas. I think my project has real value and if I invest the time and energy it requires, it could become something very wonderful.

In my new role here in Iowa, I’m in a situation where the data layer in a production application is practically nonexistent. The core business logic exists in stored procedures, and stored procedure calls are crafted mainly in the code-behind class files for the various ASP.NET pages. There’s also some stored procedure and SQL code in a massive static class. This approach is a maintenance nightmare. As a consultant brought in to get the product under control, I have to find a way to standardize the data layer without making a negative impact on the product. Easier said than done.

Throwing out the stored procedure code appeals to the architect in me ("Let’s start all over!"), but the realist in me knows that that is a thing you should never do. Given that the code that uses the stored procedures in the ASP.NET project is repetitive, I figured that using ORM would best solve this problem. I chose SubSonic over other choices like NHibernate because SubSonic wraps the stored procedure calls in objects (it does the same for our tables and other database objects). Now all the features of our database, where the business logic tragically lives, are available to us in an object-oriented way. I can refactor the existing code instead of rewriting it, and preserve the stored procedures as they are. Calling a stored proc will no longer require string-heavy usage of SqlCommand. We can use DatabaseName.SPs.StoredProcName(typed parameters).

For example, say you have your web application configured to use SubSonic. Your database connection string is named PurchasingSystem. In your actual SQL Server database, you have a stored procedure called UpdateItemList and it takes two parameters, an integer identifying the product list to update and a string that is the name of the item. (Yeah yeah, not the best example in the world, but stick with me.)

The SubSonic-generated code will let you do this:

PurchasingSystem.SPs.UpdateItemList(1, "Gears of War 2");

That is so much simpler and easier to maintain than the alternative of using ADO.NET objects.

As the kids say, ActiveRecord and scaffolding FTW.