How we manage large projects

One of our recently completed projects was a relatively complex web application. Here are some interesting facts:

• 3 primary developers, 2 others involved
• 8 months
• 90 MB of data
• 41,000+ lines of code in 510+ separate files

With projects like this one, there’s always a risk that the size of the project itself will actually hinder our progress and ability to meet the client’s needs quickly and efficiently. As a project gets larger and lasts longer, the effort required to keep things organized and moving forward increases dramatically. In fact, the chances that the project will be finished at all actually goes down substantially. For projects over 6 months in duration, the average success rate (completed on time, under budget) has been shown to be below 50%. [ Agile and Iterative Development by Craig Larman, p. 71 ]

That’s not a statistic we want our clients to have to deal with.

1. Prioritize features and functions

One of the most common reasons for failed development projects is having too broad a scope. After all, the duration of a project is directly related to the scope of its features. A recent study of large projects and their implemented features showed that over 45% of the features initially requested are never used in practice, with another 35% seldom used. Shockingly, only 20% of features in initial project scopes are actually used regularly. [ Agile and Iterative Development by Craig Larman, p. 73 ] In other words, for many projects, a large portion of the time, energy and money invested in them is essentially wasted.

We combat this danger by spending time early on helping our clients analyze their needs and choose only the most important features and functions. If the project is coming in on time and under budget, there’s always room to add those extra touches from the wish list later on. But the reality is, if we don’t intelligently pick and choose, the project is on rough ground right from the start.

2. Short development cycles

Developing a large website or web application is a lot like constructing a building. You need plans before you begin if you want to obtain a satisfactory result. And if those plans are going to change significantly, you want it to happen as early on as possible. Changing the floor plan after sheetrock is on the walls would be costly and time-consuming. In the world of software development, this risk is exacerbated. The longer a project takes, the more likely things are to change along the way. And when they do, it can cause major delays.

There are a few things we can do to lower the chances of budget-breaking changes late in the development cycle. First and foremost among these is “iterative development”. In simple terms, we break the project into smaller, more manageable pieces. Instead of tackling the entire feature set from day one, we take things in bite-size chunks of 2-3 weeks in length. During each of these development cycles, we examine the most important features we’d like to implement, work with the client to make sure we’re meeting their needs, and finish by testing and reviewing the outcome of that particular piece. This lowers the risk of failure by reducing the feature scope, shortening the duration, and giving us a chance to quickly adjust to any changes as we go along.

3. Source code management

Dealing with project source code is a regular task for developers, but one that can lead to serious problems if not handled correctly. If you’ve ever collaborated with others on a single document, you’ll know that information can be lost if more than one user is editing simultaneously. The easy way around this is to restrict editing to one user at a time. For a large project however, this limitation is unacceptable. For the project I mentioned at the beginning, we often had 3 developers working with dozens of files at any given time, and regularly needing to work on the same files simultaneously. Coordinating with each other to make sure we had the latest versions and weren’t overwriting each other’s changes would have slowed progress to a crawl.

The solution? Source control management (SCM). SCM software stores source code in a simple database called a “repository” and tracks each of the changes made to the code as development progresses. Because the SCM is keeping track, it can intelligently merge changes from multiple developers within a single source code file. If something goes wrong, it can even help undo any changes made by mistake. There are several SCM tools available, but our favorite is Git. Git is a “distributed” SCM, meaning each developer has his own code repository that he makes changes to. When we’re ready, a simple command merges all our code into the master Git repository for the project. A wonderful benefit to using an SCM is that making quick changes to a project is trivial. Any one of the developers has full access to the latest copy of the project code, and any changes he makes are instantly available to all the others. An SCM is essential for any project involving more than one developer.

4. Centralized project testing

Another area that can often slow down project teams is the test environment setup. Each developer working on a project needs to be able to quickly test the code he’s writing, often in multiple browsers to ensure compatibility. The usual way to do this is to set up a local web server on every developer’s system. Aside from the time this consumes every time a developer is added to the project, it can cause further delays any time a dependency or server extension is required in the project code. This then has to be properly installed or upgraded on each developer’s machine independently. Having multiple server environments also increases the general amount of effort required to keep things running smoothly.

At Brighten Labs, we got tired of dealing with these headaches, and decided to set up a single, central server environment for testing. All the code being edited by our developers resides on this central system. By giving the server a public IP address and each project its own subdomain, a developer can even connect securely and continue working when outside the office. Any changes can be tested instantly on this subdomain address, without even being committed to our Git repository. This takes half a dozen steps out of the usual process, speeding up development and testing substantially.

—

With over 50% of larger projects (over 6 months duration) going beyond deadlines and over budget, we want to do whatever we can to make sure our clients don’t have to feel that pain. I’ve mentioned four of the major ways we’ve found to combat “large project syndrome”, but there are many other things that can help.

Do you have a story involving trouble during a large development project? Maybe you have some strategies you’ve discovered along the way? Feel free to sound off in the comments.

Cure flat forehead phenomenon in 10 minutes

If you get stuck on a problem in your HTML, JavaScript, CSS, or other code, stop banging your head on it and ask for help. With the wildly popular website Stack Overflow, it’s faster than ever to get the answer you need. Gone are the days when you have to slog through page upon page of mailing list archives to find, finally, that the problem remained unresolved.

The genius of Stack Overflow is that it rewards those who give great answers. Programmers race to answer a question faster and better, receiving “reputation points” when you accept their answer.

Many questions are answered within minutes of being posted, and some get their first response within a matter of seconds. In some cases, however, a question will go unanswered for a long time. Here’s how to get answered quickly:

1. Use the search box at the top of any page on the site to see if the question has already been asked.
2. Log in. OpenID is supported, so you can use your Gmail or Yahoo! account to log in.
3. Carefully write the title. Remember, people are racing to find questions that they think they can answer, so make it specific and concise.
4. Describe exactly what you have tried, and include code snippets. The person attempting your question should be able to use the code you provide to reproduce the problem on their own computer.
5. Format your code snippets. Indent the code four spaces, and it will appear in a gray box with syntax highlighting. Use the preview pane below the question box to check if your question is easy to read.
6. Describe the symptoms. Never, ever say “it’s broken” or “it doesn’t work.” Instead, describe precisely what you expect to happen and what is happening instead. Include all error messages and any other clues.
7. Help them help you. If an answer isn’t helpful, add a comment describing why. People are likely to revise their answer to make it more helpful if they think they can.
8. Ignore jerks. Resist the urge to comment on rude comments or answers. If an argument starts, people are more likely to join the argument, rather than answer your question.
9. Mark your favorite answer. The person who wrote it will then get a reward of 15 points, and you’ll get 2 points. Otherwise you’ll get a low “accept rate,” which reduces the chance that anyone will answer your future questions.

Easily test websites on the iPhone / iPad Simulator

Read this article if:

• Your client wants the website to work on the iPhone or iPad but you don’t own one yet.
• You’ve been testing on an iPhone but would like to speed things up by using a keyboard and mouse.

iPhone Simulator can simulate both the iPhone and the iPad, and comes with Mobile Safari. In this article, I’ll show you how to install it.

Before you start, make sure that you are running Mac OS X version 10.6 (a.k.a. Snow Leopard) – click the Apple icon on the menu bar, then click “About this Mac.” Unfortunately, I’m not aware of a way to run iPhone Simulator on 10.5 (Leopard).

1. Register as an Apple Developer (if you haven’t already)
2. Follow links to “iPhone SDK.” If you are already logged in, this link will take you there.
3. Click a link similar to “Xcode 3.2.3 and iPhone SDK 4” to download the SDK. It is a large file, so have your newspaper or knitting at the ready.
4. Once the download is complete, expand and install it.

Unfortunately, the iPhone Simulator is difficult to find. Here’s how to add it to your Applications folder:

1. Open Finder
2. Press Command+Shift+G and enter “/Developer/Platforms/iPhoneSimulator.platform/Developer/Applications/” in the blank. This will take you directly to the folder containing “iPhone Simulator.app”
3. Control+click on “iPhone Simulator.app” and click “Make Alias”
4. Single-click on the alias and press the Return key. Type “iPhone Simulator” to rename the alias and press Return.
5. Drag the alias and drop it on the Applications folder in your dock.

Now you can easily run iPhone Simulator from your Applications folder. For a great article describing how to use it, see Using iPhone Simulator from the iPhone Development Guide.

Please comment if these instructions don’t work for you.

Tip: ending the year right

“Bad customers will drain you of passion. Really bad customers will drain you of both passion and profits. Unfortunately, most bad customers will degenerate into really bad customers if you don’t do something about it.” -Dharmesh Shah

There is no doubt that having the wrong clients can quickly sink a company and put all its clients in a bad position. Here is an exercise to help you know which clients are winners and which may be bleeding you as you launch into the new year. (I recommend doing this in a spreadsheet.)

1. Make a list of your engagements over the past year
2. Beside each engagement enter:
   1. Revenue generated
   2. Total profit (total profit = revenue – ((billable + non-billable hours) * cost per hour))
   3. Profit margin
   4. On a scale of 1-10, how much you enjoyed it
   5. Whether this client has been a one-time buyer, repeat buyer, or better yet, an advocate (refers work to you)
3. Now sort the customers by each of the columns

You may be surprised at what you discover. Often I find I favor clients based on how much revenue they produce and how much I enjoy working with them. But profit margin is critical, as well as the amount of repeat work they bring or refer. If you’d like some help measuring your profit and profit margin, stay tuned.