A Disaster Rollout

With the rollout of, technology has officially taken center stage in the healthcare industry. It may seem only natural now, but ten years ago - when I created what would eventually become FormFire - paper was still very much the only currency in use, at least in the small group market. Over that time, we've pushed, clawed and fought our way from tolerance to acceptance to being even the preferred way of working in the markets we serve. So when President Obama made technology a key part of his plan to make the Affordable Care Act a reality, I knew the industry was about to come of age. And regardless of one’s views on the Affordable Care Act itself, this was a good thing. Imagine booking a plane ticket without Travelocity or Orbitz. I've been waiting for years for the country to have the same view of healthcare. Unfortunately, the rollout has been a disaster.  Users who've attempted to access the site have been met with error pages and delays, and the PR mess surrounding hidden (actually just commented out) code in the terms and conditions didn't help. So what went wrong? is an exceedingly complicated system and the answer isn't easy either, but I’ll break it down into its most basic components as I see them:

1: Too Many Cooks in the Kitchen

I once heard a joke: What’s the only thing worse than design by committee? The answer: Design by Government committee. is the work of more than 55 contractors. That’s a lot of people across a lot of organizations to manage. Unfortunately there was no single contractor overseeing everything until mid-October. Prior to that, Centers for Medicare & Medicaid Services (CMS) attempted to manage the project. At FormFire, we use Agile development methodology. This allows us to be flexible with changing requirements while ensuring development doesn't grind to a halt. We also have organized discussions about which features make the cut. This helps discourage scope creep and unnecessary last minute changes. Clearly was not beyond last minute changes. Despite originally not requiring users to create accounts to view prices, contractors were told one month before launch they wanted all users to register first. As a software developer, I can say this is the type of things that will kill a release date, if not the project as a whole.

2: Too Little Testing

This one is big. We learned the value of extensive testing first hand when, early on, we couldn't afford a dedicated Quality Assurance team. Sure we tested the really important stuff like permissions and security-related features, but unit tests? A formal Software Development Policy? User Acceptance Testing? That was the domain of big companies with deep pockets and government agencies. Or so we thought… User feedback is the single most important influence on our development. After all, why design a system for users that doesn't work how users need it to work? To this end we allow focus groups to preview significant changes before going into to production. We also regularly survey our users after the fact to make sure we hit the mark. The government didn't perform any end to end testing until mid September, two weeks before launch. I can only imagine what a user survey now would look like. (By the way, I am one of those users. I even completed the Navigator training but couldn't finish because the identity proofing did not work.)

3: Bloated, Poorly Written Code

If we’re to believe one source, is estimated to contain about 500 million lines of code or about 10 times the amount of code in Windows XP. That’s… a lot. But a lot of code is not necessarily indicative of poorly written code.  Unfortunately everything we can see does indicate the code is written poorly and probably just bloated. For example, one researcher has found that there are over 1,000 resource calls just to load a single page. If you’d like to see a sample, click here.  This is an actual file from While you’re there, search for “lorem ipsum”, a filler text often used by programmers as a placeholder. You’ll find it 14 times. Where’s the copy that’s supposed to be there? To put it in another perspective, the registration file loads on the signup page, which in total loads about 2MB. FormFire’s signup page is one tenth the size at 200KB.

4: Too Much Money and Resources

Yes, everyone involved in the development – the contractors, government agencies, everyone – had too many resources to work with. Perhaps it’s more accurate to say there was no incentive to be lean or efficient. Writing sloppy, inefficient code? Add more servers.  Have a request to add a non-essential feature? Bring in more developers and do it. As a small startup, we had to weigh the plusses and minuses of every feature or line of code we built into the system. Adding more servers was a quick fix, but given our growth, we would still outpace our servers if we didn’t pay attention to writing quality, efficient code. In the end, this forced us to develop a discipline which serves us well to this day.

5: Lack of Infrastructure Planning

This is another lesson hard-learned. Our business has distinct annual usage cycles with the busiest occurring from late September through early December. This is when our usage triples and on a couple of occasions we got caught on the back foot because our infrastructure couldn't handle the demand. By now you probably see where I’m going with this. The government said it didn't anticipate the huge volume of usage it would see when the system launched. This is a lame excuse. Either through policy (staggered rollout) or technology (more servers or cloud capacity) they should have planned for the worst (or best?) On the first day open, only 6 people were able to enroll in coverage through Fortunately, things are looking a lot better and after some fixes, the site can supposedly now handle up to 17,000 people per hour. The number of people who have actually enrolled is to be released mid month.


Certainly every technology endeavor makes some of the same mistakes – as did FormFire. But we did not make them all at once. If we had, we’d be out of business. Instead, we had the benefit of growing organically and learning from our mistakes and are a lot wiser because of it.