Boeing, Boeing, Gone

I have mentioned a couple time in posts and in responding to comments that I believed one of the issues facing Boeing as to the Starliner program is that Boeing was too wed to cost-plus contracting. I came across a May 6, 2024, article by Eric Berger at Ars Technica that points out the same issue. The article is "The surprise is not that Boeing lost commercial crew but that it finished at all," and in it Berger observes:

    ... Boeing's space division had never won a large fixed-price contract. Its leaders were used to operating in a cost-plus environment, in which Boeing could bill the government for all of its expenses and earn a fee. Cost overruns and delays were not the company's problem—they were NASA's. Now Boeing had to deliver a flyable spacecraft for a firm, fixed price.

    Boeing struggled to adjust to this environment. Regarding complicated space projects, Boeing was used to spending other people's money. Now, every penny spent on Starliner meant one less penny in profit (or, ultimately, greater losses). This meant that Boeing allocated fewer resources to Starliner than it needed to thrive.

Berger later concluded:

In hindsight, it seems obvious that the strain of operating in a fixed-price environment was the fundamental cause of many of Boeing's struggles with Starliner and similar government procurement programs—so much so that the company's Defense, Space, & Security division is unlikely to participate in fixed-price competitions any longer. In 2023, the company's chief executive said Boeing would "never do them again."

But Boeing's size also worked against it. For instance:

    There was no single flight software team at Boeing. The responsibilities were spread out. A team at Kennedy Space Center in Florida handled the ground systems software, which kept Starliner healthy during ground tests and the countdown until the final minutes before liftoff. Separately, a team at Boeing's facilities in Houston near Johnson Space Center managed the flight software for when the vehicle took off.

    Neither team trusted one another, however. When the ground software team would visit their colleagues in Texas, and vice versa, the interactions were limited. The two teams ended up operating mostly in silos, not really sharing their work with one another. The Florida software team came to believe that the Texas team working on flight software had fallen behind but didn't want to acknowledge it. (A Boeing spokesperson denied there was any such friction.)

    In a fixed-price contract, a company gets paid when it achieves certain milestones. Complete a software review? Earn a payment. Prove to NASA that you've built a spacecraft component you said you would? Earn a payment. This kind of contract structure naturally incentivized managers to reach milestones.

    The problem is that while a company might do something that unlocks a payment, the underlying work may not actually be complete. It's a bit like students copying homework assignments throughout the semester. They get good grades but haven't done all of the learning necessary to understand the material. This is only discovered during a final exam in class. Essentially, then, Boeing kept carrying technical debt forward so that additional work was lumped onto the final milestones.

The result of this is that Boeing never ran integrated, end-to-end tests of the software for the entire planned 48-hour initial unmanned test flight. Consequently:

    Due to a software error, the spacecraft captured the wrong "mission elapsed time" from its Atlas V launch vehicle—it was supposed to pick up this time during the terminal phase of the countdown, but instead, it grabbed data 11 hours off of the correct time. This led to a delayed push to reach orbit and caused the vehicle's thrusters to expend too much fuel. As a result, Starliner did not dock with the International Space Station.

    The second error, caught and fixed just a few hours before the vehicle returned to Earth through the atmosphere, was a software mapping error that would have caused thrusters on Starliner's service module to fire incorrectly. This could have caused Starliner's service module and crew capsule to collide. Senior NASA officials would later declare the mission a "high visibility close call," or very nearly a catastrophic failure.

Problems also arose with the propulsion system, which Berger partially blames on a rocky relationship between Boeing and the propulsion subcontractor, Aerojet Rocketdyne.

    It's an interesting article, and all the more tragic because, being written in May 2024 before the launch of the crewed test mission, Berger seemed at least hopeful that Boeing had overcome its problems and would finally have a successful mission. But instead, there have been more propulsion and software issues such that NASA did not trust the Starliner capsule to safely return its astronauts to Earth and decided that they will now return on a SpaceX craft.