There is a set of attributes that describe the organizations I have been asked to lead.  All of these organizations have been associated with a Federal Agency and have a science-based mission.

I will define the organizations, which I managed or served as a change consultant, as a “performing” organization. They are expected to deliver a science-based product for customers. The products are computational models and simulations or data sets. The customers are researchers or application specialists, for example, weather forecasters.

The performing organization is embedded in a much larger organization, Federal agencies – bureaucracies. Though important functions of the Agency, they are only a small part of the Agency’s portfolio, and they are not the most important part of the Agency’s portfolio.

In all cases there has been a desire by “management” to develop a new and expanded capability that would have outcomes of consequence on Federal and National levels. If an Agency is divided, coarsely, into an Executive layer, a Program layer, and a Performing layer, the motivation for organizational change often comes from the Executive layer, perhaps, with strong input from external members of the community. The target of their interest is likely a Performing Organization. The Program Offices may or may not share the Executive’s enthusiasm for change.

The change model being analyzed is Executive management wants change in a Performing Organization. Program managers may or may not want this change. It is unlikely that all of an Agency’s Executives and Program Offices will share a common vision.

With regard to what Executive management wants to achieve, there was relevant existing capacity that provided obvious building blocks for the new capability. However, this existing capacity was viewed with doubt as the right “place” to invest. The doubt was for many reasons.

The existing capacity was fragmented and insufficient. This is a natural outcome of how the efforts had been funded over the years, often to individual principal investigators for specific narrow purposes. The principal investigators had been successful because their ideas were innovative and led to successful research outcomes – often proof of concept and journal articles. A consequence of this research-funding model is that principal investigators and their teams are focused on their individual part of a bigger problem at the expense of other parts that might be important. They were embedded in organizations with peers in a similar field. However, the desire and skill set to contribute to organizational goals was limited, often, only to opportunistic collaborations.

More problematic, however, was that the existing capacity was isolated in ways that nurtured substantive criticism. For example, an effort within NASA might have had years of success within NASA. With the desire to develop a capability of potentially National significance, the research methods and outcomes were open to broader review. In many cases this broader review would raise questions about the state of the art of the NASA research, and hence, whether or not the principal investigator’s ideas were the right ones to follow.

The isolation of the research activities had many other consequences. For example, publication rates were often weak. There were statements that the research being performed was so important that there was no time to publish.  Similarly, taking time to publish would take away from innovation.  Publication and review, however, are fundamental parts of the practice of science. If the practice of not publishing becomes too present in an organization, the organization risks becoming scientifically irrelevant. In addition, the lack of transparency erodes trust in the organization.

Another consequence of the isolation was that it is likely that the research tools of an isolated organization will become outmoded. When I assumed a leadership role of data assimilation at NASA, a common criticism that I heard was that the results coming from NASA showed great impact in NASA applications. However, these results were not being realized elsewhere because the NASA tools were not state of the art, and hence, the impact was exaggerated.

In short, the isolated nature of the existing organization had created an echo chamber of perceived excellence that needed to be disrupted. There are both structural and cultural changes needed, and the two had evolved together, sometimes over decades.

Though the existing organization was isolated and, often, quite comfortable in their isolation, there was usually discomfort by some members within the organization. In all cases, on assuming a formal leadership position, I would interview all, if possible, members of the organization.  There were usually three large groups.

The first group, generally older, was skeptical of the push to form new capabilities and to expand the mission. They felt that providing “products” for “customers” would damage “the science.” They were skeptical, therefore, of the fundamental desires of Agency Executive management. They were, also, heavily invested in the existing organization. They were conservative in the sense that the organization needed to return to the simple things that had worked in the past. That, usually, required dismissing the desires and roles of  “management,” and years of practice had made them quite adept at avoiding strategic programmatic efforts.

The second group, usually younger, was anxious to see change. They frequently came from research environments outside of NASA, and they were familiar with new tools and methods. They saw excitement in building new capabilities and the expanded mission.  They were frustrated with existing management.

The remaining group was the messy middle, who would consider change, but they needed evidence that both change could be achieved and that there would be benefits to that change.

These groups were often relatively evenly split, 1/3, 1/3, 1/3.

These three groups suggest that there is substantial resistance in the organization to change. The group of older people, who are heavily invested in the organization, will resist change. To them, the current situation is what they built and it is working. They defined the organizational culture, and they were the traditional power holders. Likely, they were vested in some of the organizational attributes or pathologies that, on the surface, seem broken. For example, fragmentation; they had learned to thrive in that world, to get what they want and to succeed.  This suggests that a time span of years, perhaps related to a significant fraction of a person’s career, is needed in order to change the culture of the organization. The time span is a substantial fraction of a generation, order, 10 years.

A question arises: why try to change such an entrenched organization, rather than demolition and starting anew? There are many reasons that motivate incremental, disruptive change rather that tearing it all down and rebuilding. Often, the existing organization has a mission that is critical and that must persist. The existing organization owns the expertise; that is, there is not a ready pool of new employees expert in the relevant field. There are personal relationships with the Program managers, and there is the common reluctance to throw away a history of investment.  Tearing down an organization is very high risk and beneficial outcomes with, largely, the same people are far from assured.

Though there is substantial internal opposition to organizational change, there are also forces to promote change.  There are the internal interests in organizational change, often represented by the newer members of the organizations. In addition to the internal interests, there are the external reasons for change represented by the Agency Executive managers.  These factors motivating change are amplified after the isolation of the organization is challenged; that is, there is peer pressure.

It is in the context of disruptive, incremental change that the experiences and lessons of my experiences are described.