Are modern enterprise software architectures doomed to produce suboptimal processes and outcomes? Today, enterprise architects value componentization perhaps more than ever before, given the mass glorification of microservices. Microservices are loosely defined as isolated, independent components designed to address a singular business need. Sounds great, until you consider that with this architecture, the creator(s) and the consumer(s) of any service are likely to become rigorously isolated from each other, the API boundary falling like an iron curtain between them.
Studies have found a “mirroring” effect between organization of code and organization of teams. Some have argued that the organization of teams causes organization of the code – an idea typically frequently referred to as “Conway’s Law.” I submit that this causation is often false, and the reverse is more likely true. An enterprise application does not have a database and a UI because the organization developing it has a database team and a UI team. More likely, the organization developing the application has a database team and a UI team because the application requires a database and a UI.
In her phenomenal book Production-Ready Microservices, Susan Fowler calls this principle “The Inverse Conway’s Law.” “Inverse Conway’s Law… means that developers will be, in some ways, just like microservices: they will be able to do one thing, and (hopefully) do that one thing very well, but they will be isolated (in responsibility, in domain knowledge, and experience) from the rest of the ecosystem.” And this is where the highly compartmentalized world of microservices leads us into trouble.
Let’s consider a simple scenario. One microservice is used by two business applications. The first application is used internally. It is available only to certain privileged users inside the company and only within its firewall. The second application is exposed to the world at large and accessible anywhere. Now, suppose the development team that owns the microservice scans the open source components of that microservice for vulnerabilities and finds a vulnerability of medium, but not high, severity. The internal business application can likely tolerate that vulnerability, at least in the short term while a fix is pending, because the scope of access to the application is sufficiently limited. For the externally available business application, an attack can be attempted without penetrating the firewall, and any vulnerability is thus far more likely to be exploited.
In the scenario above, should the team developing the microservice be the one responsible for reconciling the conflicting needs of its two consumers? If the answer is yes, then the loose coupling that makes microservices so alluring to architects is violated. The producing team must become tightly coupled to the usage needs of its consumers. Is the business need for the internal consumer so high that the update with the vulnerable component should be deployed immediately, denying access to the external consumer until the vulnerability is remedied? Or should the internal consumer be forced to wait until the vulnerability can be addressed? Of the three parties in this interaction, either of the two consumers stands to lose depending on the outcome of the decision, yet the organization we’ve proposed renders both consumers practically mute.
Consider how this compartmentalized approach contradicts the values of the famed Toyota Production System (TPS), which for decades has been regarded as the gold standard of manufacturing processes. Where TPS values “seeing for oneself,” going directly to the source (Genchi Genbutsu), compartmentalization actively discourages inspection of one team’s work by another. Where TPS values consensus, compartmentalization favors tribalism, with the API being the only facet lending itself to cross-team discussion. Where TPS famously enables any employee to stop the entire production process if s/he discovers a flaw, compartmentalization makes each step in the assembly of an application opaque to those working on the subsequent step, hiding flaws from discovery.
Perhaps a change in organizational thinking is in order. In Competing Against Luck (semi-required reading here at Black Duck), Clayton Christensen et al. propose organizing teams by the job they do for their customers rather than by any other principle.
“Through a jobs lens, what matters more than who reports to whom is how different parts of the organization interact to systematically deliver the offering that performs customers’ Jobs to be Done. When managers are focused on the customer’s Job to be Done, they not only have a very clear compass heading for their innovation efforts but they also have a vital organizing principle for their internal structure.”
Thus, in a job-focused organization, a team developing a single component toward one or more larger jobs cannot be isolated from other teams contributing to accomplishing the same job(s).
Does this mean that microservices must be put to pasture and enterprise development must return to the age of the Monolith? No. But aggressive and proactive efforts are required to prevent componentization from devolving into compartmentalization. These include:
A microservice architecture does not guarantee cross-team dysfunction any more than its absence guarantees ideal cross-team function. However, it does introduce forces that decrease the ability of job-oriented teams to ensure that the enterprise components they consume are fit for their respective jobs. These forces must be combatted viciously and rigorously. Otherwise, customers, internal or external, will lose out.
Yev Bronshteyn is a senior software engineer at Black Duck by Synopsys and a member of the SPDX Working Group’s technical team. His past engineering positions include eBay and Progress Software.