Two common, and defective, approaches in software architecture

Let me deviate—or actually, approach my target from a different angle—for two more postings before presenting one such "mundane notation" for software architecture documentation (which I have promised in my last posting).

What are the main problems with current (explicit) approaches to software architecture? Very briefly, they might be dubbed

• the "waterfall approach"; and
• the "moralistic approach".

The first one, "waterfall thinking", is the old idea that one "starts" with deciding on basic and important architectural aspects, and "then" goes on to design and write software accordingly. Some parts of software engineering might follow this pattern, but there are at least two major scenarios—or maybe forces—that lead to a different process:

• One is the fact that in almost all cases, a huge software is already in place; and the architectural problem is to modify this software "from inside out". This can and is often be done by small exploratory "experiments" in the software that prove or disprove whether some concept might be worthwhile. And in many cases, this is done implicitly and "under the hood", when some developer starts, on his or her own initiative, to introduce the first RESTful service, a "small NoSQL database on the side", or reuses some executable for production purposes that originally started out as a tool for developers only.
• The second scenario is brought on us by the typically vast capabilities of commercial and open-source frameworks or tools. When you buy SQL server instead of using Postgres (maybe for external reasons, like having a partner status with Microsoft); or when you take Angular instead of some lesser-known JS framework because some graphics library ties in better with it, you also "buy into" a huge feature set that comes with that tool. Your architectural possibilities are suddenly, and at the same time, extended by the tool's many for-free features, and also limited by the grand architectural and technological lines of it. And like your software, such a toolset is often "just there", without any possibility or even wish to ponder any underlying architectural requirements and decisions.

It is by no means clear that bottom-up approaches, as done by the hypothetical developer above, aren't on par or even better than processes that proceed from "grand architectural analyses" "down" to design and implementation. And, in real life, such bottom-up situations are unavoidable anyway. Thus, "waterfall thinking", while certainly an option, should not be the only and preconceived approach to architectural decisions.

The second problem is the "moralistic approach" to architecture (and design). Architecture and design decisions produce rules: "In our system, code on the GUI layer must access the database via an intermediate DAO layer"—or the other way round; "plugin registration happens explicitly by adding an entry to the configuration, and not implicitly by merely placing the plugin at some location"—or not. And somehow, such rules must be enforced. Most of the time, there are only two enforcement regimes in place:

• One is the "build-and-install-regime": The build and, later, installation processes of a software require that certain rules are followed. These rules are often implicit, but at least it is hard to violate them. It is also often very hard to change them.
• The other regime is the "moralistic one"—"you should", or "you must": Without support from tools, it is assumed that developers have the capabilities to follow the current rules. When, later, some disaster happens, one can more or less easily find a person that is the "culprit": "You shouldn't have added that trigger that implicitly calls itself and then fills up the audit table!", "You should not have hardcoded that connection string, but taken it from that (faraway) configuration file to keep database accesses consistent!" But of course, people will only adhere to some sub-100% percentage of rules—and this assumes that the rules are explicitly documented and consistent to begin with. And also of course, culprit-finding does not solve problems well (it might, in some cases, prevent others from violating the same rules in the near future). And finally, we are all versed in putting the fault on the shoulders of the ultimate culprit: "This has grown historically."

Both the "waterfall approach" and the "moralistic approach" are wrong in their fundamentals. But just by saying so, there is no positive alternative in place that replaces them. And, to tread somewhat more carefully, one should certainly not throw out top-down approaches (of which "waterfall" is a special case) and rules-of-thumb (an essentially "human-compatible" method for solving problems, just like "moral") from the portfolio of process building blocks for "doing software architecture": These are worthwhile at the, well, right places.

But some alternative view on "doing it" should be possible.