Is there something wrong with software architecture - or with us?

I am a software architect (one of a few) for a 20-million LOC business software, with currently a few thousand installations, developed and delivered in a Scrum process with monthly iterations, a development team of more than 40 developers, analysts, and testers. The software is in a reasonable state—we have no problems delivering new features, we have no problems integrating new team members, we envision major overhauls of the software (ranging from virtualization to running the hitherto GUI-driven application server as a SaaS system and integrating more DWH functionality into a database designed for a pure—and partly time-critical—OLTP load).

We do not have a valid software architecture documentation.

We have, I think, a solid software architecture: We are able to immerse new team members into our culture in a way that keeps our main architectural constraints in place—layers, tiers, synchronous and async calls, normalized and denormalized data, communication with central systems about licences and failures. The performance, the usability, the stability of the whole system has never been a fundamental problem: Standard design and implementation activities have kept everything in place more or less nicely: Adding covering indexes to critical database tables, rewriting small as well as large chunks of business objects—and their underlying database structures—or swapping out complete workflows with their GUIs, even when they were referenced from other parts of the system, was "work to be done", never a "crisis".

But we do not have a valid software architecture documentation.

And we are not happy about it, because we would like to base some upcoming decisions on both

• what actually goes on in our software (might it be that the application server actually calls, at some obscure place, some client-side software directly? or can we rely on the fact that this does not happen?)—what is commonly called the descriptive architecture; 
• and on our rules of our software architecture (did we actually agree, 10 years ago, that we will forever only access a single database behind the application server?)—what is sometimes called the prescriptive architecture.

At one time, we had a project Wiki. It worked somewhat nicely for maybe 3 or 4 year of our 12 year product history, then one of the main contributors left the company, and the other one—me—got more productive, i.e., wrote more code. The Wiki, which was designed in a somewhat hierarchical fashion (it always had a large catalogue of "TBD" documents), was more and more referenced as "don't read it, because it's uncontrollably wrong as well as right". We tried to revive it half a year ago, with let's say small success, even though we still remain hopeful—but this is a separate story.

What should we do?

I started reading about software architecture documentation, and software architecture in general. And I found the problem. But the solution will be hard, and most probably require serious out-of-the-box thinking. I will tell you about the problem in the next postings.

Let me start with what I learnt from "the books"—two of them, actually, and a host of articles I found on the web.

The first book I started to read was "Essential Software Architecture" by Ian Gorton, which was published in 2006. It is a good book—read it (it is somewhere on the web, whether legally or not I do not know), and take everything away with you that you didn't know, or saw differently before.

• I agree with everything it says in chapters 1 to 3.
• Chapter 4, a guide to middleware technologies, is already outdated—not so much because the described technologies have evolved so much, but rather because they have been superseded, mostly by the cloud and the REST revolution.
• With chapter 5, I disagree: This is a disguised waterfall process. Unfortunately, this is still the mainstream view in the software architecture literature, sometimes a little beautified by talk about agile cycles, but without real depth. Writers in the field will probably disagree. I will make my case in later postings.
• Then comes chapter 6: "Documenting a Software Architecture". This would be it. But it wasn't. And then there is a "case study" in chapter 7: Interesting—we should expect that if we do as is done there, we would be perfect, or at least very good. I'll look at this shortly—but let me go quickly over the rest of the chapters, to find whether there is something interesting in there.
• With Chapter 8, a second part is introduced which collects contributions for various topics from different authors. From my (probably limited) perspective, they are not relevant: Product lines in chapter 9 may be interesting, but are, in my humble opinion, just a special case of restructuring software in the long run. Aspect orientation, in chapter 10, has not fulfilled its promises—or it has been integrated in all sort of frameworks so seamlessly that we didn't notice that it has already arrived; anyway, it is, as of today, not relevant as a major architectural concept. Model driven architecture, in chapter 11, is equally absent from industrial practice. SOA, in chapter 12, is in some sense here: With web services, in the cloud, and then with micro-services, we have services everywhere. The "web service standards" shown on p.226, on the other hand, seem to lose ground: XML (still strong) declines against JSON; messaging and especially reliable messaging is nowhere seen, as are (distributed) transactions; on metadata, I do not dare say anything; but WSDL support for typical B2B services is sketchy at best. The semantic web, from chapter 13, has not arrived. And also chapter 14's software agents are a niche architectural style, at least in all IT systems I know.
• The fascinating future of chapter 15, at least, has arrived: But, when viewed from a decade after the publication, more in the sense that on the one hand, many concrete technologies and the architectural patterns and styles they were based on or promoted, have been defeated by completely new ones, for which the old diagrams do not really work. But on the other hand, this fascinating future has, as far as I can see, not brought much progress in the wide-spread use of distinctly architectural techniques and methods in many many software projects ...

... which brings me back to my original question: Is there something wrong with software architecture—or with us? ... and to chapters 6 and 7 of Gorton's book, about documentation of an architecture, and a case study.

Let me tackle these in my next posting.