Importance of Conceptual Integrity in System Design
wiki.c2.comI agree with the main point that conceptual integrity in system design is important, and it's a great article over all.
However, I really think that the immediate examples given by the author are actually not great examples of conceptual integrity, because the author too quickly conflates "conceptual integrity" with "consistency" (note the repeated use of "everything is a" in the examples given):
Smalltalk ("everything is an object", and the small set of other accompanying principles)
SQL ("all data is in tables", with keys and constraints)
Lisp ("everything is a list")
I would even go as far to say that an over-emphasis on simplistic consistency speaks more to the lack of a genuine concept, or as the classic quote has it: "consistency is the last refuge of the unimaginative".
And committees love consistency because it's a cheap sell.
On the other hand, conceptual integrity just means staying true to the concept, however that is defined, so that conceptual integrity could well mean a break from simplistic consistency where the concept demands it.
If I can give it a shot, I would say that the human body is a better example of conceptual integrity in system design: not everything is a head or a hand, not everything is a foot or a finger, not all fingers are the same size, and yet everything works together. The conceptual integrity lies more in the overall symmetry of the human body, its purpose, its functions, its senses, motion, perception, thoughts, speech, actions, and especially in its interdependent relationship to others in community, as well as everything else in the world.
the paper "a city is not a tree" is a perfect example of the fallacy and makes a point of the complexity of systems and the fallacies in systems thinking. The parallel is very useful to illustrate the stupidity that is "smart cities" engineering (& marketing). System thinkers envision highly complex "systems of systems" as tree-like structures (in the CompSci/Mathematical sense). It applies to all systems and models of systems though.
A city is not a tree http://www.bp.ntu.edu.tw/wp-content/uploads/2011/12/06-Alexa...
Thanks, Christopher Alexander is great.
Do you think there is any long-running complex system that does not have conceptual integrity? For example, does a city have conceptual integrity? Does a country have conceptual integrity?
Conversely do you think there have been any failed and therefore short-lived complex systems that did have conceptual integrity?
Everything in the human body is a collection of cells.
This is not true. Your extracelluar matrix is vital for your survival, as well as the gases dissolved into it and in your lungs. Your bones, while they have stem cells inside and have osteoblasts scattered throughout an extra cellular material. Do the cells build all this? Yes, but not everything inside of you is strictly speaking made of cells.
I knew my post had a 100% chance of a follow-up "aksually!". Thank you for the irrelevant information.
Or you could admit you spoke incorrectly like an adult and move on.
I didn't speak incorrectly. You didn't understand my point and apparently not the GP's either. Diving into the construction of the human body isn't relevant to a discussion on design philosophy weighing homogeneous vs. heterogenous constructs.
"But the body isn't homogeneous! It's just mostly homogenous" is a splitting hair argument even if I'd believed you'd kept the plot.
It is not mostly just homogenous as previously described. It is a hefty mix of cells and extra cellular material, or are you going to argue that hair is made of cells too and that distinguishing between alive and dead cells is splitting hairs?
The fact that you even acknowledge it is splitting hairs indicates that you are aware that your analogy is flawed as you can't really think of the body as just cells and that is in direct contrast to systems we discuss in which everything, not mostly, are a certain way. Your body is not mostly cells in the strictest of senses. It is lot of non living material that is a mix or organically made structures as well as various gases, liquids and solids.
I'm not making the analogy; I'm pointing to how GP's analogy fails to demonstrate their point.
But yeah, the analogy isn't perfect. And the academic nature of the metaphor's subject was obviously going to invite some rube to point out how it's not 100% spot on and how that's well worth talking about.
You're just arguing my greater point anyway: Your chosen level of perspective with these metaphors (and real issues) will invert the design philosophy. At one level, everything is different and using different means for the same goal, at another level, everything is the same and using the same means for different goals. If it's not organs, then its cells, then its "extracellular material and are dead cells different from live cells!?", then its atoms and then quarks.
But please, go ahead and educate me on how atoms aren't all the same and how that too is a critical point to be made in this conversation.
Rube? Educate you? You do realize why people are off put by you? I'm happy to engage because I was raised like this. Most people downvote and move on. That is one issue here you need to resolve.
The second, which speaks directly to the concepts talked about, is the idea that "everything in the body is a cell" is a poor analogy to use to try to articulate your idea either supporting or not supporting the original posts concept, and you got salty when called out on it rather than have people simply agree with your "amazing insight".
PS I love this game.
Your imagined version of me and this conversation is very colorful, but its luster seems to be cooking some neurons.
That's OK. Take a break. Relax. When you can think clearly, come back and we can finish our discussion of why "everything is a cell in the human body" is still, in fact, a bad analogy.
I feel like this is downvoted so harshly with only a single reply at this writing because there was some sentimental value to grandparent's post, but their analogy was fundamentally broken when they started talking about the design of human life from the high-level perspective of Aristotle's definition of a living thing as "being-at-work-being-itself", essentially.
More importantly, the perceived conceptual integrity of the human body is rather philosophical was put to question by Roger Penrose in his JRE episode where he talks about why the distribution of motor functions makes no sense ("the foot and walking are over here", "running is all the way over there", to paraphrase) all and we likely need a new physics model to explain consciousness as it arises in the physical brain.
Biology moved on from such an Aristotelian perspective for good reason, to slightly alter something Spinoza once said, we don't know what a cell can do. Our physical being is a radical aggregate that we don't even fully understand yet.
Even with a single mind having the final say over the design, it is still easy to violate conceptual integrity in software if it directly or indirectly touches the end user or “real world” in general. In this case, requirements or their understanding will inevitably change, and any time that happens in order to maintain conceptual integrity software has (effectively) to be redesigned[0].
If you’re lucky, the result of that redesign might turn out to be close enough to what you currently have[1]. What’s dangerous is skipping that taxing redesign process altogether, or losing sight of the new target while you strive to maintain software built on the now-outdated conceptual model in order to address users’ current needs faster. Keeping in mind the updated model and evolving the whole system towards it while continuing to provide functional software to existing users at all times is probably the most difficult aspect of building software for the end user.
That said, I agree that originating from a single person (or indeed very few “agreeing resonant minds”) might be a pre-requisite for cohesive design, and I think “conceptual integrity” is a very fitting term.
[0] This might sound drastic, but this is my belief so far. Slightly expanded on it in https://news.ycombinator.com/item?id=25408650.
[1] I am tempted to say that the “tighter” the design (the more tailored the solution is to the problem), the more likely a small change in requirements is to affect the product in a major way once you rethink it.
Speaking of conceptual integrity, wiki.c2.com is the original wiki, and it's still chugging along 26 years later, as of next month => https://en.m.wikipedia.org/wiki/WikiWikiWeb
Take that, all y'all whipper-snappers!
Love this quote: "Conceptual integrity in turn dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds"
That notion of resonant minds seems very important
Conceptual Integrity is hugely important, but a system architect is neither necessary nor sufficient for achieving it.
One person's system designs can lack conceptual integrity as easily as someone can make a plausible logical argument which lacks conceptual integrity.
If a design is to maintain conceptual integrity, the people who maintain it are responsible for it. That means everyone.
If everyone is responsible then nobody is.
This is true in situations where there is no social contract for the event at hand, like emergencies and other one-off situations. It doesn't apply where there is a social contract. Everybody is responsible for obeying the traffic laws, for example. And everybody can be responsible for maintaining conceptual integrity.
I think it applies here. The problem which conceptual integrity is that short-term incentives oppose it. A developer gets personal benefits for shortcuts (Yay, a 10x rockstar developer!) while the costs are shared by all. A classic tragedy of the commons situation.
Maybe you assume that random developers will speak up, point out an inconsistency, and everybody agrees to make it consistent from now on. In a small team it might work that way, but "system design" implies a larger community to me. Consensus is hard.
The whole point of responsibility is that blame and praise is focused. A focus on everyone is no focus at all.
> Everybody is responsible for obeying the traffic laws, for example.
Well, we do have traffic police and courts and lawyers.
Yes, everyone should follow the rules. But the point of having an individual responsible for conceptual integrity does not mean, no one else should care and do whatever they want, but that there is, if needed, "police and judges" in place...
But there is another point: the fewer people the easier it is to have a consensus about what the conceptual integrity entails. If you have a single judge, you are more likely to expect consistent rulings, so to speak.
So I've always felt that this is true not just of technical architecture, but also of companies and products. Does your company's strategy have conceptual integrity? Can employees and investors understand it, and is it congruent? Same for your product. Do users understand the information architecture?
That said, I've definitely worked at some pretty successful places that seem more like organized chaos. I think it's just easier and more fulfilling to work on products and at companies that have that conceptual integrity.
Agreed. One of the most difficult things to get right is conceptual integrity from the product perspective. Bad conceptual integrity leads to poor product requirements, which leads to unclear technical requirements, which leads to poor engineering and technical debt.
Thank you. I've just spent weeks trying to explain this.
Brooks’ book seems like it is percieved as a software engineering classic but it’s mostly about the management side of things.
Companies are very hierarchical so of course it makes sense for managers that most things should ideally come from one mind or a few agreeing minds. Then the average developer just have to do the grunt work.
So why is the book popular among developers? Because there is an elite cadre of developers in Brooks model which do have autonomy. I guess that cadre might also be known by other names.
[1] And the big man-month lesson of the book should be much more obvious now than it might have been back then: as you add more cooks you also have to take into account the communication overhead among all the new nodes.
It is a software engineering classic. Just like “A.I” describes an entire field within computer science, so does “Software engineering”. It’s the part of computer science that deals with managing projects, quality and people, among other things.
In the real world, software engineering is some what more loosely defined and encompasse s things like programming, system design and so on.
> the architect(s) must be egoless
Yeah, and 'the bird(s) must be wingless'. Whoever wrote this is not an architect, nor has ever met any real architects worthy of the label. Architects are driven by an overwhelming imperative to 'order' everything around them.
Also there is a reason we have "egoes". It's not a useless function of the psyche. The problem is with problematic egoes, fragile egoes, over-reaching egoes, etc. A strong, healthy, ego servers a respectable purpose.
Everybody considers themselves a designer, but it is baseless optimism that is supported by little data or precedent. Good designers are unique. Design ability is a 'talent'. Being able to conceive a complex system that maintains "conceptual integrity" is not an ability that everyone has, nor is it show (yet) that is even teachable!
Now an "egoless manager". That is a topic worth discussing. That species was never intended to take to the skies. /g (Bad managers let their egoes get in the way of actual talent, again and again. They should just manage the process and leave technical design for the architect. :-)
Also, yes it's true, you don't always need an architect. Most systems are an instance of a handful of widely repeated system patterns. What most teams need is a bit of maturity to distinguish and select the appropriate ready-made blueprint for their project. It is highly unlikely that your project requires a 'conceptual' breakthrough or innovation.
> Everybody considers themselves a designer, but it is baseless optimism that is supported by little data or precedent. Good designers are unique. Design ability is a 'talent'. Being able to conceive a complex system that maintains "conceptual integrity" is not an ability that everyone has, nor is it show (yet) that is even teachable!
It's also talent that is hard to recognize and can be valued very differently from one organization to the next. I've been in the room with developers and management looking cross-eyed at me as I walk them through a top down business/problem decomposition and system design questioning the value of the entire exercise. Not surprising, these are organizations which have extremely brittle systems full of overlapping and "hard to reason about" abstractions.
I've had that experience on multiple occasions. Initially, when younger, I did attribute it to myopism. Today, I recognize that credibility also needs to be earned. All architects are unhappy except those who manage to find the right client. The client-architect relationship is fundamental. A successful architectural undertaking is minimally a two variable equation. Client (aka stakeholders) are hugely important.
(this is related imo to your post: https://brooker.co.za/blog/2020/10/19/big-changes.html - )
On a related tangent, I've been re-reading "The Open Hand - Essays on Le Corbusier". Specifically, the "Le Corbusier at Pessac - Professional and Client Responsibilities" essay by Brian Brace Taylor.
https://www.worldcat.org/title/open-hand-essays-on-le-corbus...
Pessac: https://www.archdaily.com/940878/le-corbusiers-cite-fruges-l...
That project basically failed its intended purpose. The client, M. Frugès, was apparently an enlightened capitalist, willing to invest in the vision of the young architect, even when incurring financial losses, because of Pessac. Corbu also learned, on the job, how to, and how not to, build his modernist vision. The whole thing was a fiasco on many levels. But it was a necessary project for progress in the field.
Which is the point: architectural innovation is costly and even great architects leave a trail of failed early projects. Yes, Corbu went on to master his field, and he did deliver (opinions of course vary) on the program of modern, afforadable, mass housing for the urban working class:
https://en.wikipedia.org/wiki/Unit%C3%A9_d%27habitation
But then:
"You know, it is life that is right and the architect who is wrong." - Le Corbusier (!)
https://libquotes.com/le-corbusier/quote/lbz6x3v
It's a problematic vocation (bricks or bytes). Love it or leave it.