Software Engineering is all in the mind
Being a relatively new industry software engineering terminologies and analogies have come predominantly from civil engineering and manufacturing. Terms such as architecture, foundation, build, production, cycle time, lean… dominate.
This strategy serves the purpose when talking to someone outside the profession by helping them relate; and stops us software engineers from sounding even more nerdy than we already do.
However, it does introduce a problem when we use this language with junior software engineers entering the profession and even in general day to day professional conversation. This probably stems from the fact that our thinking is greatly affected by the language we use.
What’s the issue?
The problem with thinking in these semantics rooted in the physical world is assigning properties to software that it simply does not have and arguably should not have.
It is not that we do not want our software, for example, to be able to weather storms as you’d expect from a building or a bridge, but we want it to weather these storms by bending and twisting rather than standing immovably and inflexibly. For the simple reason that it is a better strategy and software unlike buildings can be made to behave that way; but our thinking does not allow it.
Bigger buildings tend to be more desirable than smaller buildings because you get economies of scale and a statement of stature, concrete better than straw, ornate and decorative over plain and simple (depending on your taste).
In software almost invariably the opposite is true. The bigger the software the more it holds you back. Decisions set in concrete cause total write offs. Clever and magical code always needs lots longer to understand and maintain.
The insight
Software engineering as a discipline shouldn’t be expressed in the language of the tangible — or if we do it has to be made evidently clear where the analogy stops working.
The fundamental reason for this is that unlike with disciplines rooted in the physical there is no manufacturing or production element to software. The act of typing or the “build process” is almost never the issue at hand.
The issue is that of imagination.
If I can imagine a software solution then making it real is trivial and instantaneous. Hence the claim that software engineering is all in the mind.
Really?
If you have any experience with software projects then you will be aware that they are almost always late with severe cost overruns. The world (particularly the enterprise world) is and continues to be littered with digital transformations gone wrong. How do we reconcile this fact with the claim that with software if I can imagine it then it’s already there.
Firstly, imagining a software solution is actually really hard. Humans no doubt are creative — finding innovative solutions to a problem by applying experience and knowledge, but will always struggle to think through every facet of a solution particularly in a complex environment where emergent properties dominate.
This is the real challenge which triggered the agile movement through the nineties. The simple but highly effective idea with being agile is to continually test our imagined solution against reality so we can learn and adjust our thinking on a regular basis. These principles are the best we have in thinking about the nature of software.
The real problem that we continue to face with enterprise software is the wasted time and effort that goes into maintaining and developing software because our thinking paradigms are rooted in the physical world. The typical SDLC (Software Development Lifecycle) is a perfect example of it.
We spend time asking users what they want instead of building what we think they need and involving them in the process; only to find out what they told us they wanted is not quite what they meant or we didn’t quite understand.
We spend time waiting. Waiting on approvals for hardware, approvals on design, approvals on release — hierarchical models of compliance enforced by manual approval processes.
We spend time satisfying enterprise architects so their box diagrams look neat, following lengthy and convoluted processes to get work done, but not questioning the why of it and the arbitrary decisions that have been made all the while delivering no value to the users.
We do not keep going back to our software to see what we can simplify and remove since the last time on the basis of what we learnt, rather we continue spending time debugging an increasingly complex solution and adding features that get progressively harder to develop every release.
All this is likely the impact of our model of thinking. You cannot start building works without a blueprint detailing every wall and light switch, because the foundation once cast cannot be redone. A sourcing contract with a vendor cannot be easily modified. One cannot decide to add a balcony at the last minute when you realised that the view is so compelling. You cannot redesign the floor plan after construction has started because you got inspired watching a design show on TV.
You can do all this in software. With SaaS (Software as a service) and other such offerings you can choose to switch a vendor as long as you imagined that option and designed it in. You can extend to accommodate a last minute feature because you left the seams visibly in there. You can perform a great redesign after you start coding because you have automated tests.
In other words the properties of software allow the unthinkable to happen with relation to analogies in the physical world. So, why do we constrain ourselves and waste valuable resources?
What’s the alternative?
The principles of agile software development need to be continually revisited rather than giving it lip service as we do today. We have reduced things down to “doing agile” — following mindless processes, frameworks and methodologies that have agile terminologies such as scrums, sprints, retrospectives, kanban, velocity to name just a few.
What is required really is an intuitive understanding of the nature of software development. And with that reject the practices that don’t fit and equally adopt practices that are essential.
Because developing good software needs time. Time for the people involved to absorb the domain in which the problem being solved sits, to absorb the technologies required to solve the problem, to learn from trial and error, to create and innovate, to engage and empathise with the end users, to get the structure and seams right, to continuously improve until it’s acceptable and then keep going until its expiry.
All the time spent not advancing the needs above steals valuable resources and often can do great harm eventually leading to the cancellation of the initiative altogether.
Be conscious of these dynamics, guard against the dangers and nurture the right behaviours in your next software development engagement.
