Why Type-safe Programming Languages are better than Dynamic and Lead to Faster, Safer Software at Scale
Date: 2024-03-08 | create | tech | business | software-engineering |
Type-safe programming languages are better than dynamic for building and maintaining software - especially at scale. This is a perpetual point of contention in the software community. But it shouldn't be - type-safe languages scale and dynamic languages don't for the most common and impactful bottleneck in software: the humans building it.
In this post we'll explore the differences between type-safe and dynamic programming languages and how they scale through the lens of a metaphor: wire management.
A metaphor for control flows - wires
To explain how type-safe and dynamic languages work in the real world, we're going to use a metaphor that hopefully most people have dealt with before: wire management.
To start, we're going to posit that you can represent a program as a series of wires.
- Each wire represents a control flow from one function to the next
- The thing that goes through the wire is the data (usually a variable of some sort) that is passed from one function to the next
For our own sanity we are going to assume that data only moves through a wire / control flow in one direction. This is mostly true in both domains from a high-level but saves us a lot of complex edges that lead to domain desynchronization. As a sanity check, factory games like Satisfactory, Mindustry, and Factorio all mostly do one-way traversals but complex control flows are still buildable so shouldn't break metaphor too much.
So in our metaphor, a wire represents data / control flows in our program:
- A control flow carries data - moves in one direction, may split into additional flows
- data can be ~anything - similar to programming you can theoretically pass whatever shape / size data you want
- You cannot discern what data flows through a control flow without additional info / context ab the flow
- A wire carries Payloads - moves in one direction, may split into additional wires
- A Payload can be ~anything - similar to programming you can theoretically pass whatever shape / size payload you want
- You cannot discern what Payload flows through a wire without additional info / context ab the wire
Caveat - Known Limitation: The biggest issue with this metaphor is that wires IRL are mostly static (set once, use a lot) whereas the process of software development has quite a bit more change involved. I will try to point this out as we go but keep in mind that modeling software dev in real life is akin to a building that is constantly being renovated in some way.
Type-safe vs Dynamic Wires
Recall that all wires look and feel exactly the same in our fantasy world. This raises some obvious issues like how do we ensure we don't cross our wires.
This is the core issue at play between dynamic and type-safe languages so this is a good question to keep in mind. Before we go into scenarios I first want to point out a few common strategies each uses to avoid this so we have shared context.
Dynamic Wires
With Dynamic Wires each wire looks exactly the same - same color, same connectors. This means you could connect a wire carrying electricity to one carrying water - there's nothing stopping you (or often warning you) from doing such a thing.
Common patterns for avoiding wire crossing in dynamic languages:
- Double-check the thing yourself before using it
- Wire equivalent: Turn wire on and see what comes out
- Utilize automated tests to detect if a change breaks something
- Wire equivalent: Some sort of auto checker device like an omni switchbreaker that checks water, electricity, sewage works for each device / room or smth
- Build type guards into usages
- Wire equivalent: Every wire has an omni switchbreaker in it that will turn itself off if it gets something that is not what it expects (like water in an electricity pipe or smth)
- Add descriptive variable names and doc comments
- Wire equivalent: A sticky note or label from a label maker on stuff
Type-safe Wires
So all wires in wire-land do look and feel exactly the same but I never said one couldn't upgrade wires to allow for additional info / context on them. This is essentially what Type-safe Wires do.
Each Type-Safe Wire can be:
- Color-coded
- Programming Equivalent: Red and Yellow squigglies making it obvious when you're doing smth sus
- Connectors can have different shapes so it's impossible to directly connect wires of incompatible kinds (though you could of course build an adaptor)
- Programming Equivalent: Compile errors, red squigglies
- Easy to see exactly what this wire is made for via a built-in display or label or some sort of QRCode pointing to this info from anywhere on the wire
- Programming Equivalent: Intellisense
After reading these descriptions you might say:
- This is unrealistic!
- Not a fair comparison!
- It's not fair to say that dynamic languages have to deal with impossible-to-differentiate pipes while type-safe languages get color-coded pipes!
And I would say:
- Yes, this is not a fair competition.
- One is clearly better than the other.
- Yet here we are, once again explaining this cause people keep using the bad one in prod.
Onto some scenarios.
Scenario 1: Wiring a desk
- Number of wires: 10
- Engineers: 1
- ChangesPerDay: 1
- Programming equivalent: A small software project / prototype you built yourself
Let's say we've got a pretty standard software engineer desk we need to wire:
- 1 Laptop
- 2 monitors
- Mouse and keyboard
This nets us with a pretty manageable ~7 wires. Just plug it all in and see what the results look like.
Using Dynamic Wires this is pretty easy to do. This is like wiring your desk without cable management. With this many wires, we might have a bit of a wire ball behind the desk but still pretty easy to move / switch out when you need to with just a few minutes of tracing wires to see which goes where.
Using Type-Safe wires is like using cable management from the jump. It probably took you a little longer to setup cause you had to make sure you had the right kinds of wires, you took the time to bundle and organize them, and you stored them nicely behind / under the desk. The flip side is that the next time you need to move / update a wire you'll know the exact one you need to change so save yourself the tracing.
Outcome: Draw, maybe slight win for Dynamic w few changes, slight win for Type-safe w more changes.
Both perform about the same at this very small scale. Type-safe wires takes a bit longer to setup so we could even say that Dynamic wires win in this case for time-to-initial-setup.
Recall though that software is constantly changing so this is a desk that is regularly receiving changes about once per day. At this low change velocity we won't see too much of a difference but perhaps the cord management approach is a bit less annoying day in and day out.
Scenario 2: Wiring an Office floor
- Wires: 1000
- Engineers: 10
- ChangesPerDay: 10
- Programming Equivalent: A single product for a startup
Here we've got a little office floor with several units each with:
- Some desks with electricity (needs: electricity)
- A small bathroom (needs: water, sewage, electricity)
- Maybe a kitchenette (needs: water, electricity)
Recall that software is constantly changing so in Wire land this means that Wires are changing too. For our metaphor we will say that the offices are constantly being renovated (while they're being worked in - joy) with 10 engineers making about 10 changes each day - adding wires, removing wires, changing wires etc.
Now that we're in a more complex wiring scenario, we start getting wires that should not be crossed. If they do bad things happen:
- electricity + water -> fire!
- sewage + water -> nasty!
So we don't want to do the wrong thing but we also can't stop changing the wires cause #business - the renovations must go on!
It is at this scale that we start to see Dynamic wires (and languages) start to stretch a bit at the seams. Doing what we did at desk-size worked okay but as we got more engineers changing things at once it got harder to remember what wires were doing what (cause some might have changed from yesterday!).
So we decide to implement a few things to try and prevent this because we're getting fires and nasty stuff and that makes us / our customer sad.
- Tell everyone what you did yesterday so people can keep up with what wire does what
- This works for less than one week as people zone out in daily wire status update so don't get the info (and even if they were paying attention people left out little changes they made yesterday anyway)
- Tell engineers to check the wire type before you connect them!
- But this requires a bit more time, maybe a trip to the basement to turn on / off, and you just tested it last week so probably fine right?
- Wrong! At 10 changes a day you're bound to have at least one breaking change from week-to-week so -> fire! nasty!
- Add labels to your wires
- This works great for awhile cause people are adding and reading labels. But problem is you gotta add lots of labels everywhere - like every time the wire enters a room you gotta add labels and what if we split a wire somewhere without a label we gotta add labels.
- Then one day an engineer trusts the label and it's wrong! -> fire! nasty!
- Someone must've changed this wire and didn't update the label!
- So you ditch labels cause too much work to keep them updated and you've already been burned (and nastied) by them once already
- Upgrade wires to use type guards so they don't let stuff they don't expect in
- This actually works pretty well - we stop getting so many fires and nasties cause the wires are only taking in what they can accept
- These cost a bit more to acquire and they carry a bit less of the Payload due to the typeguards needing to check every single input but mostly this works okay
- The one downside is we sometimes don't realize we've connected bad wires together until a customer starts complaining that their water isn't working or the sewage is stopped up or the power isn't on so we then have to go back and redo some of the stuff we did but better than fires and nasties
With type-safety we avoid all the fires and nasties altogether. We skip directly to the type guard phase but with a version that:
- Is built-in by default
- Does not lower Payload throughput (a free abstraction)
- Is impossible to connect wrong, so we lower amount of re-work
Of course we're still making 10 changes a day but most of these are changes we choose to do - not urgent fires and nasties we need to put out that prevent forward progress.
The one issue might be that changes sometimes take longer to make - you realize you don't have the correct color wire where you need it and thus need to do more wiring before you call a project "done". But interestingly the time to finish a project (i.e. start -> works correctly including all firefighting and re-work) is much lower cause there is far less re-work cause you did it right the first time.
Outcome: Type-Safe Wires by a long shot.
We can see that Dynamic Wires slowly started bandaiding itself with solutions to try and solve the issue of not understanding what's in the wire because it cost so much in the form of firefighting and rework. They were able to approximate versions of what Type-Safe wires provide - like preventing bad inputs entirely and easily understanding what the wire contains - but were just not able to fully catch up to what a Type-Safe native wire provides.
This may seem too fantastical and abstract but this happens in programming all the time. Dynamic languages may seem like a speed boost at the start but always (and I mean ALWAYS) ends up being a speed hindrance sooner rather than later when individual engineers cannot keep an up-to-date mental model of the full coebase due to size and change velocity. This typically becomes obvious at around 2 teams / 10 engineers and gets worse from there.
Scenario 3: Wiring an office building
- Wires: 100,000
- Engineers: 100
- ChangesPerDay: 100
- Similar to: Software product at mid-sized company
We've already been through the main differences in practice between Dynamic and Type-safe Wires. What I want to point out by going a step further in scale is that these problems only increase in magnitude.
- The speed of changes / size of mental model continues to grow - impossible to have full understanding of system in head
- The surface area of potential breakage grows exponentially - any one bad wire connection can break or start to pollute the rest of the system
- At this scale you are now dealing with people turnover - your most experienced (and knowledgeable) people leave so you have to figure out a scalable onboarding process to make new people productive quickly
We are not even at a large company / piece of software yet and yet we are running into issues with the biggest bottleneck in software development: humans.
There has to be a better way to prevent engineers from making small, unintentional, and catastrophic mistakes without needing them to understand the entire blueprint (impossible) or learning our history of custom fixes (inefficient) or learning through trial by fire (dangerous, demoralizing).
If only we had a wire that was easy to understand by anybody, immune to crossing, and could scale to the next hundred changesPerDay / Engineers / Office buildings.
Outcome: Type-safe Wires are the only ones that work.
Next
I went through several drafts of this post trying to come up with accurate, relatable metaphors that could be ~objectively used to compare dynamic and type-safe approaches in the wild. I think I ended up a bit hot on dynamic but truly it is a dumpster fire at scale so maybe this is appropriate.
I'd love to know if this was helpful / relatable or if this was wrong / confusing. That can help me iterate on making it more understandable for the future.
Also let me know if you have ideas / suggestions for other concepts you'd like explored.
If you liked this post you might also like:
Want more like this?
The best / easiest way to support my work is by subscribing for future updates and sharing with your network.