Comments (7)
Thank you for this awesome post. It addresses a topic I encounter very frequently but feel a little helpless about. The tragedy lays in that it takes education to appreciate simplicity (see Dijkstra’s quote above). This means that the ones appreciating simplicity will always be a minority. Maybe in an operational context it is easier to grasp if we look at the relationship between the number of lines of code a given product has vs. the number of service interruptions or issues it produces and/or put the lines of code in relation to the utility the product produces.
from eugeneyan-comments.
I love this article, thank you so much for writing this! 🤩
There is only one thing here I'd like to discuss and elaborate on for a bit:
It [rewarding complexity] also encourages the “not invented here” mindset where people prefer to build from scratch and shun reusing existing components even though it saves time and effort. This wastes time and resources and often leads to poorer outcomes.
I don't feel like this is quite nuanced enough, and I believe there is more to discuss here.
In my experience, the "not invented here" mindset has an equally powerful opposite bias, where people automatically prefer to reuse existing components because they already know them, it's easier relative to their individual experience, and it saves their time and effort - however, if those existing components are very complex, this can end up wasting time and resources and leading to poorer outcomes as compared to building something simpler.
If we want to emphasize simplicity, when no simple option is available, sometimes we have to invent something new.
And don't get me wrong, I'm not ignorant of the fact that this often goes wrong - we often think something could be simpler, and we end up building something that's only different but just as complex as the thing we were trying to simplify. Some problems are inherently complex, requiring essential complexity that we can't actually get rid of in practice.
My favorite "bad" example is ORMs - we keep building more, they all start out simple, and then explode in complexity. The object/relational impedance mismatch is a well documented fact, and the idea itself is inherently complex. (and if you're thinking of an ORM right now that managed to stay simple, it's probably not an ORM, and more likely just a query builder or data mapper, etc.)
By contrast, a "good/bad" example is how the PSR-6 cache API was followed by the much simpler PSR-16 cache API, which reduced the complexity and number of concepts, making the API much smaller and less opinionated - you can write a PSR-6 adapter for a PSR-16 implementation (and, for that matter, vice-versa) demonstrating the fact that none of the complexity that was removed by PSR-16 was essential to begin with.
There are many libraries and frameworks out there that go far beyond dealing with essential complexity - they often do so in the name of "developer experience", a rather lofty idea, and aptly named "developer" rather than "development", highlighting the fact that this value is often completely subjective and relative to individual experience; as opposed to real simplicity, which is an objective and measurable thing. (lines of code, number of public methods, amount of coupling, etc.)
These libraries can sometimes grow to 10x their original size over the cause of a few years - often with major breaking changes every 6-12 months and, ironically, this is perceived as "good" as well, but often indicates a complete failure to keep the scope of a package contained, focused, and small. In fact, libraries that don't exhibit growth are often perceived as being "dead", and people are often motivated to move on to packages that are more "active", another clear symptom of complexity bias.
Sometimes we have to build something new to avoid this sprawling complexity - if we can make things truly simpler, if we can bolt down the requirements and avoid sprawling growth, this can be a good thing, and sometimes it's the only way to control the infectious complexity we often inherit from "over active" libraries and frameworks.
It's a nuanced topic and we need to be careful not to feed either of these opposing biases. 🙂
from eugeneyan-comments.
@eugeneyan feel free to quote/copy/modify to your hearts content 😄
from eugeneyan-comments.
"Instead of focusing on the complexity of the solution, we should focus on the complexity of the problem."
Mmm isn't this a typo?
from eugeneyan-comments.
Why do you think so?
from eugeneyan-comments.
complexity sells better
Perhaps it's partly our responsibility as engineers to sell our solutions, even adding a veneer of accidental complexity to some solutions as needed while maintaining a simple core.
At least having made the sale, we then have the opportunity to gradually educate customers and guide them to simpler designs in future.
from eugeneyan-comments.
Hey Rasmus, thank you for taking the effort to share your perspective, and in such depth too!
💯 that "not invented here" taken too far has bad side-effects too.
May I add your comment, in full, within the article please? I feel like it adds the balance that it's lacking now. Will provide full attribution to you off course, and your comment will stay in the comments.
from eugeneyan-comments.
Related Issues (20)
- https://eugeneyan.com/writing/recsys2022/ HOT 1
- https://eugeneyan.com/writing/autoencoders-vs-diffusers/ HOT 1
- https://eugeneyan.com/writing/mechanisms-for-projects/ HOT 1
- https://eugeneyan.com/writing/what-i-did-not-learn-about-writing-in-school/ HOT 3
- https://eugeneyan.com/writing/bandits/ HOT 2
- https://eugeneyan.com/writing/labeling-guidelines/ HOT 1
- https://eugeneyan.com/writing/llm-experiments/ HOT 5
- https://eugeneyan.com/writing/open-llms/ HOT 1
- https://eugeneyan.com/writing/obsidian-copilot/ HOT 2
- https://eugeneyan.com/writing/more-patterns/ HOT 1
- https://eugeneyan.com/writing/llm-patterns/ HOT 17
- https://eugeneyan.com/writing/attention/ HOT 2
- https://eugeneyan.com/writing/finetuning/ HOT 3
- https://eugeneyan.com/writing/push/ HOT 1
- https://eugeneyan.com/writing/why-read-papers/ HOT 3
- https://eugeneyan.com/writing/georgia-tech-omscs-faq/ HOT 2
- https://eugeneyan.com/writing/unit-testing-ml/ HOT 8
- https://eugeneyan.com/writing/roam-to-obsidian/ HOT 2
- https://eugeneyan.com/writing/ai-coach/ HOT 5
- https://eugeneyan.com/writing/evals/ HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eugeneyan-comments.