I think that you're spot on. I've probably been more obsessed about "the way it should be" rather than "what's the shortest path to get our first clients using it". I suppose that it's a newbie mistake; out of fear.
Part of me couldn't accept the idea of delivering something that would explode and that I wouldn't be able to support in production. It's still something I fear, especially when I look at our code coverage, even though I do care about the quality of what I create..
As you say, switching from CouchDB to PGSQL wouldn't be easy at all.
And yep, agreed about schools and restaurants, you're probably right.
It sounds like you might not have the right mindset for being a technical startup founder.
Unit tests and code coverage is anticorrelated with success. With a startup, your goal is to iterate the fastest. You cannot afford to be spending more than 1 day of time on your continuous deploy system before you have your first paying client.
Given the size to your team, you should also have been spending near zero time on documentation. Documentation should be in the form of sticky notes and Slack messages.
I think you’re very talented technically, but you should not work on startups until you are able to emulate more of the successful technical cofounder patterns.
If your code and processes can scale to 100x of your time expected volume, this is almost universally a sign that you have failed to optimise for what matters.
This also goes even if you’re building something like AWS. AWS engineers do not build over-engineered systems that can handle 100d of its expected volume.
I find this weird. Automated testing has saved me so much time.
If I don't have tests and make a change, I have to manually go through a whole set of checks to make sure I didn't break anything. If I have a test suite, it will tell me in 2 minutes if I broke anything.
The thing I'm working on at the moment has tests on the backend but not on the UI. I can definitely say the UI work is slower because of the lack of tests.
You're essentially in prototyping phase. Inevitably you'll have to discard whole chunks of code, because your understanding of requirements/concept space changes. Then you have to discard both your code and your fine tests, wasting ~2x more time.
Yes, but writing tests isn't a huge cost if you're used to it. And once I've discarded that huge chunk of code, I immediately know what else I broke, rather than having to find it by testing manually.
If I have to write 100 lines of code to implement a feature, it really doesn't take me twice as long to write another 100 to test it. In most cases, writing the test alongside (or even before) the code helps with writing the code. I'd say writing a test adds maybe 25% to the time.
Knowing a year later than that code is still good, because that test still runs successfully, is easily worth that 25%
Of course I'm not saying 100% code coverage is worth it. Just the critical bits.
Test suite is not free to create or maintain. To your example, for a product that isn't mature, a UI test suite is a waste of time and greatly increases the cost of UI changes.
This is what I don't get - a test suite saves me time. So, yes, it is free to create and maintain. Because it's much quicker to find regression bugs with automated testing than with manual testing. And this is massively more valuable when things are changing rapidly.
I think the core of what they're trying to get at is: let the users find those bugs. The business isn't large enough to focus on details, yet. In the future, when fixing bugs takes more time than making features, you should have the money to hire more hands and focus more on testing. If you don't, then you built too much product for too little clients which is a problem in and of itself ;-)
The cost/benefit isn't there in the beginning. All the bugs are either obvious, easy to find, or quickly discovered without a test suite. So save the time writing tests to stamp out more features the customer wants!
Adding features adds complexity. Features interact with each other in complex ways. If you add a feature and don't know quickly if it broke another feature, that costs time.
Letting the users find the bugs costs time - you have to talk to the user, replicate the bug report, find the failure, fix it, and then release. Finding that same bug 2 minutes after you wrote it (and before it got released) is massively quicker.
This feels like one of those "write code 16 hours a day to make more progress!" things that experience has taught me just doesn't work in practice. I feel like I'm making a lot of progress, but after about 10 hours most of the code I write is pure shite and needs to be rewritten the next day. I've found that it's actually way faster to quit coding after 8-10 hours rather than spend 2 hours the next day rewriting everything I wrote in the last 6 hours the day before.
Tests cost money up front. Can you pay for those tests? You said you will pay for it in decreased costs once the business is running well.
Ok but how confident are you that your business will work out?
1%? you just increased your project costs 100 fold by doing tests.
50%? you just doubled your project costs by doing tests.
100%? Ok you are saving money, but if you are that confident then why haven't you gotten investors aboard that are convinced that doing it well from the start is a business edge?
The mindset and goals are much different. At the extreme, a showstopper bug in an MVP is not a big deal - you have few (if any) customers and a glitch here and there won't make any difference in the long run.
Early stage tech founder should be either a cowboy programmer or a business-minded project manager, the type who ruthlessly kills projects and blows things up on a whim.
That's completely dependent on what kind of software you are writing. (And yeah, if you caught yourself automating UI tests, you are probably way too far.)
The best phrasing is the OP's in that "automated tests are anticorrelated to success" (so is posting on HN, by the way). That doesn't mean that for your exact case they are not essential, just that there is an statistic relation.
I don’t think that you are slow or not a good developer is the right conclusion. I think you focused on the wrong things. Like code coverage, CI, etc.
You mentioned of being afraid of your codebase exploding in other comment. You know code don’t really explode, right? The worst case is you will need some refactor down the road. Nothing worse than that.
Don't be too hard on yourself or you'll start doubting your own decisions too much. Everyone screws up some things, and nails some other ones.
Here's something that helps me with development. First, decide if it's a personal project for learning/fun, or if it's a commercial solution you are developing for someone else.
Personal projects get the cleanest code. I obsess with perfection. Max optimizations. Sexy tech. I'm Michelangelo and this us my Sisteen Chapel. John Carmack himself should take notes from that code. You get the idea. This project never ships though. 9/10 times I learn what I need and then it's just about grinding, I get bored and move on.
Commercial products or projects I really need to ship get a clean design, but thats all. I spend some time to design the foundation correctly, and from there, its an ugly, duct taped, unoptimized piece of crap that I can iterate quickly on.
Also, make sure the power dynamics in your team are correct. Dont get hired to write the code. Get hired to develop the technical solution. This way you can say "no" to things.
A pattern I've noticed much later in my early endeavors: if the business people (those writing the check) are also the product managers, the project is likely doomed. You know the type of person. They will insist on which features need to be there because having money automatically makes you an expert. They typically think of the end product as the MVP.
One last thing, internalize the following:
"Make it work. Make it right. Make it fast." <- in that order.
I say this, because you mentioned code coverage, and I would like to say that my opinion is one doesn't need unit tests for the first step.
Basically, screw the tests. I know, I know, how can anyone take me seriously when I say that. But, why do you need unit tests? To catch bugs in an MVP? Who expects rock solid, bug free MVPs? Maybe its to sleep better at night because code coverage is representative of code quality? We know it's not, so why bother?
Your MVP should be small enough that it's entire functionality is critical paths through the code. This means that every time you click through your product you are testing your code and you should be able to exercise most of it quickly and often. Unit tests are basically code that you have to maintain when something changes and it decreases your velocity. You don't want that, you just want to "make it work".
Will you eventually pay the price for not having unit tests initially? Yes, of course, but, if you designed your MVP correctly, this will either be never, because the product doesn't have market fit, or right at the time when you are getting product-market fit and are growing your team. At this time you will have a great idea of what functionality to add tests for. You won't even give a damn about the code coverage, because now is when you "make it right".
Either way, exclusions apply because software is hard. Good luck with your endeavors and keep coding ;)
I think that you're spot on. I've probably been more obsessed about "the way it should be" rather than "what's the shortest path to get our first clients using it". I suppose that it's a newbie mistake; out of fear.
Part of me couldn't accept the idea of delivering something that would explode and that I wouldn't be able to support in production. It's still something I fear, especially when I look at our code coverage, even though I do care about the quality of what I create..
As you say, switching from CouchDB to PGSQL wouldn't be easy at all.
And yep, agreed about schools and restaurants, you're probably right.