At some tech companies, technical debt is dealt with reactively and on an as-needed basis. Other companies take a more proactive approach, with multiple teams working together to both prevent future debt from accruing and devising strategies to effectively deal with what’s already built up. We’ll let you guess which types of companies we spoke to about tech debt for this piece.
Continue reading to learn more about how some of LA’s fastest-growing companies define, manage and minimize tech debt.
Fast-moving startups like Builder, formerly known as Engineer.ai, sometimes have tough choices to make when it comes to technical debt. Do you continue to move quickly, even if that sometimes means taking on more tech debt? Or do you slow things down and give your team time to iron out issues that could create problems down the road? It’s not an easy call, to say the least. Here’s how Builder Engineering Lead Tooraj Helmi and his team approach it.
What is technical debt and how does your team define it?
Technical debt refers to the prioritization of short-term gains despite the longer-term troubles that may arise. It’s the understood tradeoff that results when our team decides to experiment and gather knowledge in support of immediate agility rather than plan and build with future efficiency top of mind. As a startup with global offices and engineering teams working together across the world, we are naturally prone to technical debt.
Just recently, my team here in LA had a dependency on a product being designed and developed by our team in India. Their part of the project was going to take three months, so we had two choices: put things on hold and wait for them to complete it, or create a simplified version of the product to fill the gap and move things forward for the time being. After a cost and benefit analysis, we chose the second approach, prioritizing short-term progress even though we knew that the placeholder product would be disposed of in a few months.
When technical debt does occur, what process does your team use to measure and manage it?
If there is an understood long-term disadvantage involved, we always conduct a cost and benefit analysis to see if the short-term benefits outweigh the costs. We consider the cost of reintegrating with the new system, the cost of retraining employees and customers, and the cost of transitioning from the sub-optimal product. If it makes sense to move forward regardless, we then make sure the decision to increase the debt is communicated to stakeholders who might be impacted by it.
In cases where technical debt can absolutely not be avoided, we build the short-term products with maximum reusable components.”
What proactive measures does your team take in minimizing technical debt?
We want to provide the best start-to-finish service to our customers, so our team plans for long-term optimization as much as we can. Rather than focusing on a client’s product in isolation, for example, we always try to look at the end-to-end production chain as a whole and consider what gaps may arise. If we spot something, we immediately gather the relevant stakeholders together, review our choices and come up with the optimal plan to minimize the technical debt.
In cases where technical debt can absolutely not be avoided, we build the short-term products with maximum reusable components. Using an architecture that allows for decoupling helps us easily eliminate the sub-optimal products as needed.
Technical debt isn’t something that can be effectively dealt with on a one-off or as-needed basis. Truly getting a hold of it requires companies to have processes and language in place that ensures everyone is on the same page and knows what their responsibilities are. BlackLine VP of Engineering Joshua Rhoades shared the concrete steps his company is taking to both address existing tech debt and reduce the amount they accrue going forward.
What is technical debt and how does your team define it?
Over the last two years, we’ve put a drastically heavier focus on technical debt and how it impacts us in all its forms. From our perspective, technical debt is anything that forces hard limitations onto the system or our users, degrades delivery efficiency, or artifacts that adversely impact the ability to deliver on business needs in an efficient and high-quality fashion.
When we choose a particular piece of debt for a business goal, it’s important that we do so in a conscious manner, and that we address the debt in subsequent iterations.”
When technical debt does occur, what process does your team use to measure and manage it?
One thing I have been really happy to see here are delivery teams calling out technical debt before it occurs, but that’s ideal and not always reality. The first-order problem is how to identify technical debt across the organization in a common language. Having a shared language has made us much more adept at recognizing technical debt. However, we all recognize that technical debt is part of delivering software. When we choose a particular piece of debt for a business goal, it’s important that we do so in a conscious manner, and that we address the debt in subsequent iterations.
We are also beginning to define types of technical debt: architectural, inadvertent versus induced, by originating source, aging and so on. Each type of debt will have assigned “owners.” Ownership is a vital concept to successfully tackling debt. Once categorized by type, these items will then go through a lightweight prioritization process based on impact, cost and value of improvement.
Our product teams are keenly aware of tech debt, with a dedicated 20 percent focus of sprint tickets targeted at technical debt. However, there is something that is often forgotten when it comes to technical debt: feature debt. What features are of low value or use, or are duplicative or cannibalizing other features? Is the UX causing inefficiencies to our users? Are we missing features? Feature debt and technical debt are often related or able to influence each other.
What proactive measures does your team take in minimizing technical debt?
We’re big believers in code reviews, as well as leveraging automated static analysis on merges so that the machines focus on the common areas. This helps us get to qualitative, rather than quantitative, reviews. But real proactivity is preventing tech debt before it even gets into the codebase. We focus on reusability of components and functions, and working in properly encapsulated and separated classes and methods.
Reusability, as well as hygienic code, tends to force a thought process that avoids tech debt, since a reusable method that is also properly encapsulated and abstracted is much easier to refactor without downstream impacts. Likewise, our minimum of 80 percent unit test coverage helps immediately identify breaking changes or problems in that code, which in turn gives us greater confidence in being able to update it without worrying about unknown impacts.
We also make use of heavy linting and intellisense in our integrated development environments so that our engineers get as much helpful real-time feedback as possible, such as memory leaks or scope issues. This allows us to focus our attention and build scalable, performant, resilient and maintainable code that does not couple us to bad smells or bad design decisions.
Mythical Games might be a young startup — it emerged from stealth in late 2018 — but the video game technology company is filled with a host of industry veterans, so it’s no wonder the team has a mature take on technical debt. Rather than trying to avoid it at all costs, the company acknowledges and accepts that tech debt will occur. Instead of asking engineers to hide the debt they’ve accrued, the video game technology company encourages them to ticket and track it.
Lead Platform Engineer Chris Downs shared more about Mythical Games’ proactive approach to tackling technical debt.
What is technical debt and how does your team define it?
At Mythical, technical debt is defined as pieces of a feature or system that are moved to the backlog to be finished later after the core functionality is delivered. While these pieces may be necessary for ease of future refactoring or stability, the deliverable will still work without them. An example of technical debt is documentation. Well-documented systems make onboarding new engineers and clients much easier, but it’s easy to push that into the backlog and hope you find time to do it later.
Instead of punishing engineers for creating technical debt and making them feel that they need to hide it, we promote transparency.”
When technical debt does occur, what process does your team use to measure and manage it?
We feel that it is important to accept that technical debt will ultimately occur. Instead of punishing engineers for creating technical debt and making them feel that they need to hide it, we promote transparency and try and ensure that any known debt is ticketed and tracked in Jira so that a project manager can make sure it is slotted in at a later date. When a milestone containing the integration of a bunch of features is approaching, we will often dedicate a sprint to polishing and technical debt repayment to try and keep the debt from rolling over to the next milestone.
What proactive measures does your team take in minimizing technical debt?
We try and figure out the root cause of the most common types of technical debt and provide tooling and processes to make them easier to avoid. One example is API documentation. We have made use of OpenAPI annotations in our Java APIs to add programmatic generation of documentation, allowing engineers to write documentation right alongside their code.
Another pain point has been code and deployment consistency across different services. We are making use of Maven archetypes to templatize the application portion of our Java services and Helm charts to templatize the deployment of these services to Kubernetes. These improvements vastly decrease the amount of boilerplate and copy and paste we need to add to a new service and this keeps things consistent across services to speed up onboarding.
