A Retrospective: Lessons from Early Technical Design Flaws at Unisala

For the longest time, Unisala operated without proper project management tools like jira. This was not only area where we faltered. It was just one of many spinning plates that dropped due to lack of accountability and experience.

Unfortunately, the problem wasn't isolated to project management. Immature processes and lack of oversight seeped into almost every corner of our work, leaving us vulnerable to mistakes and inefficiencies.

In this blog, i want to dissect some key areas where these issues manifested, focusing on two pivotal aspects of technical designs:

1. The database
2. Api decision tree

As a project lead, i take full responsibility for these amateur approaches. This serves as a stark reminder of the importance of high level planning and proper documentation, lessons that are shaping how we move forward.

The Database: A Messy Web of Models

What Went Wrong

There was never a proper high level database design like Entity Relationship Diagram(ERD). Every feature was developed based on a specific developer's thought process with brute force being the preferred approach to get the things working somehow. This approach worked temporarily but created long-term chaos.

Example of High Level Database Design ERD:

As more features were added:

1. Code bloated: The lack of cohesive planning led to more and more code being added, without a clear understanding of what connected to what.
2. No Documentation: We didn't have a proper documentation or references for knowledge transfer for new developers, which resulted in mix thinking processes, inconsistent practices and thus no standard convention to follow.

Today, the app has over 35 database models, and almost half of them are no longer used.

Impact:

• These unused models add to the system's complexity.
• Developers now have to sift through hundreds of files in the codebase just to figure out which models are still relevant.

This is a fundamental system design flaw. It's not just inefficient but also unsustainable.

What we should have done:

1. Create a high level database design:
2. Begin with Entity Relationship Diagram(ERD), to visualize relationship between models.
3. Review the design to periodically:
4. To omit, data redundancy and inefficiencies.
5. To ensure it aligns with the evolving system.
6. Enforce Standards:
7. A conventions for model naming, structure and use
8. Document these standard to ensure consistency across developers.
9. Audit and refactor regularly:
10. Audit periodically, to identify unused models, functions and clean up codebase.
11. Establish a process for deprecating unused models with proper control.

The Api Decision Tree: A Black Box

What went wrong

We never established a proper decision tree for our API functionalities.

An Api decision tree outlines:

1. The function executed to fulfill an API request.
2. The model and databases touched during the process of execution.

An API decision tree is more than just a technical diagram. It's a blueprint for clarity, maintainability and optimisation. It plays critical role in ensuring your system remains efficient and resilient to change.

Why an API Decision Tree?

1. Understanding ripple effect:
2. When we modify a function, we need to know where it might impact the service.
3. Identifying Optimisation opportunities:
4. A decision tree helps you visualize how functions and features interact, enabling you to spot inefficiencies.
5. For example, you might notice redundant calls to the same database model across multiple apis, which can be optimised by caching.
6. Facilitating Collaboration:
7. New developers can onboard fasters by understanding the dependencies and flow of data across the system
8. Teams can work more confidently knowing exactly what their changes will touch.
9. Streamlining System Evolution:
10. As the app grows, knowing what's connected to what becomes essential for scaling.
11. Decision trees highlight unused or low-priority functionalities, making it easier to refactor or deprecate parts of the system.

Where we went wrong?

In Unisala, we never had a decision tree for our APIs. As a result, i cannot tell you how many apis are we managing at this point.

1. Missed Optimization:
2. Redundant or poorly performing functions went unnoticed because we don't have holistic view of system
3. Lost Productivity:
4. Debugging became a guessing game, with developers sifting through layers of undocumented code to identify root causes.

What we should have done:

1. Establish a decision tree for every api:
2. Document all functions executed by api, including:
3. models
4. servies
5. database touched
6. Integrate Decision Tree updates:
7. When adding or modifying an API, update the decision tree as a part of the development process.
8. Optimisation Review:
9. Periodic reviews on decision tree to cover and refactor unused features, refactor for better performance.

Moving Forward

At Unisala, we’re now prioritising ERD, API decision trees as a core part of our technical documentation. By clearly mapping dependencies and flows, we aim to:

• Reduce bugs caused by unanticipated changes.
• Enhance collaboration among developers.
• Create a more maintainable and scalable system.

#systemDesign #ERD #decisionTree #API #design #softwareDevelopement