Application Development With Kubernetes

Kubernetes is widely regarded as the ideal platform for teams in large and mid-size organizations with high availability and autoscaling requirements. However, it has also added friction to existing developer workflows. This change can be understood as part of a broader push towards the adoption of the micro-service paradigm and the increasing complexity of how software is run. In this article, we will examine how the problem of software development has evolved with this paradigm shift.

Dev Loop

Lets start with the concept of a dev loop and its role in development workflows. The development loop for any software project consists of two separate loops that intersect over git.

These are -

• Inner dev loop - The process involves writing, compiling, and testing a piece of code on a developer's own workstation. This loop continues until a satisfactory piece of code is moved to Git.
• Outer dev loop - This loop is triggered after the code has been moved from the developer's workstation to Git. It involves running tests and deploying the final product. Ideally, this process should be set up as a continuous loop, where deployments occur with every code push that passes the tests.

These loops feed into each other and form a complete software development lifecycle.

Rise of the micro-service

The advent of micro-services has brought small, independent, and loosely coupled software components that offer better scalability and reliability. We won't discuss the advantages here since its been very well publicized, but our concerns come from the consequent increased operational complexity.

Let's consider a simple setup where the frontend calls the api, which in turn communicates with two other services called identity and db. In this scenario, our developer works on the api layer, which has both downstream (frontend) and upstream dependencies (identity, db). Let's explore how the development process for the api service changes in the two different development loops  -

Inner dev loop

• Upstream dependencies access - api service needs to talk to both identity and db services to successfully run.
• Downstream dependencies access - In order to test whether the changes being made on the api service are compatible with its downstream dependencies, it is also important to  redirect downstream requests to this service.

Outer dev loop

• Upstream dependencies - In order for an e2e test to run for the api service, there needs to be access to a running copy of identity and db services
• Downstream dependencies - There is also a need for a way to allow the frontend to access this service for testing purposes.
• Hosted endpoint - In addition to the above mentioned requirements, we also need to host this copy of the api service so that it can be directly tested

The inner dev loop problem is something that Telepresence from Ambassador Labs has solved for. We have attempted to solve the outer dev loop problem on the TrueFoundry platform by Intercepts.

Introducing TrueFoundry Intercepts

The concept of TrueFoundry Intercepts work by solving the three outer dev loop problems outlined in the previous sections. We follow the following strategy to tackle this -

• Preview URLs - Using the platform itself, it's easy to deploy a copy of an existing service at a specific commit SHA. This allows you to create preview URLs that are dedicated to a particular commit or pull request.
• Header based intercepts - Intercepts make it possible for a service to continue using the same URL while adding an additional header that redirects to the preview copy. For example, we can attach an intercept to the api service that redirects to a preview copy based on a header being passed to it. This means frontend can continue to work with the same URL just by passing an additional header during testing.

💡

To read more about TrueFoundry intercepts, please refer to our docs - https://docs.truefoundry.com/docs/intercepts

Conclusion

We have discussed the changes in development flow that have been brought about by the advent of microservices and how we are attempting to address them. This remains an open question, and we can expect even more innovative solutions in this space in the future.

Chat with us

if you're looking to maximise the returns from your LLM projects and empower your business to leverage AI the right way, we would love to chat and exchange notes.

Have a ☕️ with us

Learn how TrueFoundry helps you deploy LLMs in 5 mins:

Deploy and Fine-tune Open Source LLMs With a Few Clicks!

TrueFoundry offers an intuitive solution for LLM deployment and fine-tuning. With our Model Catalogue, companies can self-host LLMs on Kubernetes, reducing inference costs by 10x in just one click. Discover how to deploy a Dolly-v2-3b model and fine-tune a Pythia-70m using TrueFoundry in our blog.

TrueFoundry Blog Truefoundry