Cloud Hosting

Photo by Growtika on Unsplash

When I started in my new role as Manager of Data Science, little did I know about setting up a data science platform for the team. In all my previous roles, I had worked on building models and to some extent deploying models (or at least supporting the team that was deploying models), but I never needed to set up something from scratch (infra, I mean). The data science team did not exist then.

So first of my objective was to set up a platform, not just for the data science team in a silo, but that can be integrated with data engineering and software teams. This is when I was introduced to Kubernetes (k8s) directly. I had heard of it earlier but hadnt worked beyond creating docker images and someone else would deploy in some infra.

Now, why is Kubernetes required for the data science team? What are some of the challenges faced by data science teams?

Continued here:

Building a Data Science Platform with Kubernetes | by Avinash Kanumuru | Jul, 2024 - Towards Data Science