5 steps to bring DevX automation to your company
Get the guide

Deploy Pytorch Model with Okteto CLI

In this article, you will be deploying an already built model for the public that generates unique and personalized love letters with no plagiarism to send to their partners or crush.

![Can we deploy to prod?](prod-meme.jpeg)

Effective deployment of machine learning models is more of an art than science.

Text generation model with Pytorch

Text generation is a Natural Language Generation application that generates text. This application (also called a model) is trained with text and uses text to generate semantic and coherent text. It is also trained to learn the likelihood of a word based on the previous sequence of words used in the text. So we are going to clone a text generation model built with PyTorch and deploy it on okteto.

We'll start by cloning the repository housing the model:

$ git clone https://github.com/Emekaborisama/Text-Generation-with-pytorch.git

I have trained the model and deployed it using Flask API. You can find the training script in the train folder, the inference, and the API script in the app folder.

When you clone the repository, you will notice the folder arrangement, which I recommend for anyone deploying a machine learning model with Flask and Okteto.

Next, navigate to the text-generation folder, and install the required packages:

$ cd Text-Generation-with-pytorch
$ pip install -r requirements.txt

Once the installation is complete, verify that the model works by running the command:

$ python3 main.py

The command above should return an output similar to the one below:

![Command Output](output.png)

Now, let's deploy the model to Okteto.

Deploying to Okteto

Okteto is a developer platform used to accelerate the development workflow of cloud-native applications. You can make deployments to Okteto either through Okteto UI or through Okteto CLI.

Okteto Cloud

Okteto Cloud gives instant access to secure Kubernetes namespaces to enable developers to code, build, and run Kubernetes applications entirely in the cloud. With Okteto cloud, you can deploy your Machine Learning model with just a click of a button.

OKteto CLI

On the other hand, the CLI tool lets you develop your Cloud Native applications directly in Kubernetes. With the Okteto CLI, you can code locally in your preferred IDE, and Okteto will automatically sync your changes.

To install Okteto CLI, use the command below:

$ brew install okteto

Not using MacOS? Check out the guide for installing on your OS on the installation page.

Deploying to Okteto

Before using the Okteto CLI, you need to log into the service to allow you to create, delete, push and synchronize changes from your local machine. You only need to do this once.

Configure your Okteto CLI context to Okteto Cloud:

okteto context use https://cloud.okteto.com

A browser tab will open automatically, confirming that you have now logged in. If this the first time that you use Okteto, it will ask you to log in with your GitHub identity.

![You are now logged in](okteto-welcome.png)

After configuring your okteto context, we will create a manifest file okteto-stack.yaml. This file defines our application configuration using a docker-compose like format.

$ touch okteto.stack.yaml

Stacks is made for developers who don't want to deal with the complexities of Kubernetes manifests.

Add the following to your okteto-stack.yaml file:

name: textgen
 public: true
 image: okteto.dev/text_generation_with_pytorch
 build: .
 replicas: 1
 - 8080
 cpu: 2000m
 memory: 4280Mi

With the stack file in place, deploy the model to Okteto using the command:

$ okteto stack deploy --build

This command containerizes your application, builds your application based on the instructions in the okteto-stack.yaml file, and then deploys the application.

![okteto stack deploy output](deployed.png)

Now we have deployed our PyTorch generation model. The deployment command starts an application on your dashboard:

![Okteto dashboard](dashboard.png)

With the application deployed, let's generate a love letter from our terminal by sending a POST request to the /lovelettergen endpoint using CURL:

$ curl --location --request POST 'http://textgen-emekaborisama.cloud.okteto.net/lovelettergen' \
--form 'content="sweet heart"'

The response from the command above:

 finding happy. really, especially, up take calling want funny kingdom you'll deeply. send away making thrilled also, care elements sad, wife. enjoyable better whereby hanging count times. felt. nicest else; kisses constant


In this article, we have learned how to deploy a text generation PyTorch model with Flask and Okteto CLI. Okteto stacks make it super easy to deploy Machine Learning APIs on Kubernetes.

Our next article will focus on deploying the model on Okteto Cloud UI from the GitHub repository.

Automate Provisioning Any Dev Resource on Any Cloud Provider With Pulumi and Okteto

The Value It is common in today's landscape to build microservices-based applications that leverage resources like RDS databases, storage buckets, etc...

October 19, 2023
Avatar of Arsh SharmaAvatar of Arsh SharmaArsh Sharma

How Developers Can Seamlessly Collaborate When Building Microservice Apps

Building microservices based applications is inherently challenging. Given the multitude of components involved, it is unrealistic to expect any individual...

October 10, 2023
Avatar of Arsh SharmaAvatar of Arsh SharmaArsh Sharma