Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add kubernetes deployment #2236

Merged
merged 4 commits into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ Or this one to deploy Langflow 0.6.x:
<img src="https://render.com/images/deploy-to-render-button.svg" alt="Deploy to Render" />
</a>

## Deploy on Kubernetes

Follow our step-by-step guide to deploy [Langflow on Kubernetes](https://github.com/langflow-ai/langflow/blob/dev/docs/docs/deployment/kubernetes.md).

# 🖥️ Command Line Interface (CLI)

Langflow provides a command-line interface (CLI) for easy management and configuration.
Expand Down
315 changes: 315 additions & 0 deletions docs/docs/deployment/kubernetes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,315 @@
# Kubernetes

This guide will help you get LangFlow up and running in Kubernetes cluster, including the following steps:

- Install [LangFlow as IDE](#langflow-ide) in a Kubernetes cluster (for development)
- Install [LangFlow as a standalone application](#langflow-runtime) in a Kubernetes cluster (for production runtime workloads)

## LangFlow (IDE)

This solution is designed to provide a complete environment for developers to create, test, and debug their flows. It includes both the API and the UI.

### Prerequisites

- Kubernetes server
- kubectl
- Helm

### Step 0. Prepare a Kubernetes cluster

We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) for this example, but you can use any Kubernetes cluster.

1. Create a Kubernetes cluster on Minikube.
```shell
minikube start
```
2. Set `kubectl` to use Minikube.
```shell
kubectl config use-context minikube
```

### Step 1. Install the LangFlow Helm chart

1. Add the repository to Helm.
```shell
helm repo add langflow https://langflow-ai.github.io/langflow-helm-charts
helm repo update
```
2. Install LangFlow with the default options in the `langflow` namespace.
```shell
helm install langflow-ide langflow/langflow-ide -n langflow --create-namespace
```
3. Check the status of the pods
```shell
kubectl get pods -n langflow
```
```
NAME READY STATUS RESTARTS AGE
langflow-0 1/1 Running 0 33s
langflow-frontend-5d9c558dbb-g7tc9 1/1 Running 0 38s
```

### Step 2. Access LangFlow

Enable local port forwarding to access LangFlow from your local machine.

```shell
kubectl port-forward -n langflow svc/langflow-langflow-runtime 7860:7860
```

Now you can access LangFlow at [http://localhost:7860/](http://localhost:7860/).

### LangFlow version

To specify a different LangFlow version, you can set the `langflow.backend.image.tag` and `langflow.frontend.image.tag` values in the `values.yaml` file.

```yaml
langflow:
backend:
image:
tag: "1.0.0a59"
frontend:
image:
tag: "1.0.0a59"
```

### Storage

By default, the chart will use a SQLLite database stored in a local persistent disk.
If you want to use an external PostgreSQL database, you can set the `langflow.database` values in the `values.yaml` file.

```yaml
# Deploy postgresql. You can skip this section if you have an existing postgresql database.
postgresql:
enabled: true
fullnameOverride: "langflow-ide-postgresql-service"
auth:
username: "langflow"
password: "langflow-postgres"
database: "langflow-db"

langflow:
backend:
externalDatabase:
enabled: true
driver:
value: "postgresql"
host:
value: "langflow-ide-postgresql-service"
port:
value: "5432"
database:
value: "langflow-db"
user:
value: "langflow"
password:
valueFrom:
secretKeyRef:
key: "password"
name: "langflow-ide-postgresql-service"
sqlite:
enabled: false
```

### Scaling

You can scale the number of replicas for the LangFlow backend and frontend services by changing the `replicaCount` value in the `values.yaml` file.

```yaml
langflow:
backend:
replicaCount: 3
frontend:
replicaCount: 3
```

You can scale frontend and backend services independently.

To scale vertically (increase the resources for the pods), you can set the `resources` values in the `values.yaml` file.

```yaml
langflow:
backend:
resources:
requests:
memory: "2Gi"
cpu: "1000m"
frontend:
resources:
requests:
memory: "1Gi"
cpu: "1000m"
```

### Deploy on AWS EKS, Google GKE, or Azure AKS and other examples

Visit the [LangFlow Helm Charts repository](https://github.com/langflow-ai/langflow-helm-charts) for more examples and configurations.

Use the [default values file](https://github.com/langflow-ai/langflow-helm-charts/blob/main/charts/langflow-ide/values.yaml) as reference for all the options available.

Visit the [examples directory](https://github.com/langflow-ai/langflow-helm-charts/tree/main/examples/langflow-ide) to learn more about different deployment options.

## LangFlow (Runtime)

The runtime chart is tailored for deploying applications in a production environment. It is focused on stability, performance, isolation and security to ensure that applications run reliably and efficiently.

Using a dedicated deployment for a set of flows is fundamental in production environments in order to have a granular resource control.

## Import a flow

There are two ways to import a flow (or multiple flows) and deploy it with the LangFlow runtime Helm chart:

1. **From a remote location**: You can reference a flow stored in a remote location, such as a URL or a Git repository by customizing the `values.yaml` file in the section `downloadFlows`:

```yaml
downloadFlows:
flows:
- url: https://raw.github.com/langflow-ai/langflow/dev/src/backend/base/langflow/initial_setup/starter_projects/Basic%20Prompting%20(Hello%2C%20world!).json
```

When the LangFlow runtime starts, it will download the flow from the specified URL and run it.
The flow UUID to use to call the API endpoints is the same as the one in the JSON file under the `id` field.
You can also specify a `endpoint_name` field to give a friendly name to the flow.

2. **Packaging the flow as docker image**: You can add a flow from to a docker image based on Langflow runtime and refer to it in the chart.

First you need a base Dockerfile to get the langflow image and add your local flows:

```Dockerfile
FROM langflowai/langflow-backend:latest
RUN mkdir /app/flows
COPY ./*json /app/flows/.
```

Then you can build the image and push it to DockerHub (or any registry you prefer):

```bash
# Create the Dockerfile
echo """FROM langflowai/langflow-backend:latest
RUN mkdir /app/flows
ENV LANGFLOW_LOAD_FLOWS_PATH=/app/flows
COPY ./*json /app/flows/.""" > Dockerfile
# Download the flows
wget https://raw.github.com/langflow-ai/langflow/dev/src/backend/base/langflow/initial_setup/starter_projects/Basic%20Prompting%20(Hello%2C%20world!).json
# Build the docker image locally
docker build -t myuser/langflow-just-chat:1.0.0 -f Dockerfile .
# Push the image to DockerHub
docker push myuser/langflow-just-chat:1.0.0
```

### Prerequisites

- Kubernetes server
- kubectl
- Helm

### Step 0. Prepare a Kubernetes cluster

Follow the same steps as for the LangFlow IDE.

### Step 1. Install the LangFlow runtime Helm chart

1. Add the repository to Helm.
```shell
helm repo add langflow https://langflow-ai.github.io/langflow-helm-charts
helm repo update
```
2. Install the LangFlow app with the default options in the `langflow` namespace.
If you bundled the flow in a docker image, you can specify the image name in the `values.yaml` file or with the `--set` flag:
```shell
helm install my-langflow-app langflow/langflow-runtime -n langflow --create-namespace --set image.repository=myuser/langflow-just-chat --set image.tag=1.0.0
```
If you want to download the flow from a remote location, you can specify the URL in the `values.yaml` file or with the `--set` flag:
```shell
helm install my-langflow-app langflow/langflow-runtime -n langflow --create-namespace --set downloadFlows.flows[0].url=https://raw.github.com/langflow-ai/langflow/dev/src/backend/base/langflow/initial_setup/starter_projects/Basic%20Prompting%20(Hello%2C%20world!).json
```
3. Check the status of the pods
```shell
kubectl get pods -n langflow
```

### Step 2. Access the LangFlow app API

Enable local port forwarding to access LangFlow from your local machine.

```shell
kubectl port-forward -n langflow svc/langflow-my-langflow-app 7860:7860
```

Now you can access the API at [http://localhost:7860/api/v1/flows](http://localhost:7860/api/v1/flows) and execute the flow:

```bash
id=$(curl -s http://localhost:7860/api/v1/flows | jq -r '.flows[0].id')
curl -X POST \
"http://localhost:7860/api/v1/run/$id?stream=false" \
-H 'Content-Type: application/json'\
-d '{
"input_value": "Hello!",
"output_type": "chat",
"input_type": "chat"
}'
```

### Storage

In this case, the storage is not needed as our deployment is stateless.

### Log level and LangFlow configurations

You can set the log level and other LangFlow configurations in the `values.yaml` file.

```yaml
env:
- name: LANGFLOW_LOG_LEVEL
value: "INFO"
```

### Configure secrets and variables

In order to inject secrets and LangFlow global variables, you can use the `secrets` and `env` sections in the `values.yaml` file.

Let's say your flow uses a global variable which is a secret; when you export the flow as JSON, it's recommended to not include it.
When importing the flow in the LangFlow runtime, you can set the global variable using the `env` section in the `values.yaml` file.
Assuming you have a global variable called `openai_key_var`, you can read it directly from a secret:

```yaml
env:
- name: openai_key_var
valueFrom:
secretKeyRef:
name: openai-key
key: openai-key
```

or directly from the values file (not recommended for secret values!):

```yaml
env:
- name: openai_key_var
value: "sk-...."
```

### Scaling

You can scale the number of replicas for the LangFlow app by changing the `replicaCount` value in the `values.yaml` file.

```yaml
replicaCount: 3
```

To scale vertically (increase the resources for the pods), you can set the `resources` values in the `values.yaml` file.

```yaml
resources:
requests:
memory: "2Gi"
cpu: "1000m"
```

### Other examples

Visit the [LangFlow Helm Charts repository](https://github.com/langflow-ai/langflow-helm-charts) for more examples and configurations.

Use the [default values file](https://github.com/langflow-ai/langflow-helm-charts/blob/main/charts/langflow-runtime/values.yaml) as reference for all the options available.

Visit the [examples directory](https://github.com/langflow-ai/langflow-helm-charts/tree/main/examples/langflow-runtime) to learn more about different deployment options.
1 change: 1 addition & 0 deletions docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ module.exports = {
items: [
"deployment/docker",
"deployment/backend-only",
"deployment/kubernetes",
"deployment/gcp-deployment",
],
},
Expand Down