LLM OS

This project contains an initial implementation of the LLM OS proposed by Andrej Karpathy. He introduced it in this post on 𝕏 and talks more about it in this post and this video.

Architecture

Learn more about LLM-OS in Intro to Large Language Models by Andrej Karpathy.

Setup

Create virtualenv

Create your virtualenv with built-in venv package.

python -m venv .venv

Activate your environment.

source .venv/bin/activate

Install dependencies

Install dependencies using poetry.

poetry install
poetry install --with dev  # for dev

API Keys and Secrets

You have two options to create your API keys: using .env file or streamlit's secrets. Note that you can only use streamlit when running the streamlit app locally or when deployed. However, .env is advised for local development.

• Create your secrets with .env

mv .env.example .env

Edit .env file.

OPENAI_API_KEY=sk-...
EXA_API_KEY=c0...

• Create your secrets & config with streamlit

mkdir -p ./.streamlit/
touch ./.streamlit/secrets.toml  # for secret keys
touch ./.streamlit/config.toml   # for streamlit config

Edit $CWD/.streamlit/secrets.toml file.

OPENAI_API_KEY = "<get API key from platform.openai.com/api-keys>"
NEWS_API_KEY = "<get API key from https://newsapi.org/register>"

Run PgVector

We use PgVector to provide long-term memory and knowledge to the LLM OS. Please install docker desktop and run PgVector using either the helper script or the docker run command.

• Run using a helper script

./run_pgvector.sh

• OR run using the docker run command

docker run -d \
  -e POSTGRES_DB=ai \
  -e POSTGRES_USER=ai \
  -e POSTGRES_PASSWORD=ai \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v pgvolume:/var/lib/postgresql/data \
  -p 5532:5432 \
  --name pgvector \
  phidata/pgvector:16

Usage

Start your streamlit application:

streamlit run home.py

Contribution

You are very welcome to modify and use them in your own projects.

Please keep a link to the original repository. If you have made a fork with substantial modifications that you feel may be useful, then please open a new issue on GitHub with a link and short description.

License (MIT)

This project is opened under the MIT which allows very broad use for both private and commercial purposes.

A few of the images used for demonstration purposes may be under copyright. These images are included under the "fair usage" laws.