This library allows you to run selected Transformer based Large Language Models using just .NET and TorchSharp.
- No python dependency
- Nuget package references to TorchSharp and SharpToken only
- Can self provision pre-trained models from HuggingFace at runtime (uses .safetensors).
- Load GPT-2 pre-trained weights from HuggingFace and perform inference using it.
- Train your own GPT-2 style models on your own data.
- Load a Roberta pre-trained model (all-distilroberta-v1) to generate vector embeddings for storing in a vector database such as Pinecone, ChromaDb or Milvus
- Use a Roberta pre-trained model (all-distilroberta-v1) to perform semantic similarity scoring of sentences.
This started as a learning exercise following along with Andre Karpathy's Lets Build GPT: From scratch, in code, spelled out. Let's train a GPT model from the ground up in C# using TorchSharp
This is just a bunch of examples of using transformer architecture all using pure C# and torch:
PerceptivePyro {command}
{command}:
* benchmark_msmarco - Evaluates GPT2 embedding sentence similarity scoring on the MS MARCO V2.1 dataset.
* benchmark_sick - Evaluates GPT2 embedding sentence similarity scoring on the SICK dataset.
* gpt2_unconditioned - Generates unconditioned random musings by GPT2 - 124M parameter model
* gpt2_large_embeddings - Generates embeddings for a sentance -
* gpt2_large_unconditioned - Generates unconditioned random musings by GPT2 - Large parameters
* gpt2_prompted - Generates a prompted response from GPT2
* gpt3_token_counts - Counts some tokens using GPT3 encoding
* gpt4_token_counts - Counts some tokens using GPT4 encoding
* roberta_similarity - Compares sentence similarity using the all-distilroberta-v1 model.
* safetensors - Test code for loading .safetensors files
* training_shakespeare - Training a small language model on Shakespeare (CUDA GPU with 10gb or more RAM required).
NOTE: The code has the following global usings in all files:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using TorchSharp;
using TorchSharp.Modules;
using static TorchSharp.torch;