Repositories list

• llm-inference-handbook

  Public
  Everything you need to know about LLM inference

  inference-optimizationllmllm-inference+ 2

  inference-optimizationllmllm-inferenceinference-handbookinference-infrastructure

  TypeScript

  •

  Apache License 2.0

  •2525 forks•283283 stars•33 issues•11 pull request•Updated Apr 30, 2026Apr 30, 2026

• BentoML

  Public
  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  pythonmachine-learningdeep-learning+ 12

  pythonmachine-learningdeep-learningmodel-servingmultimodalmlopsml-engineeringai-inferencellmgenerative-ai+ 5

  Python

  •

  Apache License 2.0

  •958958 forks•8.6k8.6k stars•134134 issues•66 pull requests•Updated Apr 30, 2026Apr 30, 2026

• ai-gateway

  Public
  Manages Unified Access to Generative AI Services built on Envoy Gateway

  Go

  •

  Apache License 2.0

  •237237 forks•00 stars•00 issues•22 pull requests•Updated Apr 29, 2026Apr 29, 2026

• OpenLLM

  Public
  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

  llamamistralfine-tuning+ 15

  llamamistralfine-tuningmlopsbentomlvicunallmmodel-inferencellmopsllm-serving+ 8

  Python

  •

  Apache License 2.0

  •807807 forks•12k12k stars•44 issues•88 pull requests•Updated Apr 27, 2026Apr 27, 2026

• sglang

  Public
  SGLang is a fast serving framework for large language models and vision language models.

  Python

  •

  Apache License 2.0

  •5.7k5.7k forks•00 stars•00 issues•11 pull request•Updated Apr 8, 2026Apr 8, 2026

• BentoSGLang

  Public
  Python

  •11 fork•22 stars•00 issues•11 pull request•Updated Mar 26, 2026Mar 26, 2026

• openllm-models

  Public
  Python

  •1717 forks•6464 stars•00 issues•11 pull request•Updated Mar 26, 2026Mar 26, 2026

• helm-charts

  Public
  00 forks•33 stars•00 issues•00 pull requests•Updated Mar 24, 2026Mar 24, 2026

• yatai-image-builder

  Public
  🐳 Build OCI images for Bentos in k8s

  dockerkubernetesdocker-image+ 3

  dockerkubernetesdocker-imagecontainerk8scontainer-image

  Go

  •1212 forks•2020 stars•55 issues•1313 pull requests•Updated Mar 24, 2026Mar 24, 2026

• yatai-schemas

  Public
  Go

  •22 forks•11 star•11 issue•22 pull requests•Updated Mar 20, 2026Mar 20, 2026

• yatai-common

  Public
  Go

  •66 forks•11 star•00 issues•22 pull requests•Updated Mar 7, 2026Mar 7, 2026

• openai_emulator

  Public
  Python

  •

  Apache License 2.0

  •00 forks•00 stars•00 issues•00 pull requests•Updated Mar 5, 2026Mar 5, 2026

• .github

  Public
  ✨🍱🦄️

  44 forks•11 star•00 issues•00 pull requests•Updated Mar 5, 2026Mar 5, 2026

• BentoVLLM

  Public
  Self-host LLMs with vLLM and BentoML

  Python

  •

  Apache License 2.0

  •2222 forks•171171 stars•33 issues•55 pull requests•Updated Mar 3, 2026Mar 3, 2026

• bentocloud-homepage-news

  Public
  22 forks•22 stars•00 issues•00 pull requests•Updated Feb 10, 2026Feb 10, 2026

• BentoSpacy

  Public
  Python

  •00 forks•00 stars•00 issues•22 pull requests•Updated Jan 26, 2026Jan 26, 2026

• BentoTRTLLM

  Public
  Python

  •11 fork•1010 stars•22 issues•22 pull requests•Updated Jan 22, 2026Jan 22, 2026

• mise-nebius

  Public
  Shell

  •

  MIT License

  •00 forks•00 stars•00 issues•00 pull requests•Updated Jan 22, 2026Jan 22, 2026

• BentoLMDeploy

  Public
  Self-host LLMs with LMDeploy and BentoML

  Python

  •22 forks•2222 stars•11 issue•11 pull request•Updated Dec 26, 2025Dec 26, 2025

• BentoFunctionCalling

  Public
  Python

  •11 fork•99 stars•00 issues•22 pull requests•Updated Dec 26, 2025Dec 26, 2025

• lago

  Public
  Open Source Metering and Usage Based Billing API ⭐️ Consumption tracking, Subscription management, Pricing iterations, Payment orchestration & Revenue analytics

  Go

  •

  GNU Affero General Public License v3.0

  •596596 forks•00 stars•00 issues•11 pull request•Updated Nov 20, 2025Nov 20, 2025

• comfy-pack

  Public
  A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

  ai-artcomfyuigenai+ 4

  ai-artcomfyuigenaicomfyui-managercomfyui-workflowcomfyui-nodescomfyui-api

  Python

  •

  Apache License 2.0

  •3030 forks•215215 stars•1111 issues•22 pull requests•Updated Nov 10, 2025Nov 10, 2025

• BentoOCR

  Public
  Turn any OCR models into online inference API endpoint 🚀 🌖

  ocrmodel-deploymentmodel-serving+ 2

  ocrmodel-deploymentmodel-servingocr-pythonai-applications

  Python

  •44 forks•6060 stars•11 issue•00 pull requests•Updated Oct 29, 2025Oct 29, 2025

• pipelines-workflows

  Public
  Gruntwork Pipelines for BentoML

  2727 forks•00 stars•00 issues•00 pull requests•Updated Oct 23, 2025Oct 23, 2025

• lago-go-client

  Public
  Lago Go Client

  Go

  •

  MIT License

  •2424 forks•00 stars•00 issues•00 pull requests•Updated Oct 15, 2025Oct 15, 2025

• BentoYolo

  Public
  BentoML service of YOLO v8

  Python

  •

  Apache License 2.0

  •44 forks•1212 stars•11 issue•11 pull request•Updated Sep 17, 2025Sep 17, 2025

• llm-optimizer

  Public
  Benchmark and optimize LLM inference across frameworks with ease

  Python

  •

  Apache License 2.0

  •1919 forks•183183 stars•1010 issues•00 pull requests•Updated Sep 12, 2025Sep 12, 2025

• kantoku

  Public
  A Process & Socket Manager built with zmq

  Python

  •

  Other

  •262262 forks•11 star•00 issues•00 pull requests•Updated Aug 28, 2025Aug 28, 2025

• byoc

  Public
  Shell

  •11 fork•00 stars•00 issues•22 pull requests•Updated Aug 28, 2025Aug 28, 2025

• BentoProphet

  Public
  BentoML with Facebook Prophet

  Python

  •00 forks•11 star•00 issues•00 pull requests•Updated Aug 27, 2025Aug 27, 2025