Commits

Breadcrumbs

History for

onmain

User selector

All users

All time

Commit history

Commits on Jan 8, 2025

• fix: streaming resource lock (#1879)

  Show description for e8f14ce
  

  gjpower

  and

  abetlen

  authored
  e8f14ceView commit details

  Copy full SHA for e8f14ce

  View code at this pointBrowse repository at this point

Commits on Dec 9, 2024

• fix: add missing await statements for async exit_stack handling (#1858)

  gjpower

  authored
  afedfc8View commit details

  Copy full SHA for afedfc8

  View code at this pointBrowse repository at this point

Commits on Dec 6, 2024

• fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context (#1798)

  Show description for 9bd0c95
  

  gjpower

  and

  abetlen

  authored
  9bd0c95View commit details

  Copy full SHA for 9bd0c95

  View code at this pointBrowse repository at this point

• fix: added missing exit_stack.close() to /v1/chat/completions (#1796)

  Show description for 073b7e4
  Ian321

  authored
  073b7e4View commit details

  Copy full SHA for 073b7e4

  View code at this pointBrowse repository at this point

Commits on Sep 20, 2024

• feat: Add option to configure n_ubatch

  abetlen

  committed
  6c44a3fView commit details

  Copy full SHA for 6c44a3f

  View code at this pointBrowse repository at this point

Commits on Aug 29, 2024

• feat: Add server chat_format minicpm-v-2.6 for MiniCPMv26ChatHandler

  abetlen

  committed
  97d527eView commit details

  Copy full SHA for 97d527e

  View code at this pointBrowse repository at this point

Commits on Jul 17, 2024

• fix(server): Use split_mode from model settings (#1594)

  Show description for 66d5cdd
  

  grider-withourai

  and

  abetlen

  authored
  66d5cddView commit details

  Copy full SHA for 66d5cdd

  View code at this pointBrowse repository at this point

Commits on Jul 9, 2024

• fix(misc): Format

  abetlen

  committed
  c1ae815View commit details

  Copy full SHA for c1ae815

  View code at this pointBrowse repository at this point

Commits on Jul 2, 2024

• fix(misc): Fix type errors

  abetlen

  committed
  387d01dView commit details

  Copy full SHA for 387d01d

  View code at this pointBrowse repository at this point

• fix(server): Fix bug in FastAPI streaming response where dependency was released before request completes causing SEGFAULT

  abetlen

  committed
  296304bView commit details

  Copy full SHA for 296304b

  View code at this pointBrowse repository at this point

• fix(server): Update `embeddings=False` by default. Embeddings should be enabled by default for embedding models.

  abetlen

  committed
  bf5e0bbView commit details

  Copy full SHA for bf5e0bb

  View code at this pointBrowse repository at this point

Commits on Jun 13, 2024

• feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

  Show description for 320a5d7
  

  jkawamoto

  and

  abetlen

  authored
  320a5d7View commit details

  Copy full SHA for 320a5d7

  View code at this pointBrowse repository at this point

Commits on Jun 4, 2024

• feat: adding `rpc_servers` parameter to `Llama` class (#1477)

  Show description for d634efc
  

  chraac

  and

  abetlen

  authored
  d634efcView commit details

  Copy full SHA for d634efc

  View code at this pointBrowse repository at this point

Commits on May 29, 2024

• fix: fix string value kv_overrides. Closes #1487

  abetlen

  committed
  df45a4bView commit details

  Copy full SHA for df45a4b

  View code at this pointBrowse repository at this point

Commits on May 14, 2024

• feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)

  Show description for 5212fb0
  twaka

  authored
  5212fb0View commit details

  Copy full SHA for 5212fb0

  View code at this pointBrowse repository at this point

Commits on May 5, 2024

• feat(server): Add support for setting root_path. Closes #1420

  abetlen

  committed
  0318702View commit details

  Copy full SHA for 0318702

  View code at this pointBrowse repository at this point

Commits on May 3, 2024

• Merge branch 'main' of github.com:abetlen/llama_cpp_python into main

  abetlen

  committed
  f9b7221View commit details

  Copy full SHA for f9b7221

  View code at this pointBrowse repository at this point

• feat(server): Remove temperature bounds checks for server. Closes #1384

  abetlen

  committed
  0a454beView commit details

  Copy full SHA for 0a454be

  View code at this pointBrowse repository at this point

• fix(server): Propagate `flash_attn` to model load. (#1424)

  dthuerck

  authored
  2138561View commit details

  Copy full SHA for 2138561

  View code at this pointBrowse repository at this point

Commits on May 2, 2024

• feat: Add llama-3-vision-alpha chat format

  abetlen

  committed
  31b1d95View commit details

  Copy full SHA for 31b1d95

  View code at this pointBrowse repository at this point

Commits on Apr 30, 2024

• feat: Add option to enable `flash_attn` to Lllama params and ModelSettings

  abetlen

  committed
  22d77eeView commit details

  Copy full SHA for 22d77ee

  View code at this pointBrowse repository at this point

• feat: Generic Chat Formats, Tool Calling, and Huggingface Pull Support for Multimodal Models (Obsidian, LLaVA1.6, Moondream) (#1147)

  Show description for fe2da09
  abetlen

  authored
  fe2da09View commit details

  Copy full SHA for fe2da09

  View code at this pointBrowse repository at this point

Commits on Apr 26, 2024

• fix: pydantic deprecation warning

  abetlen

  committed
  fcfea66View commit details

  Copy full SHA for fcfea66

  View code at this pointBrowse repository at this point

Commits on Apr 23, 2024

• feat(server): Provide ability to dynamically allocate all threads if desired using `-1` (#1364)

  sean-bailey

  authored
  53ebcc8View commit details

  Copy full SHA for 53ebcc8

  View code at this pointBrowse repository at this point

Commits on Apr 17, 2024

• feat: add `disable_ping_events` flag (#1257)

  Show description for b73c73c
  khimaros

  authored
  b73c73cView commit details

  Copy full SHA for b73c73c

  View code at this pointBrowse repository at this point

• feat: Use all available CPUs for batch processing (#1345)

  

  ddh0

  authored and

  abetlen

  committed
  c96b2daView commit details

  Copy full SHA for c96b2da

  View code at this pointBrowse repository at this point

Commits on Apr 10, 2024

• feat: Add support for yaml based configs

  abetlen

  committed
  060bfa6View commit details

  Copy full SHA for 060bfa6

  View code at this pointBrowse repository at this point

Commits on Apr 1, 2024

• feat: add support for KV cache quantization options (#1307)

  Show description for f165048
  

  Limour-dev

  and

  abetlen

  authored
  f165048View commit details

  Copy full SHA for f165048

  View code at this pointBrowse repository at this point

Commits on Mar 31, 2024

• feat: Add logprobs support to chat completions (#1311)

  Show description for aa9f1ae
  

  windspirit95

  and

  abetlen

  authored
  aa9f1aeView commit details

  Copy full SHA for aa9f1ae

  View code at this pointBrowse repository at this point

Commits on Mar 23, 2024

• fix(server): minor type fixes

  abetlen

  committed
  d11ccc3View commit details

  Copy full SHA for d11ccc3

  View code at this pointBrowse repository at this point

Commits on Mar 19, 2024

• docs: Add chat examples to openapi ui

  abetlen

  committed
  f7decc9View commit details

  Copy full SHA for f7decc9

  View code at this pointBrowse repository at this point

Commits on Mar 9, 2024

• feat: Add endpoints for tokenize, detokenize and count tokens (#1136)

  Show description for c139f8b
  

  felipelo

  and

  abetlen

  authored
  c139f8bView commit details

  Copy full SHA for c139f8b

  View code at this pointBrowse repository at this point

Commits on Feb 28, 2024

• misc: Format

  abetlen

  committed
  727d60cView commit details

  Copy full SHA for 727d60c

  View code at this pointBrowse repository at this point

• feat: Update llama.cpp

  abetlen

  committed
  0d37ce5View commit details

  Copy full SHA for 0d37ce5

  View code at this pointBrowse repository at this point

Commits on Feb 26, 2024

• feat(server): Add support for pulling models from Huggingface Hub (#1222)

  Show description for 4d574bd
  abetlen

  authored
  4d574bdView commit details

  Copy full SHA for 4d574bd

  View code at this pointBrowse repository at this point