Discussions

• 1 You must be logged in to vote

  🙏

  Is decoding one token or two token at a time in llama.cpp?

  afsara-ben asked Apr 29, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💡

  BitNet b1.58 2B4T

  iplayfast started Apr 29, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  💡

  Microsoft released BitNet inference code bitnet.cpp, shows huge improvement over llama.cpp (100B at 7 token/s on a single M2 Ultra!)

  Dampfinchen started Oct 18, 2024 in Ideas

  

  

  

  

  2
• 3 You must be logged in to vote

  💬

  Insufficient documentation for -ot and --override-tensor flag

  Juahyori started Apr 28, 2025 in General

  

  

  1
• 1 You must be logged in to vote

  💬

  What application can be used with llama.cpp that enables the model to read/write txt file

  harisnae started Apr 29, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to make prompt processing faster for Qwen3-30B moe?

  CHNtentes asked Apr 29, 2025 in Q&A · Unanswered

  

  

  0
• 5 You must be logged in to vote

  💡

  Add energy efficiency tracking index to llama-bench tests

  RafaAguilar started Dec 2, 2023 in Ideas

  

  

  

  

  3
• 0 You must be logged in to vote

  🙏

  Slow prompt processing?

  pt13762104 asked Apr 29, 2025 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  🙌

  M-series Macs running llama.cpp in GPU-Accelerated Containers - New Benchmark Results

  AndreasKunar started Apr 16, 2025 in Show and tell

  

  

  

  

  15
• 15 You must be logged in to vote

  💬

  Performance of llama.cpp on Snapdragon X Elite/Plus

  AndreasKunar started Jul 3, 2024 in General

  

  

  

  

  101
• 8 You must be logged in to vote

  💬

  Performance of llama.cpp with Vulkan

  performance Speed related topics need feedback Testing and feedback with results are needed

  netrunnereve started Dec 18, 2024 in General

  

  

  

  

  164
• 1 You must be logged in to vote

  🙏

  error loading model: invalid model: tensor 'blk.0.attn_norm.weight' is duplicated

  gomidas asked Apr 27, 2025 in Q&A · Unanswered

  

  

  1
• 7 You must be logged in to vote

  💡

  llama_server: allow streaming tool use

  khimaros started Mar 27, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  Integration with Unreal Engine 5.5.4

  gomidas asked Apr 25, 2025 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to use function calls?

  Snowiiii asked Apr 27, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  the logic of mul_mat_vec_q in llama.cpp

  liuyao0 asked Apr 25, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Math & Code Benchmark/Testing for GGUFs

  Bobchenyx asked Apr 26, 2025 in Q&A · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  Which llama.cpp Windows package to choose for my hardware

  domasofan asked Apr 25, 2025 in Q&A · Answered

  

  

  2
• 3 You must be logged in to vote

  🙏

  how to disable built-in chat template for llama-server

  RunningLeon asked Jan 7, 2025 in Q&A · Unanswered

  

  

  

  

  2
• 3 You must be logged in to vote

  🙏

  need help on llama.cpp in docker container with arm64 (raspberry pi 5)+ vulkan + amdgpu

  zphilip asked Mar 29, 2025 in Q&A · Unanswered

  

  

  1
• 6 You must be logged in to vote

  💬

  async/parallel speculative execution

  okuvshynov started Apr 23, 2024 in General

  

  

  

  

  16
• 7 You must be logged in to vote

  💬

  [SYCL][Intel GPU] Long Term Features & Issues Tracking

  Intel GPU

  airMeng started Feb 2, 2024 in General

  

  

  

  

  168
• 2 You must be logged in to vote

  💬

  RPC Per Host Layer Count Specification

  sigkill started Apr 2, 2025 in General

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  performance difference based on number of tokens to process

  okuvshynov asked Apr 20, 2024 in Q&A · Answered

  

  

  

  

  3
• 6 You must be logged in to vote

  💡

  Support Phi 3.5 MoE

  Dampfinchen started Aug 21, 2024 in Ideas

  

  

  

  

  5