I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training. I've had a lot of people ask if they can contribute. Command used to create the GPTQ: python llama.py ehartford_Wizard-Vicuna-30B-Uncensored c4 -wbits 4 -act-order -true-sequential -save_safetensors įor further support, and discussions on these models and AI in general, join us at:.Works with text-generation-webui one-click-installers.Works with all versions of GPTQ-for-LLaMa code, both Triton and CUDA branches.
It was created without group_size to minimise VRAM usage, and with -act-order to improve inference quality. This will work with all versions of GPTQ-for-LLaMa. Once it says it's loaded, click the Text Generation tab and enter a prompt!Ĭompatible file.Click Reload the Model in the top right.Click Save settings for this model in the top right.Fill out the GPTQ parameters on the right: Bits = 4, Groupsize = None, model_type = Llama.If you see an error in the bottom right, ignore it - it's temporary.In the Model drop-down: choose the model you just downloaded, Wizard-Vicuna-30B-Uncensored-GPTQ.Click the Refresh icon next to Model in the top left.Wait until it says it's finished downloading.Under Download custom model or LoRA, enter TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ.Open the text-generation-webui UI as normal. How to easily download and use this model in text-generation-webui float16 HF format model for GPU inference and further conversions.4bit and 5bit GGML models for CPU inference.It is the result of quantising to 4bit using GPTQ-for-LLaMa. This is GPTQ format quantised 4bit models of Eric Hartford's Wizard-Vicuna 30B. Want to contribute? TheBloke's Patreon pageĮric Hartford's Wizard-Vicuna-30B-Uncensored GPTQ
0 kommentar(er)
