GPTits (v0.2c)
changelog:
Added button description on the bottom right
You can use LLAMA offline, without using the OpenAI service
(v0.2b) GPT error messages with more context
(v0.2b) LLAMA_MODEL setting removed to increase speed, to change the model edit the model name in /LAMA/main.py
(v0.2b) included empty default "models" folder inside LLAMA
(v0.2b) Fixed bug that prevented the use of LLAMA in some cases
(v0.2b) included a backup of config.txt
(v0.2c) the LLAMA_URL can be used with a URL without a PORT
How to configure LLAMA
Leave the OPENAI_API_KEY empty.
Requires Python 3 to be installed. (Tested on Python 3.11.3)
Open the LLAMA folder, run the "llama_server.bat".
You will receive the address where it is running.
Running on http://127.0.0.1:5000
Configure LLAMA_URL and LLAMA_PORT if necessary.
Put your model in the LLAMA/models folder.
Edit model in "/LAMA/main.py" to the name of the file, example:
model = "ggml-alpaca-7b-q4.bin"
You can use this model or other compatible:
https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin
Questions at discord