Import GGUF into Ollama

Import GGUF into Ollama

1.Create a Modelfile

##格式0

FROM tinyllama-my-model.gguf

    ### Set the system message
    SYSTEM """
    You are a super helpful helper.
    """

PARAMETER stop <s>
PARAMETER stop </s>

#格式0的运行方式：ollama run my-model "<s>\nQ: \nWhat is the capital of France?\nA:\n"



##格式1
FROM ./llama3-unsloth.Q8_0.gguf

    TEMPLATE """
    <|system|>
    
    <|user|>
    
    <|assistant|>
    """

SYSTEM """You are a helpful, smart, kind, and efficient AI assistant.Your name is Aila. You always fulfill the user's requests to the best of your ability."""

    PARAMETER temperature 0.8
    PARAMETER num_ctx 8192
    PARAMETER stop "<|system|>"
    PARAMETER stop "<|user|>"
    PARAMETER stop "<|assistant|>"

    ##格式2
    FROM GEITje-7B-chat-v2.gguf
    TEMPLATE """
    <|system|>
    
    </s>
    <|user|>
    
    </s>
    <|assistant|>
    """
    PARAMETER temperature 0.2
    PARAMETER num_ctx 8192
    PARAMETER stop "<|system|>"
    PARAMETER stop "<|user|>"
    PARAMETER stop "<|assistant|>"
    PARAMETER stop "</s>"

    ##格式3
    FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf
    PARAMETER temperature 0.7
    PARAMETER stop "<|im_start|>"
    PARAMETER stop "<|im_end|>"
    TEMPLATE """
    <|im_start|>system
    <|im_end|>
    <|im_start|>user
    <|im_end|>
    <|im_start|>assistant
    """
    SYSTEM """You are a helpful assistant."""

2.Build the Model in Ollama

ollama create example -f Modelfile

3. Run the Model

ollama run example