Fine Tune mistral-7b-instruct on Predibase with Your Own Data and LoRAX

In today’s rapidly advancing world of artificial intelligence, fine-tuning pre-trained language models has become an essential practice for achieving state-of-the-art performance in various natural language processing tasks. One such powerful model is mistral-7b-instruct, known for its impressive capabilities in understanding and generating human-like text. In this article, we will delve into the process of fine-tuning mistral-7b-instruct on Predibase, a platform designed to simplify the deployment and scaling of machine learning models. Additionally, we will explore how integrating LoRAX, a cutting-edge tool for enhancing model interpretability and control, can further elevate the performance and reliability of your fine-tuned model. Whether you are a seasoned data scientist or a newcomer to the field, this guide will provide you with valuable insights and practical steps to harness the full potential of mistral-7b-instruct with your own data and LoRAX.

Note: This article is a continuation of the previous article:

Revolutionizing AI with Predibase: The Future of Serverless, Fine-Tuned LLMs

Prerequisites:

1 — Get the Trial Free Account with Predibase

Fill out the form with your info and submit it:

You will receive an email with a link:

2 — Get the API Key

After you sign in, go to Settings:

Go to “My Profile”

Down on the page, click on “Create API Token.”

We will use this token in the following exercise.

Check your balance:

Check your balance at Billing to see how cheap it is to Fine-Tune and Deploy a model compared to AWS and Azure

Hands-on Project

The following hands-on is a quick start guide for fine-tuning Large Language Models (LLMs) using Predibase, specifically focusing on a code generation use case. The project demonstrates how to prompt, fine-tune, and deploy LLMs to generate code from natural language instructions. Here’s a breakdown of the code:

Step1: Installation:

The predibase library is installed using pip.

!pip install -U predibase --quiet

Step 2: Setup:

A PredibaseClient object is initialized with an API token to interact with the Predibase services.

Copy the Token

Settings/MyProfile

from predibase import PredibaseClient

# Use the API Token we got before
pc = PredibaseClient(token="{your-api-token}")

Prompting a Deployed LLM:

The following code demonstrates how to use a pre-deployed serverless mistral-7b-instruct model to generate code based on a given instruction and input. The result is printed to the console.

# Create a deployment
llm_deployment = pc.LLM("pb://deployments/mistral-7b-instruct")

Let’s ask the same question we asked Llama2:

result: list = llm_deployment.prompt("""
    Below is an instruction that describes a task, paired with an input
    that may provide further context. Write a response that appropriately
    completes the request.

    ### Instruction: Write an algorithm in Java to reverse the words in a string.

    ### Input: The quick brown fox

    ### Response:
""", max_new_tokens=256)
print(result.response)

Here is the answer:

Here is an algorithm in Java to reverse the words in a string:
```java

public static String reverseWords(String input) {
    String[] words = input.split(" ");
    StringBuilder sb = new StringBuilder();
    for (String word : words) {
        sb.append(word).append(" ");
    }
    return sb.toString().trim();
}
```

To use this algorithm, you can call the `reverseWords` method with the input string as an argument:

```java
String input = "The quick brown fox";
String reversed = reverseWords(input);
System.out.println(reversed); // Output: "fox brown quick The"
```
This algorithm works by splitting the input string into an array of words using the `split` method, then building a new string by appending each word to a `StringBuilder` with a space in between. Finally, it returns the reversed string by trimming the extra space at the end.

If you go back to the previous article, you can see the difference in answering the same question

Let’s format a question to ask the base model before fine-tuning. Since I am using my own data here and you will be using a completely different doc, I will not show my questions and answers here. Please, use the template above to play around with your questions.

Fine-tuning a Pretrained LLM:

Upload Your CSV file and create dataset

we will be using the following command

dataset = pc.upload_file('{Path to local file}', 'A Name to your Dataset')

The previous command will upload a file from which a dataset will be created.

You will need to create a csv file with two columns, one is “question” and the other is “answer”

Define the template used to prompt the model.

prompt_template = """Below is a question about .... . Please write a response that 
    accurately answers the questins.....

    ### Question: {question}

    ### Response:
"""

Specify the Huggingface LLM you want to fine-tune

llm = pc.LLM("hf://mistralai/Mistral-7B-Instruct-v0.2")

Kick off a fine-tuning job on the uploaded dataset

job = llm.finetune(
    prompt_template=prompt_template,
    target="output",
    dataset=dataset,
    
)

model = job.get()

You can monitor the progress.

Keep track of the model name, because we will use it in the deployment. The rest of the process will be the same as we discussed on the following article:

Revolutionizing AI with Predibase: The Future of Serverless, Fine-Tuned LLMs

Fine Tune mistral-7b-instruct on Predibase UI