看到 Google 這篇,這篇 report,以及 HuggingFace 這篇,就知道 Goolge 針對 Code LLM 推出了對應的版本。首先先看一下 CodeGemma 2B, 7B 與 7B Instruct 的差別。
雖然 HugginfFace 有個 Notebook 介紹如何使用 transformers 設定 inference environment,但是身為一個懶人工程師,透過目前 (2024.04) 主流的方式 (不外乎 LM Studio 或是 Ollama) ,採用 Ollama 也是合理的選擇。不多說,直接看程式。
Notebook 設定 TPU 或是 T4,下載 ollama
!curl -fsSL https://ollama.com/install.sh | sh
ollama server mode
import subprocess subprocess.Popen(["ollama", "serve"])
下載 codegemma:7b
!ollama pull codegemma:7b
準備個 function 方便後續直接呼叫 ollama serve endpoint
import requests import json model_name = "codegemma:7b" url="http://localhost:11434/api/generate" headers ={ "Content-Type":"application/json" } history=[] def generate_response(prompt): history.append(prompt) final_prompt = "\n".join(history) data = { "model":model_name, "prompt":final_prompt, "stream":False } response = requests.post(url,headers=headers, data=json.dumps(data)) if response.status_code ==200: response = response.text data=json.loads(response) actual_response = data['response'] return actual_response else: print("error",response.text)
試試看簡單的 fibonacci
prompt='You are an expert programmer that writes simple, concise code and explanations. Write a python function to generate the nth fibonacci number.' resp = generate_response(prompt) import pprint pprint.pprint(resp, width=40)
結果 (結果要印出漂亮的 code snippet 好難?)
('```python\n' 'def fibonacci(n):\n' ' """\n' ' This function takes an integer n ' 'and returns the nth fibonacci ' 'number.\n' ' The fibonacci sequence is defined ' 'as follows:\n' ' - The first two numbers are 0 ' 'and 1.\n' ' - Each subsequent number is the ' 'sum of the two previous numbers.\n' ' """\n' ' if n <= 0:\n' ' return 0\n' ' elif n == 1:\n' ' return 1\n' ' else:\n' ' # Use dynamic programming to ' 'avoid redundant computations.\n' ' fib_sequence = [0, 1]\n' ' for i in range(2, n + 1):\n' ' ' 'fib_sequence.append(fib_sequence[i - ' '1] + fib_sequence[i - 2])\n' ' return fib_sequence[n]\n' '\n' '# Example usage:\n' 'nth_fibonacci = fibonacci(10)\n' 'print(nth_fibonacci) # Output: 55\n' '```')
完整的 Notebook 在這。