Open In Colab

LLaMEA Minimal Example

This notebook shows a simple usage of LLaMEA to automatically generate and refine a Python-based optimization algorithm for a toy evaluation.

[ ]:

#this dependency is sometimes missed by poetry
!pip install swig
!pip install llamea==1.0.5

[1]:

# Cell 1: Imports
import os
import numpy as np
from llamea import LLaMEA, Gemini_LLM

Cell 1: Set up the LLM

If you haven’t already, set your OpenAI or other API key in your environment variables, e.g., export OPENAI_API_KEY="..." or export GEMINI_API_KEY="...."

You can also use Gemini in most countries for free.

[2]:

from google.colab import userdata
api_key = userdata.get('GOOGLE_API_KEY_1') # <--- Make sure you add your Google API key via the Colab secrets panel
#api_key = os.getenv("GEMINI_API_KEY")
llm = Gemini_LLM(api_key, "gemini-2.0-flash")

Cell 2: Define an evaluation function for LLaMEA

  • The function must accept a “solution” argument, which contains code, a name, etc.

  • You parse solution.solution (the raw code), dynamically load it, and run it on your problem(s).

  • You then set_scores() to record how well it did.

We’ll define a simple example with a 1D quadratic function: f(x) = (x - 2)^2 We’ll ask the solution code to search for the minimum x in [-5, 5]. We’ll then return a score based on how close x is to 2. The closer, the higher the score.

[3]:

import re
import textwrap
import math

# We implement an exception to stop algorithms that try a too large budget (to prevent infinite loops).
class OverBudgetException(Exception):
    """The algorithm tried to do more evaluations than allowed."""
    pass

def evaluate_simple(solution, explogger=None):
    code_str = solution.code  # The Python code the LLM generated
    alg_name = solution.name

    # We define our 1D search space: x in [-5, 5], budget=100
    # We'll create a small function that the generated code should optimize.
    def f(x):
        # We only allow so many function calls
        if f.call_count >= 100:
            raise OverBudgetException("Budget exceeded.")
        f.call_count += 1
        return (x - 2.0)**2

    f.call_count = 0

    # Dynamically run the generated code
    # The code is expected to define a class named alg_name, with __init__(budget, dim) and __call__(f).
    # We'll create a safe execution context to run it.
    safe_globals = {
        "OverBudgetException": OverBudgetException,
        "math": math,
        "np": np,
    }
    local_env = {}
    try:
        exec(code_str, safe_globals, local_env)
    except Exception as e:
        # If there's an error in code, set the score to 0
        solution.set_scores(0, feedback=f"Runtime/Syntax error: {e}")
        return solution

    # Instantiate the class with budget=100, dim=1
    try:
        AlgorithmClass = local_env[alg_name]
        algo = AlgorithmClass(budget=100, dim=1)
    except Exception as e:
        solution.set_scores(0, feedback=f"Instantiation error: {e}")
        return solution

    # Now run the algorithm
    best_f = math.inf
    try:
        best_f, best_x = algo(f)
    except OverBudgetException:
        # If over budget, we penalize heavily
        best_f = 9999

    # We'll convert it to a "score" where smaller f is better => we do `score = 1/(1 + best_f)`
    # so that 0 => 1/1 => 1, big f => near 0
    # Note: LLaMEA is optimizing by default! (bigger is better)
    score = 1.0 / (1.0 + best_f)

    # Provide feedback
    feedback_str = f"Algorithm {alg_name} got score={score:.4f} (bigger is better)."

    # Save the score to the solution object
    solution.set_scores(score, feedback_str)
    return solution