Problem

class iohblade.problem.Problem(logger=None, training_instances=None, test_instances=None, name='Problem', eval_timeout=6000, dependencies=None, imports=None)

Bases: ABC

Abstract problem class.

cleanup()

abstract evaluate(solution: Solution)

Evaluates a solution on training instances and updates its fitness and feedback.

Args:

solution (Solution): Solution object to be evaluated.

get_prompt()

Get the full prompt describing the problem and how to format the answer.

set_logger(logger)

Sets the logger for this problem.

abstract test(solution: Solution)

Performs a complete evaluation on test instances and returns the fitness score.

Args:

solution (Solution): Solution object to be tested.

abstract to_dict()

Returns a dictionary representation of the problem including all parameters.

Returns:

dict: Dictionary representation of the problem.

class iohblade.problem.WrappedProblem(evaluate_fn, *, name='Problem', eval_timeout=600, training_instances=None, test_instances=None, dependencies=None, imports=None, task_prompt='', example_prompt='', logger=None)

Bases: Problem

evaluate(solution: Solution)

Evaluates a solution on training instances and updates its fitness and feedback.

Args:

solution (Solution): Solution object to be evaluated.

test(solution: Solution)

Performs a complete evaluation on test instances and returns the fitness score.

Args:

solution (Solution): Solution object to be tested.

to_dict()

Returns a dictionary representation of the problem including all parameters.

Returns:

dict: Dictionary representation of the problem.

iohblade.problem.evaluate_in_subprocess(problem, conn, solution)

Evaluate a solution in a dedicated virtual environment.

iohblade.problem.simplify_subprocess_error(stderr: str, solution=None)

Parse a Python traceback string and produce a concise error summary. Optionally include the offending line of code from solution.code.

iohblade.problem.wrap_problem(evaluate_fn, *, name='Problem', eval_timeout=6000, training_instances=None, test_instances=None, dependencies=None, imports=None, task_prompt='', example_prompt='', logger=None)