Skip to content

Ops

The ops module provides sets of operations and formats for building and evolving genetic programs including graphs and trees. In the language of radiate, when using an op, it is the Allele of the GraphNode or TreeNode. An op is a function that takes a number of inputs and returns a single output. The op can be a constant value, a variable, or a function that operates on the inputs.

The op comes in five flavors, mirroring the variants of the Op<T> enum:

  1. Function (Fn): Stateless functions that take inputs and return a value (e.g. Add, Sigmoid).
  2. Variable (Var): Reads from an input vector, returning the value at that index.
  3. Constant (Const): A fixed value that does not change - returning the value when called.
  4. Value (Value): A stateful operation that holds data (a Param<T>) alongside a function, allowing for learnable parameters such as the Weight op.
  5. Pair (Pair): Like Value, but the stored data is a pair of values (a Param<(T, T)>), such as the two learnable weights of the Dot op.

Each op has an arity, defining the number of inputs it accepts. For example, the Add operation has an arity of 2 because it takes two inputs and returns their sum. The Const operation has an arity of 0 because it does not take any inputs, it just returns its value. The Var operation has an arity of 0 because it takes an index as a parameter, and returns the value of the input at that index.

Provided Ops include:

Basic ops
Name Arity Description Initialize Type
const 0 A fixed, named constant value Op::constant(name, value) Const
one / two / three 0 The constants 1, 2 and 3 Op::one(), Op::two(), Op::three() Const
erc 0 Ephemeral random constant: starts at a random value in [-1, 1] that mutation can tune Op::erc() Value
var 0 Variable. input[i] - return the value of the input at index i Op::var(i) Var
vars 0 One var for each index in a range Op::vars(range) Var
named_var 0 A var with a custom name instead of Xi Op::named_var(name, i) Var
category 0 A var for a categorical input with k possible values Op::category(name, i, k) Var
identity 1 return the input value Op::identity() Fn

Python

Python provides const (as Op.const(name, value)), var (as Op.var(i)), erc and identity. one/two/three, vars, named_var and category are Rust-only.

Basic math operations
Name Arity Description Initialize Type
Add 2 x + y Op::add() Fn
Sub 2 x - y Op::sub() Fn
Mul 2 x * y Op::mul() Fn
Div 2 x / y (1 if y ≈ 0) Op::div() Fn
Sum Any Sum of n values Op::sum() Fn
Product Any Product of n values Op::prod() Fn
Difference Any x1 - x2 - ... - xn (the first input minus the rest) Op::diff() Fn
Neg 1 -x Op::neg() Fn
Abs 1 abs(x) Op::abs() Fn
pow 2 x^y Op::pow() Fn
Sqrt 1 sqrt(x) Op::sqrt() Fn
Exp 1 e^x Op::exp() Fn
Log 1 ln(x) (0 if x ≤ 0) Op::log() Fn
Sin 1 sin(x) Op::sin() Fn
Cos 1 cos(x) Op::cos() Fn
Tan 1 tan(x) Op::tan() Fn
Max Any Max of n values Op::max() Fn
Min Any Min of n values Op::min() Fn
Ceil 1 ceil(x) Op::ceil() Fn
Floor 1 floor(x) Op::floor() Fn
Sign 1 -1, 0 or 1, matching the sign of x Op::sign() Fn
Reciprocal 1 1 / x (1 if x ≈ 0) Op::reciprocal() Fn
LogSumExp 2 ln(e^x + e^y) Op::logsumexp() Fn
Gaussian Any e^(-x²), where x is the sum of the inputs Op::gaussian() Fn
Tooth Any x mod 1 (keeps the sign of x), where x is the sum of the inputs Op::tooth() Fn
Weight 1 x * w (input multiplied by a learnable weight) Op::weight() Value
Dot 2 x * w1 + y * w2 (each input multiplied by its own learnable weight) Op::dot() Pair

All of these clamp their output and return 0 instead of NaN, so an evolved program can't overflow or produce invalid values. The clamp is ±1e7 for f32 and ±1e14 for f64. "≈ 0" means within 1e-6 of zero for f32 and 1e-12 for f64.

Weight starts at a random value in [-1, 1], and Dot starts with both weights at 1. To start from specific values, use Op::weight_with(w), Op::dot_with((w1, w2)) or Op::erc_with(v).

Python

LogSumExp and the *_with constructors are Rust-only. Every other op in this table is available in Python.

Activation Ops

These are the most common activation functions used in Neural Networks. Each accepts any number of inputs, sums them first, then applies the activation — so x in the descriptions below refers to the sum of the node's inputs.

Name Arity Description Initialize Type
Sigmoid Any 1 / (1 + e^-x) Op::sigmoid() Fn
Tanh Any tanh(x) Op::tanh() Fn
ReLU Any max(0, x) Op::relu() Fn
LeakyReLU Any x if x > 0 else 0.5x Op::leaky_relu() Fn
ELU Any x if x > 0 else 0.5(e^x - 1) Op::elu() Fn
Linear Any x (sum of inputs, identity activation) Op::sum() Fn
Softplus Any log(1 + e^x) Op::softplus() Fn
Swish Any x / (1 + e^-x) Op::swish() Fn
Mish Any x * tanh(ln(1 + e^x)) Op::mish() Fn
bool Ops
Name Arity Description Initialize Type
And 2 x && y Op::and() Fn
Or 2 x || y Op::or() Fn
Not 1 !x Op::not() Fn
Xor 2 x ^ y Op::xor() Fn
Nand 2 !(x && y) Op::nand() Fn
Nor 2 !(x || y) Op::nor() Fn
Xnor 2 !(x ^ y) Op::xnor() Fn
Greater 2 x > y Op::gt() Fn
Less 2 x < y Op::lt() Fn
GreaterEqual 2 x >= y Op::ge() Fn
IfElse 3 if x then y else z Op::if_else() Fn
AndThen 3 x && y && z Op::and_then() Fn
OrElse 3 x || y || z Op::or_else() Fn
Implies 2 !x || y Op::implies() Fn

bool_ops() returns one op per distinct function: and, or, not, xor, xnor, nand, nor, implies and if_else. It leaves out gt, ge and lt (gt and lt are each other with the arguments swapped, and ge is implies swapped) and the three-input and_then / or_else.

Removed duplicates

eq, iff, ne and le were removed: they computed the same function as xnor, xnor, xor and implies. Programs saved with those names no longer load; replace them with the equivalent op in the saved file. The same applies to the old linear (now sum) and w2 (now dot) names.

Python

The bool ops are Rust-only.

import radiate as rd

add = rd.Op.add()
sub = rd.Op.sub()
mul = rd.Op.mul()
div = rd.Op.div()

constant = rd.Op.const("my const", 42.0)
variable = rd.Op.var(0)

sigmoid = rd.Op.sigmoid()
relu = rd.Op.relu()
tanh = rd.Op.tanh()

add_result = rd.Op.add().eval(1.0, 2.0)  # result is 3.0
const_result = rd.Op.const("my const", 42.0).eval()  # result is 42.0
var_result = rd.Op.var(0).eval(5.0, 10.0)  # result is 5.0 when evaluated with inputs
// Example usage of an Op
let fn_op = Op::add();
let result = fn_op.eval(&[1.0, 2.0]); // result is 3.0

// Example usage of a constant Op
let const_op = Op::constant("fourty-two", 42.0);
let result = const_op.eval(&[]); // result is 42.0

// Example usage of a variable Op
let var_op = Op::var(0); // Read from input at index 0
let inputs = var_op.eval(&[5.0, 10.0]); // result is 5.0 when evaluated with inputs

Want to create your own Op<T>? It's pretty simple! But beware, radiate cannot serialize/deserialize custom ops. Let's create a custom Square operation that squares its input.

fn my_square_op(inputs: &[f32]) -> f32 {
    inputs[0] * inputs[0]
}

// Supply a name, arity (number of inputs), and function to create the Op.
// Square takes a single input, so its arity is Exact(1).
let square_op = Op::Fn("Square", Arity::Exact(1), my_square_op);

Now you have a new square_op which is completely compatible with the rest of the Radiate GP system and can be plugged in anywhere a regular Op can be used! For more information on creating ops, check out the API docs to see how the rest are created - it's not too crazy.

Alters

OperationMutator

Inputs

  • rate: f32 - Mutation rate (0.0 to 1.0)
  • replace_rate: f32 - Rate at which to replace an old op with a completely new one (0.0 to 1.0)
  • Purpose: Randomly mutate an operation within a TreeNode or GraphNode.

This mutator randomly changes or alters the op of a node within a TreeChromosome or GraphChromosome. It can replace the op with a new one from the node set or tune the learnable parameters of Weight, Dot and erc ops.

import radiate as rd

# Create a mutator that has a 10% chance to mutate an op and a 50% chance to replace it with a new one
mutator = rd.Mutate.op(0.1, 0.5)  # Using the dsl syntax for mutators
// Create a mutator that has a 10% chance to mutate an op and a 50% chance to replace it with a new one
let mutator = OperationMutator::new(0.1, 0.5);