Ops
The ops module provides sets of operations and formats for building and evolving genetic programs including graphs and trees. In the language of radiate, when using an op, it is the Allele of the GraphNode or TreeNode.
An op is a function that takes a number of inputs and returns a single output. The op can be a constant value, a variable, or a function that operates on the inputs.
The op comes in five flavors, mirroring the variants of the Op<T> enum:
- Function (
Fn): Stateless functions that take inputs and return a value (e.g.Add,Sigmoid). - Variable (
Var): Reads from an input vector, returning the value at that index. - Constant (
Const): A fixed value that does not change - returning the value when called. - Value (
Value): A stateful operation that holds data (aParam<T>) alongside a function, allowing for learnable parameters such as theWeightop. - Pair (
Pair): LikeValue, but the stored data is a pair of values (aParam<(T, T)>), such as the two learnable weights of theDotop.
Each op has an arity, defining the number of inputs it accepts. For example, the Add operation has an arity of 2 because it takes two inputs and returns their sum. The Const operation has an arity of 0 because it does not take any inputs, it just returns its value. The Var operation has an arity of 0 because it takes an index as a parameter, and returns the value of the input at that index.
Provided Ops include:
Basic ops
| Name | Arity | Description | Initialize | Type |
|---|---|---|---|---|
const |
0 | A fixed, named constant value | Op::constant(name, value) |
Const |
one / two / three |
0 | The constants 1, 2 and 3 | Op::one(), Op::two(), Op::three() |
Const |
erc |
0 | Ephemeral random constant: starts at a random value in [-1, 1] that mutation can tune | Op::erc() |
Value |
var |
0 | Variable. input[i] - return the value of the input at index i |
Op::var(i) |
Var |
vars |
0 | One var for each index in a range |
Op::vars(range) |
Var |
named_var |
0 | A var with a custom name instead of Xi |
Op::named_var(name, i) |
Var |
category |
0 | A var for a categorical input with k possible values |
Op::category(name, i, k) |
Var |
identity |
1 | return the input value | Op::identity() |
Fn |
Python
Python provides const (as Op.const(name, value)), var (as Op.var(i)), erc and identity. one/two/three, vars, named_var and category are Rust-only.
Basic math operations
| Name | Arity | Description | Initialize | Type |
|---|---|---|---|---|
Add |
2 | x + y | Op::add() |
Fn |
Sub |
2 | x - y | Op::sub() |
Fn |
Mul |
2 | x * y | Op::mul() |
Fn |
Div |
2 | x / y (1 if y ≈ 0) | Op::div() |
Fn |
Sum |
Any | Sum of n values | Op::sum() |
Fn |
Product |
Any | Product of n values | Op::prod() |
Fn |
Difference |
Any | x1 - x2 - ... - xn (the first input minus the rest) | Op::diff() |
Fn |
Neg |
1 | -x | Op::neg() |
Fn |
Abs |
1 | abs(x) | Op::abs() |
Fn |
pow |
2 | x^y | Op::pow() |
Fn |
Sqrt |
1 | sqrt(x) | Op::sqrt() |
Fn |
Exp |
1 | e^x | Op::exp() |
Fn |
Log |
1 | ln(x) (0 if x ≤ 0) | Op::log() |
Fn |
Sin |
1 | sin(x) | Op::sin() |
Fn |
Cos |
1 | cos(x) | Op::cos() |
Fn |
Tan |
1 | tan(x) | Op::tan() |
Fn |
Max |
Any | Max of n values | Op::max() |
Fn |
Min |
Any | Min of n values | Op::min() |
Fn |
Ceil |
1 | ceil(x) | Op::ceil() |
Fn |
Floor |
1 | floor(x) | Op::floor() |
Fn |
Sign |
1 | -1, 0 or 1, matching the sign of x | Op::sign() |
Fn |
Reciprocal |
1 | 1 / x (1 if x ≈ 0) | Op::reciprocal() |
Fn |
LogSumExp |
2 | ln(e^x + e^y) | Op::logsumexp() |
Fn |
Gaussian |
Any | e^(-x²), where x is the sum of the inputs | Op::gaussian() |
Fn |
Tooth |
Any | x mod 1 (keeps the sign of x), where x is the sum of the inputs | Op::tooth() |
Fn |
Weight |
1 | x * w (input multiplied by a learnable weight) | Op::weight() |
Value |
Dot |
2 | x * w1 + y * w2 (each input multiplied by its own learnable weight) | Op::dot() |
Pair |
All of these clamp their output and return 0 instead of NaN, so an evolved program can't overflow or produce invalid values. The clamp is ±1e7 for f32 and ±1e14 for f64. "≈ 0" means within 1e-6 of zero for f32 and 1e-12 for f64.
Weight starts at a random value in [-1, 1], and Dot starts with both weights at 1. To start from specific values, use Op::weight_with(w), Op::dot_with((w1, w2)) or Op::erc_with(v).
Python
LogSumExp and the *_with constructors are Rust-only. Every other op in this table is available in Python.
Activation Ops
These are the most common activation functions used in Neural Networks. Each accepts any number of inputs, sums them first, then applies the activation — so x in the descriptions below refers to the sum of the node's inputs.
| Name | Arity | Description | Initialize | Type |
|---|---|---|---|---|
Sigmoid |
Any | 1 / (1 + e^-x) | Op::sigmoid() |
Fn |
Tanh |
Any | tanh(x) | Op::tanh() |
Fn |
ReLU |
Any | max(0, x) | Op::relu() |
Fn |
LeakyReLU |
Any | x if x > 0 else 0.5x | Op::leaky_relu() |
Fn |
ELU |
Any | x if x > 0 else 0.5(e^x - 1) | Op::elu() |
Fn |
Linear |
Any | x (sum of inputs, identity activation) | Op::sum() |
Fn |
Softplus |
Any | log(1 + e^x) | Op::softplus() |
Fn |
Swish |
Any | x / (1 + e^-x) | Op::swish() |
Fn |
Mish |
Any | x * tanh(ln(1 + e^x)) | Op::mish() |
Fn |
bool Ops
| Name | Arity | Description | Initialize | Type |
|---|---|---|---|---|
And |
2 | x && y | Op::and() |
Fn |
Or |
2 | x || y |
Op::or() |
Fn |
Not |
1 | !x | Op::not() |
Fn |
Xor |
2 | x ^ y | Op::xor() |
Fn |
Nand |
2 | !(x && y) | Op::nand() |
Fn |
Nor |
2 | !(x || y) |
Op::nor() |
Fn |
Xnor |
2 | !(x ^ y) | Op::xnor() |
Fn |
Greater |
2 | x > y | Op::gt() |
Fn |
Less |
2 | x < y | Op::lt() |
Fn |
GreaterEqual |
2 | x >= y | Op::ge() |
Fn |
IfElse |
3 | if x then y else z | Op::if_else() |
Fn |
AndThen |
3 | x && y && z | Op::and_then() |
Fn |
OrElse |
3 | x || y || z |
Op::or_else() |
Fn |
Implies |
2 | !x || y |
Op::implies() |
Fn |
bool_ops() returns one op per distinct function: and, or, not, xor, xnor, nand,
nor, implies and if_else. It leaves out gt, ge and lt (gt and lt are each other
with the arguments swapped, and ge is implies swapped) and the three-input and_then /
or_else.
Removed duplicates
eq, iff, ne and le were removed: they computed the same function as xnor, xnor,
xor and implies. Programs saved with those names no longer load; replace them with the
equivalent op in the saved file. The same applies to the old linear (now sum) and w2
(now dot) names.
Python
The bool ops are Rust-only.
import radiate as rd
add = rd.Op.add()
sub = rd.Op.sub()
mul = rd.Op.mul()
div = rd.Op.div()
constant = rd.Op.const("my const", 42.0)
variable = rd.Op.var(0)
sigmoid = rd.Op.sigmoid()
relu = rd.Op.relu()
tanh = rd.Op.tanh()
add_result = rd.Op.add().eval(1.0, 2.0) # result is 3.0
const_result = rd.Op.const("my const", 42.0).eval() # result is 42.0
var_result = rd.Op.var(0).eval(5.0, 10.0) # result is 5.0 when evaluated with inputs
// Example usage of an Op
let fn_op = Op::add();
let result = fn_op.eval(&[1.0, 2.0]); // result is 3.0
// Example usage of a constant Op
let const_op = Op::constant("fourty-two", 42.0);
let result = const_op.eval(&[]); // result is 42.0
// Example usage of a variable Op
let var_op = Op::var(0); // Read from input at index 0
let inputs = var_op.eval(&[5.0, 10.0]); // result is 5.0 when evaluated with inputs
Want to create your own Op<T>? It's pretty simple! But beware, radiate cannot serialize/deserialize custom ops. Let's create a custom Square operation that squares its input.
fn my_square_op(inputs: &[f32]) -> f32 {
inputs[0] * inputs[0]
}
// Supply a name, arity (number of inputs), and function to create the Op.
// Square takes a single input, so its arity is Exact(1).
let square_op = Op::Fn("Square", Arity::Exact(1), my_square_op);
Now you have a new square_op which is completely compatible with the rest of the Radiate GP system and can be plugged in anywhere a regular Op can be used! For more information on creating ops, check out the API docs to see how the rest are created - it's not too crazy.
Alters
OperationMutator
Inputs
rate: f32 - Mutation rate (0.0 to 1.0)replace_rate: f32 - Rate at which to replace an oldopwith a completely new one (0.0 to 1.0)
- Purpose: Randomly mutate an operation within a
TreeNodeorGraphNode.
This mutator randomly changes or alters the op of a node within a TreeChromosome or GraphChromosome. It can replace the op with a new one from the node set or tune the learnable parameters of Weight, Dot and erc ops.