This is a comparison of C# and F# implementations of programming a simple neural network library I wrote for use in a side project.

A neural net is essentially a calculator that takes one or more numerical inputs and computes one or more numerical outputs. Simple neural net with an input layer, a hidden layer, and an output layer

Neural networks can fulfill a wide range of functions, but where they excel is finding an optimal output given various inputs.

The inputs are held in an input layer which connects to another layer — either the output layer or a hidden layer. Each layer, including the input layer consists of one or more neurons. Each neuron is connected to every neuron in the next layer and given a positive or negative weight indicating how important that connection is.

Individual neurons compute their values by calculating performing a summarizing function on all inputs from the prior layer with each input multiplied by the weight of the neuron connection. This then feeds in as an input into the next layer until they arrive in the output layer.

The output layer is what the caller of the neural net will use to evaluate the result of the network. This could represent anything from whether or not to buy a piece of stock, to the attractiveness of a move in a game, to what the dominant color in an image is or how happy a face appears.

Neural nets achieve these calculations via their inter-connected nature allowing flexibility to represent innovative solutions to problems, but neural nets are hard to interpret just by reading them.

Neural Networks typically have either a back propagation mechanism for training or are trained by some other factor such as a Genetic Algorithm, but both are beyond the scope of this article.

### Neuron

A Neuron summarizes and stores a value from other inputs.

### C# Neuron

 /// /// Represents a Neuron in a layer of a Neural Network. /// public class Neuron { /// /// Gets or sets the value of the Neuron. /// public decimal Value { get; set; } /// /// Creates a new instance of a . /// public Neuron() { OutgoingConnections = new List(); } /// /// Connects this Neuron to the /// /// The Neuron to connect to. internal void ConnectTo([NotNull] Neuron nextNeuron) { if (nextNeuron == null) throw new ArgumentNullException(nameof(nextNeuron)); OutgoingConnections.Add(new NeuronConnection(nextNeuron)); } private decimal _sum; private int _numInputs; /// /// Evaluates the values from the incoming connections, averages them by the count of connections, /// and calculates the Neuron's Value, which is then passed on to any outgoing connections. /// internal void Evaluate() { if (_numInputs > 0) { Value = _sum / _numInputs; _sum = 0; } OutgoingConnections.Each(c => c.Fire(Value)); } /// /// The list of outgoing Neuron connections /// [NotNull, ItemNotNull] public IList OutgoingConnections { get; } /// /// Receives a value from a connection. /// /// The value to receive internal void Receive(decimal value) => _sum += value; /// /// Registers an incoming connection from another neuron. /// /// The connection internal void RegisterIncomingConnection([NotNull] NeuronConnection neuronConnection) { if (neuronConnection == null) throw new ArgumentNullException(nameof(neuronConnection)); _numInputs++; } }

view raw
Neuron.cs
hosted with ❤ by GitHub

In the C# Implementation, there’s a lot of boiler-plate code for maintaining fields and properties as well as connecting to other nodes and layers. The core evaluation logic occurs in the `Evaluate` method and is fairly minimal, but supported by the connections established in the supporting methods.

### F# Neuron

 /// Represents a node in a Neural Network and Neuron ([] ?initialValue: decimal) = let mutable value = defaultArg initialValue 0M; let mutable inputs: NeuronConnection seq = Seq.empty; /// Exposes the current calculated amount of the Neuron member this.Value with get () = value and set (newValue) = value <- newValue /// Incoming connections from other Neurons (if any) member this.Inputs: NeuronConnection seq = inputs; /// Adds an incoming connection from another Neuron member this.AddIncomingConnection c = inputs <- Seq.append this.Inputs [c]; /// Adds all connections together, stores the result in Value, and returns the value member this.Evaluate(): decimal = if not (Seq.isEmpty this.Inputs) then do let numInputs = Seq.length this.Inputs |> decimal value <- Seq.sumBy (fun (c:NeuronConnection) -> c.Evaluate()) this.Inputs / numInputs; value; /// Connects this neuron to another and returns the connection member this.Connect(target: Neuron) = let connection = new NeuronConnection(this); target.AddIncomingConnection(connection); connection;

view raw
Neuron.fs
hosted with ❤ by GitHub

By contrast, the F# implementation is minimal and offers some brief property storage and some simple `Connect` and `Evaluate` methods.

One of the things I like about this is that there isn’t a lot of meaningless syntax, spacing, or irrelevant logic. The downside of this is that the functional syntax can be harder to read while scanning code.

### Layers

Layers are just collections of neurons in the same tier. The layer code is used for managing inter-connections between nodes in different layers.

### C# Layer

The `NeuralNetLayer` is honestly fairly boring. It acts as glue between the different nodes, but the implementation takes 100 lines of code to do that.

 /// /// Represents a layer in a neural network. This could be an input, output, or hidden layer. /// public class NeuralNetLayer : IEnumerable { private readonly IList _neurons; [CanBeNull] private NeuralNetLayer _nextLayer; /// /// Creates a new neural network layer with the given count of neurons. /// /// The number of neurons in the layer /// /// Thrown if was less than 1 /// public NeuralNetLayer(int numNeurons) { if (numNeurons <= 0) { throw new ArgumentOutOfRangeException(nameof(numNeurons), "Each layer must have at least one Neuron"); } _neurons = new List(numNeurons); numNeurons.Each(n => _neurons.Add(new Neuron())); } /// /// Gets the Neurons belonging to this layer. /// public IEnumerable Neurons => _neurons; /// /// Sets the values of the layer to the given values set. One value will be used for each neuron in the layer. /// /// The values to use. /// Thrown if did not have an expected values count. internal void SetValues([NotNull] IEnumerable values) { if (values == null) throw new ArgumentNullException(nameof(values)); if (values.Count() != _neurons.Count) throw new ArgumentException("The number of inputs must match the number of neurons in a layer", nameof(values)); int i = 0; values.Each(v => _neurons[i++].Value = v); } /// public IEnumerator GetEnumerator() => _neurons.GetEnumerator(); /// IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); /// /// Evaluates each node in the layer, as well as the next layer if one is present. /// /// The outputs from the Output layer internal IEnumerable Evaluate() { // Calculate all neurons. _neurons.Each(n => n.Evaluate()); // If this is the last layer, return its values, otherwise delegate to the next layer and return its results return _nextLayer == null ? _neurons.Select(n => n.Value) : _nextLayer.Evaluate(); } /// /// Connects this layer to the , forming connections between each node in this /// layer and each node in the next layer. /// /// The layer to connect to internal void ConnectTo([NotNull] NeuralNetLayer nextLayer) { _nextLayer = nextLayer ?? throw new ArgumentNullException(nameof(nextLayer)); _neurons.Each(source => nextLayer.Each(source.ConnectTo)); } /// /// Sets the weights in the layer to the values provided /// /// The weights to use to set in the connections [UsedImplicitly] public void SetWeights(IList weights) { int weightIndex = 0; _neurons.Each(neuron => neuron.OutgoingConnections.Each(c => c.Weight = weights[weightIndex++])); } }

view raw
NeuronLayer.cs
hosted with ❤ by GitHub

### F# Layer

The F# version is shorter which uses `Seq` (sequence) methods to delegate responsibilities to individual Neurons.

 /// A layer is just a series of Neurons in parallel that will link to every Neuron in the next layer (if any is present) and NeuralNetLayer(numNeurons: int) = do if numNeurons <= 0 then invalidArg "numNeurons" "There must be at least one neuron in each layer"; let neurons: Neuron seq = seq [ for i in 1 .. numNeurons -> new Neuron 0M] /// Layers should start with an empty collection of neurons member this.Neurons: Neuron seq = neurons; /// Sets the value of every neuron in the sequence to the corresponding ordered value provided member this.SetValues (values: decimal seq) = let assignValue (n:Neuron) (v:decimal) = n.Value <- v; Seq.iter2 assignValue this.Neurons values /// Evaluates the layer and returns the value of each node member this.Evaluate(): decimal seq = for n in this.Neurons do n.Evaluate() |> ignore; Seq.map (fun (n:Neuron) -> n.Value) this.Neurons; /// Connects every node in this layer to the target layer member this.Connect(layer: NeuralNetLayer): unit = for nSource in neurons do for nTarget in layer.Neurons do nSource.Connect(nTarget) |> ignore;

view raw
NeuralNetLayer.fs
hosted with ❤ by GitHub

### Neural Net

The Neural Net ties everything together into one wrapper. It arranges layers, exposes the inputs and outputs, and offers a way for callers to configure the network into a pre-determined arrangement.

### C# Neural Net

Keeping to form, the C# implementation does some basic iteration and enumeration, but has a pronounced amount of extra space devoted only to syntax.

 /// /// Represent a neural network consisting of an input layer, an output layer, and 0 to many hidden layers. /// Neural networks can compute values and return a set of output values, allowing for computation to occur /// between layers. /// public class NeuralNet { private readonly IList _hiddenLayers = new List(); /// /// Creates a new instance of a /// /// The number of nodes in the input layer /// The number of nodes in the output layer public NeuralNet(int numInputs, int numOutputs) { if (numInputs <= 0) throw new ArgumentOutOfRangeException(nameof(numInputs), "You must have at least one input node"); if (numOutputs <= 0) throw new ArgumentOutOfRangeException(nameof(numOutputs), "You must have at least one output node"); Inputs = new NeuralNetLayer(numInputs); Outputs = new NeuralNetLayer(numOutputs); } /// /// Adds a hidden layer to the neural net and returns the new layer. /// /// The number of neurons in the layer public void AddHiddenLayer(int numNeurons) { if (numNeurons <= 0) throw new ArgumentOutOfRangeException(nameof(numNeurons), "You cannot add a hidden layer without any nodes"); if (IsConnected) throw new InvalidOperationException("Cannot add a new layer after the network has been evaluated."); var layer = new NeuralNetLayer(numNeurons); _hiddenLayers.Add(layer); } /// /// Evaluates the result of the neural network given the specified set of . /// /// The inputs to evaluate. /// The values outputted from the output layer public IEnumerable Evaluate(IEnumerable inputs) { // Don't force people to explicitly connect EnsureConnected(); // Pipe the inputs into the network and evaluate the results Inputs.SetValues(inputs); return Inputs.Evaluate(); } /// /// Declares that the network is now complete and that connections should be created. /// public void Connect() { if (IsConnected) throw new InvalidOperationException("The Network has already been connected"); if (_hiddenLayers.Any()) { // Connect input to the first hidden layer Inputs.ConnectTo(_hiddenLayers.First()); // Connect hidden layers to each other if (_hiddenLayers.Count > 1) { for (int i = 0; i < _hiddenLayers.Count – 1; i++) { _hiddenLayers[i].ConnectTo(_hiddenLayers[i + 1]); } } // Connect the last hidden layer to the output layer _hiddenLayers.Last().ConnectTo(Outputs); } else { // No hidden layers, connect the input layer to the output layer Inputs.ConnectTo(Outputs); } IsConnected = true; } /// /// Determines whether or not the nodes in the network have been connected. /// public bool IsConnected { get; private set; } /// /// The input layer /// public NeuralNetLayer Inputs { get; } /// /// The output layer /// public NeuralNetLayer Outputs { get; } /// /// Gets all layers in the network, in order from first to last, including the Input layer, /// output layer, and any hidden layers. /// public IEnumerable Layers { get { yield return Inputs; foreach (var layer in _hiddenLayers) { yield return layer; } yield return Outputs; } } /// /// Sets the weights of all connections in the network. This is a convenience method for loading /// weight values from JSON and restoring them into the network. /// This will connect the network if it is not currently connected. /// /// The weight values from -1 to 1 for every connector in the network. [UsedImplicitly] public void SetWeights(IList weights) { // Setting weights makes no sense unless the network is connected, so ensure we're connected EnsureConnected(); ConnectorCount = 0; int weightIndex = 0; foreach (var layer in Layers) { foreach (var neuron in layer.Neurons) { foreach (var connection in neuron.OutgoingConnections) { // Early exit if we've run out of weights to go around if (weightIndex >= weights.Count) { break; } connection.Weight = weights[weightIndex++]; ConnectorCount++; } } } } /// /// Connects the neural net if it has not yet been connected /// private void EnsureConnected() { if (IsConnected) return; Connect(); } /// /// Gets the total connector count in the neural net. /// public int ConnectorCount { get; private set; } }

view raw
NeuralNet.cs
hosted with ❤ by GitHub

### F# Neural Net

The F# version is the largest F# class, but it’s logic is still fairly concise with small, focused methods.

 /// A high-level encapsulation of a neural net and NeuralNet(numInputs: int, numOutputs: int) = do if numInputs <= 0 then invalidArg "numInputs" "There must be at least one neuron in the input layer"; if numOutputs <= 0 then invalidArg "numOutputs" "There must be at least one neuron in the output layer"; let inputLayer: NeuralNetLayer = new NeuralNetLayer(numInputs); let outputLayer: NeuralNetLayer = new NeuralNetLayer(numOutputs); let mutable hiddenLayers: NeuralNetLayer seq = Seq.empty; let mutable isConnected: bool = false; let connectLayers (n1:NeuralNetLayer) (n2:NeuralNetLayer) = n1.Connect(n2); let layersMinusInput: NeuralNetLayer seq = seq { for layer in hiddenLayers do yield layer; yield outputLayer; } let layersMinusOutput: NeuralNetLayer seq = seq { yield inputLayer; for layer in hiddenLayers do yield layer; } /// Yields all connections to nodes inside of the network let connections = Seq.collect (fun (l:NeuralNetLayer) -> l.Neurons) layersMinusInput |> Seq.collect (fun (n:Neuron) -> n.Inputs); /// Gets the layers of the neural network, in sequential order member this.Layers: NeuralNetLayer seq = seq { yield inputLayer; for layer in hiddenLayers do yield layer; yield outputLayer; } /// Represents the input layer for the network which take in values from another system member this.InputLayer = inputLayer; /// Represents the last layer in the network which has the values that will be taken out of the network member this.OutputLayer = outputLayer; /// Connects the various layers of the neural network member this.Connect() = if isConnected then invalidOp "The Neural Network has already been connected"; Seq.iter2 (fun l lNext –> connectLayers l lNext) layersMinusOutput layersMinusInput isConnected <- true; /// Determines whether or not the network has been connected. After the network is connected, it can no longer be added to member this.IsConnected = isConnected; /// Adds a hidden layer to the middle of the neural net member this.AddHiddenLayer(layer: NeuralNetLayer) = if isConnected then invalidOp "Hidden layers cannot be added after the network has been connected."; hiddenLayers <- Seq.append hiddenLayers [layer]; /// Sets the weights on all connections in the neural network member this.SetWeights(weights: decimal seq) = if isConnected = false then do this.Connect(); Seq.iter2 (fun (w:decimal) (c:NeuronConnection) -> c.Weight <- w) weights connections; /// Evaluates the entire neural network and yields the result of the output layer member this.Evaluate(): decimal seq = if not isConnected then do this.Connect(); // Iterate through the layers and run calculations let mutable result: decimal seq = Seq.empty; for layer in this.Layers do result <- layer.Evaluate(); result;

view raw
NeuralNet.fs
hosted with ❤ by GitHub

### Closing Thoughts

While the F# syntax is more concise, it should be noted that this is an example that is almost ideal for a functional language. This is a key example of a component that could be used by C# code in other projects.

If you were looking to add F# to a project, I’d recommend starting with a small isolated slice of your application that other areas depend on for calculations or other sorts of transformation logic.

I personally feel that Functional Programming, or at least core concepts from those languages, can benefit software quality significantly, so this is an idea worth exploring.

### Where can I find this code?

All code in these examples is hosted on GitHub at https://github.com/IntegerMan/MattEland.AI

If you’re curious about MattEland.AI, it is available as a NuGet package at https://www.nuget.org/packages/MattEland.AI.Neural/

### One Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.