How does this relate to Machine Learning
Remember that the solution space here is very small with a total of only 11 x 11 = 121 values or
states.
Today's neural networks have solution spaces up to the size of 10 to the power of a trillion.
This is a number with a trillion (1012) zeros.
If I wanted to draw a diagram for that kind of space, I would need a spatial dimension for each network
parameter, i.e. trillions of axes.
These are spaces that can no longer be understood and whose solutions can only be calculated using
efficient methods such as those described here.
Finding a solution for a neural network is no different than training it.
What is GD and SA anyway
Gradient Descent (GD) is an optimization method that iteratively moves towards the minimum of a function
by following the negative gradient. It's like walking down a hill step by step towards the lowest point.
Simulated Annealing (SA) is another optimization method inspired by the annealing process in metallurgy.
It allows for probabilistic "jumps" to escape local minima, with the probability of these jumps governed
by a "temperature" parameter that gradually decreases over time. Higher temperatures allow bigger jumps
and greater exploration, while lower temperatures lead to more refined searches around the current best
solution.
But wait, isn't Gradient Decent the same as Backpropagation?
Gradient Descent is the algorithm for finding the minimum of a function.
Backpropagation is an algorithm used in training neural networks to compute the gradient of the loss
function with respect to each weight by the chain rule, enabling efficient gradient computation.
Backpropagation essentially informs GD which direction to take to reduce error in the network.
Therefore, while Gradient Descent and Backpropagation are related, they serve different purposes within
the optimization and training process of neural networks. Backpropagation calculates the gradients,
which Gradient Descent then uses to update the weights in the network.
This example does not use Backpropagation.
How was it built
This software was created using Vibe Coding by a Large Language Model LLM / chatbot
and reworked in look & feel.
Some features had to be implemented manually and
corrections and improvements had to be made.
The 3D diagram was created in a separate session and the two pieces of code were merged manually and some
fixes and enhancements had to be done.
The following Vibe Coding prompts were used on Copilot
"create a html single page with table. The let the rows be x coordinates of the hospital from -5 to 5
and the columns be the y coordinates from -5 to 5. Both with a increment of 1. draw a 11 by 11 grid on a
canvas. On the canvas draw a 2D hospital in one cell. let the user drag and drop the hospital anywhere
on the grid. Also draw 5 2D houses on the grid. Randomize the position of the houses but keep them
fixed. Make shure the hospital can not be place on a house. When the hospital is moved compute and sum
up the distance to each house when only vertical and horizontal moves are allowed. Display the distance
in a field. Initially fill the table with the distances when computed using the coordinates of the rows
anc columns."
"I can not see houses or the hospital. You can use a black box for houses and a red cross for the
hospital"
"instead of dragging the hospital just set the hospital to the cell where the mouse clicks. Also
display the distance to the current hospital postion in a field and hightlight the corresponding cell in
the table."
"set the background of each cell table use dark colors for high distances and light colors for small
distances. make shure the numbers are readable and change the font-color of cells to white if necessary.
next to the table draw a 3D line chart with three.js use the x and y axis for the x and y coordinates
and the z axis for the distance."
"the 3d line chart is black, nothing to see. The cell colors of the table are also all black, use
distinguishable colors according to the cell values. use an import map for the three.js lib and
implement an orbitcontrol"
"change the color algorythm of the table cells compute the min and max distance value first and then
use the full range of rgb luminance to reflect the cell values. Also the 3D Line chart should display
the table as a 3D representation not the houses."
"the distances have one local minima. What can I do for 2 minimas"
"create 4 barriers randomize their positions. Draw the barriers on the grid with a yellow and black
pattern like a barrier tape. In the 3D chart do only display the dots and position them according the
the grid values."
"you need to add the additional barrier cost if the barrier is in the way from the hospital to the
house not when the house sits on the barrier. Leave the table cells empty when there is no barrier.
change the cell hightlight to a red box instead of a background."
"remove the 3d chart. add an animation. when the user clicks on a table cell start from that cell and
gradient decent to the next lower value. Move on until you reach a minima. change the font-weight of the
visited cells to bold. change the background of the minima to green. Do the gradient decent async and
leave a time between the steps of 500ms."
"keep the hightlight function of the previous code when clicking on a canvas cell. Start the gradient
animation when clicking on a table cell. make shure barriers are not created where houses are also make
shure there is no barrier and no house where the initial hospital stands."
"there are bugs"
"add a checkbox when the checkbox is checked change the gradient decent to a simulated anealing to find
the absolute minima."
"create a single html page with a 3d graphic using three.js. use a import map for the three.js lib. use
an orbitcontrol. Create an array with 11x11 dimenstion and fill it with random values from 1 to 100. Let
the 3d graphic display the array data as a 3D grid using a dot for each value."
"add a 11x11 grid on the x y plane and the axis. add a title to each axis add labels to the axis"
"use a standard font do not load a font. let the grid and the data start at zero point"
"the grid is not in the xy plane and its corner does not match the axix corner. Also add units to the
axis"
"connect the datapoint dots with interpolated curves along x and y"
"replace the interpolated lines with a 3d plane that interpolates the datapoints and use a light to
give it a 3D look."
"does not work because the planeGeometry is not in the xy plane and does not align with the datapoints.
It has to have the same origin than the datapoints."
"turn the planeGeometry so it aligns with the datapoints"
"does not work"
