Deep Beliefs Network

If you stack several restricted Boltzmann machine on the top of each other then we form a deep belief network

Input layers -> suplied to 1st RBM -> inputs to 2nd RBM -> Inputs to 3rd RBM

We also need to make sure directionality is in place for all the layers except for the top two

Greedy layer wise training is:

You train this network layer by layer as RBMs.

The wake-sleep algorithm is basically you train all the way up then you train all the way down. The training as it progresses up it's wake, then the training as it goes down again is called sleep.

You stack you RBMs, you train them up, once you've got the weights, make sure these connections only work downwards.

Deep belief network has stacked RBM, then after the training has happened, all the layers except the top two are directed layers

Deep Boltzmann Machine: We don't deprive the network of the undirectedness of it's connections.