Non-negative Matrix factorisation (NMF):

Non-negative Matrix factorisation (NMF):

It's an unsupervised algorithm that simultaneously performs dimensionality reduction and clustering. It's an alternative to LDA.

we can use it in conjunction with TF-IDF to model topics across documents.

General mathematics behind NMF:

1. you are given a non negative matrix A, creating TF-IDF
2. then we want to find a K dimension approximation in terms of 2 other non negative factors W and H
3. A (n*m) = W(n * k) * H(k*m)
4. K is going to be the number of topics we choose.
5. We are going to initialise some values for the factors W and H i.e start off as random matrices.
6. First, we get some measurement of the reconstruction error between A and WH
7. then we optimize
8. so we have expectation maximisation optimisation
9. to essentially refine W and H in order to minimize that objective function.

How to implement it practically?

1. Construct vector space model
2. filter out stop words
3. we get our document term matrix, A
4. Then we apply TF-IDF, term weight normalisation to A
5. Normalize TF-IDF vector to unit length
6. Initialize the factor using NNDSVD i.e Non negative double singular value decomposition on A matrix
7. Apply projected gradient NMF (non negative matrix factorization) to A.

We end up discovering:

1. Basis vectors: the topics i.e clusters in the data
2. Coefficient matrix: membership weights for documents relative to each topic i.e clusters.