Linear Transformations & Their Applications in Machine Learning

Linear Transformation:

A function that takes a vector as input and produces another vector as an output, while preserving vector addition & scalar multiplication. Matrices are used to represent these transformations

Key Properties

1. Linearity: For vectors u & v, scalar c:
2. T(u + v) = T(u) + T(v)
3. T(cu) = c T(u)
4. Representation: Any linear transformation in n-dimensional space can be represented by an n * n matrix

we will discuss about scalar later in the notes.

Common 2D Linear Transformations

Imagine a unit square in 2D space. Different linear transformations will affect this square in different ways:

We have original square.

1. Scaling: where sx and sy are scaling factors in x and y directions.

2. Rotation (counterclockwise by angle θ): 

This represents a counterclockwise rotation of a point (x, y) by an angle θ around the origin (0, 0).

Components:

1. Rotation Matrix: [cos(θ) -sin(θ)] [sin(θ) cos(θ)]

• Input Vector:

 [x 
  y]

• Result Vector:

 [x' 
  y']

3. Shear:

Geometrical transformation that distorts the shape of an object by shifting one part of it in a specific direction relative to another part. Changes to the shape by sliding it's part parallel to a given axis while leaving the other axis unchanged. It's often described as sliding or skewing effect.

Understanding these visual representations can greatly aid in grasping the concept of linear transformations.

What is scalar?

A scalar, often denoted by 'c' in mathematical notation is simply a single number. It's called scalar to distinguish it from more complex mathematical objects like vectors or matrices.

Key points about scalar:

1. A scalar is any real number that can be used to scale (multiply) a vector or a matrix.
2. Examples: 2, -3.5, π or √2 are example of scalars.
3. When we multiply a vector by a scalar, we're scaling that vector.
4. if vector = [2, 3], & c = 2, then cv = [4, 6]
5. When we multiply a matrix by a scalar, we're scaling every element in that matrix.
6. In the context of linear transformation: T(cv) = c T(v),
7. if we scale a vector then apply transformation
8. if we apply transformation than scale it
9. we get the same result vice versa.
10. Contrast with vectors and matrices:
11. A scalar is a 0 dimensional quantity (Just a magnitude)
12. A vector is 1 dimensional array of numbers
13. A matrix is 2-dimensional array of numbers.

# Scalar
scalar = 5


# Vector
vector = [1, 2, 3]


# Matrix
matrix = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
]

Understanding scalars is crucial because they allow us to express ideas like

• Double this vector
• Shrink this transformation by half

Linear transformation & the concepts of scaling, rotation, shear, scalars play crucial roles in various machine learning techniques & algorithms.

1. Feature scaling & normalisation
2. Ensure all features contribute equally to the model
3. Dimensionality Reduction:
4. Linear transformation are core of dimensionality reduction techniques like Principal component analysis (PCA).
5. Data Argumentation:
6. In computer vision, rotation, scaling & shear are used to augment training data .
7. By applying these transformation to existing images, we can create new training samples.
8. Convolutional Neural Network(CNNs):
9. CNN can be viewed as series of linear transformation.
10. Each filter in convolutional layer performs a linear transformation on a subset of the input.
11. Natural Language Processing:
12. Word embeddings such as Word2Vec, use linear transformation to map words to dense vector representations.

#CSCE-598 #LinearTransformation #MachineLearning #Mathematics #Matrices