Tensors and TensorFlow 101

Tensors:

1. Definition:
2. A tensor is a mathematical object that describes linear relations between vectors, scalars & other tensors.
3. They can be thought of multidimensional array of numbers.
4. Order of tensors:
5. 0th order tensor: Scalar (single number)
6. 1st order tensor: Vector(1D array)
7. 2nd order tensor: Matrix (2D array)
8. 3rd order tensor: Higher-dimensional arrays

Example of third order tensor: A 2 * 3 * 2 tensor could be visualised as:
[[[1, 2], [3, 4], [5, 6]],
 [[7, 8], [9, 10], [11, 12]]]

Tensor Operations:

1. Tensor Addition: Element-wise addition of tensors of the same shape.
2. Scalar Multiplication: Multiplying each element by a scalar.
3. Tensor Contraction: Summing over one or more indices. d. Tensor Product: Outer product of two tensors.

import numpy as np

# Create a 3rd order tensor (3x3x2)
tensor = np.array([[[1, 2], [3, 4], [5, 6]],
          [[7, 8], [9, 10], [11, 12]],
          [[13, 14], [15, 16], [17, 18]]])

print(tensor.shape) # Output: (3, 3, 2)
print(tensor[1,2,0]) 
# Output: 11 (2nd row, 3rd column, 1st depth)

5th Order Tensor:

Example: Brain connectivity data in neuroscience

T[i,j,k,l,m]

Where i and j represent brain regions, k represents the type of connection, l represents time, and m represents the subject ID.

Weather Data Tensor:

A 4th order tensor representing temperature data:

T[city, day, hour, weather_parameter]

Where weather_parameter could include temperature, humidity, pressure, etc.

Video Data:

A 4th order tensor representing a color video:

V[frame, height, width, color_channel]

Language Model Embedding:

A 3rd order tensor representing word embeddings in a language model

E[word_position, batch, embedding_dimension]

This image illustrates different orders of tensors in mathematics, showing examples from 0th order (scalar) up to 4th order tensors. Let's break it down:

This progression demonstrates how tensors increase in complexity and dimensionality as their order increases. Each higher-order tensor can be thought of as an array of lower-order tensors

Tensors and Tensor-flow:

Name Tensorflow is named after tensors.

1. Name origin:
2. "Tensors" refers to the mathematical concept
3. "Flow" represents the idea of data flowing through a computational graph.
4. Core Data Structure:
5. In tensor flow, tensors are the primary data structures used to represent all data. This includes:
6. Inputs & ouputs of operation.
7. Model parameters (weights & biases)
8. Intermediate results in computations.
9. Implementation:
10. Tensorflow tensors are multi-dimensional arrays of basic data types i.e (like int32, float32, string). They're similar to NumPy arrays but with additional features for efficient computation on GPUs & TPUs

How can we implement / experiment the knowledge:

import tensorflow as tf

  # Creating tensors
  scalar = tf.constant(5)
  vector = tf.constant([1, 2, 3])
  matrix = tf.constant([[1, 2], [3, 4]])
  
  # Tensor operations
  a = tf.constant([[1, 2], [3, 4]])
  b = tf.constant([[5, 6], [7, 8]])
  
  # Element-wise addition
  add_result = tf.add(a, b)
  
  # Matrix multiplication
  matmul_result = tf.matmul(a, b)
  
  # Reshaping
  reshaped = tf.reshape(vector, [1, 3])
  
  # Slicing
  slice_result = tf.slice(matrix, [0, 1], [2, 1])
  
  print("Addition result:", add_result.numpy())
  print("Matrix multiplication result:", matmul_result.numpy())
  print("Reshaped tensor:", reshaped.numpy())
  print("Sliced tensor:", slice_result.numpy())








#TensorFlow101 #TensorBasics #MachineLearning #DeepLearning #AIFundamentals #DataScience #PythonProgramming #MLBasics #TensorMath  #TFLearning #NeuralNetworks #ComputationalGraphs #DataStructures