Handling images on apps. CDN architecture

Ways to handle image in application:

• Directly upload files/images to the server (Naive way)
• Use external buckets like S3 to put your images/files.

To create a scalable application, we should be able to handle our growing demand for images handling. One of prominent ways to store the images in application is using cloud storage like AWS S3.

In this article we will primarily compare 3 different types of architecture to handle images in the apps.

Ways to use external bucket:

• Directly use key in the UI of the application

1. Use it to upload the image to the server
2. Once the image is added in S3, S3 will return a key which you can save in api
3. Later use the key to retrieve image

1. Credentials on UI: 

Client        Server        S3
  |             |            |
  |--Upload---->|            |
  |<--S3 Creds--|            |
  |---Direct upload--------->|
  |<----Upload confirmed-----|
  |             |            |
  |  Later, fetching image   |
  |             |            |
  |--Get image->|            |
  |<-Image URL--|            |
  |---Fetch image----------->|
  |<----Image data-----------|
  |             |            |


2. Credentials on Server:

Pre-signed Url from the server is sent to access images for certain time like 1 hour, could be 7 day, or custom time set by the programmer.

1. Create a user role in aws system.
2. Assign role to your server like ec2 to access S3.
3. User uploads the image/files to the server
4. Server uploads the image to the S3
5. Server saves the key in the db for future reference.
6. When user tries to access files / images, server signs a url and send it to the client.
7. Client uses the pre-signed url to access the image.

Client        Server        S3
  |             |            |
  |--Upload---->|            |
  |             |--Upload--->|
  |             |<--Confirm--|
  |<--Confirm---|            |
  |             |            |
  |  Later, fetching image   |
  |             |            |
  |--Get image->|            |
  |             |--Fetch---->|
  |             |<-Image data|
  |<-Image data-|            |
  |             |            |

• Improved security by not exposing S3 credentials directly in the client
• Controlled access to S3 resources

The second approach sounds good enough. So we implemented it on our architecture.

But as the image uploading starts picking up, it will start being slow and hard to manage as well as keep it up, as every single image

Every single image => Extra work to the api to call extra function.

3. Hybrid Server-CDN Delivery Model

Client        Server        S3           CDN
  |             |            |             |
  |             |            |             |
  |--Upload---->|            |             |
  |             |--Upload--->|             |
  |             |<--Confirm--|             |
  |<--Confirm---|            |             |
  |             |--Store key-|             |
  |             |            |             |
  |  Later, when retrieving the image      |
  |             |            |             |
  |--Get image->|            |             |
  |             |--Gen URL-->|             |
  |             |<--CDN URL--|             |
  |<--CDN URL---|            |             |
  |                                        |
  |  Server's role ends here               |
  |  CDN takes over for delivery           |
  |                                        |
  |--Fetch image-------------------------->|
  |<--Deliver image------------------------|
  |             |            |             |
  |                                        |
  |  CDN handles:                          |
  |  - Caching                             |
  |  - Global distribution                 |
  |  - Fast delivery                       |
  |                                        |

1. Upload is handled by server
2. Fetched / retrieval is handled by cloud distribution network (CDN) sitting separate from the code.
3. Removes unnecessary work in caching the image.
4. Reduced load on server/machine as image is loaded from CDN.

Server-side upload:

• Reduced client-side complexity
• Better control over file validation and processing

CDN fetching:

• Faster content delivery to end-users
• Reduced load on your origin server

Code optimisation:

• Removal of extra code, leading to a cleaner codebase
• Potentially improved performance due to reduced overhead