This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.
Cloud-Computing
This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.
AWS Command Line Interface (CLI) is a powerful tool that allows users to interact with AWS services directly from the terminal. It simplifies managing cloud resources by providing commands for a wide range of AWS services, enabling tasks such as provisioning, managing, and automating workflows with ease.
LocalStack is a fully functional, local testing environment for AWS services. It enables developers to simulate AWS services on their local machines, facilitating the development and testing of cloud-based applications without needing access to an actual AWS account.
we understood how to deploy a Virtual Machine (VM) on a cloud server using LocalStack.
Now, let’s level up and dive into AWS S3 — one of the most powerful and essential cloud storage services!
Before we jump into the setup, I’ll explain what S3 is, why it’s useful, and how we can use it for storing files, hosting static websites, backups, and more.
What is AWS S3?
Amazon S3 (Simple Storage Service) is an object storage service that offers scalability, security, and high availability. It allows users to store and retrieve any amount of data at any time, making it ideal for backups, website hosting, and data lakes.
Secure and highly available storage solution
Supports various storage classes for cost optimization
Used for website hosting, backup, and disaster recovery
Provides strong access control and encryption options
Integrates seamlessly with other AWS services
Use Cases for S3
Amazon S3 is widely used across industries for a variety of purposes, including storing media files, hosting websites, and serving as a data lake for big data analytics.
Storing images, videos, and static website assets
Hosting static websites and content delivery
Backup and disaster recovery solutions
Log storage for analytics and monitoring
Machine learning and big data storage
S3 Storage Types
Amazon S3 provides different storage classes to meet various needs, balancing cost and performance. Each class is designed for a specific use case, from frequent access to long-term archival.
S3 Standard: High durability and performance for frequently accessed data
S3 Intelligent-Tiering: Automatically moves data to lower-cost tiers when not accessed
S3 Standard-IA (Infrequent Access): Lower cost for data that is accessed less frequently
S3 Glacier & Glacier Deep Archive: Cost-effective long-term archival storage
S3 One Zone-IA: Lower-cost storage for infrequently accessed data in a single availability zone
Step-by-Step Guide
Step 1: Start LocalStack
Run the following command to start LocalStack:
localstack start
Alternatively, use Docker:
docker run --rm-it-p 4566:4566 localstack/localstack
Start Docker Desktop
Launch Docker Desktop and wait until it indicates that "Docker is running."
LocalStack will simulate AWS services on port 4566, allowing local cloud development without an actual AWS account.
Step 2: Creating an S3 Bucket in LocalStack
Commands Breakdown:
List Existing Buckets:
aws --endpoint-url=http://localhost:4566 --region us-east-1 s3 ls
Purpose: This command is used to list the existing S3 buckets in your LocalStack environment, running at localhost:4566.
Output: No output is shown initially, meaning there are no buckets created at that point.
Purpose: This command creates a new S3 bucket named my-test-bucket in the LocalStack environment.
Output:make_bucket: my-test-bucket — This confirms the successful creation of the my-test-bucket.
List Buckets Again:
aws --endpoint-url=http://localhost:4566 --region us-east-1 s3 ls
Purpose: This command lists the existing S3 buckets again, this time after creating the new bucket.
Output:
2025-01-23 13:09:41 my-test-bucket
— This confirms that the my-test-bucket is now listed and was successfully created.
Detailed Explanation:
LocalStack’s Purpose:
LocalStack emulates AWS services locally, providing an environment for developers to simulate AWS without needing an actual AWS account or incurring costs. It mimics APIs for various services such as S3, EC2, Lambda, etc., making it a useful tool for development and testing.
CLI Commands and Output:
The AWS CLI interacts directly with LocalStack through the endpoint localhost:4566, which is where the LocalStack service is running. Every time you run commands like aws s3 mb or aws s3 ls, LocalStack processes these commands as if they were sent to the actual AWS cloud.
After running the aws s3 mb command, LocalStack creates the my-test-bucket bucket in its emulated environment. The successful creation is confirmed by the output: make_bucket: my-test-bucket.
When you list the buckets with aws s3 ls, it shows the newly created bucket as my-test-bucket, confirming the operation was successful.
Why the localhost:4566 Page Remains Empty:
localhost:4566 is not intended to serve a graphical user interface (GUI) like the AWS Management Console. Instead, it acts as an endpoint for API requests from tools like the AWS CLI or SDKs.
LocalStack emulates AWS services by providing HTTP-based API endpoints. These APIs handle requests and respond accordingly, but LocalStack does not serve a web-based dashboard. Therefore, accessing localhost:4566 via a browser will result in a blank or empty page, which is completely normal.
To interact with LocalStack, users should rely on the AWS CLI, SDKs, or other infrastructure management tools (like Terraform). These tools send API calls to LocalStack, which then processes and responds with the requested information or performs actions such as creating S3 buckets.
Why the S3 Bucket Is Created:
The aws s3 mb command in LocalStack simulates the creation of an S3 bucket, just as it would in AWS. In your case, the bucket my-test-bucket was created locally within LocalStack.
Even though LocalStack is emulating AWS S3 locally, the functionality is designed to be very similar to the actual AWS S3 service. Therefore, the aws s3 ls command shows the bucket because LocalStack has processed the request successfully and maintains an internal record of the bucket.
Key Takeaways:
LocalStack is functional: The successful creation and listing of the my-test-bucket confirm that LocalStack is operating correctly and the AWS CLI is interacting with it as expected.
No graphical interface is provided at localhost:4566: LocalStack is designed to work through API calls, not through a web dashboard.
Use CLI for interaction: For operations like creating buckets, listing them, or interacting with other AWS services, the AWS CLI or other tools should be used to send requests to LocalStack's API endpoints.
Step 3: Steps to Store and Access Images & CSV in S3 (LocalStack)
1. Upload Files to S3 Bucket
Navigate to the directory where your images and CSV file are located:
Learning AWS S3 on Localhost – Best Practices with Boto3 and LocalStack
An insightful guide on working with AWS S3 locally using Boto3 and LocalStack. The visuals I’ve used in this guide are taken from this article — thanks @r0mymendez for this helpful resource! 🙌
AI and Cloud Services is such a booming field, and it's really important for people to take the time to analyze and understand its impact. I really enjoy seeing posts like this — even though I’m not planning to venture into it myself right now, it’s always insightful to read! 😊
Great to hear! 😄 I too read a lot of posts even if I may or may not venture into it at the moment — it’s always good to explore 🔍. And whenever you decide to try it out, feel free to reach out if you hit any bumps! 💡💬
Thank you for sharing part of my post :)