☁️ Local Cloud Setup: LocalStack + Docker + AWS CLI ⚡
#devops#tutorial#cloud#docker
Madhurima Rawat

Madhurima Rawat @madhurima_rawat

About: 🚀 Passionate about tech & growth | 💻 Data Scientist & Frontend Dev | C, C++, Python, R | 📊 InfluxDB, Grafana | 🤖 ML Enthusiast | 🐙 GitHub & Open Source | 📚 Learning & Sharing

Location:
Bhilai, Chhattisgarh, India
Joined:
Apr 11, 2024

☁️ Local Cloud Setup: LocalStack + Docker + AWS CLI ⚡

Publish Date: Mar 26
8 0

☁️ Local Cloud Computing: No Credit Card Needed! 🚀

Hey Devs! 👋

Cloud computing is exciting, but the idea of linking your credit card just to explore services? Not so much. Maybe you're afraid of waking up to a $1000 bill for doing nothing, or you just want a hands-on experience without financial stress.

That's where Local Cloud Computing comes in! 🌐💡

This is the first article in my series:

"☁️ Local Cloud Computing with LocalStack + Docker + AWS CLI 💡"

In this series, I will guide you through setting up a local cloud environment to run various cloud services locally on your system. 🚀

Here, I'll guide you through setting up everything so you can run cloud services locally—no cloud costs, no surprises.

💾 All code, docs, and resources are available in my GitHub repository:

GitHub logo madhurimarawat / Cloud-Computing

This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.

Cloud-Computing

This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.

Repo Size GitHub Stars GitHub Forks GitHub Issues Closed Issues Open Pull Requests Closed Pull Requests GitHub Discussions GitHub Contributors Top Language License Last Commit Repository Age Workflow Status GitHub Watchers Visitors

Tools and Technologies ⚙️💻

1. AWS CLI

AWS Command Line Interface (CLI) is a powerful tool that allows users to interact with AWS services directly from the terminal. It simplifies managing cloud resources by providing commands for a wide range of AWS services, enabling tasks such as provisioning, managing, and automating workflows with ease.

2. LocalStack

LocalStack is a fully functional, local testing environment for AWS services. It enables developers to simulate AWS services on their local machines, facilitating the development and testing of cloud-based applications without needing access to an actual AWS account.

3. Docker

Docker is a containerization platform that allows developers to build, share, and run applications in isolated environments called…


View on GitHub


🚨 Heads up! This is going to be a long article.

I’ll start by explaining the core concepts—LocalStack and its key components—so you have a clear understanding before diving into the setup.

Then, I’ll walk you through the step-by-step installation and configuration. And since no setup is complete without a few hiccups, I’ll also cover troubleshooting tips for every issue I encountered along the way.

💻 I did this entire setup on Windows, so if you're using Windows too, this guide should be extra helpful!

So, let’s dive in! 🚀

LocalStack

LocalStack is a local AWS cloud stack that provides a seamless development and testing environment. Here's a concise breakdown of its features and benefits:

  • AWS Service Simulation: Mimics AWS services like S3, EC2, Lambda, DynamoDB, and more.
  • Local Development: Enables developers to test and build cloud applications on their local machine without needing internet connectivity.
  • Cost-Free Environment: No AWS account or associated costs are required for running services locally.
  • API Compatibility: Provides APIs that replicate AWS functionalities, allowing easy integration with existing tools and workflows.
  • Isolated Testing: Facilitates isolated development and testing before deploying to the actual AWS cloud.
  • Automation and CI/CD: Perfect for automating workflows, testing infrastructure, and integrating with continuous integration/continuous deployment pipelines.

Installation Guide

Docker Setup

Since LocalStack uses Docker to simulate AWS services locally, follow these steps to install it:

1. Install Docker on the System

  • Download and install Docker Desktop for Windows from the official Docker website.
  • During installation, ensure the following options are selected:
    • "Install required Windows components for WSL 2" (if using WSL 2).
    • Restart the machine if prompted to complete the installation.

2. Verify Docker Installation

  • Open a command prompt or PowerShell and run the following command to check if Docker is installed: 
  docker --version
Enter fullscreen mode Exit fullscreen mode
  • If Docker is successfully installed, a version number will appear, for example: Docker version 24.0.2, build abcdef.

3. Start Docker Desktop

  • Launch Docker Desktop and wait until it indicates that "Docker is running."

4. Enable WSL 2 Integration (If Required)

  • If Docker is not running on Windows, enable WSL 2 by following these steps:
    1. Open Docker Desktop settings.
    2. Go to General and check the option "Use the WSL 2-based engine."
    3. Apply the changes and restart Docker Desktop.

LocalStack Installation

  1. Install LocalStack:

  • LocalStack can be installed via pip:

     pip install localstack

  • Alternatively, it can be pulled from Docker:

     docker pull localstack/localstack
  1. Start LocalStack:

  • For running LocalStack with Docker:

     docker run -d -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

  • If LocalStack is installed locally:

     localstack start
  1. Install AWS CLI:

  • If the AWS CLI is not already installed:

     pip install awscli
  1. Configure AWS CLI for LocalStack:

  • The AWS CLI can be configured using default credentials, as LocalStack does not require actual AWS credentials:

     aws configure
    • Access Key ID: test
    • Secret Access Key: test
    • Region: us-east-1

  • To configure the AWS CLI to connect to LocalStack:

     export AWS_ENDPOINT_URL=http://localhost:4566

During the aws configure setup, the AWS CLI will prompt for configuration details, including the output format, which controls the display style of results from AWS CLI commands.

Common Output Formats:

  1. JSON (default): A structured data format, ideal for automation and scripting.

Example:

   {
     "Key": "Value",
     "AnotherKey": "AnotherValue"
   }
Enter fullscreen mode Exit fullscreen mode
  1. Table: A human-readable table format, ideal for quick reviews.

Example:

   +------------+----------------+
   |   Key      |    Value       |
   +------------+----------------+
   | Key1       | Value1         |
   | Key2       | Value2         |
   +------------+----------------+
Enter fullscreen mode Exit fullscreen mode
  1. Text: A simple plain-text format, suitable for scripts or command-line parsing.

Example:

   Key1 Value1
   Key2 Value2
Enter fullscreen mode Exit fullscreen mode

Choosing the Output Format

During the aws configure setup, users can:

  • Accept the default format by pressing Enter to select json.
  • Alternatively, specify a different format by typing one of the following:
    • json
    • table
    • text

Example:

Default output format [json]: table
Enter fullscreen mode Exit fullscreen mode

How to Change the Output Format Later

If a user wishes to change the output format after the initial configuration, they can:

  1. Manually edit the AWS CLI configuration file:
  • Open ~/.aws/config (on Linux/macOS) or %USERPROFILE%\.aws\config (on Windows).

  • Modify the output setting:

     [default]
 output = text
  1. Use the --output flag when running commands: 
   aws s3 ls --output table
Enter fullscreen mode Exit fullscreen mode

Resolving the Failed to connect to localhost port 4566 Error

When encountering the error Failed to connect to localhost port 4566, it typically indicates that no service is running on that port. To resolve this issue:

  1. Check if LocalStack is Running:

  • To verify if LocalStack is running, users can check the container or process status by executing the following command:

     docker ps
  1. Verify Port Mapping and LocalStack Startup:

  • If using Docker, ensure the correct ports are mapped. If running LocalStack locally, users should confirm its successful startup by checking the logs or re-running the following command:

     localstack start

Limitations in LocalStack

  • LocalStack should be considered a basic simulation of AWS EC2, as it does not support actual VM provisioning.
  • For working with real VMs locally, hypervisors such as VirtualBox or VMware should be used, or LocalStack should be integrated with these tools for a hybrid setup.

This setup should be considered suitable for testing and development in a simulated cloud environment, allowing API interactions and configuration testing without incurring cloud service costs.

Troubleshooting

Error Encountered: 

C:\Windows\System32>pip install localstack
WARNING: Ignoring invalid distribution -treamlit (c:\users\rawat\
appdata\local\programs\python\python310\lib\
site-packages)
Collecting localstack
  Using cached localstack-4.0.3.tar.gz (5.7 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting localstack-core (from localstack)
  Using cached localstack_core-4.0.3-py3-none-any.whl.
  metadata (5.4 kB)
Collecting localstack-ext==4.0.3 (from localstack)
  Using cached localstack_ext-4.0.3.tar.gz (6.2 MB)
ERROR: Could not install packages due to an OSError:
[Errno 2] No such file or directory:
'C:\\Users\\rawat\\AppData\\Local\\Temp\\pip-install-
q450br39\\localstack-ext_771d2730449b461b8323becea
3af2ce5\\localstack/pro/core/services/lambda_/invocation
/endpoint_injection/java/SdkV2DisableCertificateValidation/
src/main/java/cloud/localstack/HttpClientTransformer.java'
HINT: This error might have occurred since this system does
not have Windows Long Path support enabled.
You can find information on how to enable this at
https://pip.pypa.io/warnings/enable-long-paths
Enter fullscreen mode Exit fullscreen mode

The error indicates that Windows Long Path Support is not enabled, which is necessary because some files in the localstack installation exceed the default path length limit of 260 characters on Windows.

Here are the steps to resolve this:

Steps to Enable Windows Long Path Support

  1. Enable Long Paths in the Windows Registry:
  • Open the Run dialog (Win + R), type regedit, and press Enter.

  • Navigate to the following key:

     HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem

  • Look for the LongPathsEnabled entry. If it does not exist, create a new DWORD (32-bit) value named LongPathsEnabled.

  • Set its value to 1.

  1. Enable Long Paths Using Group Policy (for Pro, Education, or Enterprise Editions):
  • Open the Run dialog (Win + R), type gpedit.msc, and press Enter.

  • Navigate to:

     Computer Configuration > Administrative Templates > System > Filesystem

  • Double-click on Enable Win32 long paths.

  • Set it to Enabled, then click OK.

  1. Restart the System:
  • After making the necessary changes, users should restart their computer to apply them.

Retry Installing localstack

Once Long Path Support is enabled, users can retry the installation with the following command:

pip install localstack
Enter fullscreen mode Exit fullscreen mode

Additional Tips

  • Upgrade Pip: It’s recommended to ensure that the latest version of pip is being used: 
  python -m pip install --upgrade pip
Enter fullscreen mode Exit fullscreen mode
  • Use a Virtual Environment: To avoid making system-wide changes, users can create a virtual environment: 
  python -m venv venv
  venv\Scripts\activate
  pip install localstack
Enter fullscreen mode Exit fullscreen mode
  • Check Python Compatibility: Ensure that the Python version is compatible with the localstack version being installed.

Issue with the export Command on Windows

The export command is used in Unix-based systems like Linux and macOS to set environment variables. However, it is not recognized in Windows Command Prompt (CMD). In CMD, the set command should be used instead to set environment variables.

To resolve the issue, replace the export command with the set command in CMD:

set AWS_ENDPOINT_URL=http://localhost:4566
Enter fullscreen mode Exit fullscreen mode

Notes:

  1. If using PowerShell, the environment variable can be set with the following command: 
   $env:AWS_ENDPOINT_URL = "http://localhost:4566"
Enter fullscreen mode Exit fullscreen mode
  1. For a persistent environment variable across sessions, use the setx command to set it permanently: 
   setx AWS_ENDPOINT_URL "http://localhost:4566"
Enter fullscreen mode Exit fullscreen mode

Caution: The variable set with setx will only be available in new CMD or PowerShell sessions.

To verify the environment variable is set correctly, run the following:

echo %AWS_ENDPOINT_URL%
Enter fullscreen mode Exit fullscreen mode

This should display:

http://localhost:4566
Enter fullscreen mode Exit fullscreen mode

Starting Services

1. Ensure the Service is Running

  • If LocalStack or another AWS emulator is being used, it's essential to confirm that it has started successfully.

    • For LocalStack: 
    localstack start
    • For Docker (if LocalStack is running via Docker): 
    docker run -p 4566:4566 -d localstack/localstack

    After executing this, the following command can be used to confirm the container is running:

    docker ps

2. Verify Port Availability

  • To check if another service is using port 4566, the following command should be run: 
  netstat -ano | find "4566"
Enter fullscreen mode Exit fullscreen mode
  • If no output is returned, it indicates the port is free.

  • If output is returned, it's necessary to identify which process is using the port by executing:

    tasklist /FI "PID eq <PID>"

    Replace <PID> with the process ID found from the netstat output.

3. Check Firewall or Antivirus Settings

  • It should be ensured that no firewall or antivirus software is blocking connections to port 4566.
  • Temporarily disabling the firewall or antivirus can help in testing.

4. Confirm localhost Works

  • To check if localhost is working correctly on the system, the following command can be used: 
  ping localhost
Enter fullscreen mode Exit fullscreen mode
  • If there is no response, replacing localhost with 127.0.0.1 is recommended: 
  set AWS_ENDPOINT_URL=http://127.0.0.1:4566
Enter fullscreen mode Exit fullscreen mode

5. Validate curl Setup

  • It is crucial to confirm that curl is installed and functioning correctly by running: 
  curl --version
Enter fullscreen mode Exit fullscreen mode

6. Restart System or Docker

  • A system restart and/or restarting Docker could help resolve any lingering issues.
  • After restarting, the desired service (e.g., LocalStack) should be started again.

7. Manually Test Another Port

  • To manually test connectivity to localhost, a simple web server can be run using the following command: 
  python -m http.server 4566
Enter fullscreen mode Exit fullscreen mode

Then, the server can be accessed with:

  curl http://localhost:4566
Enter fullscreen mode Exit fullscreen mode
  • If this works, it suggests the issue lies with the service running on port 4566.
  • If it doesn’t work, there may be a network configuration issue.

Troubleshooting Docker Desktop Sign-Out Issues

If Docker Desktop keeps signing out automatically, several common causes and fixes should be considered:

1. Enable Trusted Root Certificates

  • Docker Desktop may fail to authenticate if the required root certificates are missing or untrusted.

  • Solution:

    1. The system’s date and time should be confirmed to be correct.
    2. The certificates can be updated by running the following command: 
     certutil -generateSSTFromWU roots.sst
  1. The roots.sst file can then be double-clicked to install the certificates.

2. Clear Docker Credentials

  • Cached credentials might be causing the sign-out issues.

  • Solution:

    1. Navigate to the Docker configuration folder: 
     C:\Users\<YourUsername>\.docker
  1. The config.json file should be deleted.
  2. Docker Desktop should be restarted, and the user should sign in again.

If Docker is not running, it should automatically troubleshoot and restart itself.

LocalStack Container Image Download Process

The following output indicates that LocalStack is attempting to pull the container image localstack/localstack to run in Docker mode, which suggests that the setup is working up to this point. Here's what's happening:

C:\Users\rawat>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
Pulling container image localstack/localstack
Enter fullscreen mode Exit fullscreen mode

What’s Happening?

  1. LocalStack CLI Detected Docker Mode:
  • [11:20:31] starting LocalStack in Docker mode confirms that LocalStack is trying to run using Docker.
  1. Container Image Not Found Locally:
  • [11:25:13] container image not found on host indicates that Docker is downloading the localstack/localstack image from Docker Hub since it’s not found locally.
  1. Pulling the Container Image:
    • ⠧ Pulling container image localstack/localstack confirms that the image download process is ongoing.

What to Expect Next

  • If the internet connection is stable, Docker should complete the download in a few minutes.
  • After the image is downloaded, LocalStack will initialize, and the services (e.g., S3, DynamoDB) will start. This will be confirmed by further logs.

How to Verify It’s Working

  1. After the image download is complete, the user can check if the container is running with the following command: 
   docker ps
Enter fullscreen mode Exit fullscreen mode

The container should be listed with the name localstack.

  1. To verify that LocalStack is running, the endpoint should be accessed: 
   curl http://localhost:4566
Enter fullscreen mode Exit fullscreen mode

A response should be returned, confirming that LocalStack is active.

If Pulling Takes Too Long

  • Check the internet speed and Docker's ability to pull images: 
  docker pull localstack/localstack
Enter fullscreen mode Exit fullscreen mode
  • If the pull succeeds, proceed to rerun the LocalStack command: 
  python "C:\Users\rawat\AppData\Local\Programs\Python\Python310\Scripts
  \\localstack" start
Enter fullscreen mode Exit fullscreen mode

After successfully pulling the image, run the following to verify:

C:\Users\rawat>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
localstack/localstack latest b686f3948f42 4 hours ago 1.18GB
hello-world latest 74cc54e27dc4 31 hours ago 10.1kB
Enter fullscreen mode Exit fullscreen mode

If issues persist after pulling hello-world, follow these steps:

  1. Start Docker Desktop.
  2. Then, execute: 
   localstack start
Enter fullscreen mode Exit fullscreen mode

✅ Hopefully, your LocalStack will now start successfully in the terminal with "LocalStack" displayed, and Docker will show all running containers.

📄 Want to see the output step by step? Check it out here:

🔗 Experiment 1 Output (PDF)

This document walks you through the output after a successful installation. 🚀

Useful Resources for Working with LocalStack

1. Articles on LocalStack and AWS S3

2. Official Docker Documentation

3. LocalStack Tutorials

  • LocalStack Official Tutorials

    Tutorials | Docs

    These tutorials enhance your comprehension of LocalStack's functionality by providing detailed information on how it works for specific use cases using diverse resources. These guides help you set up and build your local cloud development & testing environment with the help of LocalStack, using local AWS services, integrations, and tools, helping create a mental model for how LocalStack works. For community contributed tutorials, check out the [LocalStack Community Tutorials](https://hashnode.localstack.cloud/).

    favicon docs.localstack.cloud

4. Blog Post on LocalStack Setup and Usage

  • Getting Started with LocalStack: Overview, Setup, and Practical Usage Guide

    Getting Started with Localstack: Overview, Setup, and Practical Usage Guide | Ruan Bekker's Blog

    Discover LocalStack: Your local AWS cloud stack for efficient and cost-effective development and testing. Learn how to emulate AWS services like S3, DynamoDB, Lambda, and more on your local machine. Enhance your cloud development workflows with faster iteration cycles, comprehensive testing, and offline capabilities. Read our comprehensive guide on why LocalStack is a must-have tool for every developer working with AWS.

    favicon ruan.dev

💡 I hope this guide helped you! If you run into any issues, drop a comment—we’ll solve them together! 🕵️‍♂️

🔥 Stay tuned for the next article, where I'll explain how to set up a VM in this local cloud setup! 🚀

💬 Let me know your thoughts in the comments! 📝 Did you find this helpful? Was the setup smooth for you? This took me a lot of time to put together, and the setup was pretty hectic—especially on Windows! 😵‍💫 So, drop a like and a comment to show some love! 💛😊

Comments 0 total

    Add comment