Comparing Services for Cheap Cloud Hosting and Storage (Cloud / AWS / S3 / Amazon Cloudfront / ... ???)

Publish Date: Nov 9 '19

16 10

Hi Dev.to! 👋👋👋

Some questions for all you performance aficionados and AWS / Cloud experts out there.

I'm looking for a cheap (as close to free as possible) service for:

1. Hosting AND serving images. These images will be used on a website, in emails, etc. I want to plan for:

100GB of added storage / month
100M image views (GET requests) / month
100K new image uploads (PUT / POST requests) / month

2. CDN / Edge caching - so as to serve requests as fast as possible. Here I am looking to reduce response times and website load times that end-users will experience.

AWS both has an amazing suite of products and at the same time is very difficult to get started with.AWS S3's pricing model is confusing. I did also play a bit with their calculator, but it's hard to say if I'm entering the numbers in correctly.

Q1: In the AWS ecosystem:

For S3: What is "Storage pricing" vs "Request Pricing"?
What is S3 Select and how is it different from S3?
What is S3 Intelligent-Tiering?
What is S3 Glacier?
And what about Amazon CloudFront?

Q2: Is AWS the best (and cheapest) available option? What about services like:

Cloudflare
Cloudinary
Photon by Jetpack etc?
Versus using my Linode server itself for hosting and serving images?
versus the 1000+ other options out there?

Thoughts on what service I should be using? Looking for advice from folks that are knowledgeable on the matter. 🙏🙏🙏

Comments 10 total

Talz CloningNov 10, 2019
Hi KP,

My thoughts:
1. Storage pricing is the storage you used throughout the month, charge based on actual storage used. Example you have 100GB of data and added 100GB throughout the month. You’ll be charged 200 x .025 (.025 based on Singapore Region) or (5 $) for the storage bill. Request pricing defines "POST,PUT, LIST" for uploading and "GET" for downloading files in your website its a small amount in your bill.
2. S3 Select added new capabilities for S3 explanation here S3 Select. Its more on the applications pull out only the data you need from an object instead retrieve the entire object.
3. Intelligent-Tiering its more on optimization storage costs. Imagine your object is accessed once in the first week and then never accessed again. AWS will handle this for you using S3 Intelligent-Tiering would move it to an infrequent access tier 30 days after its last access.
4. S3 Glacier for archiving using cheaper storage. Example your BACKUP, or infrequent access objects.
5. Amazon CloudFront is CDN features by AWS. Global Edge Network to distribute your web app globally to serve requests faster
Based on your requirements I can recommend AWS, cost-effective solution and security capabilities.

General S3 FAQs
- KPNov 10, 2019
  
  @geektalz thank you so much for the detailed reply. Wow - AWS is capable of a lot! You really explained it like I'm 5, which is very helpful and I appreciate it. It also gives me confidence in using AWS going forward!
Austin S. HemmelgarnNov 10, 2019
For S3: What is "Storage pricing" vs "Request Pricing"?

Storage pricing is the cost of actually storing the data. It's measured in GB of space usage per month. Pretty much, this is the cost of disk-space.

Request pricing is how much you have to pay for each API request made. If you just upload your data and then let it sit there, this works out to essentially zero. This is paying for usage of the API itself. It's typically billed per thousand or million requests by request type.

Note that you have to pay separately for bandwidth usage from both storage pricing and request pricing. Inbound bandwidth is (usually mostly) free, but outbound data transfer is pro-rated just like storage usage (you pay per GB of usage per month).

What is S3 Select and how is it different from S3?

I can't comment on this one, as I've never looked into S3 select.

What is S3 Intelligent-Tiering?

The general concept is pretty simple. S3 offers a couple of different options for storage to optimize based on access patterns. The two big ones are the Standard tier (which is better optimized for frequent access of smaller amounts of data) and the Infrequent Access (or IA) tier (which is better optimized for infrequent access of larger amounts of data (costs less per GB stored, but more per request to access it).

Intelligent Tiering is a special storage option that will look at your access patterns, and automatically use the tier that will save you the most money. It costs a bit extra, but will usually end up saving you money for cases where you don't know ahead of time what your access patterns will look like.

What is S3 Glacier?

Near-line archival data storage. Storing data in Glacier costs almost nothing compared to other S3 storage options (most of the US regions cost less than a third of the standard tier for actual storage), but you can't immediately retrieve it, and there's a minimum retention period you're always billed for. Pretty much, it's for stuff like off-site backups where you need to store a very large amount of data, but almost never need to access it. In practice, it's similar to Backblaze's 'B2 Cloud Storage' offering (though Backblaze is a bit cheaper per request and doesn't have the minimum retention requirements or retrieval delays).

What makes Glacier so cheap is that they save the data, and then remove the physical media it's stored on from the computer and store it in a warehouse somewhere (so they're not paying for power, and the chance of media failure is far lower). The upshot is that to actually access the data, they have to retrieve the media and load it into a system (though this is probably automated, otherwise their expedited retrieval would not be realistically possible).

And what about Amazon CloudFront?

A CDN that you can use to serve data from other AWS services. Think of it like CloudFlare, and you'll get the idea.

As far as pricing, I'd look at pricing out:
- AWS
- GCP (Google's AWS equivalent)
- A VPS provider (Linode in your case) plus CloudFlare
Those are the most statistically likely to be inexpensive while still meeting your actual usage needs. Note that you will need to provide your own upstream storage with any CDN like Cloudflare.

I personally am fond of AWS for stuff like you seem to be talking about, but part of that is that I can do everything in one place (including DNS hosting (Route 53, excellent pricing compared to most other DNS providers), VPS hosting (Lightsail (works like Linode/DigitalOcean/Vultr) or EC2 (more expensive, but you pay for exactly what you use), container management (multiple options), and more).
- KPNov 10, 2019
  I wish I could give each of these replies multiple likes, because they're so, so good. Thank you @ahferroin7 for the phenomenal explanation.
  
  Some follow on questions to your comment:
  
  I've never fully understood Cloudflare...I understand they're a CDN, but do they cache / speed up access to ENTIRE sites, or images only, or JS/CSS etc?
  
  Why would I need DNS hosting, like Route53? Aren't DNS services free??
  
  What are some use-cases for EC2 (elastic compute capacity)? Is it essentially a replacement of a VPS like Linode? I did take a look at this, btw.
  
  Given that I'm on a VPS (Linode), does it make sense to be using that, and AWS for:
  
  SES + SNS for emails
  
  S3 for storage and serving media files
  
  Amazon CloudFront for the CDN?
  - Austin S. HemmelgarnNov 10, 2019
    
    I've never fully understood Cloudflare...I understand they're a CDN, but do they cache / speed up access to ENTIRE sites, or images only, or JS/CSS etc?
    
    It depends on how you configure it, but usually it's used for entire sites. The big thing for them (and AWS CloudFront) though is geo-caching. That is, they have dozens of endpoints around the world, and requests through them get routed through the geographically closest endpoint, so you have minimal latency for stuff that's already cached.
    
    Why would I need DNS hosting, like Route53? Aren't DNS services free??
    
    If, for example, you wanted to use your own domain name to serve data from. AWS also provides domain registrar services through Route 53, so you can use them to register your own domain as well (and it's essentially at cost for that, they have very little markup). Some VPS offerings like Linode provide really basic free DNS, but you have to go through their specific naming scheme. S3 has a similar thing where you don't need a domain to serve data from it, but you have to live with their domain layout.
    
    What are some use-cases for EC2 (elastic compute capacity)? Is it essentially a replacement of a VPS like Linode? I did take a look at this, btw.
    
    The big thing with EC2 is that unlike all-in-one offerings like Linode (or AWS Lightsail), billing is itemized per resource. IOW, you pay separately for the storage for the VM (note that this is different from the pricing for S3), the network usage for the VM (which is the same as for S3), and the compute time plus memory for the VM. EC2 has four major benefits over the all-in-one offerings:
    
    If you have a workload that only needs one out of the three in large amounts, it will often be a bit cheaper than an all-in-one offering. For example, if you mostly need bandwidth and not storage capacity or computing power, you can pay for only a small disk, a very lightweight VM, and a lot of bandwidth. You have to have a very asymmetrical workload for this to pay off though.
    
    It offers much higher capacities than many all-in-one VPS offerings. Linode is the only one that I know of that comes close (with it's 48-core offering) to the biggest EC2 instances (with 96 cores). There are also much higher memory options (up to 3/4 of a TB of RAM).
    
    It provides better options for automatic scaling of infrastructure. This is really the biggest selling point for EC2, as you can have limits set up to automatically scale up or down by starting or stopping extra instances of a VM as your actual usage changes. Because of how the bill is itemized, this is much easier to keep track of when you actually pay the bill at the end of the month.
    
    On a similar note to scaling, EC2 provides a rather neat feature called 'spot instances', which are essentially a way to say 'run X many instances when the price of running them is less than Y per hour'. Essentially, you end up renting spare computing power, which varies in price depending on actual utilization of resources in the region.
    
    Note, however, that if your needs closely match up with a specific offering from an all-in-one provider, EC2 will usually be more expensive than going with the all-in-one offering.
    
    SES + SNS for emails
    
    Probably not. SES and SNS are really designed for very large scale messaging management, and the interfaces for working with them really reflect that. Unless you're talking about dealing with hundreds of thousands of emails to send, they're probably overkill.
    
    S3 for storage and serving media files
    
    Probably, but I'd suggest looking very closely at pricing. There's a nice calculator web app provided by AWS that can help you figure out what your costs would look like. Most likely, it will be worth it for your usage, but keep in mind that your monthly bill will fluctuate based on actual usage.
    
    Amazon CloudFront for the CDN?
    
    Possibly. Even if you're using S3 for storage, I'd generally recommend CloudFlare over AWS CloudFront in most cases. Keep in mind that you will probably need your own domain though for either of them. Route 53 is what I'd recommend for that, as it will effectively only cost a few USD a month plus the registration fees for whatever domain you pick (which are billed yearly, and typically only cost 10-20 USD depending on what TLD you're using).
    - KPNov 11, 2019
      
      @ahferroin7 Austin, phenomenal explanation, thank you so much for this.
      I finally understand a lot about what these services do, thanks to you. Also, thanks for the tip about Cloudflare vs Amazon Coudfront. It shows you've got a ton of experience in this domain, and you're exactly the type of person whose opinion I needed.
      I read your explanation twice.
      
      Route 53: I already have a domain (renewal costs $15/yr), and DNS set up with Linode (costs $10/mo). The domain (and all it's subdomains) resolve fine. So, I don't see the need for Route 53....unless I am missing something.
      
      Is there a downside to using SES + SNS for emails, if the number of emails is in the hundreds only (small numbers)? I ask because I already have it configured and set up (which was quite painful if I'm being honest).
      
      Thank you, again!
      - Austin S. HemmelgarnNov 11, 2019
        
        Route 53: I already have a domain (renewal costs $15/yr), and DNS set up with Linode (costs $10/mo). The domain (and all it's subdomains) resolve fine. So, I don't see the need for Route 53....unless I am missing something.
        
        Ignoring the transfer fees, Route 53 is probably less expensive unless you're seeing many millions of queries. It's 0.50 USD per hosted domain per month, and 0.10 USD per million standard queries per month. The registration costs are probably the same (though worth looking into, because they might not be). I think there's a one-time transfer cost if you want to switch registration to AWS (which would make other Route 53 setup easier), but I'm not sure.
        
        So, unless you're really busy, you'd probably end up paying less than 1 USD a month through Route 53 (provided you don't use any of the fancy features like monitoring and failover). It may also be marginally faster for people who aren't located close to Linode datacenters (AWS has a lot more datacenters, and they automatically route the queries through the closest datacenter).
        
        Is there a downside to using SES + SNS for emails, if the number of emails is in the hundreds only (small numbers)? I ask because I already have it configured and set up (which was quite painful if I'm being honest).
        
        If you've already got it set up and it works for you, there's probably not much of a downside unless you can find a less expensive alternative. The biggest issue with it is how much effort it takes to set up.
        
        KPNov 11, 2019
        
        Thanks @ahferroin7 ! It definitely looks like I should give Route 53 a go at some point in the future when I have more time. It's low onthe priority list, though.
        
        All said and done, your tips have been a great help. Following you on DEV, and I hope you don't mind the odd question on AWS every now and then..
        
        Austin S. HemmelgarnNov 11, 2019
        
        Always glad to share my knowledge!
        
        KPNov 11, 2019
        
        Thank you!

Add comment

KP @kp