Recently, I was looking for a simple way to share photos amongst friends and family. But no way I would pay $14.99 a month for a turnkey solution! - This goes against rule #422 of being a developer:

Rule #422: I'm a developer, I can build that myself - how hard can it be?

After all, it's just uploading and displaying some photos...

A friend already helped me out with a prototype, but things quickly started to break once I added 3000 high resolution images.

One important lesson before we get into it:

Just pay for the damn service! Don't be so cheap. I spend 5 days on what I estimated to be a 5 hour project.

Rule #675 of being a developer:

Rule #675: Your estimates will be wrong. By a lot.

Okay it was still fun, so here are some lessons:

1. Where to store your images

The prototype was build on Cloudflare's object storage R2. Cloudflare also offers an image service, that comes with compression and resizing - 5,000 transformations are included, after that it's $5 per 100,000 images stored per month and $1 per 100,000 images delivered per month. I won't pay for that - I'm a developer, I can do that myself, how hard can it be?

2. Smooth scrolling through thousands of images

The trivial solution - render 3000 images on your site.

<img v-for="image in images" :key="image.id" :src="image.src" />

In general, I'd recommend to always start with the trivial solution and work your way up. Btw. I am building this with Nuxt 3 - so example code will be Nuxt, Vue and TypeScript.

At a few hundred images the browser started to hickup and scrolling wasn't smooth anymore. In very large lists, it is common practice to work with a virtual dom and only render visible elements.

I found vue-virtual-scroller but for reasons I can't remember, it did not work immediately so I got frustrated and went back to rule #422:

I can build that!

I started with rendering empty divs instead of imgs. All devices i tested with had no issue to handle thousands of divs and scrolling through worked smoothly. Might be an issue on old devices, but we have to draw a line somewhere...

I used an intersection observer to keep track of visible elements on screen, and replaced my divs with imgs whenever they entered the viewport:

<template>
<div>
 <!--...-->

 <div v-for="(image,index) in images" :key="image.id" :data-index="index">
   <img v-if="intersectingIndices.has(index)" :src="image.src" />
</div>

</div>
<template>


<script setup lang="ts">

// ...

onMounted(()=>{
   const observer = new IntersectionObserver(handleIntersection, {
      root:null, // Use the viewport as the root
      threshold: 0.1, // Trigger when 10% of the image is visible
   });
})

const intersectingIndices = new Set<number>();
function handleIntersection(entries: IntersectionObserverEntry[]) {
  for (const entry of entries) {
    const indexAttr = entry.target.getAttribute("data-index");
    if (indexAttr == null) continue;
    const index = Number(indexAttr);
    if (entry.isIntersecting) {
      intersectingIndices.add(index);
    } else {
      intersectingIndices.delete(index);
    }
  }
}
</script>

It worked, but scrolling was stuttering. With the research and previous failed attempts, I am approaching the 5 hours deadline. Luckily, we live in modern times and ChatGPT pointed me towards batching DOM updates with requestAnimationFrame(applyPendingUpdates). It worked!

function handleIntersection(entries: IntersectionObserverEntry[]) {
  for (const entry of entries) {
    const indexAttr = entry.target.getAttribute("data-index");
    if (indexAttr == null) continue;
    const index = Number(indexAttr);
    if (entry.isIntersecting) {
      intersectingIndices.add(index);
    } else {
      intersectingIndices.delete(index);
    }
  }
  // Batch the update to visibleImages using requestAnimationFrame
  if (rafId === null) {
    rafId = requestAnimationFrame(applyPendingVisibleUpdates);
  }
}

Beautiful.

Scrolling now worked buttery smooth. Praise the AI lords!

3. Improve Image Load time

Next issue, I could scroll, but the unprocessed images were too large and load time was whack. I used Nuxt Image in order to compress and downsize the images on the fly. Nuxt Image ships with a builtin image transformation library that can be used locally, so F U Cloudflare image service!

Well, not so fast. CPU and Memory spiked on my server with frequent image transformations. For my limited user count the app handled it well, but at some point this is something to keep in mind.

I assumed Nuxt image would store transformed files in memory, so my strategy to keep the cache alive was: don’t code bugs so I don’t have to deploy and everything is fine.

4. Honorable Bug Mentions and Challenges

The Back Button

I knew ahead of time, that the app would probably be used 90% on mobile. What I did not know: Some phones have a back button and it get's used a lot. The app could show full res images, but it relied on in memory variables to trigger the detail view. A click on the back button sent users to the homepage and they had to navigate back to the images page and scroll through 1000 images again to get back to where they started. I had to rewrite that to show the detail view on a new page while still preserving the scroll position (also if the page was reloaded).

Swiping

Another mobile issues. Users are just used to swiping gestures on their phones. To view the next image, to close the detail view. I tried to make it work in the beginning, but it quickly turned out to be a bad idea, since gestures overlapped with browser native gestures (swipe down to reload, swipe right to navigate back). After a few hours of fine tuning I had my solution: Users need to unlearn their swiping behavior. It's a won't fix. Guess there still is some UX benefit in native apps compared to web.

Nuxt

I really like Nuxt and have been reliably using the framework for years. It's just natural, that before you assume a bug in your framework, you usually spent a lot of time to question your own capabilities and whether you chose the wrong career. I spent about 4 hours testing the app from front to back until I figured out, there was a bug in Nuxt, that caused my app to crash. Luckily, they released a new version just 2 days earlier, that fixed it!

Cloudflare Cache

To save on storage and bandwidth, I used a custom script to convert all of my images to webp before uploading them to Cloudflare. I used sharp for the image transformation. Every now and then you take a picture with your phone and it is rotated 90 degrees. The orientation is stored in the metadata of the image. Sharp strips all metadata by default. I only noticed this, after my images had been uploaded to Cloudflare, so I deleted the bucket, transformed everything again and reuploaded my images (side info: I was visiting relatives at that time. The router is in the basement, I was on first floor (European first floor, American second), I measured 2 MBit/s upload - You do the calculation for 10GB of images...).

After reuploading the images were still rotated and I spend hours comparing metadata, clearing browser cache and going through reuploading again and again (this time with smaller sample sizes) until I noticed, that the old images were still served from Cloudflare cache. Be smarter than I was. Don't forget the cache.

Photo Upload

I gave my friends an upload button. And they uploaded. Or at least they tried. Some of them tried to upload a lot of images at once - I had no restrictions. Who needs restrictions anyways? Aren't we all born as free human beings? If you asked me, I wouldn't put a restriction on your uploads. But I should have. These big uploads reliably pushed the server into memory limits and broke the application. How did I fix it? I ran the app on localhost and uploaded the images myself. LOL.

5. Deploying

Okay, I have to tell ahead of time, I am biased here. But this is just the perfect project to run on sliplane.io. Sliplane is a cheap and dead simple way to run Docker containers on a VPS - no DevOps required. This means we can basically run whatever we want (frontend and backend with image processing) and my cache in memory strategy can actually work for a while! (At least until the machine will be rebooted.) Take that serverless!

Summary

It's a rabbit hole and it's deeper than it looks from the outside. Don't be cheap. Pay for that damn photo sharing service.