Nestjs(fastify, multer). Uploading & cropping image.
#nestjs#node#javascript#multer
Sergey Telpuk

Sergey Telpuk @sergey_telpuk

About: Experienced PHP/NodeJS/Golang Developer with a demonstrated history of working in the information technology and services industry.

Location:
Minsk
Joined:
Apr 23, 2018

Nestjs(fastify, multer). Uploading & cropping image.

Publish Date: Aug 10 '19
58 11

Hello friends!

At this article, I wanna show you how we can download and crop image with the help of Multer and in the same time to use different adapters, for instance, I'm gonna show two adapters FTP, AWS.

In the beginning, adapt Multer to Nest, main.js can look bellow:

import {NestFactory} from '@nestjs/core';
import {AppModule} from './app.module';
import {INestApplication} from '@nestjs/common';
import fs from 'fs';
import {FastifyAdapter, NestFastifyApplication} from '@nestjs/platform-fastify';
import fmp from 'fastify-multipart';

(async function bootstrap() {
    const fastifyAdapter = new FastifyAdapter({
        http2: true,
        logger: true,
        https: {
            allowHTTP1: true, // fallback support for HTTP1
            key: fs.readFileSync(APP.HTTPS_SERVER_KEY),
            cert: fs.readFileSync(APP.HTTPS_SERVER_CRT),
        },
    });

    fastifyAdapter.register(fmp, {
        limits: {
            fieldNameSize: 100, // Max field name size in bytes
            fieldSize: 1000000, // Max field value size in bytes
            fields: 10,         // Max number of non-file fields
            fileSize: 100,      // For multipart forms, the max file size
            files: 1,           // Max number of file fields
            headerPairs: 2000,   // Max number of header key=>value pairs
        },
    });
    const app: INestApplication = await NestFactory.create<NestFastifyApplication>(
        AppModule,
        fastifyAdapter,
    );
    app.enableCors();

    await app.listen(APP.PORT, APP.HOST);
})();

Now, Nest can parse multipart/form-data.

After add some abstraction for our storage.
Create UploadImageFactory:

export const UploadImageFactory: FactoryProvider = {
    provide: 'IUploadImage',
    useFactory: () => {
        return StorageFactory.createStorageFromType(TYPE_STORAGE);
    },
};

IUploadImage looks like:

export interface IUploadImage {
    /**
     *
     * @param value
     */
    setFilename(value: string);

    /**
     *
     * @param value
     */
    setCroppedPrefix(value: string): IUploadImage;

    /**
     *
     * @param value
     */
    setCroppedPayload(value: CropQueryDto): IUploadImage;

    getMulter(): any;
}

StorageFactory looks like:

export class StorageFactory {
    static createStorageFromType(type: string): IUploadImage {
        switch (type) {
            case TYPE_STORAGE.FTP:
                return new FtpStorageAdapter({
                        fileFilter(req, file, cb) {
                            //TODO validate files on mime-type

                            cb(null, true);
                        },
                    },
                );
            case TYPE_STORAGE.AWS: {
                return new AwsStorageAdapter({
                    fileFilter(req, file, cb) {
                          //TODO validate files on mime-type

                        cb(null, true);
                    },
                });
            }
            default:
                return null;
        }
    }
}

Create the first adapter for FTP:
FtpStorageAdapter

import FtpStorage from 'multer-ftp';
import fs from 'fs';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import multer from 'fastify-multer';

export class FtpStorageAdapter extends StorageAbstract implements StorageEngine {

    private readonly storage;
    private readonly storageForCropping;

    constructor(options: Options | undefined) {
        super();

        this.setMulter(multer(
            {
                ...options,
                storage: this,
            },
        ).single('file'));

        this.storage = new FtpStorage({...FTP_STORAGE});
        this.storageForCropping = new FtpStorage({...FTP_STORAGE});
    }

    async _handleFile(req, file, cb) {
        const filePath = await this.saveAsTemp(file);

        await this.resize(filePath).then((resizedFile) => {
            this.storageForCropping.opts.destination = (inReq, inFile, inOpts, inCb) => {
                inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
            };
            this.storageForCropping._handleFile(req, {
                ...file,
                stream: fs.createReadStream(resizedFile as string),
            }, (err, destination) => {
                if (err) {
                    Promise.reject(err);
                }
                Promise.resolve(true);
            });
        });

        const storage: any = await new Promise((resolve, reject) => {
            this.storage.opts.destination = (inReq, inFile, inOpts, inCb) => {
                inCb(null, this.filename + path.extname(inFile.originalname));
            };
            this.storage._handleFile(req,
                {
                    ...file,
                    stream: fs.createReadStream(filePath as string),
                },
                (err, destination) => {
                    resolve(() => cb(err, destination));
                });
        });

        this.reset();

        storage();
    }

    async _removeFile(req, file, cb) {
        this.reset();
    }
}

Create the second adapter for AWS:
AwsStorageAdapter

import AwsStorage from 'multer-s3';
import fs from 'fs';
import AwsS3 from 'aws-sdk/clients/s3';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import {StorageAbstract} from '../storage.abstract';
import multer from 'fastify-multer';

export class AwsStorageAdapter extends StorageAbstract implements StorageEngine {

    private readonly storage;
    private readonly storageForCropping;

    private readonly AWS_CONFIG = {
        s3: new AwsS3({
            credentials: {
                accessKeyId: AWS_STORAGE.AWS_ACCESS_KEY_ID,
                secretAccessKey: AWS_STORAGE.AWS_SECRET_ACCESS_KEY,
            },
            s3ForcePathStyle: AWS_STORAGE.AWS_S3_FORCE_PATH_STYLE,
            s3BucketEndpoint: AWS_STORAGE.AWS_S3_BUCKET_ENDPOINT,
            endpoint: AWS_STORAGE.AWS_ENDPOINT,
        }),
        acl: AWS_STORAGE.AWS_ACL,
        bucket: AWS_STORAGE.AWS_BUCKET,
    };

    constructor(options: Options | undefined) {
        super();

        this.setMulter(multer(
            {
                ...options,
                storage: this,
            },
        ).single('file'));

        this.storage = new AwsStorage({
            ...this.AWS_CONFIG,
        });

        this.storageForCropping = AwsStorage({
            ...this.AWS_CONFIG,
        });
    }

    async _handleFile(req, file, cb) {

        const filePath = await this.saveAsTemp(file);

        await this.resize(filePath).then((resizedFile) => {
            this.storageForCropping.getKey = (inReq, inFile, inCb) => {
                inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
            };

            this.storageForCropping.getContentType = (inReq, inFile, inCb) => {
                inCb(null, inFile.mimetype);
            };

            this.storageForCropping.getMetadata = (inReq, inFile, inCb) => {
                inCb(null, {fieldName: inFile.fieldname});
            };

            this.storageForCropping._handleFile(req, {
                ...file,
                stream: fs.createReadStream(resizedFile as string),
            }, (err, destination) => {
                if (err) {
                    Promise.reject(err);
                }
                Promise.resolve(true);
            });
        });

        const storage: any = await new Promise((resolve, reject) => {
            this.storage.getKey = (inReq, inFile, inCb) => {
                inCb(null, this.filename + path.extname(inFile.originalname));
            };

            this.storage.getContentType = (inReq, inFile, inCb) => {
                inCb(null, inFile.mimetype);
            };

            this.storage.getMetadata = (inReq, inFile, inCb) => {
                inCb(null, {fieldName: inFile.fieldname});
            };

            this.storage._handleFile(req,
                {
                    ...file,
                    stream: fs.createReadStream(filePath as string),
                },
                (err, destination) => {
                    resolve(() => cb(err, destination));
                });
        });

        this.reset();

        storage();
    }

    async _removeFile(req, file, cb) {
        this.reset();
    }
}

StorageAbstract looks like:

import fs from 'fs';
import sharp from 'sharp';

export abstract class StorageAbstract implements IUploadImage {

    protected filename: string;
    protected croppedPayload: CropQueryDto;
    protected croppedPrefix: string;

    private multer: any;

    protected constructor() {
    }

    protected setMulter(multer: any) {
        this.multer = multer;
    }

    setFilename(value): IUploadImage {
        this.filename = value;
        return this;
    }

    setCroppedPrefix(value: string): IUploadImage {
        this.croppedPrefix = value;
        return this;
    }

    setCroppedPayload(value: CropQueryDto): IUploadImage {
        this.croppedPayload = value;
        return this;
    }

    getMulter(): any {
        return this.multer;
    }

    protected async saveAsTemp(file): Promise<string> {
        return new Promise((resolve, reject) => {
            const tmpFile = '/tmp/' + uuid4();
            const writeStream = fs.createWriteStream(tmpFile);
            file.stream
                .pipe(writeStream)
                .on('error', error => reject(error))
                .on('finish', () => resolve(tmpFile));
        });
    }

    protected async resize(file: string): Promise<string> {
        return new Promise((resolve, reject) => {
            const tmpFile = '/tmp/' + uuid4();
            let readStream = fs.createReadStream(file as string);
            const writeStream = fs.createWriteStream(tmpFile);

            if (Object.keys(this.croppedPayload).length !== 0) {
                const {cw, ch, cl, ct} = this.croppedPayload;
                readStream = readStream.pipe(sharp().extract({
                    left: cl,
                    top: ct,
                    width: cw,
                    height: ch,
                }));
            }

            readStream
                .pipe(writeStream)
                .on('error', error => reject(error))
                .on('finish', () => resolve(tmpFile));

        });
    }

    protected reset() {
        this.setFilename(null);
        this.setCroppedPrefix(null);
        this.setCroppedPayload({
            ch: 0,
            cl: 0,
            cw: 0,
            ct: 0,
        });
    }
}

Add IUploadImage into a contrived controller:

...
import {BadRequestException, Post,Req,Res} from '@nestjs/common';
...
    constructor(
        @Inject('IUploadImage')
        private readonly uploadImage: IUploadImage,
    ) {

    }
    @Post('/')
    async upload(
        @Query() avatarCropDto: CropQueryDto,
        @Req() req,
        @Res() res,
....
    ): Promise<void> {
        try {
...
             return new Promise((resolve, reject) => {
               this.uploadImage
                .setFilename(uuid)
                .setCroppedPrefix(croppedPrefix)
                .setCroppedPayload(cropPayload)
                .getMulter()(req, res, (err) => {
                    if (err) {
                        reject(err);
                    }
                    resolve(req.file);
                });
        });
        } catch (e) {
            throw new BadRequestException(e.message);
        }
    }

After those manipulations, We can not only download but also crop images.

Done repository.

Comments 11 total

  • Ruslan Gonzalez
    Ruslan GonzalezAug 12, 2019

    I was looking something about uploading to S3 AWS... thanks... I'll try following a little about your setup and hope will works. If you have recommendations would be much appreciated. My challenge is to just host files to AWS with some Multer Validations (I think it already do very good)...

  • Wahaj Aayani
    Wahaj AayaniOct 31, 2019

    Couldn't get it running. How to export the UploadImageFactory to a module?

    • Sergey Telpuk
      Sergey TelpukOct 31, 2019

      Hi, You can do it like below:

      import { Module } from '@nestjs/common';

export const UploadImageFactory: FactoryProvider = {
    provide: 'IUploadImage',
    useFactory: () => {
        return StorageFactory.createStorageFromType(TYPE_STORAGE);
    },
};

@Module({
  controllers: [],
  providers: [UploadImageFactory],
})
export class CatsModule {}
      • Wahaj Aayani
        Wahaj AayaniNov 1, 2019

        Thanks for your response. Figured that out earlier. I was somewhat trying to make it a dynamic module to be exported and used elsewhere in the code. I can see that you have hard-coded AWS config in it's adapter itself which is ofc not the best of practices, and now if you want to take it out of there and pass in from the module itself. How would you recommend doing that?!

        • Sergey Telpuk
          Sergey TelpukNov 1, 2019

          To be honest, I made this solution in a hurry. You know, I start thinking about making a library MulterCropper.
          You are right about hard-coded config, This solution sucks, As for me the best way is to take it out and pass via a constructor.

          • Wahaj Aayani
            Wahaj AayaniNov 1, 2019

            It really helped me to say the least. Thank you. That's a nice idea for a library though.

  • Cedric Mariano
    Cedric MarianoMar 31, 2020

    great implementation indeed, couldn't think any better on how to implement this, as I created one with the traditional, controller service approach with providers. one question though, have you tried handling multiple images upload on single request for this one? if yes, how many files do you recommend to be uploaded once or how big is the size for example on the s3 storage?

    • Sergey Telpuk
      Sergey TelpukMar 31, 2020

      Hello, by default this solution doesn't support multiple files, about size and how big it depends on your aims

  • Nutchapong
    NutchapongJun 14, 2020

    Thank you so much for your guide. I have tested it out and it works great so far. The response from the controller is replied with the original image detail uploaded to S3 AWS. Is it possible to get the detail of both the cropped image and the original image as a response after finish uploading?

Add comment