VENNStart
Guide

What bit me doing S3 presigned URL uploads in NestJS

Presigned PUT or upload through the API, what the ticket signs, checking the object with HeadObject, and the env var that freezes every stored URL.

Lê Anh Tuấn4 min read

A successful S3 PUT doesn't finish an upload in my post scheduler. The API still has to verify the stored object, reserve workspace quota and save a URL the publishing platforms can fetch. The code is from a side project that schedules posts to Instagram and Threads: a NestJS API, Prisma, an S3-compatible bucket on Neon, @aws-sdk/client-s3 3.1146.

Presign big files the server never reads

With a presigned URL, the browser PUTs straight to the bucket and calls the API to confirm. With multipart, multer spools it to the API's disk. I started with presigning so the bytes wouldn't cross the network twice. I dropped it from the composer quickly, for two reasons unrelated to S3.

Uploading on attach filled the bucket with files from unfinished posts, so files now stay in the page as blob: URLs until Save. The API also needs the whole video to read its duration, and a phone .mov usually keeps its moov box at the end. A confirm step that reads the first few hundred KB can't see it.

So the web app uploads through the API. Multer uses diskStorage because two 500 MB videos in memory means an OOM, and uploads go straight to the API because a Vercel Function refuses bodies over 4.5 MB. I'd still presign big files the server never reads, and send anything that needs processing through the API, which is much simpler.

The ticket signs the size and a staging key

Before signing, the endpoint checks MIME type, declared size and workspace quota. The server builds the key as <workspaceId>/<uuid><ext>, with the extension lowercased, stripped to [a-z0-9.] and cut at 12 characters. The client's filename never becomes the key.

import { PutObjectCommand, S3Client } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

const s3 = new S3Client({
  region: process.env.AWS_REGION,
  endpoint: process.env.AWS_ENDPOINT_URL_S3 || undefined, // S3-compatible providers
  forcePathStyle: process.env.S3_FORCE_PATH_STYLE === 'true',
});

export async function presignUpload(key: string, contentType: string, size: number) {
  const url = await getSignedUrl(
    s3,
    new PutObjectCommand({
      Bucket: 'example-bucket',
      Key: `incoming/${key}`, // a staging key, never the final one
      ContentType: contentType,
      ContentLength: size, // signed: a body of any other size is rejected
    }),
    { expiresIn: 300 },
  );
  return { url, headers: { 'content-type': contentType } };
}

The first version left ContentLength unsigned. I assumed a forbidden browser header couldn't match a signature. That was wrong. The browser computes the header from the body, and the body is the File with the declared size. Unsigned, a ticket for 1 KB accepted a 5 GB PUT. Signed, a body one byte longer gets 403 SignatureDoesNotMatch.

The ticket that comes with the URL is stateless, which saves a pending-uploads table and cleanup job for a five-minute permission.

import { createHmac, timingSafeEqual } from 'node:crypto';

type Ticket = { workspaceId: string; key: string; mimeType: string; size: number; expiresAt: number };

// Derived with a purpose label, so this signature means nothing anywhere else
const ticketKey = createHmac('sha256', process.env.JWT_ACCESS_SECRET!).update('media-upload-ticket:v1').digest();
const mac = (body: string) => createHmac('sha256', ticketKey).update(body).digest('base64url');

export const signTicket = (t: Ticket) => {
  const body = Buffer.from(JSON.stringify(t)).toString('base64url');
  return `${body}.${mac(body)}`;
};

export function verifyTicket(token: string, workspaceId: string): Ticket | null {
  const [body, sig] = token.split('.');
  if (!body || !sig) return null;
  const a = Buffer.from(sig), b = Buffer.from(mac(body));
  if (a.length !== b.length || !timingSafeEqual(a, b)) return null;
  try {
    const t = JSON.parse(Buffer.from(body, 'base64url').toString()) as Ticket;
    if (t.expiresAt < Date.now() || t.workspaceId !== workspaceId) return null;
    return t.key.startsWith(`${workspaceId}/`) ? t : null;
  } catch {
    return null;
  }
}

Any failure returns the same null and one 400.

The incoming/ prefix exists because a presigned PUT can be reused, and S3 replaces an object that already has the key. Originally the URL pointed at the final key, so anyone holding it could overwrite a file after confirm. Now the PUT goes to a staging key that nothing serves, and confirm copies it across.

Confirm trusts HeadObject, not the client

Confirm ignores the declared size and uses the one HeadObjectCommand returns.

async confirm(workspaceId: string, token: string) {
  const ticket = verifyTicket(token, workspaceId);
  if (!ticket) throw new BadRequestException({ code: 'media.uploadTicketInvalid' });

  const existing = await this.prisma.media.findFirst({ where: { workspaceId, storageKey: ticket.key } });
  if (existing) return existing; // confirm is idempotent

  const staged = `incoming/${ticket.key}`;
  const copied = await this.storage.copy(staged, ticket.key);
  await this.storage.remove(staged);

  const head = copied ? await this.storage.stat(ticket.key) : null; // HeadObjectCommand
  if (!head || head.size <= 0) throw new NotFoundException({ code: 'media.uploadMissing' });
  if (head.size !== ticket.size || head.size > MAX_BYTES) {
    await this.storage.remove(ticket.key);
    throw new PayloadTooLargeException({ code: 'media.tooLarge' });
  }

  const dims = await this.readDimensions(ticket.key); // ranged GET of the first 512 KB
  return this.quota.reserve(workspaceId, head.size, (tx) =>
    tx.media.create({
      data: { workspaceId, storageKey: ticket.key, size: head.size, mimeType: ticket.mimeType,
              width: dims?.width ?? null, height: dims?.height ?? null,
              url: this.storage.publicUrl(ticket.key) },
    }),
  );
}

stat maps NotFound, NoSuchKey and the bare 404 of some S3-compatible endpoints to null. Image dimensions come from the first 512 KB through a Range request, and they're null when that isn't enough.

quota.reserve takes a per-workspace lock. The full service checks for an existing media row again inside that lock, a check omitted from the trimmed snippet. The lock alone wouldn't stop two confirms from creating rows. copy falls back to streaming through the API on a 501.

The stored URL comes from one env var

The URL written at upload stays on the row. At first the API served files itself and stored ${PUBLIC_BASE_URL}/uploads/<key>, and production had never set that variable. Rows pointed at http://localhost:4001/uploads/..., and Meta can't fetch from localhost.

I took the API off the file-serving path and deleted the local driver, which wrote to a disk Render wipes on every deploy. The bucket went public_read (set in Neon's console, since PutBucketPolicy and PutBucketAcl return 501 there), rows store the bucket address, and one function picks the host.

export function resolveMediaBaseUrl(env: {
  MEDIA_PUBLIC_BASE_URL: string; // a CDN or custom domain in front of the bucket
  S3_BUCKET: string;
  AWS_ENDPOINT_URL_S3: string; // set for S3-compatible providers
  AWS_REGION: string;
}): string | null {
  const trim = (v: string) => v.replace(/\/+$/, '');
  if (env.MEDIA_PUBLIC_BASE_URL) return trim(env.MEDIA_PUBLIC_BASE_URL);
  if (!env.S3_BUCKET) return null; // never fall back to the API's own host

  return env.AWS_ENDPOINT_URL_S3
    ? `${trim(env.AWS_ENDPOINT_URL_S3)}/${env.S3_BUCKET}` // path-style
    : `https://${env.S3_BUCKET}.s3.${env.AWS_REGION}.amazonaws.com`; // virtual-hosted
}

Changing MEDIA_PUBLIC_BASE_URL later only affects new rows. Old ones need a backfill from storageKey, and I'm still not sure freezing the URL is right. A presigned GET can't go on the row, since a scheduled post may publish a week later and SDK presigned URLs last seven days at most.

The API used to refuse to boot without a bucket, and a Render deploy stayed down because S3_BUCKET is sync: false in render.yaml and nobody had set it on the dashboard. It's a boot warning now, and the resolver returns null instead of guessing a host.

On Neon, <bucket>.<host> doesn't resolve in DNS, so forcePathStyle is true and public URLs use the path form. A browser PUT also needs CORS for PUT and content-type, which Neon allows by default and AWS leaves to you.

Orphans and a useful 413

A pg-boss watchdog cron runs every ten minutes and deletes anything under incoming/ older than an hour. Staged bytes aren't rows, so the quota can't see them until confirm.

Once a post is out everywhere, its files are released, object first, then the row (unless the platform returned a cover image). Done the other way round, a failed delete left bytes the sweep for unreleased rows never found. DeleteObject succeeds on a missing key, so retrying is safe.

The limit is MAX_UPLOAD_MB, 500 by default and capped at 2048 because Media.size is an Int. Multer aborts at limits.fileSize and @nestjs/platform-express wraps it in a PayloadTooLargeException with multer's message, so the client got 413 {"message":"File too large"} in English, under a Vietnamese toast, with no numbers.

@Catch(PayloadTooLargeException)
export class UploadTooLargeFilter implements ExceptionFilter {
  catch(_e: PayloadTooLargeException, host: ArgumentsHost) {
    const maxMb = Number(process.env.MAX_UPLOAD_MB ?? 500);
    host.switchToHttp().getResponse<Response>().status(413).json({
      statusCode: 413,
      error: 'Payload Too Large',
      code: 'media.tooLarge', // what the client translates
      params: { maxMb },
      message: `The file is larger than the ${maxMb} MB per-file limit`,
    });
  }
}

// on the upload routes only
@Post()
@UseFilters(UploadTooLargeFilter)
@UseInterceptors(FileInterceptor('file', { storage: diskStorage({ destination: './uploads' }), limits: { fileSize: MAX_BYTES } }))
upload(@UploadedFile() file: Express.Multer.File) { /* ... */ }

It's on the upload routes only because the app also caps JSON bodies at 2 MB, and a global rule would answer a 3 MB JSON body with a sentence about videos. Multer stops reading at the limit, so only the client has the real size. GET /media/limits gives it the cap up front, and the composer refuses with both numbers.

  • NestJS
  • AWS S3
  • File uploads
  • Node.js
  • Backend

I'm Tuấn (Venn), a web developer in Ho Chi Minh City. I mostly build Next.js sites and web apps, and I take freelance work.

Work with me

More posts

  1. 01Guide4 min read

    Deploying Next.js and NestJS to AWS EC2 with Docker Compose

    How I deploy a pnpm monorepo to one EC2 box. Images get built in GitHub Actions and Docker Compose pulls them from GHCR. The server has no code.

  2. 02Guide4 min read

    How I store OAuth tokens for social media APIs in NestJS

    OAuth for Facebook, TikTok and YouTube in a NestJS API. State and PKCE, encrypted tokens, refreshing before each publish, and when to ask for a reconnect.

  3. 03vennee.dev3 min read

    llms.txt on my Next.js site, and now Lighthouse grades it

    vennee.dev has a hand-written llms.txt and an llms-full.txt built from the blog. Notes on the spec, .txt routes in Next.js, and Lighthouse 13's new audit.