Bootstrapping a Microservice with Node.js, Fastify, TypeScript, Prisma, MongoDB and gRPC

Introduction

A new microservice often needs to be started from scratch. That exact need came up not long ago — and of course there is also the temptation to try something new.

The stack was chosen first. The process itself is not new, yet it still came with a number of pitfalls. That is how the idea for this tutorial arose.

A link to the repository with the code appears at the end.

Step 1. Initializing the TS project

First, TypeScript itself has to be initialized and set up.

Starting a TypeScript project from scratch is a fairly trivial task, but for some reason the setup keeps running into problems. Nest is very convenient in this regard, since everything comes out of the box, but it is fairly heavy and is almost a programming language in its own right, so that is not the path taken here.

Run the following commands in the terminal:

npm init -y
yarn add typescript tsconfig-paths ts-node @types/node nodemon concurrently --dev
mkdir src && touch src/index.ts
npx tsc --init --rootDir src --outDir build --esModuleInterop --resolveJsonModule --module commonjs --allowJs true --noImplicitAny true --target esnext --moduleResolution node
touch .gitignore && echo "node_modules \n.vscode\nbuild" >> .gitignore
touch nodemon.json
echo "console.log('Hello world')" >> src/index.ts

Next, extend the typescript.json config file. The initial settings will not be touched — they are quite enough for our purposes:

{
  "compilerOptions": {...},
  "include": ["src"],
  "exclude": [
    "node_modules",
    "dist",
    "examples",
  ],
}

Fill in the scripts section of package.json:

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "tsc",
    "dev": "export NODE_ENV=development TS_NODE_BASEURL=./dist && concurrently -k -p \"[{name}]\" -n \"TypeScript,App\" -c \"yellow.bold,cyan.bold\" \"tsc -w\" \"nodemon\""
  },

nodemon will be used in dev mode, so it also needs to be configured in nodemon.json:

{
  "ignore": [
    "**/*.test.ts",
    "**/*.spec.ts",
    ".git",
    "node_modules"
  ],
  "watch": [
    "src"
  ],
  "exec": "tsc && node -r tsconfig-paths/register -r ts-node/register build/index.js",
  "ext": "ts, js"
}

Final check for this step: after starting it, the console should show:

yarn dev

[App] [nodemon] starting `tsc && node -r tsconfig-paths/register -r ts-node/register build/index.js`
[App] Hello world
[App] [nodemon] clean exit - waiting for changes before rest

The initial, basic setup is now done.

Step 2. Adding Fastify

Next, the server itself has to be added. To answer the question "why Fastify?" first:

Fastify is fairly light, and REST endpoints are quick and easy to write on it. It has a convenient plugin system — so why not.

Add the package and create a file for our server:

yarn add fastify
touch src/app.ts

At this stage the server is simple enough that building the application itself can go into a separate file, app.ts:

import Fastify, { FastifyServerOptions } from 'fastify'

export type AppOptions = Partial<FastifyServerOptions>;

async function buildApp(options: AppOptions = {}) {
  const fastify = Fastify(options);

  return fastify;
}

export { buildApp }

Describe the server startup in index.ts:

import { buildApp, AppOptions } from './app';

const options: AppOptions = {
  logger: true,
};

const start = async () => {
  const app = await buildApp(options);

  try {
    await app.listen({
      port: 3000,
      host: 'localhost',
    });

  } catch (err) {
    app.log.error(err);
    process.exit(1);
  }
};

start();

To check, start the server in dev mode; the console should show the following:

yarn dev

[App] {"level":30,"time":1676434938643,"pid":39600,"hostname":"Anatolys-MacBook-Pro.local","msg":"Server listening at http://[::1]:3000"}

Step 3. Adding Prisma

MongoDB will be used as the database. And why Prisma? Prisma was tried for the first time not long ago, and it was impressive: you describe a schema and it generates tons of logic on its own, which greatly simplifies working with the database.

You need to run MongoDB locally or pull it from Docker Hub: https://hub.docker.com/_/mongo.

Assume MongoDB is already running at mongodb://localhost:27017.

First install the required packages:

yarn add prisma @prisma/client fastify-plugin
npx prisma init --datasource-provider mongodb

Then define the database schema in prisma/schema.prisma:

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "mongodb"
  url      = "mongodb://localhost:27017/example"
}

model Post {
  id       String    @id @default(auto()) @map("_id") @db.ObjectId
  slug     String    @unique
  title    String
  body     String
  author   User      @relation(fields: [authorId], references: [id])
  authorId String    @db.ObjectId
}

model User {
  id      String   @id @default(auto()) @map("_id") @db.ObjectId
  email   String   @unique
  name    String?
  address Address?
  posts   Post[]
}

Run the generation of the database business logic and apply the changes to the database itself. This has to be done every time the schema changes, so these commands can be moved into the scripts section of package.json:

yarn prisma generate
npx prisma db push

Prisma will be connected through a Fastify plugin: touch src/prisma.plugin.ts

import { PrismaClient } from '@prisma/client';
import fp from 'fastify-plugin';

async function initDatabaseConnection(): Promise<PrismaClient> {
  const db = new PrismaClient();
  await db.$connect();
  return db;
}

// Use TypeScript module augmentation to declare the type of server.prisma to be PrismaClient
declare module 'fastify' {
  interface FastifyInstance {
    prisma: PrismaClient
  }
}

const prismaPlugin = fp(async (server) => {
  const prisma = await initDatabaseConnection();

  // Make Prisma Client available through the fastify server instance: server.prisma
  server.decorate('prisma', prisma);
  server.addHook('onClose', async () =>  {
    await server.prisma.$disconnect();
  });
});

export default prismaPlugin;

Next, register the plugin in app.ts:

import Fastify, { FastifyServerOptions } from 'fastify'
import prismaPlugin from './prisma.plugin';

export type AppOptions = Partial<FastifyServerOptions>;

async function buildApp(options: AppOptions = {}) {
  const fastify = Fastify(options);
  fastify.register(prismaPlugin);

  return fastify;
}

export { buildApp }

Add a simple handler to check that it works: touch src/services.ts

import { PrismaClient } from '@prisma/client';

async function main(prisma: PrismaClient) {
  await prisma.user.deleteMany();
  await prisma.user.create({ data: { email: '[email protected]' } });
  const usersCount = await prisma.user.count();
  console.log({ users_count: usersCount });

}

export { main };

Finally, update index.js:

import { buildApp, AppOptions } from './app';
import { main } from './services';

const options: AppOptions = {
  logger: true,
};

const start = async () => {
  const app = await buildApp(options);

  try {
    await app.listen({
      port: 3000,
      host: 'localhost',
    });

    await main(app.prisma);
  } catch (err) {
    app.log.error(err);
    process.exit(1);
  }
};

start();

Final check for step 3: the console should show the following:

yarn dev

[App] { users_count: 1 }

Step 4. Adding gRPC

We are approaching the finish line — all that is left is adding the gRPC server. What do we actually need gRPC for?

gRPC is good for backend-to-backend interactions: it is faster than REST, but harder to implement.

First install the required packages:

yarn add @grpc/grpc-js @grpc/proto-loader
mkdir proto
touch proto/example.proto

Then describe our server from the proto file side, example.proto. For the example it is enough to describe one service that returns an array of users:

syntax = "proto3";

package example;

message User {
  string id = 1;
  string email = 2;
};

message GetUsersResponse {
  repeated User users = 1;
}

message GetUsersRequest {}

service UserService {
  rpc GetUsers(GetUsersRequest) returns (GetUsersResponse);
}

Once the proto file is described, TypeScript types need to be generated. For that, use the proto-loader-gen-types generator built into proto-loader. Add the required command to the scripts section of package.json right away:

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "tsc",
    "dev": "export NODE_ENV=development TS_NODE_BASEURL=./dist && concurrently -k -p \"[{name}]\" -n \"TypeScript,App\" -c \"yellow.bold,cyan.bold\" \"tsc -w\" \"nodemon\"",
    "gen-proto": "$(npm bin)/proto-loader-gen-types --longs=String --enums=String  --oneofs --grpcLib=@grpc/grpc-js --outDir=src/proto/interfaces proto/*.proto"
  },

Then run the type generation: yarn gen-proto

All generated types end up in src/proto/interfaces.

The gRPC server is also connected through Fastify plugins. Create the file touch src/grpc.plugin.ts and put the following server implementation in it:

import { GetUsersResponse } from './proto/interfaces/example/GetUsersResponse';
import { GetUsersRequest__Output } from './proto/interfaces/example/GetUsersRequest';
import fp from 'fastify-plugin';
import { join } from 'path';
import * as grpc from '@grpc/grpc-js';
import * as protoLoader from '@grpc/proto-loader';
import { ProtoGrpcType } from './proto/interfaces/example';

declare module 'fastify' {
  interface FastifyInstance {
    grpcServer: {
      start: () => void,
    },
  }
}

const grpcServerOptions = {
  keepCase: false,
  longs: String,
  enums: String,
  defaults: false,
  oneofs: true,
};

const grpcServerPlugin = fp(async (fastify) => {
  // load proto files from directory
  const packageDefinition = protoLoader.loadSync([join(__dirname, '../proto/example.proto')], grpcServerOptions);
  const proto = grpc.loadPackageDefinition(packageDefinition) as unknown as ProtoGrpcType;
  const grpcServer = new grpc.Server();

  // mapping between handlers and rpc services
  grpcServer.addService(proto.example.UserService.service, {
    GetUsers: async (
      req: grpc.ServerUnaryCall<GetUsersRequest__Output, GetUsersResponse>,
      res: grpc.sendUnaryData<GetUsersResponse>) => {
        return res(null, {
          users: [{
            id: 'test',
            email: 'test',
          }],
        })
    },
  });

  function start(opts: { port: number } = { port: 50501 }) {
    return grpcServer.bindAsync(
      `0.0.0.0:${opts.port}`,
      grpc.ServerCredentials.createInsecure(),
      (err, port) => {
        if (err) {
          console.error(err);
          return;
        }

        grpcServer.start();
        console.log(`GRPC Server listening on ${port}`);
      },
    );
  }

  fastify.decorate('grpcServer', { start });
});

export { grpcServerPlugin };

The GetUsers endpoint handler response is mocked for demonstration purposes.

Once the plugin is ready, register it in app.ts:

import Fastify, { FastifyServerOptions } from 'fastify'
import { grpcServerPlugin } from './grpc.plugin';
import prismaPlugin from './prisma.plugin';

export type AppOptions = Partial<FastifyServerOptions>;

async function buildApp(options: AppOptions = {}) {
  const fastify = Fastify(options);
  fastify.register(prismaPlugin);
  fastify.register(grpcServerPlugin);

  return fastify;
}

export { buildApp }

The plugin is registered; all that remains is starting the server in index.ts:

import { buildApp, AppOptions } from './app';
import { main } from './services';

const options: AppOptions = {
  logger: true,
};

const start = async () => {
  const app = await buildApp(options);

  try {
    await app.listen({
      port: 3000,
      host: 'localhost',
    });

    app.grpcServer.start();

    await main(app.prisma);
  } catch (err) {
    app.log.error(err);
    process.exit(1);
  }
};

start();

To check, start the server with yarn dev; the console should show the following:

yarn dev

[App] {"level":30,"time":1676521988065,"pid":69311,"hostname":"Anatolys-MacBook-Pro.local","msg":"Server listening at http://127.0.0.1:3000"}
[App] GRPC Server listening on 50501
[App] { users_count: 1 }

The response can be checked either with a test or through a third-party application; bloomRPC was used here.

The response should return the mocked payload.

Conclusion

To summarize, the microservice template can accept gRPC and REST requests at the same time. REST runs on port 3000, gRPC on 50501.

Prisma is used as the ORM. The only thing missing is tests, but that is already beyond the scope of this tutorial!

Link to the project repo: https://github.com/iseekyouu/habr-tfpg