Setting up a timeout for HTTP requests can prevent the connection from hanging forever, waiting for the response. It can be set on the client side to improve user experience, and on the server side to improve inter-service communication.

axios package provides a timeout parameter for this functionality.

const HTTP_TIMEOUT = 3000;
const URL = 'https://www.google.com:81';
(async () => {
  try {
    await axios(URL, {
      timeout: HTTP_TIMEOUT,
    });
  } catch (error) {
    console.error('Request timed out', error.cause);
  }
})();

Use this snippet also to simulate aborted requests.

Demo

The demo with the mentioned example is available here.

Profiling refers to analyzing and measuring an application's performance characteristics.

Profiling helps identify performance bottlenecks in a Node.js app, such as CPU-intensive tasks like cryptographic operations, image processing, or complex calculations.

This post covers running a profiler for various Node.js apps in Chrome DevTools.

Prerequisites

• Google Chrome installed

• Node.js app bootstrapped

Setup

• Run node --inspect app.js to start the debugger.

• Open chrome://inspect, click Open dedicated DevTools for Node and then navigate to the Performance tab. Start recording.

• Run load testing via autocannon package using the following command format npx autocannon <COMMAND>.

• Stop recording in Chrome DevTools.

Profiling

On Perfomance tab in Chrome DevTools open Bottom-Up subtab to identify which functions consume the most time.

Look for potential performance bottlenecks, such as synchronous functions for hashing (pbkdf2Sync) or file system operations (readFileSync).

k6 is a performance testing tool. This post explains types of performance testing and dives into k6 usage, from configuration to running tests.

Load and stress testing

Load and stress testing are two types of performance testing used to evaluate how well a system performs under various conditions.

Load testing determines how the system performs under expected user loads. The purpose is to identify performance bottlenecks.

Stress testing assesses how the system performs when loads are heavier than usual. The purpose is to find the limit at which the system fails and to observe how it recovers from such failures.

Prerequisites

• k6 installed

• Script (Node.js) file with configuration and execution function

Configuration

Configuration is stored inside options variable, which allows you to set up different testing scenarios:

• constant user load, the number of virtual users (vus) remains constant throughout the test period

export const options = {
  vus: 30,
  duration: '10m'
};

• variable user load, the number of users increases and decreases over time

export const options = {
  stages: [
    {
      duration: '1m',
      target: 30
    },
    {
      duration: '10m',
      target: 30
    },
    {
      duration: '5m',
      target: 0
    }
  ]
};

Environment variables can be passed through the command line and are accessible within the script via the __ENV object.

k6 -e TOKEN=token run script.js

Execution function

This function defines what virtual users will do during the test. This function is called for each virtual user and typically includes steps that simulate user actions on the app.

export default function() {
  http.get(URL);
  // Add more actions as required
}

Test report

k6 generates a report that provides detailed insights into various benchmarks, such as the number of virtual users, requests per second, request durations and error rates.

Example

This example utilizes k6 to conduct a load test using a variable user load approach:

• User simulation: The script ramps up to 1,000 users, maintains that level to simulate sustained traffic, and gradually reduces to zero.

• Request handling: During the test, each virtual user sends a POST request to an API, with pauses between requests to mimic real user behavior.

• Performance insights: After the test, k6 provides a report that shows key information, such as how fast the app responds and how many requests fail.

Run it via k6 -e TOKEN=1234 run script.js command.

// script.js
import { check, sleep } from 'k6';
import { scenario } from 'k6/execution';
import http from 'k6/http';
export const options = {
  stages: [
    // Ramp up to 1000 users over 10 minutes
    {
      duration: '10m',
      target: 1000
    },
    // Hold 1000 users for 30 minutes
    {
      duration: '30m',
      target: 1000
    },
    // Ramp down to 0 users over 5 minutes
    {
      duration: '5m',
      target: 0
    }
  ]
};
export default () => {
  const response = http.post(
    URL,
    JSON.stringify({
      iteration: scenario.iterationInTest
    }),
    {
      headers: {
        Authorization: __ENV.TOKEN,
        'Content-Type': 'application/json'
      }
    }
  );
  check(response, {
    'response status was 200': (res) => res.status === 200
  });
  sleep(1);
};

The demo with the mentioned example is available here.

nvm facilitates switching between different Node versions across projects. This post covers its overview from installation to version management.

Installation

To install nvm, execute the following commands in your terminal. This example uses zsh, but the process is similar for other shells like bash.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.zshrc

Version management

• Install the specific version. Including the v prefix is optional.

  nvm install v21.7.2

• Install the latest version

  nvm install node

• Install the latest one for the specified major version

  nvm install 22

• Switch to a specific installed version

  nvm use 21

• Add .nvmrc file inside the project directory and run nvm use command to use the specified installed version.

  v21.7.2

• Get the list of locally installed versions

  nvm ls

• Get the list of available versions for installation

  nvm ls-remote

Boilerplate

Here is the link to the template I use for the development.

Debugging with a debugger and breakpoints is recommended rather than using console logs. Chrome provides a built-in debugger for JavaScript-based apps.

This post covers configuring and running a debugger for various Node.js apps in Chrome DevTools.

Prerequisites

• Google Chrome installed

• Node.js app bootstrapped

Setup

Open chrome://inspect, click Open dedicated DevTools for Node and open the Connection tab. You should see a list of network endpoints for debugging.

Run node --inspect-brk app.js to start the debugger, it will log the debugger network. Choose the network in Chrome to open the debugger in Chrome DevTools.

Debugging basics

The variables tab shows local and global variables during debugging.

The step over next function call option goes to the following statement in the codebase, while the step into next function call option goes deeper into the current statement.

Add logs in the debug console via logpoints by selecting the specific part of the codebase so it logs when the selected codebase executes.

Boilerplate

Here is the link to the boilerplate I use for the development.

To send e-mails in a production environment, use services like Sendgrid.

Verify the e-mail address on the Sender Management page and create the SendGrid API key on the API keys page.

import nodemailer from 'nodemailer';
(async () => {
  const emailConfiguration = {
    auth: {
      user: process.env.EMAIL_USERNAME, // 'apikey'
      pass: process.env.EMAIL_PASSWORD
    },
    host: process.env.EMAIL_HOST, // 'smtp.sendgrid.net'
    port: process.env.EMAIL_PORT, // 465
    secure: process.env.EMAIL_SECURE, // true
  };
  const transport = nodemailer.createTransport(emailConfiguration);
  const info = await transport.sendMail({
    from: '"Sender" <sender@example.com>',
    to: 'recipient1@example.com, recipient2@example.com',
    subject: 'Subject',
    text: 'Text',
    html: '<b>Text</b>'
  });
  console.log('Message sent: %s', info.messageId);
})();

Demo

The demo with the mentioned example is available here.

Web scraping means extracting data from websites. This post covers extracting data from the page's HTML tags.

Prerequisites

• cheerio package is installed

• HTML page is retrieved via an HTTP client

Usage

• create a scraper object with load method by passing HTML content as an argument

  • set decodeEntities option to false to preserve encoded characters (like &) in their original form

  const $ = load('<div><!-- HTML content --></div>', { decodeEntities: false });

• find DOM elements by using CSS-like selectors

  const items = $('.item');

• iterate through found elements using each method

  items.each((index, element) => {
    // ...
  });

• access element content using specific methods

  • text - $(element).text()

  • HTML - $(element).html()

  • attributes

    • all - $(element).attr()
    • specific one - $(element).attr('href')

  • child elements

    • first - $(element).first()
    • last - $(element).last()
    • all - $(element).children()
    • specific one - $(element).find('a')

  • siblings

    • previous - $(element).prev()
    • next - $(element).next()

Disclaimer

Please check the website's terms of service before scraping it. Some websites may have terms of service that prohibit such activity.

Demo

The demo with the mentioned examples is available here.

GitHub provides GraphQL API to create integrations, retrieve data, and automate workflows.

Prerequisites

• GitHub token (Settings → Developer Settings → Personal access tokens)

Integration

Below is an example of retrieving sponsorable users by location.

export async function getUsersBy(location) {
  return fetch('https://api.github.com/graphql', {
    method: 'POST',
    body: JSON.stringify({
      query: `query {
        search(type: USER, query: "location:${location} is:sponsorable", first: 100) {
          edges {
            node {
              ... on User {
                bio
                login
                viewerCanSponsor
              }
            }
          }
          userCount
        }
      }`
    }),
    headers: {
      ContentType: 'application/json',
      Authorization: `Bearer ${process.env.GITHUB_TOKEN}`
    }
  })
    .then((response) => response.json())
    .then((response) => response.data?.search?.edges || []);
}

Demo

The demo with the mentioned example is available here.

Web scraping means extracting data from websites. This post covers extracting data from the page's HTML when data is stored in JavaScript variable or stringified JSON.

The scraping prerequisite is retrieving an HTML page via an HTTP client.

Examples

The example below moves data into a global variable, executes the page scripts and accesses the data from the global variable.

import jsdom from 'jsdom';
fetch(URL)
  .then((res) => res.text())
  .then((response) => {
    const dataVariable = 'someVariable.someField';
    const html = response.replace(dataVariable, `var data=${dataVariable}`);
    const dom = new jsdom.JSDOM(html, {
      runScripts: 'dangerously',
      virtualConsole: new jsdom.VirtualConsole()
    });
    console.log('data', dom?.window?.data);
  });

The example below runs the page scripts, and access stringified JSON data.

import jsdom from 'jsdom';
fetch(URL)
  .then((res) => res.text())
  .then((response) => {
    const dom = new jsdom.JSDOM(response, {
      runScripts: 'dangerously',
      virtualConsole: new jsdom.VirtualConsole()
    });
    const data = dom?.window?.document?.getElementById('someId')?.value;
    console.log('data', JSON.parse(data));
  });

Disclaimer

Please check the website's terms of service before scraping it. Some websites may have terms of service that prohibit such activity.

Boilerplate

Here is the link to the boilerplate I use for the development.

Gumroad allows verifying license keys via API calls to limit the usage of the keys. It can be helpful to prevent the redistribution of products like desktop apps.

Allow generating a unique license key per sale in product settings, and the product ID will be shown there. Below is the code snippet for verification.

try {
  const requestBody = new URLSearchParams();
  requestBody.append('product_id', process.env.PRODUCT_ID);
  requestBody.append('license_key', process.env.LICENSE_KEY);
  requestBody.append('increment_uses_count', true);
  const response = await fetch('https://api.gumroad.com/v2/licenses/verify', {
    method: 'POST',
    body: requestBody
  });
  const data = await response.json();
  if (data.purchase?.test) {
    console.log('Skipping verification for test purchase');
    return;
  }
  const verificationLimit = Number(process.env.VERIFICATION_LIMIT);
  if (data.uses >= verificationLimit + 1) {
    throw new Error('Verification limit exceeded');
  }
  if (!data.success) {
    throw new Error(data.message);
  }
} catch (error) {
  if (error?.response?.status === 404) {
    console.log("License key doesn't exist");
    return;
  }
  console.log('Verifying license key failed', error);
}

Demo

The demo with the mentioned example is available here.

Gotenberg is a Docker-based stateless API for PDF generation from HTML and Markdown files.

To get started, configure Docker compose and run the docker-compose up command.

version: '3.8'
services:
  gotenberg:
    image: gotenberg/gotenberg:7
    ports:
      - 3000:3000

API is available on http://localhost:3000 address.

Run the following commands to generate PDFs

• from the given URL

  curl \
    --request POST 'http://localhost:3000/forms/chromium/convert/url' \
    --form 'url="https://sparksuite.github.io/simple-html-invoice-template/"' \
    --form 'pdfFormat="PDF/A-1a"' \
    -o curl-url-response.pdf

• from the given HTML file

  curl \
    --request POST 'http://localhost:3000/forms/chromium/convert/html' \
    --form 'files=@"./index.html"' \
    --form 'pdfFormat="PDF/A-1a"' \
    -o curl-html-response.pdf

PDF/A-1a format is used for the long-term preservation of electronic documents, ensuring that documents can be accessed and read even as technology changes.

Demo

The demo with the mentioned examples is available here. It also contains examples using Fetch API and Axios package.

Handlebars is a template engine that can create server-side views, e-mail templates, and invoice templates by injecting JSON data into HTML.

Resolving all variables in a Handlebars template is essential to maintain the accuracy of the displayed information and to prevent incomplete content or layout problems.

The following snippet checks for missing variables by overriding the default nameLookup function. It logs a warning for unresolved variables and sets the default value, empty string, in this case.

// ...
const originalNameLookup = Handlebars.JavaScriptCompiler.prototype.nameLookup;
Handlebars.JavaScriptCompiler.prototype.nameLookup = function(
  parent,
  name,
  type
) {
  if (type === 'context') {
    const messageLog = JSON.stringify({
      message: `Variable is not resolved in the template: ${name}`,
      level: WARNING_LEVEL
      // ...
    });
    return `${parent} && ${parent}.${name} ? ${parent}.${name} : (console.log(${messageLog}), ''`;
  }
  return originalNameLookup.call(this, parent, name, type);
};
// ...
const result = Handlebars.compile(template)(data);

Extending package declarations locally is one of the options for outdated package typings.

Create a declaration file .d.ts (e.g., handlebars.d.ts), and put it inside the src directories.

Find the exact name of the package namespace inside the node_modules types file (e.g. handlebars/types/index.d.ts).

Extend the found namespace with your needed properties, like classes, functions, etc.

// handlebars.d.ts
declare namespace Handlebars {
  export class JavaScriptCompiler {
    public nameLookup(parent: string, name: string, type: string): string | string[];
  }
  export function doSomething(name: string): void;
  // ...
}

Notion is a versatile workspace tool combining note-taking, task management, databases, and collaboration features into a single platform.

It also supports integration with Notion content, facilitating tasks such as creating pages, retrieving a block, and filtering database entries via API.

Prerequisites

• Notion account
• Generated Integration token (Settings & Members → Connections → Develop or manage integrations → New integration)
• Notion database ID (open database as full page, extract database ID from the URL (https://notion.so/<USERNAME>/<DATABASE_ID>?v=v))
• Added Notion connection (three dots (...) menu → Add Connections → choose created integration)
• @notionhq/client package installed

Integration

Below is an example of interacting with Notion API to create the page (within the chosen database) with icon, cover, properties, and child blocks.

const { Client } = require('@notionhq/client');
const notion = new Client({ auth: process.env.NOTION_INTEGRATION_TOKEN });
const response = await notion.pages.create({
  parent: {
    type: 'database_id',
    database_id: process.env.NOTION_DATABASE_ID
  },
  icon: {
    type: 'emoji',
    emoji: '🆗'
  },
  cover: {
    type: 'external',
    external: {
      url: 'https://cover.com'
    }
  },
  properties: {
    Name: {
      title: [
        {
          type: 'text',
          text: {
            content: 'Some name'
          }
        }
      ]
    },
    Score: {
      number: 42
    },
    Tags: {
      multi_select: [
        {
          name: 'A'
        },
        {
          name: 'B'
        }
      ]
    },
    Generation: {
      select: {
        name: 'I'
      }
    }
    // other properties
  },
  children: [
    {
      object: 'block',
      type: 'bookmark',
      bookmark: {
        url: 'https://bookmark.com'
      }
    }
  ]
});

Check out Sparxno for more Notion-related content.

Puppeteer is a headless browser for automating browser tasks. Here's the list of some of the features:

• Turn off headless mode

  const browser = await puppeteer.launch({
    headless: false
    // ...
  });

• Resize the viewport to the window size

  const browser = await puppeteer.launch({
    // ...
    defaultViewport: null
  });

• Emulate screen how it's shown to the user via the emulateMediaType method

  await page.emulateMediaType('screen');

• Save the page as a PDF file with a specified path, format, scale factor, and page range

  await page.pdf({
    path: 'path.pdf',
    format: 'A3',
    scale: 1,
    pageRanges: '1-2',
    printBackground: true
  });

• Use preexisting user's credentials to skip logging in to some websites. The user data directory is a parent of the Profile Path value from the chrome://version page.

  const browser = await puppeteer.launch({
    userDataDir: 'C:\\Users\\<USERNAME>\\AppData\\Local\\Google\\Chrome\\User Data',
    args: [],
  });

• Use Chrome instance instead of Chromium by utilizing the Executable Path from the chrome://version URL. Close Chrome browser before running the script

  const browser = await puppeteer.launch({
    executablePath: puppeteer.executablePath('chrome'),
    // ...
  });

• Get value based on evaluation in the browser page

  const shouldPaginate = await page.evaluate((param1, param2) => {
    // ...
  }, param1, param2);

• Get HTML content from the specific element

  const html = await page.evaluate(
    () => document.querySelector('.field--text').outerHTML,
  );

• Wait for a specific selector to be loaded. You can also provide a timeout in milliseconds

  await page.waitForSelector('.success', { timeout: 5000 });

• Manipulate with a specific element and click on some of the elements

  await page.$eval('#header', async (headerElement) => {
    // ...
    headerElement
      .querySelectorAll('svg')
      .item(13)
      .parentNode.click();
  });

• Extend execution of the $eval method

  const browser = await puppeteer.launch({
    // ...
    protocolTimeout: 0,
  });

• Manipulate with multiple elements

  await page.$$eval('.some-class', async (elements) => {
    // ...
  });

• Wait for navigation (e.g., form submitting) to be done

  await page.waitForNavigation({ waitUntil: 'networkidle0', timeout: 0 });

• Trigger hover event on some of the elements

  await page.$eval('#header', async (headerElement) => {
    const hoverEvent = new MouseEvent('mouseover', {
      view: window,
      bubbles: true,
      cancelable: true
    });
    headerElement.dispatchEvent(hoverEvent);
  });

• Expose a function in the browser and use it in $eval and $$eval callbacks (e.g., simulate typing using the window.type function)

  await page.exposeFunction('type', async (selector, text, options) => {
    await page.type(selector, text, options);
  });
  await page.$$eval('.some-class', async (elements) => {
    // ...
    window.type(selector, text, { delay: 0 });
  });

• Press the Enter button after typing the input field value

  await page.type(selector, `${text}${String.fromCharCode(13)}`, options);

• Remove the value from the input field before typing the new one

  await page.click(selector, { clickCount: 3 });
  await page.type(selector, text, options);

• Expose a variable in the browser by passing it as the third argument for $eval and $$eval methods and use it in $eval and $$eval callbacks

  await page.$eval(
    '#element',
    async (element, customVariable) => {
      // ...
    },
    customVariable
  );

• Mock response for the specific request

  await page.setRequestInterception(true);
  page.on('request', async function (request) {
    const url = request.url();
    if (url !== REDIRECTION_URL) {
      return request.continue();
    }
    await request.respond({
      contentType: 'text/html',
      status: 304,
      body: '<body></body>',
    });
  });

• Intercept page redirections (via interceptor) and open them in new tabs rather than following them in the same tab

  await page.setRequestInterception(true);
  page.on('request', async function (request) {
    const url = request.url();
    if (url !== REDIRECTION_URL) {
      return request.continue();
    }
    await request.respond({
      contentType: 'text/html',
      status: 304,
      body: '<body></body>',
    });
    const newPage = await browser.newPage();
    await newPage.goto(url, { waitUntil: 'domcontentloaded', timeout: 0 });
    // ...
    await newPage.close();
  });

• Intercept page response

  page.on('response', async (response) => {
    if (response.url() === RESPONSE_URL) {
      if (response.status() === 200) {
        // ...
      }
      // ...
    }
  });

Boilerplate

Here is the link to the boilerplate I use for the development.

Image upscaling can be done using Real-ESRGAN, a super-resolution algorithm. Super-resolution is the process of increasing the resolution of the image.

Real-ESRGAN provides Linux, Windows and MacOS executable files and models for Intel/AMD/Nvidia GPUs.

The snippet below demonstrates bulk image upscaling with scale factor 4 and using the realesrgan-x4plus-anime model.

(async () => {
  const inputDirectory = path.resolve(path.join(__dirname, 'pictures'));
  const outputDirectory = path.resolve(
    path.join(__dirname, 'pictures_upscaled')
  );
  const modelsPath = path.resolve(path.join(__dirname, 'resources', 'models'));
  const execName = 'realesrgan-ncnn-vulkan';
  const execPath = path.resolve(
    path.join(__dirname, 'resources', getPlatform(), 'bin', execName)
  );
  const scaleFactor = 4;
  const modelName = 'realesrgan-x4plus-anime';
  if (!fs.existsSync(outputDirectory)) {
    await fs.promises.mkdir(outputDirectory, { recursive: true });
  }
  const commands = [
    '-i',
    inputDirectory,
    '-o',
    outputDirectory,
    '-s',
    scaleFactor,
    '-m',
    modelsPath,
    '-n',
    modelName
  ];
  const upscaler = spawn(execPath, commands, {
    cwd: undefined,
    detached: false
  });
  upscaler.stderr.on('data', (data) => {
    console.log(data.toString());
  });
  await timers.setTimeout(600 * 1000);
})();

Demo

The demo with the mentioned example is available here.

Releasing Electron desktop apps can be automated with Electron Forge and GitHub Actions. This post covers the main steps for automation.

Prerequisites

• bootstrapped Electron app
• GitHub personal access token (with repo and write:packages permissions) as a GitHub Action secret (GH_TOKEN)

Setup

Run the following commands to configure Electron Forge for the app release.

npm i @electron-forge/cli @electron-forge/publisher-github -D
npm i electron-squirrel-startup
npx electron-forge import

The last command should install the necessary dependencies and add a configuration file.

Update the forge.config.js file with the bin field containing the app name and ensure the GitHub publisher points to the right repository.

Put Windows and MacOS icons paths in the packagerConfig.icon field, Windows supports ico files with 256x256 resolution, and MacOS supports icns icons with 512x512 resolution (1024x1024 for Retina displays). Linux supports png icons with 512x512 resolution, also include its path in the BrowserWindow constructor config within the icon field.

// forge.config.js
const path = require('path');
module.exports = {
  packagerConfig: {
    asar: true,
    icon: path.join(process.cwd(), 'main', 'build', 'icon'),
  },
  rebuildConfig: {},
  makers: [
    {
      name: '@electron-forge/maker-squirrel',
      config: {
        bin: 'Electron Starter'
      }
    },
    {
      name: '@electron-forge/maker-dmg',
      config: {
        bin: 'Electron Starter'
      },
    },
    {
      name: '@electron-forge/maker-deb',
      config: {
        bin: 'Electron Starter',
        options: {
          icon: path.join(process.cwd(), 'main', 'build', 'icon.png'),
        },
      }
    },
    {
      name: '@electron-forge/maker-rpm',
      config: {
        bin: 'Electron Starter',
        icon: path.join(process.cwd(), 'main', 'build', 'icon.png'),
      }
    }
  ],
  plugins: [
    {
      name: '@electron-forge/plugin-auto-unpack-natives',
      config: {}
    }
  ],
  publishers: [
    {
      name: '@electron-forge/publisher-github',
      config: {
        repository: {
          owner: 'delimitertech',
          name: 'electron-starter'
        },
        prerelease: true
      }
    }
  ]
};

Upgrade the package version before releasing the app. The npm script for publishing should use publish command. Set productName field to the app name.

// package.json
{
  // ...
  "version": "1.0.1",
  "scripts": {
    // ...
    "publish": "electron-forge publish"
  },
  "productName": "Electron Starter"
}

GitHub Action workflow for manually releasing the app for Linux, Windows, and MacOS should contain the below configuration.

# .github/workflows/release.yml
name: Release app
on:
  workflow_dispatch:
jobs:
  build:
    strategy:
      matrix:
        os:
          [
            { name: 'linux', image: 'ubuntu-latest' },
            { name: 'windows', image: 'windows-latest' },
            { name: 'macos', image: 'macos-latest' },
          ]
    runs-on: ${{ matrix.os.image }}
    steps:
      - name: Github checkout
        uses: actions/checkout@v4
      - name: Use Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - name: Publish app
        env:
          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
        run: npm run publish

Windows startup events

Add the following code in the main process to prevent Squirrel.Windows launches your app multiple times during the installation/updating/uninstallation.

// main/index.js
if (require('electron-squirrel-startup') === true) app.quit();

Boilerplate

Here is the link to the boilerplate I use for the development. It contains the examples mentioned above with more details.

Formatting helps to stay consistent with code style throughout the whole codebase. Include format script in pre-hooks (pre-commit or pre-push). This post covers Prettier setup with JavaScript and TypeScript code.

Start by installing the prettier package as a dev dependency.

npm i prettier -D

Specify rules inside the .prettierrc config file.

{
  "singleQuote": true,
  "trailingComma": "all"
}

Add format script in the package.json file.

{
  "scripts": {
    // ...
    "format": "prettier --write \"{src,test}/**/*.{js,ts}\""
  }
}

Notes

If you use Eslint, install the eslint-config-prettier package as a dev dependency and update the Eslint configuration to use the Prettier config.

// eslint.config.js
// ...
import eslintConfigPrettier from 'eslint-config-prettier';
export default [
  // ...
  eslintConfigPrettier
];

Using Visual Studio Code, you can install a prettier-vscode extension and activate formatting when file changes are saved.

Demo

The demo with the mentioned examples is available here.

Regarding microservices observability, tracing is important to catch bottlenecks of the services like slow requests and database queries.

OpenTelemetry is a set of monitoring tools that support integration with distributed tracing platforms like Jaeger, Zipkin, and NewRelic, to name a few. This post covers Jaeger's tracing setup for Node.js projects.

Start by setting up the Docker compose via the docker-compose up command. Jaeger UI will be available at http://localhost:16686.

version: '3.8'
services:
  jaeger:
    image: jaegertracing/all-in-one:1.46
    environment:
      - COLLECTOR_ZIPKIN_HTTP_PORT=:9411
      - COLLECTOR_OTLP_ENABLED=true
    ports:
      - 6831:6831/udp
      - 6832:6832/udp
      - 5778:5778
      - 16685:16685
      - 16686:16686
      - 14268:14268
      - 14269:14269
      - 14250:14250
      - 9411:9411
      - 4317:4317
      - 4318:4318

The code below shows setting up the tracing via Jaeger. Jaeger doesn't require a separate exporter package since OpenTelemetry supports it natively. Others need to use an exporter package. Filter traces within Jaeger UI by service name or trace ID stored within the logs.

Use resources and semantic resource attributes to set new fields for the trace, like service name or service version. Auto instrumentation identifies frameworks like Express, protocols like HTTP, databases like Postgres, and loggers like Winston used within the project.

Process spans (units of work in distributed systems) in batches to optimize tracing performance. Also, terminate the tracing during the graceful shutdown.

import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
const traceExporter = new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/traces',
});
const sdk = new NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: `<service-name>-${process.env.NODE_ENV}`,
    [SemanticResourceAttributes.SERVICE_VERSION]: process.env.npm_package_version ?? '0.0.0',
    env: process.env.NODE_ENV || '',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
  spanProcessor: new BatchSpanProcessor(traceExporter),
});
sdk.start();
process.on('SIGTERM', () => {
  sdk.shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.error('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

Import tracing config as the first thing inside the entry file.

import './tracing';
// ...

Search → Service menu should show the service name in Jaeger UI. Happy tracing!

Demo

The demo with the mentioned examples is available here.

Streaming is useful when dealing with big files in web apps. Instead of loading the entire file into memory before sending it to the client, streaming allows you to send it in small chunks, improving memory efficiency and reducing response time.

The code snippet below shows streaming the binary CSV and base64-encoded PDF files with NestJS. Use the same approach for other types of files, like JSON files.

Set content type and filename headers so files are streamed and downloaded correctly. Base64 file is converted to a buffer and streamed afterward. Read files from a file system or by API calls.

import { Controller, Get, Param, Res } from '@nestjs/common';
import { Response } from 'express';
import { createReadStream } from 'fs';
import { readFile } from 'fs/promises';
import { join } from 'path';
import { Readable } from 'stream';
@Controller('templates')
export class TemplatesController {
  @Get('csv')
  getCsvTemplate(@Res() res: Response): void {
    const file = createReadStream(join(process.cwd(), 'template.csv'));
    res.set({
      'Content-Type': 'text/csv',
      'Content-Disposition': 'attachment; filename="template.csv"'
    });
    file.pipe(res);
  }
  @Get('pdf/:id')
  async getPdfTemplate(
    @Param('id') id: string,
    @Res() res: Response
  ): Promise<void> {
    const fileBase64 = await readFile(
      join(process.cwd(), 'template.pdf'),
      'base64'
    );
    // const fileBase64 = await apiCall();
    const fileBuffer = Buffer.from(fileBase64, 'base64');
    const fileStream = Readable.from(fileBuffer);
    res.set({
      'Content-Type': 'application/pdf',
      'Content-Disposition': `attachment; filename="template_${id}.pdf"`
    });
    fileStream.pipe(res);
  }
}

Boilerplate

Here is the link to the boilerplate I use for the development.

Node.js version 20 brings a stable test runner so you can run tests inside *.test.js files with node --test command. This post covers the primary usage of it regarding spies and mocking for the unit tests.

Spies are functions that let you spy on the behavior of functions called indirectly by some other code while mocking injects test values into the code during the tests.

mock.method can create spies and mock async, rejected async, sync, chained methods, and external and built-in modules.

• Async function

import assert from 'node:assert/strict';
import { describe, it, mock } from 'node:test';
const calculationService = {
  calculate: () => // implementation
};
describe('mocking resolved value', () => {
  it('should resolve mocked value', async () => {
    const value = 2;
    mock.method(calculationService, 'calculate', async () => value);
    const result = await calculationService.calculate();
    assert.equal(result, value);
  });
});

• Rejected async function

const error = new Error('some error message');
mock.method(calculationService, 'calculate', async () => Promise.reject(error));
await assert.rejects(async () => calculateSomething(calculationService), error);

• Sync function

mock.method(calculationService, 'calculate', () => value);

• Chained methods

mock.method(calculationService, 'get', () => calculationService);
mock.method(calculationService, 'calculate', async () => value);
const result = await calculationService.get().calculate();

• External modules

import axios from 'axios';
mock.method(axios, 'get', async () => ({ data: value }));

• Built-in modules

import fs from 'fs/promises';
mock.method(fs, 'readFile', async () => fileContent);

• Async and sync functions called multiple times can be mocked with different values using context.mock.fn and mockedFunction.mock.mockImplementationOnce.

describe('mocking same method multiple times with different values', () => {
  it('should resolve mocked values', async (context) => {
    const firstValue = 2;
    const secondValue = 3;
    const calculateMock = context.mock.fn(calculationService.calculate);
    calculateMock.mock.mockImplementationOnce(async () => firstValue, 0);
    calculateMock.mock.mockImplementationOnce(async () => secondValue, 1);
    const firstResult = await calculateMock();
    const secondResult = await calculateMock();
    assert.equal(firstResult, firstValue);
    assert.equal(secondResult, secondValue);
  });
});

• To assert called arguments for a spy, use mockedFunction.mock.calls[0] value.

mock.method(calculationService, 'calculate');
await calculateSomething(calculationService, firstValue, secondValue);
const call = calculationService.calculate.mock.calls[0];
assert.deepEqual(call.arguments, [firstValue, secondValue]);

• To assert skipped call for a spy, use mockedFunction.mock.calls.length value.

mock.method(calculationService, 'calculate');
assert.equal(calculationService.calculate.mock.calls.length, 0);

• To assert how many times mocked function is called, use mockedFunction.mock.calls.length value.

mock.method(calculationService, 'calculate');
calculationService.calculate(3);
calculationService.calculate(2);
assert.equal(calculationService.calculate.mock.calls.length, 2);

• To assert called arguments for the exact call when a mocked function is called multiple times, an assertion can be done using mockedFunction.mock.calls[index] and call.arguments values.

const calculateMock = context.mock.fn(calculationService.calculate);
calculateMock.mock.mockImplementationOnce((a) => a + 2, 0);
calculateMock.mock.mockImplementationOnce((a) => a + 3, 1);
calculateMock(firstValue);
calculateMock(secondValue);
[firstValue, secondValue].forEach((argument, index) => {
  const call = calculateMock.mock.calls[index];
  assert.deepEqual(call.arguments, [argument]);
});

Running TypeScript tests

Add a new test script, --experimental-transform-types flag requires Node version >= 22.10.0

{
  "type": "module",
  "scripts": {
    "test": "node --test",
    "test:ts": "NODE_OPTIONS='--experimental-transform-types --disable-warning=ExperimentalWarning' node --test ./src/**/*.{spec,test}.ts"
  }
}

Demo

The demo with the mentioned examples is available here.

Async API documentation is used for documenting events in event-driven systems, like Kafka events. All of the event DTOs are stored in one place. It supports YAML and JSON formats.

It contains information about channels and components. Channels and components are defined with their messages and DTO schemas, respectively.

{
  "asyncapi": "2.6.0",
  "info": {
    "title": "Events docs",
    "version": "1.0.0"
  },
  "channels": {
    "topic_name": {
      "publish": {
        "message": {
          "schemaFormat": "application/vnd.oai.openapi;version=3.0.0",
          "payload": {
            "type": "object",
            "properties": {
              "counter": {
                "type": "number"
              }
            },
            "required": ["counter"]
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "EventDto": {
        "type": "object",
        "properties": {
          "counter": {
            "type": "number"
          }
        },
        "required": ["counter"]
      }
    }
  }
}

Autogeneration

Async API docs can be autogenerated by following multiple steps:

• define DTOs and their required and optional fields with ApiProperty and ApiPropertyOptional decorators (from the @nestjs/swagger package), respectively
• generate OpenAPI docs from the defined DTOs
• parse and reuse component schemas from generated OpenAPI documentation to build channel messages and component schemas for Async API docs

Validation

Use AsyncAPI Studio to validate the written specification.

Preview

There are multiple options

• AsyncAPI Studio

• VSCode extension asyncapi-preview, open the command palette, and run the Preview AsyncAPI command.

UI generation

• Install @asyncapi/cli and corresponding template package (e.g., @asyncapi/html-template, @asyncapi/markdown-template)
• Update package.json with scripts

{
  "scripts": {
    // ...
    "generate-docs:html": "asyncapi generate fromTemplate ./asyncapi/asyncapi.json @asyncapi/html-template --output ./docs/html",
    "generate-docs:markdown": "asyncapi generate fromTemplate ./asyncapi/asyncapi.json @asyncapi/markdown-template --output ./docs/markdown"
  }
}

Boilerplate

Here is the link to the boilerplate I use for the development.

Linting represents static code analysis based on specified rules. Please include it in the CI pipeline.

Setup

Run the following commands to generate the linter configuration using the eslint package.

npm init -y
npm init @eslint/config

Below is an example of the configuration. Some rules can be ignored or suppressed as warnings, ignore the files using ignores field.

// eslint.config.js
import globals from 'globals';
import pluginJs from '@eslint/js';
export default [
  { languageOptions: { globals: globals.node } },
  pluginJs.configs.recommended,
  {
    ignores: ['dist/**/*.js']
  },
  {
    rules: {
      'no-console': ['off'],
      'no-unused-vars': ['warn']
    }
  }
];

Linting

Configure and run the script with the npm run lint command. Some errors can be fixed automatically with the --fix option.

// package.json
{
  "scripts": {
    // ...
    "lint": "eslint .",
    "lint:fix": "npm run lint -- --fix"
  }
}

Demo

The demo with the mentioned examples is available here.

I recently migrated the Node.js app from Heroku to Fly.io, mainly due to reduced costs.

This blog post will cover the necessary steps in the migration process.

Prerequisites

• Heroku app running

• Use the exact versions for dependencies and dev dependencies in package.json so installation and build steps can pass successfully

• Use the same Node.js version in Dockerfile, package.json, and GitHub Actions workflow

• Use API gateway or custom domain for the service so web apps and mobile apps don't get affected by changing the URL of the service

Migration steps

• Migrate environment variables and secrets

• Migrate the Postgres database with the following commands (the ssl field in database configuration options is not needed)

fly secrets set HEROKU_DATABASE_URL=$(heroku config:get DATABASE_URL)
fly ssh console
apt update && apt install postgresql-client
pg_dump -Fc --no-acl --no-owner -d $HEROKU_DATABASE_URL | pg_restore --verbose --clean --no-acl --no-owner -d $DATABASE_URL
exit
fly secrets unset HEROKU_DATABASE_URL

• Migrate the Redis database if it's used

• Include the deployment step in the GitHub Actions workflow

References

• Deploying Node.js apps to Heroku

• Deploying Node.js apps to Fly.io

ChatGPT is a large language model (LLM) that understands and processes human prompts to produce helpful responses. OpenAI provides an API to interact with the ChatGPT model (gpt-3.5-turbo).

Prerequisites

• OpenAI account
• Generated API key
• Enabled billing

Integration

Below is an example of interacting with ChatGPT API based on a given prompt.

const handlePrompt = async (prompt) => {
  const response = await axios.post(
    'https://api.openai.com/v1/chat/completions',
    {
      model: 'gpt-3.5-turbo',
      messages: [
        {
          role: 'user',
          content: prompt
        }
      ]
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`
      }
    }
  );
  return response?.data?.choices?.[0]?.message?.content;
};