Collecting Nodejs winston logs

If you are using winston as your logging library in your Nodejs application, you can export these logs to SigNoz very easily using various transports provided by winston.

const logger = winston.createLogger({
  level: 'debug',
  // ... other configurations
});

How to collect logs from Node.js Winston without the requiring Otel collectors

• First we will install a few dependencies using npm

  npm install @opentelemetry/api-logs @opentelemetry/sdk-logs @opentelemetry/exporter-logs-otlp-http @opentelemetry/winston-transport @opentelemetry/resources winston dotenv
  

• logger.js

  const logsAPI = require('@opentelemetry/api-logs')
  const { LoggerProvider, SimpleLogRecordProcessor } = require('@opentelemetry/sdk-logs')
  const { OTLPLogExporter } = require('@opentelemetry/exporter-logs-otlp-http')
  const { OpenTelemetryTransportV3 } = require('@opentelemetry/winston-transport')
  const { Resource } = require('@opentelemetry/resources')
  const winston = require('winston')
  require('dotenv').config()
  
  // Initialize the Logger provider
  const loggerProvider = new LoggerProvider({
    resource: new Resource({
      'service.name': 'winston-logger',
      'service.version': '1.0.0',
      'deployment.environment': process.env.NODE_ENV || 'development',
    }),
  })
  
  // Configure OTLP exporter for SigNoz
  const otlpExporter = new OTLPLogExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
    headers: {
      'signoz-access-token': process.env.SIGNOZ_ACCESS_TOKEN,
    },
  })
  
  // Add processor with the OTLP exporter
  loggerProvider.addLogRecordProcessor(new SimpleLogRecordProcessor(otlpExporter))
  
  // Set the global logger provider
  logsAPI.logs.setGlobalLoggerProvider(loggerProvider)
  
  // Create Winston logger with Console and OpenTelemetry transports
  const logger = winston.createLogger({
    level: 'debug',
    format: winston.format.combine(
      winston.format.timestamp(),
      winston.format.errors({ stack: true }),
      winston.format.metadata(),
      winston.format.json()
    ),
    defaultMeta: {
      service: 'winston-logger',
      environment: process.env.NODE_ENV || 'development',
    },
    transports: [
      new OpenTelemetryTransportV3({
        loggerProvider,
        logAttributes: {
          'service.name': 'winston-logger',
          'deployment.environment': process.env.NODE_ENV || 'development',
        },
      }),
    ],
  })
  
  module.exports = logger
  

• example index.js

  const express = require('express')
  const logger = require('./logger')
  const { v4: uuidv4 } = require('uuid')
  
  const PORT = process.env.PORT || 5555
  const app = express()
  
  app.use((req, res, next) => {
    const requestId = uuidv4()
    req.requestId = requestId
  
    logger.info({
      message: 'Incoming Request',
      method: req.method,
      path: req.path,
      requestId: requestId,
      ipAddress: req.ip,
      userAgent: req.get('User-Agent'),
    })
  
    req.startTime = Date.now()
  
    res.on('finish', () => {
      const duration = Date.now() - req.startTime
  
      logger.info({
        message: 'Request Processed',
        method: req.method,
        path: req.path,
        requestId: requestId,
        statusCode: res.statusCode,
        responseTime: `${duration}ms`,
      })
    })
  
    next()
  })
  
  app.use(express.json())
  
  app.get('/', (req, res) => {
    try {
      logger.debug({
        message: 'Accessing root endpoint',
        requestId: req.requestId,
      })
  
      res.json({
        method: req.method,
        message: 'Hello World',
        timestamp: new Date().toISOString(),
        requestId: req.requestId,
      })
    } catch (error) {
      logger.error({
        message: 'Error in root endpoint',
        error: error.message,
        stack: error.stack,
        requestId: req.requestId,
      })
      res.status(500).json({ error: 'Internal Server Error' })
    }
  })
  
  app.get('/404', (req, res) => {
    logger.warn({
      message: '404 Route Accessed',
      requestId: req.requestId,
    })
  
    res.status(404).json({
      error: 'Not Found',
      requestId: req.requestId,
    })
  })
  
  app.get('/user', (req, res) => {
    try {
      // Simulating an authentication failure
      throw new Error('Authentication Failed')
    } catch (error) {
      logger.error({
        message: 'User Authentication Error',
        error: error.message,
        stack: error.stack,
        requestId: req.requestId,
      })
  
      res.status(401).json({
        error: 'Unauthorized',
        requestId: req.requestId,
      })
    }
  })
  
  app.use((err, req, res, next) => {
    logger.error({
      message: 'Unhandled Error',
      error: err.message,
      stack: err.stack,
      requestId: req.requestId || 'unknown',
    })
  
    res.status(500).json({
      error: 'Unexpected Server Error',
      requestId: req.requestId || 'unknown',
    })
  })
  
  const server = app.listen(PORT, () => {
    logger.info({
      message: 'Server Started',
      port: PORT,
      environment: process.env.NODE_ENV || 'development',
    })
  
    console.log(`Listening on http://localhost:${PORT}`)
  })
  
  process.on('SIGTERM', () => {
    logger.warn('SIGTERM received. Shutting down gracefully.')
    server.close(() => {
      logger.info('Server closed. Process terminating.')
      process.exit(0)
    })
  })
  
  module.exports = app
  

• Now you can run your application by running node index.js

• If there are no errors your logs will be visible on SigNoz UI.

Collecting Nodejs logs when application is deployed on Docker or Kubernetes

When your application is deployed in Docker or a Kubernetes cluster, the logs from the console are automatically collected and stored in the node. The SigNoz collector will automatically collect the logs and they will be visible on the SigNoz UI.

Console Transport Configuration

You can easily add a console transport to your Winston logger with the following code:

logger.add(new winston.transports.Console(options))

The Console transport supports several configuration options:

• level: Specifies the level of messages this transport should log (default: level set on parent logger).
• silent: A boolean flag to suppress output (default is false).
• eol: Defines the end-of-line characters to use (default is os.EOL).
• stderrLevels: An array of log levels to be sent to stderr instead of stdout. For example: ['error', 'debug', 'info'] (default is an empty array).
• consoleWarnLevels: An array of log levels to use console.warn() or stderr (in Node.js) instead of stdout. For example: ['warn', 'debug'] (default is an empty array).

Example configuration:

logger.add(
  new winston.transports.Console({
    level: 'info',
    format: winston.format.simple(),
    silent: false,
    stderrLevels: ['error'],
  })
)

By using these configuration options, you can fine-tune how your logs are output and collected when deployed in containerized environments like Docker or Kubernetes.

Collecting Nodejs logs when application is deployed on a Host

When you run your application directly on the host, you will be required to add a intermediary medium ex:- a file, where you can export your logs and the otel-collector can read them and push to signoz.

You can add a file transport very easily as stated here.

logger.add(new winston.transports.File({
  filename: path.join(process.cwd(), 'logs', 'application.log'),
  level: 'debug',
  handleExceptions: true,
  maxsize: 5242880, // 5MB
  maxFiles: 5,
  tailable: true,
}));

Once you run your application and the logs are added to a file, you can configure otel collector to read from that file.

For configuring it you can follow the guide here.

Once you configure the otel collector the logs will be visible on the UI.