Debugging on production #

Logs #

Node.js process #

The output of the dist-server's process is written to files by PM2. Usually, these should be in ~/.pm2/logs/dist-server-out.log (normal) and ~/.pm2/logs/dist-server-error.log (errors).

Tip: add an alias in ~/.bash_aliases to easily tail the dist-server logs:

alias taildslog="tail -f -n 30 ~/.pm2/logs/dist-server-out.log"
alias taildserr="tail -f -n 30 ~/.pm2/logs/dist-server-error.log"

You can also use pm2 log to view the logs realtime, or if you set up the web-based dashboard at https://app.pm2.io, you can view the realtime logs online, view memory usage, or even reset the app, among other things.

If the Node.js process is stuck, you should be able to restart it using pm2 reload dist-server or pm2 restart dist-server. Otherwise, find the process' pid file at ~/.pm2/pids. cat the pid file corresponding with dist-server, then then kill the process using kill -SIGKILL <pid>, or -SIGTERM to try a soft kill first.

service wrapper #

If the pm2-<user> systemd service fails, try journalctl -u pm2-<user>.service or journalctl -xe. Error messages may also have been logged to ~/.pm2/logs/dist-server-error.log.

Slack #

Useful notifications are logged to Slack. You need to invite the Slack app you've created to the channel you want to log into, otherwise Slack logging will fail.

Dashboard #

If the dashboard is enabled, it'll display the history, current builds and queued builds.