Gateway server

This page provides details on the tools and interfaces provided to monitor and diagnose issues with the Gateway Server

Glances / top

These tools are provided to visually display the performance and load of the operating system and to view running services. Administrators should be cognisant of:

• High CPU load and any associated warnings;
• High memory usage and any associated warnings - including swap file usage;
• High file system usage;
• Running processes, or processes not running that should be (note: a script is provided for this purpose).

Glances

Click here to learn more about Glances.

There are 2 ways to access Glances on the Gateway Server

• Method 1: Access via the Web Interface, or
• Method 2: Start an ssh session and at the prompt: <user>@prod_tms_server:~$ glances.

top

Click here to learn more about top.

To access top, start an ssh session and at the prompt: <user>@prod_tms_server:~$ top.

Application Logs

The Gateway Server provides comprehensive logs for all running services. The logging level is set to provide debug and error information and administrators therefore can expect that in some cases, the logs are quite large despite log rotation in operation.

RabbitMQ Interface

Without doubt, the RabbitMQ broker is the main component of the Gateway Server.

An interface is provided for users to monitor:

• Overview of all queued messages and message rates;
• Active connections including network message rate;
• Active channels and there connected services, including message rates;
• All live exchanges, including inbound/outbound message rates;
• Active message queues.

Note: The value of using the RabbitMQ interface to diagnose issues cannot be underestimated:

Where for example, a user has reported that the forecast service is not updating or the TD interface is not updating, RabbitMQ should always be the first port of call; It will be able to demonstrate if the problem is with the external services or within acumen by simply checking if the messages are being received and being consumed by acumen application servers.

Acumen Service Management Script

For convenience, a script is provided that guides administrators to the status of vital services on the Gateway Server. To access the script, start an ssh session and at the prompt:

<user>@prod_tms_server:~$ cd /var/www/server_tasks

<user>@prod_tms_server:/var/www/server_tasks$ ./service_script.sh

Administrators should observe the following:

Administrators should note that all referenced services are configured as system services and are setup to restart on failure therefore it would be unusual to observe any particular service down however, An administrator should attempt to start a service and consult the relevant logs if a service is observed to be down.

Also, selecting X against each service entry shows the systemd logs for the relevant service; this would be useful to diagnose any issues that the operating system has had in starting and running the service.

MQIPT

This is the component of the gateway Server that is responsible for maintaining the connection to TD.net, you can learn more about what MQIPT is by clicking here.

Where an Administrator has identified that there is an issue with the TD.net inbound feeds, then an attempt should be made to diagnose the issue and re-start the MQIPT service.