Administration: Difference between revisions

From acumen Wiki
Jump to navigation Jump to search
Line 93: Line 93:
:- Check Glances to see how the system is working.
:- Check Glances to see how the system is working.
:- Check the Application logs to see if there are any errors.
:- Check the Application logs to see if there are any errors.
:- Use the RabbitMQ Interface to see if the problem is with the external services or within acumen.


{| class="wikitable"
{| class="wikitable"

Revision as of 13:24, 30 July 2020

This section of the acumen wiki is written for the express purpose of providing information and guidance to acumen administrators.

Simplified Architecture Overview

Acumen architecture.png

The application is deployed to 3 servers, as follows:

Host Name IP Address Containerised Deployment? Purpose
gateway.tms-uk-rail.co.uk 77.68.28.115 No Agregate all inbound message feeds and serve via RabbitMQ Broker to application servers.
acumen.tms-uk-rail.co.uk 77.68.118.126 Yes Serve the acumen application (main server)
acumen-training.tms-uk-rail.co.uk 77.68.3.32 Yes Serve the acumen application (training server)

All servers are hired VPS - Virtual Private Server, with the sevice provider being IONOS by 1&1 under several contracts. All servers are located within the UK at a data centre in London.

Gateway Server

The functionality of this component is to aggregate various data feeds and make any resulting messages available to be consumed by one or more application servers.

There are 2 distinct feeds that the gateway server connects to:

  • Network Rail TD.net feed;
  • National Rail Enquiries Darwin Feed.

TD.net feed uses MQIPT protocols for message publication and consumption, whilst NRE Darwin subscription utilises a protocol known as STOMP; The gateway server manages the connections to these services and publishes all messages to a message broker, for consumption by the application server(s).

RabbitMQ was chosen as the message broker component within the gateway - all message orientated services on the application servers connect to this broker.

- The interface is here. Note: It will not open with zscaler.
- More information can be found here Gateway server#RabbitMQ Interface.

There are several tools and interfaces provided to administrators for the purposes of monitoring and diagnosing issues with the gateway server.

Application Server

Each application server runs a single instance of the acumen application which consists of numerous containerised services which make up the application - acumen utilises a microsystem architecture where each service is responsible for defined or limited tasks and as a whole, make up the application.

Services running on the application server can conceptionaly be grouped thus:

  • Middleware - those services which are needed by the application to function;
  • UI services - services which are responsible for creating and managing front end (UI rendering and user interaction) functionality;
  • CIF Processing - the service which takes the POINTA CIF from PPTE and maintains the TSDB (Train Service Database);
  • Forecasting - services which maintain the train service forecast, including conflicts.

Glances / top

These tools are provided to visually display the performance and load of the operating system and to view running services. Administrators should be cognisant of:

  • High CPU load and any associated warnings;
  • High memory usage and any associated warnings - including swap file usage;
  • High file system usage;
  • Running processes, or processes not running that should be (note: a script is provided for this purpose).

Glances

Click here to learn more about Glances.

Start an ssh session and at the prompt: <user>@prod_tms_server:~$ glances.

top

Click here to learn more about top.

To access top, start an ssh session and at the prompt: <user>@prod_tms_server:~$ top.

User Administration

The User Management section provides guidance and instructions for Administrators and Superusers in managing the users of acumen.

Diagnosing Issues

Under development

The gateway server should be the first port of call is something goes wrong.

- Check Glances to see how the system is working.
- Check the Application logs to see if there are any errors.
- Use the RabbitMQ Interface to see if the problem is with the external services or within acumen.
Prompt Initial actions
Missing movement reports Check Gateway
No train movement reports docker ps falling over
Many services showing overdue Check RabbitMQ TRUST feeds
502 Bad Gateway error message UI services
Wrong schedule (hasn't had STP overlay) CIF Processing

Reporting Issues

Should a fault be diagnosed with an external service, the following table provides information on who the Administrator should contact:

Component / Service Organisation primary contact for first response
TD.net Network Rail 4C's helpdesk Calls can be logged by dialling 085 51600 (internal) or 01270 721600 (external).
PPTE (CIF) Network Rail 4C's helpdesk Calls can be logged by dialling 085 51600 (internal) or 01270 721600 (external).
Darwin Feed National Rail Enquiries dsg_darwind3.support@caci.co.uk

All Administration Web Pages

Administration wiki pages

  • Gateway server - including Glances, Top, application logs, RabbitMQ, acumen service management scripts (are services UP or DOWN), MQIPT

Other helpful wiki pages