Middleware
This page relates to the middleware components deployed to application servers.
Note: Here the host name in PuTTY is likely to be acumen.tms-uk-rail.co.uk. (Rather than gateway.tms)
Click here to learn more about middleware.
Database
Central to the application is the database server; acumen utilises a database server called MariaDB - the database server runs within its own Docker Container
Checking the container status
- Start an ssh session to the relevant application server;
- At the prompt, type the following commands:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker ps | grep "STATUS\|mysql"
The program will output information about the status of the container, for example:
In the above example, we can observer that the STATUS of the container was created 12 days ago and has been 'Up' for 12 days, which indicates that there is no issue with the container.
- To see a list of all containers, simply type:
<user>@acumen-app-server:~$ docker ps
What to look for
The Created and Status timings should be the same. If the Status only shows times of a few seconds or minutes then this is an indication the container keeps falling over.
Container logging
Where an issue with the database container has been observed then the Administrator should check the logs:
- Start an ssh session to the relevant application server;
- At the prompt, type the following commands:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker-compose logs -f mysql
This action will stream the logs to the terminal; observing this output for a short time should provide some indications to any potential issues.
Press [CTRL+C] to stop the logging output and return to the prompt.
Note: It is usual to see [Warning] entries within the container logs - if any entries are classified [Error] or [Critical], then please copy any errors and add to the issue ticket.
Starting the container
If the container is Down - then the following commands will bring the container back up:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker-compose up -d mysql"
What to look for
Following a prompt of no train movement reports for example, if the message broker (RabbitMQ) is ok and mysql is ok, then check Redis.
Redis
The application server uses an in-memory key/value store called Redis - this is used to temporarily store information and data used by the application and to act as a low latency cache.
Numerous services across the application make use of Redis and it is therefore considered a critical service within the middleware group of services; without it, the application cannot run. For example, the forecast goes into Redis.
Note: Redis is a web interface and cannot be accessed with zscaler turned on.
There are numerous ways to establish the health and status of the Redis service:
Checking the container status (command line)
- Start an ssh session to the relevant application server;
- At the prompt, type the following commands:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker ps | grep "STATUS\|redis"
The program will output information about the status of the container, for example:
Checking the Redis Queues (Web Interface)
Another method to confirm the state of the Redis Container is to scrutinise the information provided using RQ Dashboard.
The application makes use of task queues within Redis and the RQ Dashboard displays the contents of each queue during processing and perhaps more importantly, maintains a list of failures to process.
RQ Dashboard is accessable on port 9181 - administrators should append ':9181' to the server ip address in the browser address bar, for example: http://77.68.118.126:9181
Note: May have to open in Internet Explorer vice Google Chrome.
If the RQ Dashboard service is not available using the access method stated above, The Administrator should start an ssh session and undertake the following at the prompt:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ make rq-dashboard"
Once access to RQ Dashboard is gained, Administrators should peruse the Failed Queue. Clicking Requeue All will requeue all the failed items. Any persisting failures should be copied into the support ticket.
Container logging
Where an issue with the database container has been observed then the Adminsitrator should check the logs:
- Start an ssh session to the relevant application server;
- At the prompt, type the following commands:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker-compose logs -f redis"
This action will stream the logs to the terminal; observing this output for a short time should provide some indications to any potential issues.
Press [CTRL+C] to stop the logging output and return to the prompt.
Starting the container
If the container is Down - then the following commands will bring the container back up:
<user>@acumen-app-server:~$ cd ~/dockerise/<user>@acumen-app-server:~$ docker-compose up -d redis"