If services have restarted without core dump them one should look at /var/log/monit.log
Monit in the only one which can restart services. Monit would do it if a process-id (PID) it
is tracking (that of infmgr, vsmd, confd) is no longer there. So if one of these processes
does graceful exit, then monit will detect that the process is no longer running and will restart
services. Now next question will be why did that process exit if there was no core. If the system
is running out of memory, then Ubuntu will kill the process that is consuming large amount of memory.
Most likely it will be vsmd. When vsmd is killed, there wont be any cores. How do we confirm
if vsmd was killed? You can look in versa-service.log and you will likely see “killed” somewhere
near the “versa-service poststop” line. You can then look in /var/log/syslog and search for OOM or oom
(out of memory) and will likely see Ubuntu killed a process. These are two most common reasons for
service restart. Also, you can look at the alarms around the time of restart. if it was more than 90%
then likely it is due to high memory usage.
Hope this helps.
admin@shiva-vcsn-a1:~$ ps aux | grep infmgr
versa 11065 0.0 0.2 69808 18508 ? S Nov26 0:07 /opt/versa/bin/versa-infmgr -N
admin@shiva-vcsn-a1:~$ sudo kill -9 11065
admin@shiva-vcsn-a1:~$ sudo tail -f /var/log/monit.log
[PST Nov 26 13:24:00] info : 'shiva-vcsn-a1' monitor action done
[PST Nov 29 07:08:34] info : Shutting down Monit HTTP server
[PST Nov 29 07:08:34] info : Monit HTTP server stopped
[PST Nov 29 07:08:34] info : Monit daemon with pid [11327] stopped
[PST Nov 29 07:08:34] info : 'shiva-vcsn-a1' Monit 5.17.1 stopped
[PST Nov 29 07:08:37] info : Starting Monit 5.17.1 daemon with http interface at [localhost]:2812
[PST Nov 29 07:08:37] info : Monit start delay set -- pause for 30s
[PST Nov 29 07:09:07] info : Starting Monit HTTP server at [localhost]:2812
[PST Nov 29 07:09:07] info : Monit HTTP server started
[PST Nov 29 07:09:07] info : 'shiva-vcsn-a1' Monit 5.17.1 started
>>>>>>> See the below log once I kill infmgr <<<<<<<
[PST Nov 29 13:29:45] error : 'versa-infmgr' process is not running
[PST Nov 29 13:29:45] info : 'versa-infmgr' trying to restart
[PST Nov 29 13:29:45] info : 'versa-infmgr' start: /opt/versa/scripts/async_restart.sh
[PST Nov 29 13:29:46] error : 'versa-infmgr' failed to start (exit status 0) -- /opt/versa/scripts/async_restart.sh: warning: commands will be executed using /bin/sh
job 335 at Thu Nov 29 13:29:00 2018
[PST Nov 29 13:29:46] info : Shutting down Monit HTTP server
[PST Nov 29 13:29:46] info : Monit HTTP server stopped
[PST Nov 29 13:29:46] info : Monit daemon with pid [16865] stopped
[PST Nov 29 13:29:46] info : 'shiva-vcsn-a1' Monit 5.17.1 stopped