Deployment Server crashed and nothing to see in the Utility Manager

YW
Yannick Weijenberg
Centric Netherlands BV

Like the title says, for some reason the deployment sever crashed in the middle of the night and there was nothing to see in the Utility Manager of Soti. When seeing the logs, Im guessing that my server requirements aren't good enough. 

This is part of the MCdeplSvr.log. Opinions?

2019-01-25 00:01:44,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:45,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:46,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:47,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:48,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:49,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:50,499 (0x00001ff0) [INFO] CWinLogFile::WriteAllLog() PulseCheck enter: SendPulse=0
2019-01-25 00:01:50,499 (0x00001ff0) [ERROR] CWinLogFile::WriteAllLog() CommMgr::Pulse: Pulse timeout (index=960 ()), disconnect
2019-01-25 00:01:50,499 (0x00001fec) [ERROR] CWinLogFile::WriteAllLog() ConnThread: index=960 (): Error 1236 receiving data
2019-01-25 00:01:50,499 (0x00001fec) [INFO] CWinLogFile::WriteAllLog() Excessive queue length: 42, Threads: 233, Working: 233 (in 233). Add 1 new thread(s).
2019-01-25 00:01:50,499 (0x00001ff0) [ERROR] CWinLogFile::WriteAllLog() CommMgr::Pulse: Pulse timeout (index=988 ()), disconnect
2019-01-25 00:01:50,499 (0x00001fdc) [ERROR] CWinLogFile::WriteAllLog() ConnThread: index=988 (): Error 1236 receiving data
2019-01-25 00:01:50,499 (0x00001ff0) [INFO] CWinLogFile::WriteAllLog() PulseCheck exit: SendPulse=0, total=916, sent=0, disconnected=2, finished in 0 ms
2019-01-25 00:01:50,499 (0x00003e94) [INFO] CWinLogFile::WriteAllLog() Worker thread started
2019-01-25 00:01:50,499 (0x00003e94) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: ConnId [1023]
2019-01-25 00:01:50,499 (0x00003e94) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: Cannot find device associated with ConnId [1023]
2019-01-25 00:01:50,499 (0x00003e94) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: ConnId [960]
2019-01-25 00:01:50,499 (0x00003e94) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: Cannot find device associated with ConnId [960]
2019-01-25 00:01:50,499 (0x00003e94) [INFO] CWinLogFile::WriteAllLog() Too many working threads 234/234, shutting down thread
2019-01-25 00:01:50,499 (0x00003e94) [INFO] CWinLogFile::WriteAllLog() Worker thread terminated
2019-01-25 00:01:50,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:51,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:52,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:53,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:54,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:55,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:56,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:57,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:58,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:01:59,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:00,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:01,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:02,936 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:03,718 (0x00001fdc) [INFO] CWinLogFile::WriteAllLog() Excessive queue length: 42, Threads: 233, Working: 233 (in 233). Add 1 new thread(s).
2019-01-25 00:02:03,718 (0x00001b44) [INFO] CWinLogFile::WriteAllLog() Worker thread started
2019-01-25 00:02:03,718 (0x00001b44) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: ConnId [988]
2019-01-25 00:02:03,718 (0x00001b44) [DEBUG] ServerUtils::OnDeviceDisconnected() OnDeviceDisconnected: Cannot find device associated with ConnId [988]
2019-01-25 00:02:03,718 (0x00001b44) [INFO] CWinLogFile::WriteAllLog() Too many working threads 234/234, shutting down thread
2019-01-25 00:02:03,718 (0x00001b44) [INFO] CWinLogFile::WriteAllLog() Worker thread terminated
2019-01-25 00:02:03,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:04,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:05,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:06,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:07,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:08,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:09,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:10,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:11,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:12,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:13,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:14,937 (0x00002ab8) [DEBUG] CJobDispatcher::DoWork() Throttled jobs: 80 m_nJobs: 101 m_nJobsInQueue: 0
2019-01-25 00:02:15,187 (0x00001fdc) [INFO] CWinLogFile::WriteAllLog() New connection entry, index=1023, name=*, host=95.129.123.180, sock=5204, port=56981
2019-01-25 00:02:15,187 (0x00001fdc) [DEBUG] CCommConnectionList::AddConn() CCommConnectionList::AddConn: index=1023 seg=16 segidx=15
2019-01-25 00:02:15,187 (0x00001fdc) [INFO] CWinLogFile::WriteAllLog() Excessive queue length: 42, Threads: 233, Working: 233 (in 233). Add 1 new thread(s).
2019-01-25 00:02:15,187 (0x00003508) [INFO] CWinLogFile::WriteAllLog() Worker thread started
2019-01-25 00:02:15,187 (0x00003508) [INFO] CWinLogFile::WriteAllLog() Too many working threads 234/234, shutting down thread
2019-01-25 00:02:15,187 (0x00003508) [INFO] CWinLogFile::WriteAllLog() Worker thread terminated

6 years ago
Windows
ANSWERS
S
SSMOD@Soti
6 years ago

Hello Yannick,

Analyzing the logs it seems that queue length and working threads increased, which in turn crashed the deployment service. These logs are not enough to get the reason for high queue length. Improper resource allocation on the server can be one of the reasons for increased queue length, to start, I would suggest you go through the system requirements for MobiControl.

I have provided a link for system requirements below:

https://www.soti.net/mc/help/v14.0/en/setup/gsrindex.html?gsrid=installing/system_requirements.html.