Machines with a managed SEP client installed randomly disconnect from, and reconnect to, the Symantec Endpoint Protection Manager (SEPM) for no apparent reason.
The SEPM has been installed to a Windows Server 2008 machine. Communication between the manager and the clients has been established. After a short period of time, it becomes noticable that clients are disconnecting and reconnecting to the manager (the green dot on the shield icon in the system tray disappears and reappears). In the SCM-SERVER-0.LOG file, an error similar to the following one appears:
2009-10-09 09:32:05.078 SEVERE: Unknown Exception in: com.sygate.scm.server.task.ClientTransportInfoTask java.lang.Exception: HTTP 503 Service Unavailable, URL: http://localhost/secars/secars.dll?action=36 at com.sygate.scm.common.communicate.Communicator.sendRequest(Communicator.java:303) at com.sygate.scm.server.task.ClientTransportInfoTask.run(ClientTransportInfoTask.java:116) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462) com.sygate.scm.common.communicate.CommunicationException: Unexpected server error. ErrorCode: 0x10010000 at com.sygate.scm.common.communicate.Communicator.sendRequest(Communicator.java:324) at com.sygate.scm.server.task.ClientTransportInfoTask.run(ClientTransportInfoTask.java:116) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462) 2009-10-09 09:32:08.683 SEVERE: IISCacheTask connect to secars failed: Service Unavailable
In the IIS logs, you may find an HTTP code similar to the following one:
Smc 503 2 0 1
In the SEPM, errors with the following verbage may appear:
Variables that may contribute to the situation:
SEPM is installed to a Windows Server 2008 machine, which is running IIS 7
There are a large number of machines in the environment; 5000+ machines
SEPM is configured for the PUSH communication mode
The HTTP code 503.2 is specific to IIS 7. The cause for this code is "concurrent request limit exceeded." According to Microsoft, "The appConcurrentRequestLimit property is set to a value that is lower than the current number of concurrent requests. IIS 7.0 does not allow more concurrent requests than the value of the appConcurrentRequestLimit property."
To properly troubleshoot this problem, you will need to have logging enabled in IIS 7 for the website that the SEPM components are installed to. For SEPM installations with MR3 and newer, this website is Symantec Web Server.
Follow these instructions to enable logging in IIS
How to turn on Logging in IIS 7:
Open up the Server Manager
Type Server Manager in the search bar
Click Server Manager under Programs at the top-left of the start menu
In the left-pane expand Roles
Then expand Web Server (IIS)
Highlight Internet Information Services (IIS) Manager
In the next pane to your right under the Connections header expand the server name
Expand SymantecWeb Server
Select the first virtual directory under the web site.
In the right-pane under the IIS heading double-click Logging
Be sure that logging is set to daily and take note of the directory it is set to.
On the right-hand side, under Actions, click Enable to start logging.
Enable logging for the remaining virtual directories
If everything is grayed out here you will need to install the logging option of IIS.
To install the logging feature for IIS do the following:
In the far left-pane, under Roles, click on Web Server (IIS)
In the right-pane, next to the heading Role Services, click Add Role Services
Under Health and Diagnostics put a check next to HTTP Logging
Click the Install button
NOTE: A reboot is required before changes will take effect
Logs are located by default in the \inetpub\logs\LogFiles directory
With IIS logging enabled, open the LOG file with today's "modified" date. Look for a code similar to this one: Smc 503 2 0 1. If you are receiving a 503.2 code, then the server is hitting the "concurrent request limit" threshold.
- - - - - -
To resolve this problem, you will need to increase the value of the appConcurrentRequestLimit property. The default value is 5000.
This property is controlled by the <serverRuntime> element. There is no User Interface (UI) for this element. Any changes must be performed using Command Prompt. Speak with your system administrator, contact Microsoft, or refer to the documents in the References section of this article for information and instructions on how to change the value of the appConcurrentRequestLimit property.
- - - - - -
An alternative to changing the appConcurrentRequestLimit value would be to change the client communication mode from push to pull.
Difference Between "Push" and "Pull" Communication Modes:
Clients that use Push Mode download policies and content as soon as they become available. With Push Mode, a connection is kept open so that the manager can contact the client immediately when data is available. Using this mode requires greater network bandwidth, but ensures that clients receive policy and content updates immediately.
Clients that use Pull Mode download policies and content based on the Heartbeat Interval setting, which is set to 5 minutes by default. In Pull Mode, a connection is established at every heartbeat and terminated as soon as the required communication is over. This mode uses less network bandwidth, but clients do not receive policy and content updates until they connect to the manager to retrieve the data.