Large number of Symantec Critical System Protection (SCSP) agents go off offline. This includes both Windows and Unix agents. This issue occurs on older and newer versions of the SCSP agents.
A red circle icon appears in the management console GUI and the agent is not reporting events to the management server.
25 2011-06-22 13:29:35.301 [ERROR] [FetchHandler:23] [SQL:1105] Could not allocate space for object 'dbo.CWT'.'CWT_PrimaryKey' in database 'tempdb' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.
java.sql.SQLException: Could not allocate space for object 'dbo.CWT'.'CWT_PrimaryKey' in database 'tempdb' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.
It appears that the issue is related to tempdb in the database being full. The logs show that tempdb in SQL Server is full and is running out of disk space. Because of it being full no new data can be inserted or updated. This is causing the agents to go offline and not allowing the agent to report events to the manager.
Steps to backup the database and shrink the tempdb; You will find this under System Databases in SQL Server Management Studio.
First back up the database (steps 1-5)
1. Use ‘sa’ username to login to SQL Server Database using SQL Server Management Studio.
2. Expand Databases, and select tempdb database.
3. Right click the database and select Tasks->Backup
4. Select Backup Type as Full
5. Click on Add and select a backup destination on an external disk.
• This will back up the entire database and might take a while to execute.
• Please ensure that you have sufficient amount of disk space on external drive.
• Press OK.
How to shrink the transaction log (steps 6-13)
6. Right-click the instance, and then click Properties, which opens the Database Properties dialog box.
7. In the Select a Page pane, click Options.
8. The current recovery model is displayed in the Recovery model list box.
9. Change Recovery model from Full to Simple. Press OK
10. Right-click the tempdb database, and then select ->Tasks->Shrink->Files
11. Change File type from Data to Log. Ensure that
• Filename is changed to tempdblog
• File location ends with tempdbBlog.ldf
12. Set Shrink action to Release Unused Space. Press OK
• The above steps should shrink the tempdb.
• Verify the transaction log file size on disk.
13. Repeat steps 6 to 9 and set recovery model back from Simple to Full.
• SCSP 126.96.36.199
• SCSP 188.8.131.52
• Windows(R) Server 2003 Standard x64 Edition
• Total Physical Memory: 2,047 MB
• Failover enabled