Troubleshoot issues with a specific web site proxied by Edge SWG (ProxySG) or ASG appliance

Products

Advanced Secure Gateway Software - ASG ProxySG Software - SGOS ASG-S200 ASG-S400 ASG-S500 ISG Proxy SG-300 SG-S200 SG-S200-RP SG-S400 SG-S400-RP SG-S500 SG-S500-RP SG-VA SGVA

Issue/Introduction

Caution: The content of this KB article is meant to help troubleshoot URL issues, but is by no means to be considered as a fix or a permanent solution unless Support recommends it in very specific cases.

How to use: The CPL below applies bypasses for most of the proxy functions. If the CPL script(s) succeeds in having the site or program working again, the script should be modified to narrow down the failure to a single component. The script can then be modified or a VPM rule can be created to allow the wanted traffic to pass.

Example: we apply the script and remove the bypasses until we learn our problematic URL fails to load because of SSL Interception, then we only apply the SSL exemption to that URL. In other situations, it may be necessary to disable more than one feature.

Once the Magic Script is applied, if it fixes the problem, start removing the bypass for the different features until the site or service breaks again. At that point, add that piece of code back to the script. This will allow only the proxy functions that break the URL or service will be bypassed.

Cause

Website or service times-out, does not function as expected or works intermittently when going through the Proxy.
Media does not respond or is slow.
The proxy's cache is serving old objects.
Proxy functions (such as Authentication, SSL Interception, Detect Protocol, ICAP, etc.) result in unexpected behavior with websites.

Resolution

This technique disables specific appliance features and should be considered only if the data does not clearly determine the cause of the problem. Symantec strongly recommends following these guidelines:

Limit the source IP address to a testing workstation. Make sure the problem can be reproduced from this workstation prior to using the CPL below.
Limit the destination of ALL rules to the problematic URL.
If the complete CPL helps to work around the issue, the different bypass-rules it contains should then be disabled (commented out) one at a time so that only the necessary bypass-rule(s) are left in place. Leaving too many bypasses active might turn off the functionality required by other rules in the policy or represent an unnecessary lost of security over that content.

When troubleshooting site slowness, the key is to find out what specific piece of the proxy's services might be causing issues with the specific site. To assist in troubleshooting, refer to the following sample CPL that can be used to disable many features and services. (If you are unsure of how to add CPL to Edge SWG policy, see Add CPL to a local policy file on the Edge SWG).

Support might require policy traces and packet captures (PCAPs) to help determine the root cause of a problem.

Note: Please be sure the formatting is the same as shown below before installing the below CPL. As an example, if there are any line breaks in the rules that occur in the copy/paste process, be sure to correct them or the policy will not work as intended.

First script, disable Proxy functions for a client workstation:

; ################# BEGIN Magic Script - Test to Disable Proxy Features Based on IP #################
; Disclaimer: Some of these options will disable authentication and virus scanning.
; Make sure this is only applies to one machine to minimize the risk.
; This script disables a variety of internal proxy features.
; To disable features on an exception basis, simply add a ";" symbol before each feature

<proxy>
Condition=TestIP trace.destination("Troubleshooting_trace") trace.request(yes)

<proxy>
Condition=TestIP authenticate(no) ALLOW

<proxy>
Condition=TestIP detect_protocol(none)

<proxy>
Condition=TestIP http.client.persistence(no) http.server.persistence(no) bypass_cache(yes) http.request.version(1.0) http.response.version(1.0) server_url.dns_lookup (ipv4-only)

<cache>
Condition=TestIP request.icap_service(no) response.icap_service(no)

<cache>
Condition=TestIP pipeline(no) cache(no)

<SSL-Intercept>
condition=TestIP ssl.forward_proxy(no)

<SSL>
condition=TestIP client.certificate.validate(no) server.certificate.validate(no)

define condition TestIP
; Put the testing machine IP instead of "ip.address.of.test_workstation"
client.address=ip.address.of.test_workstation
end

; ############### END Magic Script - Test to Disable Proxy Features Based on IP ###############

Second script, to disable proxy functions for specific domains (not clients):

;::::::::::::::::::: BEGIN Bypass Proxy Based on URL :::::::::::::::::::
; Disclaimer: Some of these options will disable authentication and virus scanning.
; Make sure this only applies to the intended URLs to minimize the risk.
; This script disables a variety of internal proxy features.
; To disable features on an exception basis, simply add a ";" symbol before each feature
 
<proxy>
condition=ByPassProxyByURL authenticate(no) ALLOW
 
<proxy>
condition=ByPassProxyByURL detect_protocol(none)

<proxy>
condition=ByPassProxyByURL http.client.persistence(no) http.server.persistence(no) bypass_cache(yes) http.request.version(1.0) http.response.version(1.0) server_url.dns_lookup(ipv4-only)

<cache>
condition=ByPassProxyByURL request.icap_service(no) response.icap_service(no)

<cache>
condition=ByPassProxyByURL pipeline(no) cache(no)

<ssl-intercept>
condition=ByPassProxyByURL ssl.forward_proxy(no)
 
<SSL>
condition=ByPassProxyByURL server.certificate.validate(no)
 
define condition ByPassProxyByURL
; Put the domain you need to test with instead of "example.com". All subdomains of this domain will be affected by this script unless you specify it. You can add more lines like the one below in case you need to disable these features for different domains.
    url.domain="example.com"
end condition ByPassProxyByURL

;::::::::::::::::::: END Bypass Proxy Based on URL :::::::::::::::::::

If a Full Bypass is needed in order to discard the Proxy as the cause of the problem, please refer to the articles:

How to bypass in a Transparent environment

How to bypass in an Explicit environment

Additional Information

Frequently Asked Questions

Q1: Will this affect the corporate infrastructure?
A1: No. This will only affect the traffic for the specified test machine/s or URL/s.. Global options are commented out by default.

Q2: Will this stop authentication and virus scanning?
A2: Yes. This has a security implication but it's meant to be a temporary test that only affects one computer or URL. Our intention is for the final solution to be the most restrictive as possible, using these scripts as a starting point.

Q3: Can I do this on production environment?
A3: Yes. It does not affect the overall performance of the proxy and does not lock admin accounts.

Q4: What exactly will this CPL do?
A4: This is a temporary test to isolate the problem. It disables features that could affect the flow of traffic like authentication, http persistence, pipelining, etc. For details, refer to the CPL in the appropriate Content Policy Language Reference.

Q5: How will this solve the problem experienced?
A5: The objective of this test is to isolate the problem. It may or may not represent a fix, but it will help you in defining the cause.

Q6: How can I be sure the CPL is effective?
A6: If the results are not immediately noticed, take a Policy Trace and make sure the CPL is overriding all other VPM/policy decisions.

Q7: How can I identify the exact feature causing the problem and apply an exemption only to that?
A7: If the sample CPL code solves the issue, start re-enabling some of the disabled features, by using a semi-colon in front of the line that call the function.

Enabled
<proxy>
Condition=TestIP authenticate(no) ALLOW
Disabled
;<proxy>
;Condition=TestIP authenticate(no) ALLOW