This article offers a way in which customers of the Email Security .cloud Service (ESS) can implement an alternative to the Anti-Spam service Newsletter Detection feature by using Data Protection, where they can configure exceptions and filters based on the header and body contents of the emails.

The solution is designed for situations where the white listing capability of the Anti-Spam service (addresses, domains and IPs/IP Ranges) cannot fully cover all of the particular newsletter type email filtering requirements.

NOTE: The Data Protection policy based solution does not fully replace the advanced newsletter detection algorithms that are found within the Anti-Spam service Newsletter Detection feature. Tweaking and monitoring are required to achieve optimal results. 

Data Protection service available

1. Create Data Protection policy

• Access Services > Data Protection
• Create a New Policy
  • Give it a descriptive Name: Newsletter Detection
  • Apply to: Inbound email only
  • Execute if: ALL rules are met
  • Action: To be decided according to company policy and preference
  • Click Edit next to the Notification option, check the box Use custom notification and disable all notifications, click Edit
   
• Add a new Rule - This rule will contain the conditions which will verify whether an email is a newsletter
  • ​Rename it from Rule 1 to Detection Conditions
  • Execute if: ANY conditions are met
  • Add a new condition: Content Regular Expression List - This condition uses regular expressions to identify newsletter type information in the email headers
    • Click on Create a new Regular Expression List
    • Name the list: Newsletter RegEx
    • Add the following lines to the list using copy/paste into the Add list items section, click Add and then click Save
      ^List-u?n?subscribe:
[^<]+<newsletters?@
      
      You can add additional entries to cover more languages or variants
       
    • Condition options:
      • Email contains: a number of matches for the regexes in the selected lists
      • At least: 1
      • Count only unique matches: No
      • Case sensitive: No
      • Look in: Header​​​​​​​
       
  • Add a new condition: Content Keyword List - This condition uses key words and phrases to search the email body for newsletter specific content
    • Click on Create a new Keyword List 
    • Name the list: Newsletter Keyword
    • Add the following lines to the list using copy/paste into the Add list items section, click Add and then click Save
      to unsubscribe
      
      Again, you can add additional entries to cover more languages or variants
       
    • Condition options:
      • Email contains: a number of matches for the keywords in the selected lists
      • At least: 1
      • Count only unique matches: No
      • Case sensitive: No
      • Look in: Body​​​​​​​, Subject
       
• Add a new Rule - This rule will contain the exceptions to this policy, based on sending domain or email body content
  • ​Rename it from Rule 2 to Exception Conditions
  • Execute if: ALL conditions are met
  • Add a new condition: Content Keyword List - This condition uses key words and phrases to search the email body and subject for content specific to the newsletter type emails that you want your users to receive
    • Click on Create a new Keyword List 
    • Name the list: Newsletter Keywords
    • Add key words or phrases to the Add list items section that are significantly unique to the newsletters that you want to be excluded from this policy's action; after every entry click Add and when done click Save
    • Condition options:
      • Email contains: none of the keywords in the selected lists
      • At least: 1
      • Count only unique matches: No
      • Case sensitive: No
      • Look in: Body​​​​​​​, Subject
       
  • Add a new condition: Sender Domain List - This condition uses key words and phrases to search the email body and subject for content specific to the newsletter type emails that you want your users to receive
    • Click on Create a new Domain List  
    • Name the list: Newsletter Domains
    • Add to the Add list items section the domain names that are found in the Envelope From field of the newsletter type emails that you want your users to receive; we recommend that you add the domains found in the Anti-Spam Approved Senders list for the purpose of allowing newsletters and after remove them from there; after every entry click Add and when done click Save
    • Condition options:
      • Domain of the sender: is in none of the selected lists
   
• Scroll to the bottom of the policy configuration page and click on Save

To create an exclusion condition with emails, we first need to create a custom group

• Access Users and Groups > User Groups
  • Click on Create new group
  • Enter group name: Newsletter Addresses
  • Type the third party addresses that you want to receive newsletters from in the New users field and then click Add >>; we recommend that you add the addresses found in the Anti-Spam Approved Senders list for the purpose of allowing newsletters and after remove them from there
  • Click on Save and exit
     
• Access Services > Data Protection and open the Newsletter Detection policy
  • Scroll to the bottom of the configuration page and add a new condition: Sender Group
    • Click on Browse for a Group, select the Newsletter Addresses group and click on Add
    • Condition options:
      • Email sender: is in none of the selected groups
  • Click on Save
• Verify that all of the addresses and domains that used to be configured in the Approved Senders and are relevant to newsletter approvals are listed. Find the policy in the policy list (automatically created at the end) and click on Activate next to it.

2. Disable Newsletter Detection

Access Services > Anti-Spam and uncheck the option next to Use Newsletter / Marketing detection.

3. Monitor and tweak the policy

To achieve the best results you will need to periodically add exceptions and detection criteria to the policy configuration. There is room for improvement and refinement in all of the conditions of this policy, according to your preferences and requirements. Below are clarifications regarding how to work with the conditions.

• Rule 1 - Detection Conditions - Will match if any of its conditions match
  • Condition 1 - Regular Expression Lists
    • Should you find some unwanted newsletters are not caught and have common patterns in their headers, a regular expression that matches that content can be added to the list here in order to start detecting them
  • Condition 2 - Keyword Lists
    • Should you find some unwanted newsletters are not caught and have common patterns in their bodies, a new keyword or short phrase that should preferably clearly identify them can be added to the list here in order to start detecting them
   
• Rule 2 - Exception Conditions - Will match if all of its conditions match
  All conditions configured here are negative lookups, meaning that they will match if none of the parameters configured within the filters is matched. If any of the criteria within them matches, one of the conditions will return negative and as a result the rule will return negative.
  • Condition 1 - Keyword Lists
    • This condition looks for specific keywords or phrases within the body or subject of the emails which should preferably clearly identify emails that you want your users to receive
  • Condition 2 - Sender Domain Lists
    • This condition verifies if the Envelope Sender domain matches the contents of the list, allowing the emails through if matched
  • Condition 3 - Sender Groups
    • This condition verifies if the Envelope Sender address matches the contents of the list, allowing the emails through if matched