After trying to setup TLS, the connection fails because  of unrecoverable key exception.

SmtpPrevent_operational0.log:

[SEVERE] (SMTP_CONNECTION.5204) Peer disconnected unexpectedly (tid=30 cid=1 local=<> remote=<> reason=End of stream)[SEVERE] (SMTP_CONNECTION.5204) Peer disconnected unexpectedly (tid=33 cid=2 local=<> remote=<> reason=java.io.IOException: java.security.UnrecoverableKeyException: Cannot recover key)

In the RequestProcessor0.log:

SEVERE: RPT(33): Returning fatal response and terminating connections due to unhandled exception.
java.lang.NullPointerException
 at com.vontu.mta.rp.ESMTPPeer.recv(ESMTPPeer.java:660)
 at com.vontu.mta.rp.ESMTPRequestProcessorThread._readPeer(ESMTPRequestProcessorThread.java:743)
 at com.vontu.mta.rp.ESMTPRequestProcessorThread._process(ESMTPRequestProcessorThread.java:816)
 at com.vontu.mta.rp.ESMTPRequestProcessorThread.run(ESMTPRequestProcessorThread.java:1130)
 at java.lang.Thread.run(Thread.java:619)

This problem can happen if the keypair password and the keystore password are not the same.

When the command to create the keypair is run, for example:

keytool -genkeypair -alias < SMTPPreventHostName > -dname "CN=<SMTPPreventHostName>, OU=Corporate, O=<CompanyName>, L=<location>, S=<State>, C=<country>" -keyalg RSA -validity <NumberOfDays> -keysize 2048 -keystore C:\ProgramData\Symantec\DataLossPrevention\DetectionServer\<version>\Protect\keystore\prevent.ks

The user will be asked to enter a password for the keypair and the keystore.

Make sure keystore and keypair passwords are same.

The keystore password will be configured in the management console.