There was an openssl update recently that caused people who were using alpine to not be able to send mail. When they tried to send, they got this message:

Trying to connect to mailserver to send in alpine, get this message in alpine

There was an SSL/TLS failure for the server

                                              mailserver.example.com

The reason for the failure was

                                           SSL negotiation failed

This is just an informational message. With the current setup, SSL/TLS will not work. If this error re-occurs
every time you run Alpine, your current setup is not compatible with the configuration of your mail server.
You may want to add the option

                                                   /notls

to the name of the mail server you are attempting to access. In other words, wherever you see the characters

                                              mailserver.example.com

in your configuration, replace those characters with

                                           mailserver.example.com/notls

Type RETURN to continue.

Turning off tls is not an option.

After reading the openssl update info, I found that one of the changes was that the software was requiring the minimum Diffie-Hellman key size be 768 bits. And that in the future, it would be 1024 bits. So if I could just figure out how to increase the key size, I thought I’d be all set.

One issue I have is that our current mailserver is running the older RHEL5 system. It’s up-to-date with patches, so I’m not worried about the security of the system. But I wasn’t sure that the version of openssl would even work with systems having the newer version of openssl.

I played around with an RHEL7 system, thinking that this version should have everything set by default. However, this was not the case. I got the same error when trying to send mail through a server running RHEL7. In looking at the logs, I saw these lines:

Jun 16 10:51:04 new sendmail[3301]: STARTTLS=client, error: connect failed=-1, reason=dh key too small, SSL_error=1, errno=0, retry=-1
Jun 16 10:51:04 new sendmail[3301]: ruleset=tls_server, arg1=SOFTWARE, relay=mailserver.example.com, reject=403 4.7.0 TLS handshake failed.
Jun 16 10:51:04 new sendmail[3301]: t5GFoxp8003299: to=<user@example.com>, delay=00:00:05, xdelay=00:00:05, mailer=esmtp, pri=120325, relay=example.com. [192.168.1.101], dsn=4.0.0, stat=Deferred: 403 4.7.0 TLS handshake failed.

The “reason=dh key too small” confirmed that I needed to increase the key size. Having no idea how to do this, I googled around a bit and found the fix.

[~]# cd /etc/pki/tls/certs
[certs]# openssl dhparam -out dh_params.pem 2048
(This command takes a few minutes to run.)

Now I just need to tell sendmail to use those Diffie-Hellman options. I added the following to my /etc/mail/sendmail.mc file.

dnl # Use DH parameters with 2048 bit key
define(`confDH_PARAMETERS',`/etc/pki/tls/certs/dh_params.pem')

Then ran:

[mail]# make -C /etc/mail
make: Entering directory `/etc/mail'
make: Leaving directory `/etc/mail'
[mail]# /etc/rc.d/init.d/sendmail restart
Shutting down sm-client:                                   [  OK  ]
Shutting down sendmail:                                    [  OK  ]
Starting sendmail:                                         [  OK  ]
Starting sm-client:                                        [  OK  ]

Now, using alpine to remotely read mail worked again. And I could stay up-to-date with openssl.