What exactly did you do with your self-signed certificate using MCadmin to make the warning on web-console session from the browser go away?   In general, the warning will go only if your SSL certificate is a strong enough certificate bought from a reputable CA.    What did you set for the common name of your self-signed certificate and the device-management address with your MCadmin.exe utility?

Irrespective of the notification shown on the devices, did your work-managed devices get successfully enrolled?

Hi Raymond, thanks for the quick reply.

We imported our root certificate in MCAdmin, and then imported the certificate containing the computer name of the server in "Deployment Server Extensions & Web Console" (see screenshot below).

As for enrolling new devices: this still seems to work, but at a given point, I get the 'invalid certificate' warning on the device, which I would really like to get rid of...

As you are using your  private "computer name"  rather than a public FQDN in your self-signed certificate,  there is no way to validate that your server is actually the authentic server the agent is supposed to be communicating with. Hence, the warning you are concerned about is inevitable.

If your devices are for iOS and Windows 10 platform, they can't even be enrolled nor controlled any more when Apple and Microsoft has respectively tightened the certificate requirements for any enterprise-grade EMM.  I believe Google will eventually follow suit and won't allow any Android Enterprise devices to be enrolled to an MDM/EMM server without a strong third-party certificate from a reputable CA.

Update:

by installing the root certificate on the mobile device manually, the issue is gone.
But this involves some manual steps (navigating to an url containing the .cer file, installing it, & setting a device pincode/password, which seems to be required to download & install certificates)...

If there is no further downside on the warning dialog in the device agent, I think we can live with this minor annoyance...

Does it means we need to acquire a non Soti generated certificate in order to avoid it ?

Considering 2 DS and one EMM (3 servers), we need 3 different certificates ?

The usual free approach is to use the open-source OpenSSL to generate the required root and self-sgned SSL certificates.  I have used this a lot for Androd+ platforms, but haven't tried it on AE-DO devices in a closed corporate network.

For HA system with multiple Deployment and/or Managerment Servers,  you likely need only one SSL certificate if all servers are behind a load-balancer, as the system should be configured such that all devices reference any available server with one name only.   If there is no load-balancer in the system, then I believe one can choose either to generate one self-signed certificate per server, or generate a single SAN SSL certificate that includes the names of all servers involved.  OpenSSL supports both approaches.

Try Raymond's path that seems right to me. To help you discover missmatch errors you can also use SSL Labs (https://www.ssllabs.com/ssltest/) which will bring you a complete detail of the FQDN analysis.

You can also use this free tool to test the certificate on specific ports like 5494 and 5495 (https://www.digicert.com/StaticFiles/DigiCertUtil.zip)

It seems to me like a missmatch, look for the source and your problem is solved.