I'm testing TLS connections on SQL Anywhere 16.0.0.2158 and am getting a TLS handshake failed, error code 20 error message. I've read the following documents with no help:
http://dcx.sap.com/index.html#sa160/en/dbadmin/tls-j22-s-5729723.html
and
http://dcx.sap.com/index.html#sa160/en/dbadmin/gencert-ml-ref1.html
and
http://dcx.sap.com/index.html#sa160/en/dbadmin/ml-tls-s-6232604.html
here is the client log
Fri Sep 25 2015 14:52:03
14:52:03 Attempting to connect using:
UID=ficsro;PWD=********;DBN=fics;ServerName=billytest;CON=SQL_DBC_4c382e5200;ENC='TLS(tls_type=rsa;fips=n;trusted_certificate=C:\ssl\public.pem)';LOG=c:\ssl\ssl.log;LINKS='tcpip(HOST=web1)';CPOOL=NO
14:52:03 Attempting to connect to a running server...
14:52:03 Attempting TCPIP connection (address 192.168.4.112:2638 found in sasrv.ini cache)
14:52:03 Looking for server with name billytest
14:52:03 Trying to find server at cached address 192.168.4.112:2638 without broadcasting
14:52:03 Found database server billytest on TCPIP link
14:52:03 Connected using client address 192.168.5.150:52913
14:52:03 Connected to server over TCPIP
14:52:03 Connected to SQL Anywhere Server version 16.0.0.2158
14:52:03 Application information:
14:52:03 IP=192.168.5.150;HOST=staging11;OSUSER=estatuswebsvc;OS='Windows 2012R2 Build 9200 ';EXE=C:\ColdFusion11\estatuswebsvc\bin\coldfusion.exe;PID=0xab4;THREAD=0xaec;VERSION=16.0.0.2158;API=iAnywhereJDBC;TIMEZONEADJUSTMENT=-300
14:52:03 Connected to the server, attempting to connect to a running database...
14:52:03 The TLS handshake failed, error code 20
14:52:03 Communication function SQLPresSyncPoint code 8
14:52:03 unknown error 0
14:52:03 Client disconnected
14:52:03 Disconnected from server
here are the server startup options which startup just fine:
-c 128M
-ec none,simple,TLS(identity=c:\db\identity.pem;identity_password=fics)
-n billytest
-x tcpip
c:\db\fics.db
here is the test certificate info:
C:\Program Files\SQL Anywhere 16\Bin64>createcert -t rsa
SQL Anywhere X.509 Certificate Generator Version 16.0.0.2158
Warning: The certificate will not be compatible with older versions
of the software including version 12.0.1 prior to build 3994 and version 16.0
prior to build 1691. Use the -3des switch if you require compatibility.
Enter RSA key length (512-16384): 2048
Generating key pair...
Country Code: US
State/Province: TX
Locality: ADDISON
Organization: FICS,INC
Organizational Unit: FICS
Common Name: web1
Enter file path of signer's certificate:
Certificate will be a self-signed root
Serial number [generate GUID]:
Generated serial number: f3dde00072d04f319b17cd429769b75e
Certificate valid for how many years (1-100): 99
Certificate Authority (Y/N) [N]:
1. Digital Signature
2. Nonrepudiation
3. Key Encipherment
4. Data Encipherment
5. Key Agreement
6. Certificate Signing
7. CRL Signing
8. Encipher Only
9. Decipher Only
Key Usage [1,3,4,5]: 3,4,5
Enter file path to save certificate: c:\db\public.pem
Enter file path to save private key: c:\db\private.pem
Enter password to protect private key: fics
Enter file path to save identity: c:\db\identity.pem