TLS/SSL Configuration for Servers
TLS (Transport Layer Security) and its predecessor SSL (Secure Sockets Layer) define the cryptographic protocols that protect data in transit between servers and clients. This reference covers the technical structure of TLS/SSL configuration, the protocol versions and cipher suite classifications in active use, the regulatory frameworks governing minimum configuration requirements, and the decision boundaries that separate compliant from non-compliant deployments. It serves infrastructure engineers, compliance officers, and security professionals responsible for server certificate management and encrypted transport policy.
Definition and scope
TLS/SSL configuration for servers is the set of technical controls that determine which protocol versions, cipher suites, certificate authorities, and key exchange mechanisms a server will accept or reject during encrypted session negotiation. The scope spans web servers, mail servers, database connection endpoints, API gateways, and any network-facing service that handles sensitive data over TCP.
SSL is formally deprecated. NIST SP 800-52 Revision 2, published by the National Institute of Standards and Technology, prohibits SSL 2.0, SSL 3.0, and TLS 1.0 in federal information systems, and strongly recommends against TLS 1.1. The current baseline for compliant deployments is TLS 1.2, with TLS 1.3 required or recommended by a growing body of federal and industry standards. The Payment Card Industry Data Security Standard (PCI DSS), governed by the PCI Security Standards Council, has required retirement of TLS 1.0 since June 2018 and treats TLS 1.2 as the minimum acceptable version.
The configuration landscape on any given server is described by four categories of parameter: protocol version support, cipher suite selection, certificate chain integrity, and session resumption behavior. Each category carries independent compliance obligations and distinct attack surfaces. Professionals navigating the broader server security service landscape can orient to the full scope of controls through the Server Security Providers.
How it works
A TLS handshake establishes an encrypted channel through a sequence of negotiated parameters between client and server. The discrete phases are:
- ClientHello — The client transmits the highest TLS version it supports, a random nonce, and an ordered list of supported cipher suites.
- ServerHello — The server selects the TLS version and cipher suite from the client's list based on server-configured preference order.
- Certificate exchange — The server presents its X.509 certificate, signed by a Certificate Authority (CA) trusted by the client's trust store.
- Key exchange — Depending on the negotiated cipher suite, a shared session key is derived. Under TLS 1.3, only ephemeral Diffie-Hellman (ECDHE) key exchange is permitted, providing forward secrecy by default. TLS 1.2 permits RSA key exchange, which does not provide forward secrecy.
- Session establishment — Symmetric encryption begins using the negotiated algorithm (e.g., AES-128-GCM or AES-256-GCM).
TLS 1.3, standardized in IETF RFC 8446, eliminates the cipher suite complexity inherited by TLS 1.2. RFC 8446 reduces the handshake to 1 round trip (versus 2 in TLS 1.2), removes support for weak algorithms including RC4, 3DES, and MD5, and mandates authenticated encryption with associated data (AEAD) ciphers. Server administrators configuring TLS 1.2 must explicitly disable legacy cipher suites; TLS 1.3 enforces a restricted suite by protocol design.
Certificate management is a parallel control layer. Certificates issued by publicly trusted CAs must comply with CA/Browser Forum Baseline Requirements, which cap maximum certificate validity at 398 days as of September 2020. Expired, self-signed, or mis-issued certificates produce handshake failures regardless of correct protocol version configuration.
Common scenarios
HTTPS web server hardening is the most widespread deployment context. An Apache or Nginx server serving public web traffic must be configured to reject TLS 1.0 and 1.1, enforce a cipher suite preference list that excludes NULL, EXPORT, and RC4 ciphers, and present a certificate chain that includes any required intermediate CA certificates. The Mozilla SSL Configuration Generator, maintained by the Mozilla Foundation, produces server-specific configuration blocks for Apache, Nginx, HAProxy, and other common platforms aligned to three defined profiles: modern, intermediate, and old.
Internal API and microservice traffic frequently operates with mutual TLS (mTLS), in which both client and server present certificates for authentication. This model is common in zero-trust architectures and is referenced in NIST SP 800-207 on Zero Trust Architecture as a mechanism for device and service identity verification.
Database server encryption in transit governs connections between application servers and database backends (PostgreSQL, MySQL, Microsoft SQL Server). Each database engine exposes its own TLS configuration parameters, and misconfiguration at this layer is a documented vector for credential interception. PCI DSS Requirement 4.2.1 explicitly addresses encryption of cardholder data in transit across all transmission types, including internal network segments.
Mail server TLS (STARTTLS and SMTP MTA-STS) addresses opportunistic versus enforced encryption for email delivery. STARTTLS allows downgrade attacks if not paired with MTA-STS (RFC 8461), which publishes a DNS-based policy requiring strict TLS for inbound SMTP. Federal email infrastructure is subject to CISA Binding Operational Directive 18-01, which mandated STARTTLS deployment across federal executive branch mail systems.
Decision boundaries
The primary configuration decision is protocol version floor. NIST SP 800-52 Rev 2 establishes TLS 1.2 as the mandatory minimum for federal systems and TLS 1.3 as the recommended standard. PCI DSS 4.0, released by the PCI Security Standards Council, carries forward the TLS 1.2 minimum. For systems subject to HIPAA Security Rule requirements (45 CFR §164.312(e)(1)), HHS does not specify a TLS version but references NIST guidance as the accepted implementation standard, making SP 800-52 Rev 2 the effective benchmark for covered entities.
The contrast between TLS 1.2 and TLS 1.3 deployments involves a concrete tradeoff:
| Parameter | TLS 1.2 | TLS 1.3 |
|---|---|---|
| Forward secrecy | Optional (requires ECDHE/DHE suite selection) | Mandatory |
| Cipher suite count | 37+ defined suites, many deprecated | 5 AEAD-only suites |
| Handshake round trips | 2 | 1 (0-RTT optional) |
| Key exchange algorithms | RSA, DH, ECDH | ECDHE only |
| Downgrade protection | Requires explicit disabling of weak suites | Built into protocol |
The decision to support TLS 1.2 alongside TLS 1.3 is a compatibility concession, not a security equivalence. Cipher suite hygiene on TLS 1.2 requires active maintenance; TLS 1.3 enforces equivalent controls by specification.
Certificate pinning presents a second decision boundary. HTTP Public Key Pinning (HPKI) has been deprecated by major browsers. The replacement mechanism, Certificate Transparency (RFC 9162), requires that publicly trusted certificates be logged to auditable append-only ledgers. Servers relying on certificates from CAs that do not comply with CT logging requirements will fail validation in modern browsers.
Administrators assessing where TLS configuration fits within a broader hardening program can reference the structural overview at Server Security Provider Network Purpose and Scope. For guidance on navigating professional service categories within server security, the How to Use This Server Security Resource page maps available reference categories.