There is significant mathematics behind each of the primitives in TLS, and time and time again we’re shown that the mathematics is the only robust component. Part of the problem is that most of these primitives were designed from a purely mathematical standpoint without consideration of the implementation. This inevitably led to the situation whereby software implementations are brittle, buggy, and have many side-channel attacks unless the developers are extraordinarily careful.
There are numerous examples of the fragility of crypto implementations. In 2003, two researchers discovered that OpenSSL’s implementation of RSA decryption was vulnerable to timing attacks and allowed the researchers to recover RSA private keys in two hours. In 2006, Debian developers removed a seemingly innocuous line of code from OpenSSL that referenced an uninitialized variable. The consequence was CVE-2008-0166 and it compromised all SSH keys generated in Debian and Ubuntu over a two year period. In December 2010, a group gained root access of the PlayStation 3 because Sony forgot to regenerate a variable in its implementation of ECDSA. Then in 2013, two researchers discovered a timing attack in most implementations of AES_CBC, which led them to recover complete plaintext from TLS 1.2. The list just goes on.
Of course, it’s easy to blame the implementer. It’s easy to blame the state of OpenSSL’s codebase. It’s easy to say that OpenSSL’s developers should have written constant-time RSA code or that Sony’s security team should have had a deeper understanding of ECDSA. While the implementers did make mistakes, I don’t consider them at fault. After all, crypto algorithms are notoriously difficult to implement correctly. Chances are you’ve got it wrong if you try to roll your own. Blaming the implementer for not addressing “obvious” pitfalls, corner-cases, and side-channels is a naïve argument and does not address the fundamental problem. The implementation doesn’t have to be tricky if the primitive is designed properly in the first place. Otherwise, brittle crypto primitives will inevitably lead to fragile implementations even in the most popular libraries.
Consider authenticated encryption, wherein the plaintext is simultaneously encrypted and integrity protected. There are three competing encryption modes for authenticated encryption: OCM, CCM, and GCM. OCM is a very strong mode, but it is patented and is legally challenging to use in practice. CCM is a two-pass mode and thus won’t work for streaming applications like Netflix, Twitch, or YouTube. This leaves GCM, which is commonly used in SSL/TLS. GCM is straightforward to implement, but it’s brittle. To quote Peter Gutmann,
The GCM slides provide a list of pros and cons to using GCM, none of which seem like a terribly big deal, but misses out the single biggest, indeed killer failure of the whole mode, the fact that if you for some reason fail to increment the counter, you're sending what's effectively plaintext (it's recoverable with a simple XOR). It's an incredibly brittle mode, the equivalent of the historically frighteningly misuse-prone RC4, and one I won't touch with a barge pole because you're one single machine instruction away from a catastrophic failure of the whole cryptosystem, or one single IV reuse away from the same. This isn't just theoretical, it actually happened to Colin Percival, a very experienced crypto developer, in his backup program tarsnap. You can't even salvage just the authentication from it, that fails as well with a single IV reuse.
The fact that GCM is intrinsically brittle means that a strong cipher suite such as TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 can be trivially broken if the GCM implementation makes that simple mistake.