I've read every single one of Aphyr's tests. Aphyr's analyses probe a broader range of distributed consistency problems in replicated stateful databases. Consensus is a necessary mechanism to achieve distributed consistency, but if you look at the problems highlighted in products like CockroachDB and TiDB, they relate to secondary effects of failures to ensure linearizability of reads and writes.
You can have a perfectly functioning consensus algorithm and still battle inconsistent reads across nodes, loss of data on node failure or network splits, write skew, and so on. For example, TiKV uses Raft, yet the problems uncovered in TiKV have nothing to do with Raft itself. Conversely, those found in Elasticsearch are absolutely caused by its consensus algorithm, Zen, which was invented in ignorance of modern consensus research (Paxos, etc.).
If you read my comment carefully, you'll note that it's above all an endorsement of Raft, which is a milestone in that it democratizes advanced consensus and makes it a tool anyone can use. My point isn't that anyone can trivially whip up an infinitely scalable, consistent database because consensus is a solved problem, but that we have solid foundational theory to build on, to the point where it is a well-understood problem (albeit with many complicated implementations) and not black magic.
Also, I did not use the term "easy". But I also think the OP's phrasing "extremely hard" is not true, unless your requirements are extremely hard – which, again, may be the case with Kafka; scalability tends to be diametrically opposed to consistency.
You can have a perfectly functioning consensus algorithm and still battle inconsistent reads across nodes, loss of data on node failure or network splits, write skew, and so on. For example, TiKV uses Raft, yet the problems uncovered in TiKV have nothing to do with Raft itself. Conversely, those found in Elasticsearch are absolutely caused by its consensus algorithm, Zen, which was invented in ignorance of modern consensus research (Paxos, etc.).
If you read my comment carefully, you'll note that it's above all an endorsement of Raft, which is a milestone in that it democratizes advanced consensus and makes it a tool anyone can use. My point isn't that anyone can trivially whip up an infinitely scalable, consistent database because consensus is a solved problem, but that we have solid foundational theory to build on, to the point where it is a well-understood problem (albeit with many complicated implementations) and not black magic.
Also, I did not use the term "easy". But I also think the OP's phrasing "extremely hard" is not true, unless your requirements are extremely hard – which, again, may be the case with Kafka; scalability tends to be diametrically opposed to consistency.