EnqueueZero Techshack 2019-09

Intro to Redis Cluster Sharding – Advantages, Limitations, Deploying & Client Connections

highscalability.com

Redis Cluster is the native sharding implementation available within Redis that allows you to automatically distribute your data across multiple nodes without having to rely on external tools and utilities.

The entire keyspace in Redis Clusters is divided in 16384 slots (called hash slots) and these slots are assigned to multiple Redis nodes. A given key is mapped to one of these slots, and the hash slot for a key is computed as: HASH_SLOT = CRC16(key) mod 16384. Multi-key operations are supported on Redis Clusters as long as all the keys involved in a single command execution belong to the same hash slot.

The benefits of Redis Cluster are high performance, high availability, horizontal scalability, and as a native solution. The downside is that it requires client support, limited multi-key operation support, and only supports one database.

The Redis Cluster solution can be a good fit for you if you need a sharded Redis solution. Typically, people start looking at sharding their Redis deployments when they’ve started to saturate a standalone Redis node with writes and want to spread writes out to multiple nodes.

An async "framework" within a single Tweet

twitter.com

from select import select
from collections import deque

R={};W={};Q=deque()

def p():
 r,w,_=select(R,W,[])
 Q.extend(R.pop(s) for s in r);Q.extend(W.pop(s) for s in w)

def go():
 while Q or p() or Q:
  try:(w,s),w[s]=(t:=Q.popleft()).send(None),t
  except StopIteration:pass

Usage:

Q.append(coro)
go()

The Right Way to Bundle Your Assets for Faster Sites over HTTP/2

medium.com

This article goes over the basics of HTTP/2 and shows benchmark data to support some simple development guidelines to ensure that your site is optimized for HTTP/2.

Though concatenation has been considered to be bad in a HTTP/2 environment because HTTP/2 is designed to transfer multiple small files simultaneously without much overhead. The benchmark test showed that asset concatenation is still good for improving loading performance in HTTP/2.

Key Takeaways:

  • Always concatenate files into several bundles.
  • Consider HTTP/1.1 compatibility.
  • Use image sprites.
  • Carefully use data URIs.

OpenSSL Strategic Architecture

www.openssl.org

This document outlines the OpenSSL strategic architecture. Below is the as-is architecture.

openssl components

Below is the to-be architecture.

openssl components

Understanding Real-World Concurrency Bugs in Go

songlh.github.com

The paper conducts the first comprehensive, empirical study on 171 real-world Go concurrency bugs from two orthogonal dimensions.

  • Root Causes of Blocking Bugs:
    • (mis)Protection of Shared Memory.
    • Misuse of Message Passing.
  • Root Causes of Non-blocking Bugs:
  • Failing to Protect Shared Memory.
  • Errors during Message Passing.

Best Deep Learning Books: Updated for 2019

blog.floydhub.com

  • Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville
  • Grokking Deep Learning by Andrew W. Trask
  • Deep Learning with Python by Francois Chollet
  • Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron
  • The Hundred-Page Machine Learning Book by Andriy Burkov
  • Reinforcement Learning: An Introduction (2nd Edition) by Richard S. Sutton, Andrew G. Barto
  • Deep Reinforcement Learning Hands-On by Maxim Lapan
  • Learning From Data by Yaser S. Abu-Mostafa, Malik Magdon-Ismail, Hsuan-Tien Lin.
  • The Book of Why by Judea Pearl, Dana Mackenzie.
  • Machine Learning Yearning by Andrew Ng.
  • Interpretable Machine Learning by Christoph Molnar.
  • Neural Networks and Deep Learning by Michael Nielsen.

Your migrations are bad, and you should feel bad

djrobstep.com

People invented versioned migrations to handle database schema changes, such as Rails migrations, Django migrations, Alembic migrations, Go migrations, etc etc, although nobody really cares about old versions. We can simplify it to:

Empty schema → Dev schema → Live schema

The author introduced a tool migra that can generate diffs for your database schemas. Every time there is a change, use the tool generate a diff and apply that to the database. No files, no version numbers: One operation.

How Discord Scaled Elixir to 5,000,000 Concurrent Users

blog.discordapp.com

  • While Discord is rich with features, most of it boils down to pub/sub. Users connect to a WebSocket and spin up a session process (a GenServer), which then communicates with remote Erlang nodes that contain guild (internal for a “Discord Server”) processes (also GenServers). When anything is published in a guild, it is fanned out to every session connected to it. When meet scaling problem, Erlang processes are effectively single threaded, and the only way to parallelize the work is to shard them. Manifold distributes the work of sending messages to the remote nodes of the PIDs (Erlang process identifier), which guarantees that the sending processes at most only calls send/2 equal to the number of involved remote nodes.
  • When using consistent hashing, the Erlang process responsible for controlling the ring would start to get so busy that it would fail to keep up with requests to the ring, and the whole system would become overloaded. The solution was to introduce mochiglobal, a module that exploits a feature of the VM: if Erlang sees a function that always returns the same constant data, it puts that data into a read-only shared heap that processes can access without copying the data.
  • Use :ets.update_counter/4, which performs atomic conditional increment operations on a number inside an ETS key.

Cloud Native Application Architecture

medium.com

Cloud Native Design Principles

  • Designed As Loosely Coupled Microservices
  • Developed With Best-of-breed Languages And Frameworks
  • Centred Around APIs For Interaction And Collaboration
  • Stateless And Massively Scalable
  • Resiliency At The Core Of the Architecture
  • Packaged As Lightweight Containers And Orchestrated
  • Elastic — Dynamic scale-up/down