EnqueueZero Techshack 2019-08
The Curious Case of BEAM CPU Usage
Key takeaway is BEAM’s busy wait settings do have a significant impact on CPU usage. The highest impact was observed on the instance with the most available CPU capacity. When running BEAM on an OS kernel shared with other software, it makes sense to turn off busy waiting, to avoid stealing time from non-BEAM processes. It would also make sense to not use busy waiting when running on burstable performance instances in the cloud.
Cloud Native Application Architecture
- Designed As Loosely Coupled Microservices;
- Developed With Best-of-breed Languages And Frameworks;
- Center Around APIs For Interaction And Collaboration;
- Stateless And Massively Scalable;
- Resiliency At The Core Of the Architecture;
- Packaged As Lightweight Containers And Orchestrated;
- Agile DevOps & Automation Using CI/CD;
- Elastic — Dynamic scale-up/down.
- Strategies for implementing resiliency:
- Retry transient failures ;
- Load balance across instances ;
- Degrade gracefully;
- Throttle high-volume tenants/users;
- Use a circuit breaker;
- Apply compensating transactions.
Will Node.js forever be the sluggish Golang?
Express.js is the oldest web framework for Node.js, building on top of out-of-box features Node.js provide, adding a nice App-centered programming interface to manage URL routes, parameters, methods and the like. Though it's very slow.
An update about Redis developments in 2019
Redis roadmap 2019: RESP3, ACL, Multi threading, Better persistence, Data structures.
How we used delayed replication for disaster recovery with PostgreSQL
Delayed replication can be used as a first resort to recover from accidental data loss and lends itself perfectly to situations where the loss-inducing event is noticed within the configured delay. However, Replication is not a backup mechanism.
Delayed replication is the idea of applying time-delayed changes from the WAL. That is, a transaction that is committed at physical time X is only going to be visible on a replica with delay d at time X + d.
- A cold backup is useful to recover from a disaster.
- The purpose of replication is mostly to guard database availability against failures of individual database servers and to distribute load.
The curse of the data lake monster
Software is best developed in thin, vertical slices that emphasize use cases and user outcomes, and data-intensive projects are no exception.
Data lake often implies a centralized repository of data, or something built and maintained by data engineers so that data scientists can consume data and focus on developing ML use cases etc. Designing a data lake in a top-down fashion, without an eye on the end use cases, will almost inevitably result in a poor problem/solution fit.
The article proposed a more bottom-up approach to realizing the data lake — one that builds one vertical slice at a time. There’s no single, one-size-fits-all definition of a data lake. You need to work on articulated use-cases and measurable business goals.