A Bloom Filter is a probabilistic data structure used for testing whether an element is a member of a set or not. It's space-efficient compared to other data structures like hash tables, but it may give false positives (indicating that an element is in the set when it's not) and never gives false negatives (indicating that an element is not in the set when it actually is not)...

Bloom Filter

Conflict-free Replicated Data Types (CRDTs) (aka convergent replicated data type or commutative replicated data type) are data structures that can be replicated across multiple computers in a network, where the replicas can be updated independently and concurrently without coordination between them, and enable operations to always converge to a final state consistent among all replicas.

Introduction to CRDT

Sequential read/write operations involve accessing data in a continuous, linear manner. This typically occurs when transferring large files or accessing a large file on the drive. In sequential read/write operations, the drive can read or write data from a series of blocks, allowing for faster performance...

SQL and how it relates to Disk Reads and Writes

Structured Query Language (SQL) is widely used for managing data in relational database management systems (RDBMS). In the context of SQL, the optimization of database queries forms the foundation for efficient data retrieval, providing quicker response times and increasing overall database performance. One key strategy to optimize SQL queries involves the concept of "Sargable" queries...

SQL Saragable Queries and Their Impact on Database Performance

Kafka has been using Zookeeper for a variety of important functions. It uses Zookeeper to keep track of which brokers are part of the Kafka cluster. This is a critical task, as it enables Kafka to ensure that each broker is working properly and that the entire cluster is functioning as intended...

The removal of Apache Kafka's dependency on Zookeeper

To improve and strongly go-live the data pipeline, besides apply best practices and pillar for Data Pipeline Native Solution, a design framework and pattern are robustly help us in...

Data Pipeline Design Framework

We are the head of census bureau for the state of California and tasked with finding the finding the population of all cities in California. All the resources we want are ready but we have only four months to finish the task. Calculating the population of all cities for a big state like california is not an easy task. The sensible thing to do is to divide the state by city and make individuals in charge of each city to calculate the population of each city where he is in charge of...

MapReduce

The motivation of this thoughts that trying to discover the LLMs and Generative AI...

Quick Learning Vector Database

Google Data Fusion is a fully managed, cloud-native data integration service that enables users to efficiently build and manage ETL/ELT data pipelines. It is designed to streamline data engineering tasks for enterprise users and is built on top of the open-source project CDAP.

Google Data Fusion

Google Dataproc is a fast and easy-to-use, fully managed cloud service for running Apache Spark and Hadoop workloads. It allows users to process and analyze large-scale data efficiently and at scale.

Google Dataproc

A comprehensive exploration of HTMX, a JavaScript library that extends HTML as a hypermedia, addressing the limitations of legacy HTML applications.

Introducing HTMX - Navigating the Advantages and Concerns

A code generation layer that bridges the gap between frontend and backend more effectively.

Typesafe Client Server

When working with websites and managing the computers that show them, we run into two important ideas: redirect URLs and rewrite URLs. These ideas are like tools that help us make websites work better for people. They help us make websites easier to find on search engines, and they help us make the paths on websites smoother. In this guide, we're going to explore these tools together. We'll learn what makes them different and how we can use them to make websites even cooler.

Playground > 01_literature > Engineering