data
15 notes tagged data in the Data topic. Data engineering: pipelines, warehouses, time series and storage choices.
- Building a scalable social listening data pipeline
We partnered with Plot to build a robust data pipeline that processes millions of social media posts daily, providing AI-powered insights into brand performance.
- Migrate regular tables into TimescaleDB hypertables to improve query performance
How do we migrate normal table to timescale table to optimized data storage
- Building a data archive and recovery strategy for high-volume trading system
A guide to implementing data archival and recovery strategies for high-volume transactional application.
- Building data pipeline for OGIF transcriber
A technical case study of creating an automated system that downloads videos, processes audio, and generates transcripts using AI services like Groq and OpenAI.
- Building better Binance transfer tracking
A deep dive into building a robust transfer tracking syste m for Binance accounts, transforming disconnected transaction logs into meaningful fund flow narratives…
- Reconstructing historical trading PnL: a data pipeline approach
A detailed look at how we rebuilt historical trading PnL data through an efficient data pipeline approach, transforming a complex problem into a maintainable solution.
- Visualizing crypto market performance: BTC-Alt dynamic indicators in Golang
Implementing a Golang-based visualization for crypto market performance indicators, focusing on Bitcoin vs Altcoin dynamics and trading strategy effectiveness through…
- Project reports system: a case study
An in-depth look at Dwarves' monthly Project Reports system - a lean, efficient system that transforms communication data into actionable intelligence for Operations…
- Building a data-driven project reporting system: A lens into modern data engineering
Learn how to transition from application development to data engineering by building a modern project reporting system. Discover essential mindset shifts, best…
- Data pipeline design framework
To improve and strongly go-live the data pipeline, besides apply best practices and pillar for Data Pipeline Native Solution, a design framework and pattern are robustly…
- Sql and how it relates to disk reads and writes
Sequential read/write operations involve accessing data in a continuous, linear manner. This typically occurs when transferring large files or accessing a large file on…
- Materialized view pattern
Materialized view is the cache of views. It pre-computes, stores, and optimizes data access when created, and automatically refreshes to ensure real-time data…
- ¶ MapReduce
We are the head of census bureau for the state of California and tasked with finding the finding the population of all cities in California. All the resources we want…