Apache Parquet

• Columnar: данные хранятся по колонкам, не по rows
• Compression: Snappy, Gzip, ZSTD, LZ4 per-column
• Encoding: dictionary, RLE, bit-packing, delta для efficient storage
• Predicate pushdown: WHERE date > 2026-01-01 уровня reader, не scan
• Schema embedded в file → self-describing

# Python (pandas)
import pandas as pd
df = pd.read_csv('sales.csv')
df.to_parquet('sales.parquet', compression='snappy')

# Read only needed columns (column pruning)
df = pd.read_parquet('sales.parquet', columns=['date', 'amount'])

# DuckDB — query parquet directly
duckdb.sql("SELECT SUM(amount) FROM 'sales.parquet' WHERE date > '2026-01-01'")

• Data Lake
• Iceberg
• Avro