Search results
14 resultsPrivacy-First Data Design — PII Handling Patterns
Tokenisation, pseudonymisation, encryption at rest, and right-to-deletion workflows.
Designing a Data Lake on AWS S3
Folder structure, naming conventions, lifecycle policies, and access patterns.
API Gateway — Responsibilities and Implementation Patterns
Authentication, rate limiting, routing, request aggregation, and when not to use a gateway.
Batch vs Streaming Pipelines — Choosing the Right Pattern
Lambda architecture, Kappa architecture, and practical guidance for choosing.
Real-Time Analytics Architecture Patterns
Lambda, Kappa, HTAP, and choosing the right pattern for sub-second analytics.
Container Registry Management and Image Lifecycle
Tagging conventions, vulnerability scanning, retention policies, and registry options.
Secrets Management for Data Platforms
HashiCorp Vault, AWS Secrets Manager, and patterns for rotating credentials safely.
Serverless Architecture — When Functions Work and When They Don't
Cold starts, event-driven patterns, cost model, and the right use cases.
OAuth 2.0 and OIDC — Implementing Secure Third-Party Auth
Authorization code flow, PKCE, scopes, and integrating with identity providers.
Message Queue Patterns — SQS, RabbitMQ, and Dead Letter Queues
Fan-out, work queues, priority queues, and poison message handling.
Orchestrating Pipelines with Apache Airflow
DAGs, operators, scheduling, and production best practices for Airflow.
Feature Stores — Bridging Data Engineering and ML
What a feature store is, online vs offline stores, and when to build vs buy.
Infrastructure as Code for Data Platforms with Terraform
Managing cloud data infrastructure reproducibly with Terraform.
Amazon Redshift — Architecture and Query Optimization
Distribution styles, sort keys, VACUUM, ANALYZE, and WLM tuning.