2026-06 Monthly Summary — apache/doris Key features delivered: - Configurable session-visible commit status for ETL insert timeouts: Introduced a session variable insert_visible_timeout_return_mode that lets ETL paths choose between returning COMMITTED or ERR when publish-timeout occurs for internal-table inserts. This fixes visibility gaps in downstream pipelines and improves ETL reliability and data lineage visibility. Commit: 8de3a2fba9fa75c5d403912958be5aed6cc7e949. Release notes and test coverage included in PR (#63919). - External table metadata preloading to reduce lock contention during mixed queries: Implemented a preload phase for external metadata before acquiring internal table locks in Nereids planning. Reduces plan-time read lock duration, improving performance for mixed internal/external queries. Commit: 98a7a7e95489338e2b38a2e7dcc46e9b6b8033b6. Supports Hive/Hudi, Iceberg, Paimon and JDBC; new table trait and PreloadExternalMetadata rule. - Username-based shared workload scheduling metric for frontend and backend: Extended workload policies to evaluate username on BE side as well as FE, enabling policy-driven scheduling using username in combination with BE metrics (e.g., query_time, be_scan_rows, be_scan_bytes, be_memory). Commit: 818bc6d9376f45e46c7be046ec691f5fd372a121. Includes FE/BE metadata propagation and unit/regression tests. Major bugs fixed: - Fixed incorrect handling of publish timeout visibility in ETL inserts by adding a configurable mode, preventing downstream pipelines from reading false-success states when data is not yet visible. This directly improves data correctness in ETL workflows. Overall impact and accomplishments: - Improved data pipeline reliability and visibility for ETL workflows, reducing risk of empty reads and data gaps. - Lowered internal lock contention and improved latency for mixed queries, enabling higher concurrency and throughput. - Enabled cross-layer workload management with username-aware policies, supporting more nuanced resource sharing and policy enforcement. Technologies/skills demonstrated: - FE/Nereids planning optimizations, external metadata handling, and session variable design. - Cross-layer policy extension (FE/BE) and workload management enhancements. - Comprehensive test strategies (unit, regression, manual) and release-notes documentation integration.

May 2026 monthly summary focusing on business value and technical achievements: Delivered real-time, unified query runtime statistics across the BE->FE pipeline, enabling consistent, real-time visibility for operators and frontend interfaces. Strengthened reliability in offline tooling and improved streaming load stability under Jetty 12. Documented monitoring improvements to empower faster diagnostics and better SRE/ops guidance.

April 2026 monthly summary for apache/doris focusing on reliability and correctness in FileScanner when handling mixed Native/JNI splits. Delivered a critical fix to preserve predicate filters across readers, preventing data leakage for queries with non-pushdown predicates. Strengthened test coverage with a backend unit test to guard against regression; linked to a concrete commit and PR for traceability.

In March 2026, delivered a targeted improvement to Apache Doris that enables selective File Cache Admission Control for remote storage, aimed at reducing cache pollution and preserving hot data. The feature allows data to be admitted into the File Cache based on user identity and table context, improving cache hit rates under heavy ETL and ad-hoc workloads. Core deliverables include FE-driven admission decision, propagation through split/scan-range generation, and BE-aware enforcement, with clear visibility into decision cost in the query plan.

February 2026 highlights: Delivered essential documentation for the Cache Query Limit feature on the doris-website to help users cap file cache usage per query, improving cache resource governance in multi-tenant environments. Fixed a bug in the Task Controller to accurately calculate running time under the workload policy, increasing reliability of query-time measurements. Strengthened overall quality with regression, unit, and manual tests validating timing logic. These efforts improve product guidance, operational reliability, and readiness for production deployments.

January 2026 monthly summary for apache/doris focusing on File Cache Query Limit work. Delivered end-to-end feature enhancement with usage entry point, independent quota controls, and full test coverage; added a robust FE/BE parameter flow (enable_file_cache_query_limit, file_cache_query_limit_percent_soft, and policy/session overrides) with Thrift transmission; implemented per-query FileCacheContext with capacity allocation (max(capacity * percent / 100, min(capacity/2, 256MB))) ensuring isolation and resource safety; expanded test suite and addressed core framework gaps related to entry point, quota independence, and test coverage.

December 2025 monthly summary for apache/doris-website focusing on improving observability and performance tuning through SqlDigest-based slow query analysis. Implemented usage of SqlDigest for analyzing slow SQL, added an audit threshold to generate SqlDigest aggregates, and prepared cross-version and bilingual documentation to drive adoption and actionable insights across environments (dev, 4.x, 3.x, 2.1).

2025-11 Monthly Summary: Delivered File Cache Observability Metrics in apache/doris, adding readsize and hitsize metrics to file_cache statistics and exposing them via backend counters. Implemented regression tests to validate the metrics integration and prevent regressions. This work enhances cache observability, enables data-driven tuning, and improves overall caching performance and reliability across the system.

October 2025 performance summary focused on strengthening File Cache reliability, observability, and deployment correctness across Doris builds. The month delivered targeted improvements to metrics accuracy, enhanced monitoring, and a robust testing baseline for cache behavior, delivering clear business value in reliability and operational insight.

July 2025 Monthly Summary: Delivered Regression Testing Documentation Enhancements for apache/doris-website to streamline and standardize regression test environments. The update adds details on pre-container startup network configuration checks, enhanced descriptions for Hive-related Docker Compose files, and clearer container log viewing commands. These changes reduce setup time, minimize configuration errors, and improve testing reliability across environments.

June 2025 monthly summary focusing on features delivered and impact across Doris repos. Highlights include documentation improvements for segcompaction_batch_size in the website and enhanced file cache metrics exposure in Doris core, enabling better observability and data-driven performance tuning. No critical bugs fixed this month; efforts centered on reducing operator toil and strengthening release quality.

May 2025 monthly summary focusing on targeted reliability improvements and user-facing clarity across Doris core and website. Delivered fixes to metrics reporting and UI text, plus documentation updates to reflect current toolchain requirements, enhancing data integrity, user experience, and developer onboarding.

April 2025: Focused on improving build reproducibility and developer onboarding for Doris website docs. Delivered targeted guidance for JDK 17 and build environment to support Doris 3.0+ and master builds, clarified environment requirements across versions, updated Maven download references, and noted ARM compilation considerations. These changes streamline local setup for contributors and reduce build-related friction, contributing to faster release readiness.