As an AI enthusiast and industry professional, I had the incredible opportunity to attend the Data + AI Summit 2025, where Databricks unveiled some of the most transformative innovations in enterprise AI. Among the highlights were two groundbreaking technologies — Agent Bricks and Mosaic AI — that promise to revolutionize how organizations build, evaluate, and deploy AI agents tailored to their unique data environments.

This article shares my first-hand experience from the summit. It provides a comprehensive, detailed exploration of these technologies, their technical underpinnings, and their profound implications for the future of enterprise AI.

Setting the Stage: The Enterprise AI Challenge

At the summit, it became clear that enterprises face significant hurdles when trying to operationalize AI agents:

• Complexity in building agents that can reason reliably over vast, heterogeneous data sources.
• Difficulty in evaluating and optimizing agent quality in a repeatable, objective manner.
• High costs and resource demands are associated with large-scale AI deployments.
• Need for strong governance and security to comply with industry regulations.

Databricks’ announcements addressed these challenges head-on with a vision to simplify and automate the entire AI agent lifecycle, turning what was once a daunting technical endeavor into an accessible, scalable, and cost-effective reality.

.

Mosaic AI: The Intelligent Foundation

Mosaic AI serves as the foundational platform enabling AI systems to reason deeply over enterprise data. It empowers organizations to build domain-specific agents that deliver actionable insights, automate workflows, and enhance decision-making.

Thousands of customers already use Mosaic AI to integrate intelligent agents into their operations. However, as the summit showcased, building production-grade agents with Mosaic AI alone still required navigating complex model tuning, evaluation, and optimization processes.

The Game-Changer: Agent Bricks

The highlight of the summit was the introduction of Agent Bricks, a product designed to automate and simplify the creation, evaluation, and optimization of AI agents. The core idea is revolutionary: instead of wrestling with technical complexity, users focus on defining the agent’s purpose and providing natural language feedback on agent quality.

How Agent Bricks Transforms AI Agent Development

Agent Bricks automates every step of the agent lifecycle:

• Automatic generation of task-specific evaluation suites: The platform synthesizes benchmarks tailored to the enterprise’s domain and data, enabling robust, repeatable quality assessment without the need for large labeled datasets.
• LLM-based judges for objective scoring: Using large language models as evaluators, Agent Bricks provides unbiased, consistent quality metrics that replace subjective human judgment.
• Automated multi-dimensional optimization: It systematically tunes prompts, retrieval methods, vector filtering, and agent orchestration to optimize for accuracy and cost.
• Agent Learning from Human Feedback (ALHF): Domain experts provide rich natural language guidance — such as ignoring outdated data or refining retrieval logic — which the system translates into technical improvements automatically.
• Built-in governance and security: Agent Bricks integrates seamlessly with Databricks’ data governance tools, ensuring compliance and operational control.
• Integration with Databricks ecosystem: Works in harmony with MLflow 3.0 for lifecycle management, Unity Catalog for data governance, and Mosaic AI Vector Search for scalable retrieval-augmented generation (RAG).

Technical Insights: Why Agent Bricks Matters

From a technical standpoint, Agent Bricks addresses critical enterprise needs:

• Repeatable, objective evaluation: By automating benchmark creation and leveraging LLM judges, it eliminates guesswork and ensures production-grade quality from day one.
• Cost efficiency: Benchmarks at the summit showed Agent Bricks achieving up to 10x lower inference costs compared to prompt-optimized proprietary LLMs, making large-scale deployments economically viable.
• Scalability: The underlying vector search infrastructure supports billions of vectors with storage-compute separation, enabling massive semantic search and RAG applications.
• Continuous improvement: Integrated monitoring and feedback loops allow agents to evolve post-deployment, adapting to changing data and requirements.
• Democratization of AI development: ALHF empowers domain experts without deep technical skills to influence agent behavior directly, bridging the gap between AI infrastructure and business expertise.

Agent Learning from Human Feedback (ALHF)

A standout innovation within Agent Bricks is Agent Learning from Human Feedback (ALHF). Traditional feedback mechanisms often rely on simplistic thumbs-up or thumbs-down signals, which do not provide actionable insights into which components of an agent should be adjusted. ALHF overcomes this by accepting rich natural language guidance from domain experts, such as instructions to ignore outdated data or refine specific retrieval methods.

The system intelligently translates this guidance into technical optimizations, adjusting retrieval algorithms, prompt engineering, vector database filtering, or even modifying the agent’s operational patterns. This approach democratizes agent development, allowing non-technical domain experts to directly influence agent quality, thus bridging the gap between AI infrastructure complexity and business expertise.

Key Use Cases and Industry Applications

Agent Bricks is optimized for several critical enterprise AI scenarios:

• Structured Information Extraction: Converts complex unstructured documents (emails, PDFs, reports) into structured data fields such as names, dates, and product details. For example, retail companies can extract product pricing and descriptions from supplier documents despite heterogeneous formats.
• Reliable Knowledge Assistance: Delivers fast, accurate, and cited answers grounded in enterprise data, solving the common problem of generic or incorrect chatbot responses. Manufacturing firms empower technicians with instant access to SOPs and maintenance manuals.
• Multi-Agent Orchestration: Enables building sophisticated multi-agent systems that coordinate tasks like intent detection, document retrieval, and compliance checks. Financial services firms can create personalized, compliant advisor-client interactions by orchestrating multiple specialized agents.
• Custom Text Transformation: Supports industry-specific content generation and chat agents, such as marketing teams generating brand-aligned copy, blogs, or press releases.

The Future of Enterprise AI with Agent Bricks and Mosaic AI

Agent Bricks represents a fundamental shift in enterprise AI development — from managing overwhelming technical complexity to focusing on strategic outcomes and continuous quality improvement. Powered by ongoing research from the Databricks Mosaic AI Research team, Agent Bricks incorporates cutting-edge methods to keep agents at the forefront of performance and reliability.

As AI agents become central to enterprise operations, the need for robust, systematic approaches to building and optimizing these systems grows. Agent Bricks meets this demand by providing an automated, feedback-driven platform that ensures agents are not only high-quality at launch but continue to improve over time.

Together, Mosaic AI and Agent Bricks form a powerful ecosystem for enterprises to harness the full potential of their data through intelligent agents. They enable organizations to transition from prototype demos to production-grade AI solutions that drive real-world impact efficiently and securely.

In short, Agent Bricks represents a technical breakthrough by automating the complex, costly, and error-prone tasks of building, evaluating, and optimizing AI agents on enterprise data. By integrating synthetic data generation, LLM-based evaluation, automated optimization, and rich human feedback in a governed environment, it enables enterprises to rapidly deploy trustworthy, cost-efficient AI agents at scale. Presented at the Data + AI Summit 2025, Agent Bricks and the broader Mosaic AI stack set a new standard for production-grade AI agents, empowering organizations to harness their data intelligence with unprecedented speed, quality, and control. This innovation not only addresses the critical technical challenges of AI agent deployment but also democratizes AI development by allowing domain experts to directly influence agent behavior without deep technical expertise, marking a new era of practical and powerful enterprise AI.

This detailed technical article synthesizes the latest announcements and insights from Databricks’ official releases and the Data + AI Summit 2025 presentation.