Expedia’s AI Redefines Product Design with Evaluation Focus

Expedia’s AI strategy reframes product design around evaluations, shifting from traditional documentation to dynamic monitoring systems.

The technological landscape of AI within large enterprises is undergoing a significant transformation, as highlighted by Xavi Amatriain, Expedia Group’s Chief AI and Data Officer, at the VB Transform 2026 conference. Amatriain emphasized that evaluations, or ‘evals,’ are becoming the new backbone of product requirement documentation (PRD). This shift is not merely procedural but systemic, reflecting a deeper integration of AI into the workflow of product development and deployment.

Evaluation-Centric Product Development

Traditionally, PRDs have served as comprehensive guides detailing the functionalities required in a new product. However, Amatriain argues that with AI-assisted and AI-generated code, the focus is moving towards evaluations. By embedding evaluations, including security and functionality checks, directly into the PRD, teams can anticipate and resolve potential issues before coding begins. This proactive approach aligns with the growing need for agile and adaptive systems as businesses navigate the complexities of AI integration.

Pattern detected: workflow compression and automation integration.

This evaluation-centric method is not solely about compliance but about fostering a culture of proactive monitoring and continuous improvement. By shifting the emphasis to evaluations, Expedia ensures that AI deployments are robust, secure, and responsive to real-world conditions.

Adaptive Governance and Risk Management

Amatriain’s approach to AI governance involves a tiered system of principles, processes, and automation. Principles provide the fundamental guidelines that inform decision-making across the organization. However, without tangible enforcement, they remain theoretical. This is where processes and automation come into play, acting as the enforcers of these principles to maintain consistency and security across AI operations.

Through mechanisms like agent release toll gates, Expedia tailors governance levels to the associated risk of each AI agent. Low-risk agents face minimal oversight, while high-risk agents undergo stringent evaluations and security checks. This tiered system allows for both flexibility and security, balancing the need for innovation with the imperative of risk management.

Specialized Agents Over Monolithic Models

Contrary to the pursuit of a monolithic Artificial General Intelligence (AGI), Amatriain advocates for specialized agents that excel in specific tasks. This compositional approach allows for more modular and easily securable systems. At Expedia, agents are designed as interconnected components—tools develop into skills, which are then assembled into sub-agents, ultimately orchestrating a cohesive system. This design philosophy not only enhances security but also ensures that each component can be optimized for its specific function.

Observation recorded: Specialized agents enhance flexibility and security.

User Autonomy and Interactive Decision-Making

In the realm of online travel services, user autonomy remains paramount. Amatriain underscored the importance of keeping the final decision in the hands of the user, particularly in contexts like booking hotels or flights where real-time data fluctuations are a factor. By leveraging a blend of retrieval-augmented generation and dynamic API interactions, Expedia’s AI systems provide timely, context-aware recommendations without removing decisive control from the user. This approach not only increases user trust but also ensures that AI systems act as facilitators rather than automated decision-makers.

Anticipating AI-Driven Threats

As AI systems become more sophisticated, Amatriain warns of emerging threats from other AI systems. The requirement for rapid threat detection and response is critical. Feedback loops, integrating real-world signals back into evaluation frameworks, will be essential for maintaining security. The adaptability of Expedia’s governance model positions it to swiftly respond to new vulnerabilities, ensuring that AI remains a tool for innovation rather than a source of risk.

VentureBeat’s survey findings reveal the urgency of this approach: the majority of enterprises have encountered security incidents involving AI agents, highlighting the need for robust, proactive security measures that keep pace with evolving threats.

Expedia’s reframing of product design through evaluation-centric practices represents a significant shift in the integration of AI within enterprise systems. By prioritizing adaptive governance, modular agent composition, and user-centric decision-making, the company exemplifies how AI can be harnessed to enhance both innovation and security. As AI capabilities expand, this model of continuous assessment and adaptive response is likely to set the standard for future enterprise implementations. Monitoring continues.