ElevenLabs unveiled a sweeping set of new models, agents and enterprise tools at the ElevenLabs NYC Summit on November 11, positioning the voice AI company as the builder of what it calls the “interaction layer for AI.” The announcements span speech models, agent memory and automation, a new workplace app and a partner program that now counts more than 550 partners in 44 countries.

“Within a few years, conversational agents will become critical infrastructure for every enterprise, as essential as cloud or payments,” co-founder and chief executive Mati Staniszewski said on stage. “Our goal is to build the interaction layer for AI, and today’s announcements make it possible for businesses to use their data to craft experiences that resonate in their unique circumstances.” Mumbrella counted 13 new products and upgrades in the package.

New models: Scribe v3 and Scribe v3 Medical

The centrepiece on the model side is Scribe v3, ElevenLabs’ latest and most accurate speech-to-text model, which powers dictation and meeting notes in the company’s new ElevenAI app. Alongside it comes Scribe v3 Medical, a transcription model tuned for health care that recognises thousands of complex medical terms. ElevenLabs’ own example is the difference between cycloserine and cyclosporine, two drugs whose names sound nearly identical but whose mix-up would matter a great deal.

ElevenLabs also said it will open up fine-tuning in the coming months: businesses will be able to fine-tune selected open-source language models and ElevenLabs voice models on data from their own live deployments, optimising for metrics they choose, such as customer satisfaction or reliability. A domain-optimised model for financial services is already in development, following the health care model.

Over the last month the company has also shipped three new voice models, according to Mumbrella: Scribe v3, Eleven v4, a text-to-speech model built for expressive speech that follows the tone, pacing and context of a script, and Eleven Realtime, a new class of speech-to-speech models for fast voice agents.

ElevenLabs announcement lineup
What ElevenLabs announced at its New York Summit.

Interaction Engine: cheaper models where they fit, frontier models where they count

The most strategically interesting announcement may be Interaction Engine, a system that orchestrates voice models and large language models across a conversation. It manages turn-taking and interruptions and picks the right model for each part of an exchange: a cheaper model for triage, a frontier model for the complex parts.

“Interaction Engine orchestrates all this under the hood, so you can be confident you’re delivering the right outcome for the customer, without wasting money on tokens,” Staniszewski said. The pitch is aimed squarely at enterprises worried about balancing model performance against token costs as voice agents move from pilots into production.

ElevenLabs voice stack
ElevenLabs’ upgraded voice stack.

ElevenAgents gains memory, routines and a new app

ElevenAgents, the company’s platform for voice and chat agents, gets two major upgrades. Memory gives agents the full context of every returning customer, distilling conversations into structured records that update over time. ElevenLabs’ examples: remembering that a customer prefers Spanish and only talks after 5pm, or that they moved house in March, so nobody has to repeat themselves.

Routines, a new automation layer, runs multi-step work before, during and after every conversation. The company cites mortgage pre-screening with a callback in minutes, appointment reminders, report emails, CRM entries and follow-up calls as the kind of work agents can now handle proactively.

Alongside the platform upgrades, ElevenLabs opened early access to ElevenAI, a new app presented as an AI teammate for enterprise teams. It listens to spoken conversations, including in-person meetings, and turns them into notes and next steps, with notice and consent mechanisms built in. Early access is free from the announcement.

Partners, McKinsey and Nvidia

To reach large, complex enterprises, ElevenLabs launched a Partner Program with more than 550 partners in 44 countries, including Google Cloud, IBM, Deloitte, DXC and Genesys. As part of the program it announced an alliance with McKinsey, pairing ElevenAgents with the consultancy’s enterprise transformation expertise and its QuantumBlack AI practice.

The company also expanded its partnership with Nvidia: Nemotron 3.5 Super VL is now available in ElevenAgents, and ElevenLabs will add 15 MW of new Vera Rubin compute capacity. The two companies will also work on an on-premises voice agent for Poland’s national e-Health platform, designed to help patients who only have a landline book appointments.

Rounding out the announcements, ElevenLabs is launching a Marketplace of industry-specific AI and voice applications built by technology partners, covering health care, financial services, insurance, education and retail. From next year, businesses will be able to buy agents through the Marketplace using their existing ElevenLabs commitment. Enterprises already putting ElevenAgents to work include Yahoo, Westgate and KPN.

Why it matters

ElevenLabs started as a voice generation company and has spent the past year pushing up the stack, from models into agents and now into the tooling that lets enterprises shape those agents with their own data. The summit announcements make that ambition explicit: the company wants to own the layer where AI talks to customers and employees, and to let each business tune that layer to its own context rather than accept a generic model.

The bet is that conversational agents become infrastructure, and that the winners will be the companies that make those agents cheap enough to run, accurate enough to trust and specific enough to be useful. With Scribe v3 Medical, a financial-services model on the way and fine-tuning open to enterprise data, ElevenLabs is arguing that specialisation, not scale alone, is the way there.

Sources