Multi-Voice Systems

In complex enterprise scenarios, a single voice isn't enough. Multi-voice systems deploy multiple distinct voices — for different departments, characters, or languages. The challenge: consistency, routing, and seamless transitions.

Multiple Distinct Agent Voices

Why Multi-Voice?

• Department recognition: Customers know by sound whether they're with sales, support, or billing
• Brand identity: Different product lines → different voices
• Language diversity: German, English, French — each language with a native voice
• Personality: Different agent personas for different concerns

Creating a Voice Portfolio

Implementation

const voiceAgents = {
  sales: {
    voiceId: 'sophia-voice-id',
    systemPrompt: 'You are Sophia, our sales consultant...',
    tools: ['check_pricing', 'schedule_demo', 'send_proposal'],
  },
  support: {
    voiceId: 'max-voice-id',
    systemPrompt: 'You are Max, our technical support...',
    tools: ['check_ticket', 'remote_diagnostics', 'escalate'],
  },
  billing: {
    voiceId: 'elena-voice-id',
    systemPrompt: 'You are Elena, our billing specialist...',
    tools: ['check_invoice', 'process_refund', 'update_payment'],
  },
}

Voice Routing

Intelligent Routing

Voice routing determines which voice the user hears — based on context:

Incoming call
  ↓
Greeting (System voice)
  ↓
Intent recognition: "How can I help you?"
  ↓
├── "I'd like to buy something" → Sophia (Sales)
├── "My internet isn't working" → Max (Support)
├── "I have a question about my invoice" → Elena (Billing)
└── "I'm new here" → Kai (Onboarding)

Routing Rules

Character-Based Voice Assignment

Consistency Across Sessions

A customer should always hear the same voice when speaking with the same department:

async function getVoiceForCustomer(customerId: string, department: string) {
  // Check if customer already has an assigned agent
  const assignment = await db.voiceAssignments.findOne({
    customerId,
    department,
  })

  if (assignment) {
    return assignment.voiceId // Known voice
  }

  // New assignment based on availability and customer profile
  const voiceId = await selectBestVoice(customerId, department)
  await db.voiceAssignments.create({ customerId, department, voiceId })
  return voiceId
}

Voice Consistency Rules

• Same customer → Same voice (within a department)
• Same brand → Same voice style (across all touchpoints)
• Language switch: Same timbre, different language
• Emotional consistency: The agent always sounds like "the same person"

Voice Consistency Management

The Challenge

ElevenLabs voices can sound slightly different with different texts. For enterprise use, consistency must be guaranteed:

Strategies for Consistency

1. Standardize voice settings:

{
  "stability": 0.65,
  "similarity_boost": 0.80,
  "style": 0.20,
  "use_speaker_boost": true
}

2. Prompt consistency:

• Same system prompt → similar output style
• Anchor tonality instructions in the prompt
• Example dialogs in the prompt as reference

3. Audio post-processing:

• Normalize volume (LUFS standard: -16 LUFS for telephony)
• EQ preset per voice (same sound character)
• Noise suppression for telephony quality

Monitoring

Practical tip: Create a "Voice Style Guide" — similar to a brand style guide, but for voices. Define for each agent: voice, settings, tonality, do's and don'ts. Share it with all developers and stakeholders.