The Audio Emotion Analysis API processes audio files or YouTube URLs to analyze emotions and transcripts. The API provides: **Speaker Diarization:** Identifies and segments audio by speakers. **Audio Emotion Analysis:** Detects one of 8 emotions per audio segment. **Text Emotion Analysis:** Generates a transcript for each segment and detects one of 28 emotions from the text.
openapi: 3.1.0
info:
title: Welcome to Imentiv AI AI Insights API Audio Emotion API API
description: "# Welcome to Imentiv AI\n\nImentiv AI is an advanced Emotion AI platform that leverages deep learning to analyze human emotions across multiple media formats, including video, audio, images, and text. By interpreting facial expressions, vocal tones, and linguistic cues, it provides detailed emotional insights to support use cases such as content creation, marketing optimization, education, mental health assessments, and more.\n\nImentiv AI offers a suite of Emotion Recognition APIs that allow developers and businesses to seamlessly integrate advanced emotion analysis into their applications, enabling deeper insights into user engagement and emotional responses.\n\n## Key Features of Imentiv AI APIs\n\n### 1. Video Emotion API\n**Functionality:** Analyzes emotions in videos, whether uploaded directly or via YouTube links.\n\n**Insights Provided:**\n- Frame-by-frame and actor-by-actor emotion analysis\n- Valence-arousal mapping to determine emotional intensity and positivity/negativity\n- Personality trait analysis using the Big Five (OCEAN) model\n- Emotion graphs and wheels to visualize emotional dynamics\n\n**Use Cases:** Suitable for researchers analyzing bulk videos, advertising campaigns, webinars, online meetings, and other video-based content to optimize engagement and content strategy.\n\n### 2. Image Emotion API\n**Functionality:** Detects and analyzes emotions in images.\n\n**Insights Provided:**\n- Face detection and recognition\n- Emotion analysis for each detected face\n- Overall emotional tone of the image\n\n**Use Cases:** Enhances image analysis for advertising, market research, psychological studies, and social media content.\n\n### 3. Audio Emotion API\n**Functionality:** Analyzes emotions in audio recordings, either via direct uploads or by extracting audio from YouTube videos.\n\n**Insights Provided:**\n- Differentiation between multiple speakers in a conversation\n- Emotion detection from vocal cues (e.g., happiness, sadness, anger, surprise)\n- Transcription and detection of up to 28 nuanced emotions (e.g., admiration, curiosity, disappointment, pride)\n- Audio summaries and detailed analytics\n\n**Use Cases:** Provides emotional insights for podcasts, voice messages, call analysis, and other audio-based content.\n\n### 4. Text Emotion API\n**Functionality:** Analyzes emotions in textual content.\n\n**Insights Provided:**\n- Detection of 28 distinct emotions, including curiosity, disappointment, and excitement\n- Overall sentiment analysis and identification of dominant emotions at sentence level\n\n**Use Cases:** Integrates into chatbots, social media platforms, and other text-based applications for sentiment and emotion analysis.\n\n---\n\n## Authorization\n\nLearn how to obtain your API keys to access Imentiv AI APIs.\n\n### Generate API Key:\n\n1. **Sign up** at [imentiv.ai](https://imentiv.ai) and log in to your account.\n2. Navigate to the **'My Profile'** tab to obtain your API key.\n3. Use this API key to authorize your API calls.\n\n> **Note:** Keep your API key secure and do not share it publicly. Include it in your API requests using the `Authorization` header or as specified in the endpoint documentation.\n "
version: 1.0.0
tags:
- name: Audio Emotion API
description: "The Audio Emotion Analysis API processes audio files or YouTube URLs to analyze emotions and transcripts. The API provides:\n\n**Speaker Diarization:** Identifies and segments audio by speakers.\n\n**Audio Emotion Analysis:** Detects one of 8 emotions per audio segment.\n\n**Text Emotion Analysis:** Generates a transcript for each segment and detects one of 28 emotions from the text.\n "
paths:
/v2/audios:
post:
tags:
- Audio Emotion API
summary: Upload Audio
description: 'Analyze emotions and speech patterns from audio content.
Submit audio from multiple sources and receive detailed emotion analysis,
speaker identification, and transcription with emotional insights.
**Input Sources:**
- **YouTube**: Provide any YouTube video URL (audio will be extracted)
- **Instagram**: Instagram Reels and video posts (public content)
- **Twitter/X**: Twitter video posts and media content
- **Facebook**: Facebook video posts (public content)
- **Cloud Storage**: Dropbox, Google Drive, or other cloud service links
- **Direct URL**: Any publicly accessible audio or video link
- **File Upload**: Upload audio files directly from your device
**Audio Extraction:**
Video files are supported for URLs only (YouTube, Instagram, Twitter, Facebook,
Dropbox, direct links) - the audio track will be automatically extracted and analyzed.
For file uploads, use audio formats only (MP3, WAV, AAC, M4A).
**Analyze Specific Segments:**
Focus on specific portions using `start_millis` and `end_millis` parameters.
This helps target key moments and optimize credit usage.
**How It Works:**
1. Submit your audio and receive an immediate response with an audio ID
2. Audio is queued for processing (typically completes within minutes)
3. Optionally receive a webhook notification via `callback_url` when complete
4. Retrieve results using the audio ID
**Supported Formats:**
- File Upload: MP3, WAV, AAC, M4A
- Media URLs: MP3, WAV, AAC, M4A, MP4, AVI, MOV (audio extracted from video URLs)
**Credit Usage:**
Processing time is billed based on the analyzed audio duration.
**Extra Parameters:**
- **speaker_diarization** (bool): Enable or disable speaker diarization for identifying different speakers in the audio. Default is True.
- **text_emotion_analysis** (bool): Enable or disable transcript emotion analysis for the transcribed text. Default is True.
- **language** (str, optional): Language code for the audio (e.g., ''en''). If not provided, language will be auto-detected.'
operationId: upload_audio_v2_audios_post
requestBody:
content:
multipart/form-data:
schema:
$ref: '#/components/schemas/Body_upload_audio_v2_audios_post'
required: true
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/AudioEmotionResponseV2'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- APIKeyHeader: []
/v1/audios:
get:
tags:
- Audio Emotion API
summary: List Audios
description: API to get audios processed by user with pagination.
operationId: list_audios_v1_audios_get
security:
- APIKeyHeader: []
parameters:
- name: offset_audio_id
in: query
required: false
schema:
type: string
description: The ID of the reference audio from which to paginate forward/backward
title: Offset Audio Id
description: The ID of the reference audio from which to paginate forward/backward
- name: page_size
in: query
required: false
schema:
type: integer
description: 'Number of results to return (default: 20, max: 100)'
default: 10
title: Page Size
description: 'Number of results to return (default: 20, max: 100)'
- name: direction
in: query
required: false
schema:
type: string
description: 'Pagination direction: forward (default) or backward'
default: forward
title: Direction
description: 'Pagination direction: forward (default) or backward'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PaginatedResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}:
get:
tags:
- Audio Emotion API
summary: Get Audio Emotion Insights
description: Retrieve Single Audio based on audio_id.
operationId: get_audio_emotion_insights_v1_audios__audio_id__get
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: The unique ID of the uploaded audio
title: Audio Id
description: The unique ID of the uploaded audio
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/Audio'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
put:
tags:
- Audio Emotion API
summary: Reprocess Audio
description: Reprocess an audio file based on audio id.
operationId: reprocess_audio_v1_audios__audio_id__put
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio to reprocess
title: Audio Id
description: Unique identifier of the audio to reprocess
requestBody:
content:
application/x-www-form-urlencoded:
schema:
$ref: '#/components/schemas/Body_reprocess_audio_v1_audios__audio_id__put'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/AudioReprocessResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
delete:
tags:
- Audio Emotion API
summary: Delete An Audio
description: Deletes a specific audio file and all associated emotion analysis data from the system. This action is irreversible.
operationId: delete_an_audio_v1_audios__audio_id__delete
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio to be deleted
title: Audio Id
description: Unique identifier of the audio to be deleted
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/AudioDeleteResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v2/audios/{audio_id}/multimodal-analytics:
get:
tags:
- Audio Emotion API
summary: Get Audio Multimodal Insights
description: 'Get complete multimodal analytics for an audio file including:
- Audio emotion analysis (speaker-by-speaker)
- Text sentiment analysis from transcription
- Detected speakers
- Overall summaries'
operationId: get_audio_multimodal_insights_v2_audios__audio_id__multimodal_analytics_get
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: The unique ID of the audio
title: Audio Id
description: The unique ID of the audio
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/FullAudioAnalyticsResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/{segment_index}:
delete:
tags:
- Audio Emotion API
summary: Delete Audio Segment
description: 'Deletes a specific segment of an audio file based on the segment index. After deletion,
the remaining segments are reindexed to fill the gap — e.g., if segment 1 is deleted, segment 2 becomes 1'
operationId: delete_audio_segment_v1_audios__audio_id___segment_index__delete
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio file
title: Audio Id
description: Unique identifier of the audio file
- name: segment_index
in: path
required: true
schema:
type: integer
description: The index of the segment to delete (0-based indexing)
title: Segment Index
description: The index of the segment to delete (0-based indexing)
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/AudioSegmentDeleteResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/transcript:
put:
tags:
- Audio Emotion API
summary: Update Audio Transcripts Segments
description: 'Update the transcript and timestamp data for specific segments within an audio file.
This is useful when refining speaker diarization, correcting transcription errors,
or uploading external transcript data.'
operationId: update_audio_transcripts_segments_v1_audios__audio_id__transcript_put
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio file
title: Audio Id
description: Unique identifier of the audio file
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/TranscriptSegment'
description: List of transcript segments to update.
title: Transcript Segments
responses:
'200':
description: Successful Response
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/TranscriptSegment'
title: Response Update Audio Transcripts Segments V1 Audios Audio Id Transcript Put
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/speakers/names:
put:
tags:
- Audio Emotion API
summary: Update Speaker Names
description: 'Update the names of detected speakers in an audio file. This is useful for labeling anonymized
speaker IDs with human-readable names (e.g., "John", "Customer", "Agent").'
operationId: update_speaker_names_v1_audios__audio_id__speakers_names_put
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio file
title: Audio Id
description: Unique identifier of the audio file
requestBody:
required: true
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/SpeakerUpdate'
description: List of speaker ID and name pairs to update
title: Speaker Updates
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/SpeakerUpdateResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/metadata:
put:
tags:
- Audio Emotion API
summary: Edit Title Or Description
description: 'Update the metadata of an audio file.
This endpoint allows you to edit the title (which is auto-generated from the file name by default)
and add or update a description.'
operationId: edit_title_or_description_v1_audios__audio_id__metadata_put
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Provide the audio ID to edit the title or description of the audio
title: Audio Id
description: Provide the audio ID to edit the title or description of the audio
requestBody:
content:
application/x-www-form-urlencoded:
schema:
$ref: '#/components/schemas/Body_edit_title_or_description_v1_audios__audio_id__metadata_put'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/AudioEditResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/report:
get:
tags:
- Audio Emotion API
summary: Download Audio Emotion Report
description: 'Download a detailed report of the emotional insights extracted from a specific audio file.
The report includes average emotions, segment-level analysis, speaker information,
and summary—formatted as a downloadable file (e.g., PDF or CSV).'
operationId: download_audio_emotion_report_v1_audios__audio_id__report_get
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio file
title: Audio Id
description: Unique identifier of the audio file
responses:
'200':
description: Successful Response
content:
application/json:
schema: {}
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
/v1/audios/{audio_id}/transcript/replace:
put:
tags:
- Audio Emotion API
summary: Replace Audio Transcript
operationId: replace_audio_transcript_v1_audios__audio_id__transcript_replace_put
security:
- APIKeyHeader: []
parameters:
- name: audio_id
in: path
required: true
schema:
type: string
description: Unique identifier of the audio file
title: Audio Id
description: Unique identifier of the audio file
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/ReplaceTranscriptRequest'
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/ReplaceTranscriptResponse'
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
components:
schemas:
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
PaginatedResponse:
properties:
documents:
title: Documents
count:
type: integer
title: Count
type: object
required:
- documents
- count
title: PaginatedResponse
description: "Model for paginated audio emotion responses.\n\nAttributes:\n documents (List[AudioEmotionResponse]): List of audio responses.\n count (int): Total count of documents."
EmotionAnalysis:
properties:
overall:
$ref: '#/components/schemas/OverallEmotions'
examples:
- audio:
angry: 0.047
happy: 0.185
neutral: 0.638
text:
admiration: 0.015
joy: 0.018
neutral: 0.75
video:
angry: 0.059
happy: 0.175
neutral: 0.408
sad: 0.049
faces:
anyOf:
- additionalProperties:
additionalProperties:
type: number
type: object
type: object
- type: 'null'
title: Faces
examples:
- John Smith:
happy: 0.175
neutral: 0.408
sad: 0.049
face_2:
angry: 0.087
happy: 0.184
neutral: 0.314
speakers:
additionalProperties:
$ref: '#/components/schemas/SpeakerEmotions'
type: object
title: Speakers
examples:
- Interviewer:
audio:
angry: 0.05
happy: 0.18
neutral: 0.65
text:
curiosity: 0.08
neutral: 0.85
John Smith:
audio:
angry: 0.045
happy: 0.19
neutral: 0.625
text:
admiration: 0.02
joy: 0.025
neutral: 0.72
type: object
required:
- overall
title: EmotionAnalysis
description: Hierarchical emotion analysis structure.
Body_upload_audio_v2_audios_post:
properties:
title:
type: string
minLength: 1
title: Title
description: A descriptive title for your audio (e.g., 'Customer Call' or 'Team Meeting').
description:
type: string
title: Description
description: Optional notes about the audio content or context.
media_url:
type: string
title: Media Url
description: URL to your audio/video source (YouTube, Instagram, Twitter, Facebook, Dropbox, or any direct media link). Audio will be extracted if source is video. Leave empty if uploading a file instead.
default: ''
start_millis:
type: number
title: Start Millis
description: Start point in milliseconds to begin analysis. Use 0 to start from the beginning. If provided (>0), end_millis is required with minimum 1-second (1000ms) gap.
default: 0
end_millis:
type: number
title: End Millis
description: End point in milliseconds to stop analysis. Use 0 only when analyzing the full audio (start_millis must also be 0). If start_millis is provided, end_millis is required and must be at least 1000ms greater than start_millis.
default: 0
callback_url:
type: string
title: Callback Url
description: Your webhook URL to receive a notification when analysis completes.
default: ''
audio_file:
title: Audio File
description: Upload an audio file directly (mp3, wav, aac, m4a). File upload takes priority over media_url.
speaker_diarization:
type: boolean
title: Speaker Diarization
description: Speaker diarization. Enabled by default.
default: true
text_emotion_analysis:
type: boolean
title: Text Emotion Analysis
description: Text emotion analysis. Enabled by default.
default: true
language:
anyOf:
- type: string
- type: 'null'
title: Language
description: Language code for analysis. Auto-detected if not provided.
type: object
required:
- title
title: Body_upload_audio_v2_audios_post
SpeakerUpdateResponse:
properties:
speakers:
items:
$ref: '#/components/schemas/SpeakerUpdate'
type: array
title: Speakers
type: object
required:
- speakers
title: SpeakerUpdateResponse
description: "Response model after updating speakers.\n\nAttributes:\n speakers (List[SpeakerUpdate]): List of updated speakers."
FullAudioAnalyticsResponse:
properties:
title:
type: string
title: Title
default: ''
examples:
- Podcast Episode 42
status:
type: string
title: Status
default: ''
examples:
- completed
description:
type: string
title: Description
default: ''
examples:
- Discussion about AI ethics
created_at:
type: string
title: Created At
default: ''
examples:
- '2025-12-01T10:15:00Z'
media_type:
type: string
title: Media Type
default: audio
examples:
- audio
audio_id:
type: string
title: Audio Id
default: ''
examples:
- 1154940c-473f-46cb-a32d-923e6f23bedc
text_id:
type: string
title: Text Id
default: ''
examples:
- c74a3fe1-e8a3-4c7a-98c9-a496ee4a3fb8
duration:
type: number
title: Duration
default: 0
examples:
- 3245.5
speaker_count:
type: integer
title: Speaker Count
default: 0
examples:
- 2
media_source_url:
type: string
title: Media Source Url
default: ''
examples:
- https://storage.googleapis.com/.../audio.mp3
youtube_url:
anyOf:
- type: string
- type: 'null'
title: Youtube Url
examples:
- https://youtube.com/watch?v=...
detected_speakers:
items:
additionalProperties: true
type: object
type: array
title: Detected Speakers
examples:
- - name: Alice Johnson
speaker_id: 1
- name: Bob Williams
speaker_id: 2
emotion_analysis:
$ref: '#/components/schemas/EmotionAnalysis'
examples:
- overall:
audio:
angry: 0.025
happy: 0.28
neutral: 0.55
text:
admiration: 0.045
joy: 0.04
neutral: 0.45
speakers:
Alice Johnson:
audio:
angry: 0.02
happy: 0.32
neutral: 0.52
text:
admiration: 0.06
joy: 0.055
neutral: 0.4
Bob Williams:
audio:
angry: 0.03
happy: 0.24
neutral: 0.58
text:
amusement: 0.07
neutral: 0.5
type: object
required:
- emotion_analysis
title: FullAudioAnalyticsResponse
description: Complete multimodal analytics response for audio content.
AudioReprocessResponse:
properties:
id:
type: string
title: Id
status:
type: string
title: Status
message:
type: string
title: Message
type: object
required:
- id
- status
- message
title: AudioReprocessResponse
description: "Response model for audio reprocessing.\n\nInherits:\n TextReprocessResponse: Reuse response format for audio reprocessing."
ReplaceTranscriptRequest:
properties:
transcript:
type: string
minLength: 1
title: Transcript
format:
type: string
const: text
title: Format
default: text
type: object
required:
- transcript
title: ReplaceTranscriptRequest
AudioEditResponse:
properties:
title:
anyOf:
- type: string
- type: 'null'
title: Title
description:
anyOf:
- type: string
- type: 'null'
title: Description
type: object
title: AudioEditResponse
description: "Response model for editing audio data.\n\nInherits:\n ImageEditResponse: Contains status and metadata after editing."
AudioDeleteResponse:
properties:
id:
type: string
title: Id
status:
type: string
title: Status
type: object
required:
- id
- status
title: AudioDeleteResponse
description: "Response model after deleting audio.\n\nInherits:\n ResponseStatus: Generic success/error status model."
SpeakerEmotions:
properties:
audio:
additionalProperties:
type: number
type: object
title: Audio
examples:
- angry: 0.045
happy: 0.19
neutral: 0.625
text:
additionalProperties:
type: number
type: object
title: Text
examples:
- admiration: 0.02
joy: 0.025
neutral: 0.72
type: object
title: SpeakerEmotions
description: Speaker-specific emotions across audio and text modalities.
Body_reprocess_audio_v1_audios__audio_id__put:
properties:
callback_url:
type: string
title: Callback Url
description: The callback url to notify audio reprocessing is completed. Give a valid URL.
reprocess:
type: boolean
title: Reprocess
description: Set to 'True' to reprocess an already completed audio; defaults to 'False'.
default: false
type: object
title: Body_reprocess_audio_v1_audios__audio_id__put
TranscriptSegment:
properties:
start_millis:
type: number
minimum: 0
title: Start Millis
description: Start time in milliseconds (>= 0)
end_millis:
type: number
minimum: 0
title: End Millis
description: End time in milliseconds (>= 0)
segment:
type: string
title: Segment
speaker_id:
type: integer
title: Speaker Id
type: object
required:
- start_millis
- end_millis
- segment
- speaker_id
title: TranscriptSegment
description: "Represents a single segment of transcribed audio with timing and speaker ID.\n\nAttributes:\n start_millis (float): Start time of the segment in milliseconds (must be >= 0).\n end_millis (fl
# --- truncated at 32 KB (39 KB total) ---
# Full source: https://raw.githubusercontent.com/api-evangelist/imentiv-ai/refs/heads/main/openapi/imentiv-ai-audio-emotion-api-api-openapi.yml