admin/dify
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add the API documentation for streaming TTS (Text-to-Speech) (#6382)

Browse Source
This commit is contained in:
chenxu9741
2024-07-17 19:44:16 +08:00
committed by GitHub
parent f3f052ba36
commit a6dbd26f75
8 changed files with 137 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
web/app/components/develop/template/template.en.mdx
View File

@@ -103,6 +103,16 @@ The text generation application offers non-session support and is ideal for tran
- `metadata` (object) Metadata
- `usage` (Usage) Model usage information
- `retriever_resources` (array[RetrieverResource]) Citation and Attribution List
- `event: tts_message` TTS audio stream event, that is, speech synthesis output. The content is an audio block in Mp3 format, encoded as a base64 string. When playing, simply decode the base64 and feed it into the player. (This message is available only when auto-play is enabled)
- `task_id` (string) Task ID, used for request tracking and the stop response interface below
- `message_id` (string) Unique message ID
- `audio` (string) The audio after speech synthesis, encoded in base64 text content, when playing, simply decode the base64 and feed it into the player
- `created_at` (int) Creation timestamp, e.g.: 1705395332
- `event: tts_message_end` TTS audio stream end event, receiving this event indicates the end of the audio stream.
- `task_id` (string) Task ID, used for request tracking and the stop response interface below
- `message_id` (string) Unique message ID
- `audio` (string) The end event has no audio, so this is an empty string
- `created_at` (int) Creation timestamp, e.g.: 1705395332
- `event: message_replace` Message content replacement event.
When output content moderation is enabled, if the content is flagged, then the message content will be replaced with a preset reply through this event.
- `task_id` (string) Task ID, used for request tracking and the below Stop Generate API
@@ -185,6 +195,8 @@ The text generation application offers non-session support and is ideal for tran
data: {"event": "message", "message_id": : "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " meet", "created_at": 1679586595}
data: {"event": "message", "message_id": : "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " you", "created_at": 1679586595}
data: {"event": "message_end", "id": "5e52ce04-874b-4d27-9045-b3bc80def685", "metadata": {"usage": {"prompt_tokens": 1033, "prompt_unit_price": "0.001", "prompt_price_unit": "0.001", "prompt_price": "0.0010330", "completion_tokens": 135, "completion_unit_price": "0.002", "completion_price_unit": "0.001", "completion_price": "0.0002700", "total_tokens": 1168, "total_price": "0.0013030", "currency": "USD", "latency": 1.381760165997548}}}
data: {"event": "tts_message", "conversation_id": "23dd85f3-1a41-4ea0-b7a9-062734ccfaf9", "message_id": "a8bdc41c-13b2-4c18-bfd9-054b9803038c", "created_at": 1721205487, "task_id": "3bf8a0bb-e73b-4690-9e66-4e429bad8ee7", "audio": "qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq"}
data: {"event": "tts_message_end", "conversation_id": "23dd85f3-1a41-4ea0-b7a9-062734ccfaf9", "message_id": "a8bdc41c-13b2-4c18-bfd9-054b9803038c", "created_at": 1721205487, "task_id": "3bf8a0bb-e73b-4690-9e66-4e429bad8ee7", "audio": ""}
```
</CodeGroup>
</Col>
@@ -495,29 +507,29 @@ The text generation application offers non-session support and is ideal for tran
### Request Body
<Properties>
<Property name='message_id' type='str' key='text'>
For text messages generated by Dify, simply pass the generated message-id directly. The backend will use the message-id to look up the corresponding content and synthesize the voice information directly. If both message_id and text are provided simultaneously, the message_id is given priority.
</Property>
<Property name='text' type='str' key='text'>
Speech generated content。
</Property>
<Property name='user' type='string' key='user'>
The user identifier, defined by the developer, must ensure uniqueness within the app.
</Property>
<Property name='streaming' type='bool' key='streaming'>
Whether to enable streaming output, true、false。
</Property>
</Properties>
</Col>
<Col sticky>
<CodeGroup title="Request" tag="POST" label="/text-to-audio" targetCode={`curl -o text-to-audio.mp3 -X POST '${props.appDetail.api_base_url}/text-to-audio' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{\n "text": "Hello Dify",\n "user": "abc-123",\n "streaming": false\n}'`}>
<CodeGroup title="Request" tag="POST" label="/text-to-audio" targetCode={`curl -o text-to-audio.mp3 -X POST '${props.appDetail.api_base_url}/text-to-audio' \\\n--header 'Authorization: Bearer {api_key}' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{\n "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290",\n "text": "Hello Dify",\n "user": "abc-123"\n}'`}>
```bash {{ title: 'cURL' }}
curl -o text-to-audio.mp3 -X POST '${props.appDetail.api_base_url}/text-to-audio' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{
"message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290",
"text": "Hello Dify",
"user": "abc-123",
"streaming": false
"user": "abc-123"
}'
```