> ## Documentation Index > Fetch the complete documentation index at: https://developers.nlpearl.ai/llms.txt > Use this file to discover all available pages before exploring further. # Stream WebSocket > The second step of a stream call. [Create Stream Token](/api-reference/v2/stream/create-token) returns the `url` and a `token`: open `url?token=`, for example `wss://agents.nlpearl.ai/Stream/WebSocket?token=wst_...`, within 60 seconds, and the conversation runs on this connection until the call ends. Always open the `url` that Create Stream Token returns rather than a fixed address. No subprotocol is needed. From a server you can send the token in the `Authorization: Bearer wst_...` header instead of the query. A browser does not show the status of a refused connection: the socket closes at once with code `1006`. ## Send Your app sends two kinds of messages: **binary** messages with the end user's audio, and **text** messages with JSON events. | Message | Kind | When | | --- | --- | --- | | Audio frame | Binary | Every 20 ms, from `ready` until the call ends | | `hangup` | Text (JSON) | To end the call | | `ping` | Text (JSON) | Optional, to time the round trip | ### Audio frames Send 20 ms of the end user's audio in each binary message, one message every 20 ms, from `ready` until the call ends. Keep sending while the end user is quiet or muted: send silence, not nothing. What one message holds depends on the call's `audioFormat`: | `audioFormat` | One message | | --- | --- | | `opus_24000`, `opus_8000` | One Opus packet, mono, encoded at the call's sample rate. Turn discontinuous transmission (DTX) off, so silence is encoded too. Packets of up to 120 ms are accepted. | | `ulaw_8000` | 160 bytes of μ-law: 160 samples. | | `pcm_24000` | 960 bytes of 16-bit little-endian PCM: 480 samples. | The raw formats also accept other frame lengths, as long as they hold whole samples. A message is at most 16 KB. Audio formats, below, gives each format's encoding, sample rate and silence. ### `hangup` Ends the call. Pearl answers with `ended` (`client_hangup`), then closes the connection. From your backend, [Hang Up Call](/api-reference/v2/call/hangup-call) ends it too. ```json { "type": "hangup" } ``` ### `ping` Optional. Pearl answers with `pong`, which echoes `t`, so you can time the round trip. `t` is any number you choose, such as your clock in milliseconds, and may be left out. ```json { "type": "ping", "t": 1730624400123 } ``` You never need `ping` to keep the call alive: any message, audio included, resets the 60-second `no_audio` timer. Pearl ignores unknown event types, and any text that is not a JSON object with a string `type`. ## Receive What Pearl sends: `ready` first, then audio and a `pong` for each of your pings, then at the end an optional `transfer` and one `ended`. ### `ready` Sent once, right after you connect. ```json { "type": "ready", "protocol": 1, "callId": "6720f3c1a8b4e5d6f7a8b9c0", "audio": { "encoding": "opus", "sampleRate": 24000, "channels": 1, "frameMs": 20 } } ``` - `protocol`: the protocol version, `1`. - `callId`: the call's id, as in the Create Stream Token response. - `audio`: the format of every binary message, in both directions: the same object as in the Create Stream Token response. ### Audio 20 ms of Pearl's voice, as a binary message, **every 20 ms for the whole call**, with silence while it listens. On an Opus call, each message is one 20 ms Opus packet. - Play the frames through a small jitter buffer of 60 to 100 ms. If it grows past about 300 ms, after a network stall, drop the oldest audio. - There is no "clear" or interruption event, and you need none: when the end user talks over Pearl, it stops speaking and the next frames are already silence. ### `pong` The answer to a `ping`, sent between two audio frames, within about 20 ms. `t` is the `t` of your `ping`, or `null` when it had none. A `ping` sent before the call's audio starts gets no answer. ```json { "type": "pong", "t": 1730624400123 } ``` ### `transfer` The flow's [Transfer Call](/pages/transfer_call) node transferred the call. It is sent once Pearl has finished its transfer announcement, and is followed at once by `ended` (`transfer`). Carrying the call on, to a person, a queue or a phone number, is up to you. ```json { "type": "transfer", "target": "+14155550123", "triggerDescription": "The caller asks for a human agent", "warmTransferMessage": "Dana is calling about order A-1234", "sipHeaders": {}, "variables": { "firstName": "Dana", "orderId": "A-1234" } } ``` - `target`: the destination set on your Pearl's transfer, as entered: usually a phone number or a SIP address. - `triggerDescription`: why the flow transferred. - `warmTransferMessage`: the warm-transfer message set for the receiving side, or `null`. - `sipHeaders`: SIP headers, only when the transfer is set to forward them. - `variables`: the data Pearl has collected so far, by variable name. ### `ended` The call is over. Pearl then closes the connection with code `1000` and the reason as the close description. `durationSec` is the whole seconds the call ran, from connection to end. ```json { "type": "ended", "reason": "agent_hangup", "durationSec": 94 } ``` | `reason` | Meaning | | --- | --- | | `client_hangup` | You sent `hangup`, closed the connection, or it dropped. | | `agent_hangup` | Pearl ended the call: end of the flow, its silence handling, and so on. | | `operator_hangup` | The call was hung up with [Hang Up Call](/api-reference/v2/call/hangup-call) or from the NLPearl platform. | | `transfer` | The flow transferred the call, right after `transfer`. | | `max_duration` | Your Pearl's maximum call duration was reached. | | `no_audio` | No message from you for 60 seconds. | | `error` | The call failed on NLPearl's side. | If you close the connection yourself, or it drops, nothing more arrives: the call is recorded as `client_hangup`. ## Audio formats `audioFormat`, chosen in Create Stream Token, sets the format of every frame, the same in both directions. The same format has a name in each place it appears: | `audioFormat` (Create Stream Token) | `audio.encoding` (response and `ready`) | Sample rate | Samples | One 20 ms frame | Silence | `audioQuality` (call history) | | --- | --- | --- | --- | --- | --- | --- | | `opus_24000` | `opus` | 24000 Hz | Opus, mono | One Opus packet, its size varies | Encoded silence | `hd` | | `opus_8000` | `opus` | 8000 Hz | Opus, mono | One Opus packet, its size varies | Encoded silence | `phone` | | `ulaw_8000` (default) | `mulaw` | 8000 Hz | G.711 μ-law, 8-bit, mono | 160 bytes | `0xFF` bytes | `phone` | | `pcm_24000` | `pcm_s16le` | 24000 Hz | Signed 16-bit PCM, little-endian, mono | 960 bytes (480 samples) | `0x00` bytes | `hd` | ## Limits | Item | Limit | | --- | --- | | Token | Single use, valid 60 seconds | | Connections per call | 1 | | Concurrent calls | Your Pearl's [agents](/pages/agents), shared with its phone calls. One is taken when the token is created. | | Call duration | Your Pearl's **Max Call Duration**, from 1 to 120 minutes; 30 minutes when it is not set. Reaching it ends the call with `max_duration`. | | Silence from your side | 60 seconds without any message ends the call with `no_audio` | | Message size | 16 KB | ## Versioning `ready.protocol` is `1`. New event types may be added within a version, which is why unknown types are ignored. A change to the frame format would raise the version. ## OpenAPI ````yaml get /Stream/WebSocket openapi: 3.0.1 info: title: NLPearl Client API description: A sample API that demonstrates features in the OpenAPI specification version: v2 servers: - url: https://api.nlpearl.ai description: Production Environement security: - bearer_auth: [] - oauth2_client_credentials: [] tags: - name: Pearl description: >- Manage your Pearls, the AI agents that handle your inbound and outbound calls. Create and configure Pearls, retrieve their details and settings, run or pause them, list and bulk-export their calls, monitor calls in progress, get analytics, and reset a customer's conversation memory. - name: Speech description: >- Turn text into speech with your voices, as a single clip or as a stream, and clone new voices from a recording. paths: /Stream/WebSocket: get: tags: - Stream summary: Stream WebSocket description: "The second step of a stream call. [Create Stream Token](/api-reference/v2/stream/create-token) returns the `url` and a `token`: open `url?token=`, for example `wss://agents.nlpearl.ai/Stream/WebSocket?token=wst_...`, within 60 seconds, and the conversation runs on this connection until the call ends.\r\n\r\nAlways open the `url` that Create Stream Token returns rather than a fixed address. No subprotocol is needed. From a server you can send the token in the `Authorization: Bearer wst_...` header instead of the query. A browser does not show the status of a refused connection: the socket closes at once with code `1006`.\r\n\r\n## Send\r\n\r\nYour app sends two kinds of messages: **binary** messages with the end user's audio, and **text** messages with JSON events.\r\n\r\n| Message | Kind | When |\r\n| --- | --- | --- |\r\n| Audio frame | Binary | Every 20 ms, from `ready` until the call ends |\r\n| `hangup` | Text (JSON) | To end the call |\r\n| `ping` | Text (JSON) | Optional, to time the round trip |\r\n\r\n### Audio frames\r\n\r\nSend 20 ms of the end user's audio in each binary message, one message every 20 ms, from `ready` until the call ends. Keep sending while the end user is quiet or muted: send silence, not nothing.\r\n\r\nWhat one message holds depends on the call's `audioFormat`:\r\n\r\n| `audioFormat` | One message |\r\n| --- | --- |\r\n| `opus_24000`, `opus_8000` | One Opus packet, mono, encoded at the call's sample rate. Turn discontinuous transmission (DTX) off, so silence is encoded too. Packets of up to 120 ms are accepted. |\r\n| `ulaw_8000` | 160 bytes of μ-law: 160 samples. |\r\n| `pcm_24000` | 960 bytes of 16-bit little-endian PCM: 480 samples. |\r\n\r\nThe raw formats also accept other frame lengths, as long as they hold whole samples. A message is at most 16 KB. Audio formats, below, gives each format's encoding, sample rate and silence.\r\n\r\n### `hangup`\r\n\r\nEnds the call. Pearl answers with `ended` (`client_hangup`), then closes the connection. From your backend, [Hang Up Call](/api-reference/v2/call/hangup-call) ends it too.\r\n\r\n```json\r\n{ \"type\": \"hangup\" }\r\n```\r\n\r\n### `ping`\r\n\r\nOptional. Pearl answers with `pong`, which echoes `t`, so you can time the round trip. `t` is any number you choose, such as your clock in milliseconds, and may be left out.\r\n\r\n```json\r\n{ \"type\": \"ping\", \"t\": 1730624400123 }\r\n```\r\n\r\nYou never need `ping` to keep the call alive: any message, audio included, resets the 60-second `no_audio` timer.\r\n\r\nPearl ignores unknown event types, and any text that is not a JSON object with a string `type`.\r\n\r\n## Receive\r\n\r\nWhat Pearl sends: `ready` first, then audio and a `pong` for each of your pings, then at the end an optional `transfer` and one `ended`.\r\n\r\n### `ready`\r\n\r\nSent once, right after you connect.\r\n\r\n```json\r\n{\r\n \"type\": \"ready\",\r\n \"protocol\": 1,\r\n \"callId\": \"6720f3c1a8b4e5d6f7a8b9c0\",\r\n \"audio\": { \"encoding\": \"opus\", \"sampleRate\": 24000, \"channels\": 1, \"frameMs\": 20 }\r\n}\r\n```\r\n\r\n- `protocol`: the protocol version, `1`.\r\n- `callId`: the call's id, as in the Create Stream Token response.\r\n- `audio`: the format of every binary message, in both directions: the same object as in the Create Stream Token response.\r\n\r\n### Audio\r\n\r\n20 ms of Pearl's voice, as a binary message, **every 20 ms for the whole call**, with silence while it listens. On an Opus call, each message is one 20 ms Opus packet.\r\n\r\n- Play the frames through a small jitter buffer of 60 to 100 ms. If it grows past about 300 ms, after a network stall, drop the oldest audio.\r\n- There is no \"clear\" or interruption event, and you need none: when the end user talks over Pearl, it stops speaking and the next frames are already silence.\r\n\r\n### `pong`\r\n\r\nThe answer to a `ping`, sent between two audio frames, within about 20 ms. `t` is the `t` of your `ping`, or `null` when it had none. A `ping` sent before the call's audio starts gets no answer.\r\n\r\n```json\r\n{ \"type\": \"pong\", \"t\": 1730624400123 }\r\n```\r\n\r\n### `transfer`\r\n\r\nThe flow's [Transfer Call](/pages/transfer_call) node transferred the call. It is sent once Pearl has finished its transfer announcement, and is followed at once by `ended` (`transfer`). Carrying the call on, to a person, a queue or a phone number, is up to you.\r\n\r\n```json\r\n{\r\n \"type\": \"transfer\",\r\n \"target\": \"+14155550123\",\r\n \"triggerDescription\": \"The caller asks for a human agent\",\r\n \"warmTransferMessage\": \"Dana is calling about order A-1234\",\r\n \"sipHeaders\": {},\r\n \"variables\": { \"firstName\": \"Dana\", \"orderId\": \"A-1234\" }\r\n}\r\n```\r\n\r\n- `target`: the destination set on your Pearl's transfer, as entered: usually a phone number or a SIP address.\r\n- `triggerDescription`: why the flow transferred.\r\n- `warmTransferMessage`: the warm-transfer message set for the receiving side, or `null`.\r\n- `sipHeaders`: SIP headers, only when the transfer is set to forward them.\r\n- `variables`: the data Pearl has collected so far, by variable name.\r\n\r\n### `ended`\r\n\r\nThe call is over. Pearl then closes the connection with code `1000` and the reason as the close description. `durationSec` is the whole seconds the call ran, from connection to end.\r\n\r\n```json\r\n{ \"type\": \"ended\", \"reason\": \"agent_hangup\", \"durationSec\": 94 }\r\n```\r\n\r\n| `reason` | Meaning |\r\n| --- | --- |\r\n| `client_hangup` | You sent `hangup`, closed the connection, or it dropped. |\r\n| `agent_hangup` | Pearl ended the call: end of the flow, its silence handling, and so on. |\r\n| `operator_hangup` | The call was hung up with [Hang Up Call](/api-reference/v2/call/hangup-call) or from the NLPearl platform. |\r\n| `transfer` | The flow transferred the call, right after `transfer`. |\r\n| `max_duration` | Your Pearl's maximum call duration was reached. |\r\n| `no_audio` | No message from you for 60 seconds. |\r\n| `error` | The call failed on NLPearl's side. |\r\n\r\nIf you close the connection yourself, or it drops, nothing more arrives: the call is recorded as `client_hangup`.\r\n\r\n## Audio formats\r\n\r\n`audioFormat`, chosen in Create Stream Token, sets the format of every frame, the same in both directions. The same format has a name in each place it appears:\r\n\r\n| `audioFormat` (Create Stream Token) | `audio.encoding` (response and `ready`) | Sample rate | Samples | One 20 ms frame | Silence | `audioQuality` (call history) |\r\n| --- | --- | --- | --- | --- | --- | --- |\r\n| `opus_24000` | `opus` | 24000 Hz | Opus, mono | One Opus packet, its size varies | Encoded silence | `hd` |\r\n| `opus_8000` | `opus` | 8000 Hz | Opus, mono | One Opus packet, its size varies | Encoded silence | `phone` |\r\n| `ulaw_8000` (default) | `mulaw` | 8000 Hz | G.711 μ-law, 8-bit, mono | 160 bytes | `0xFF` bytes | `phone` |\r\n| `pcm_24000` | `pcm_s16le` | 24000 Hz | Signed 16-bit PCM, little-endian, mono | 960 bytes (480 samples) | `0x00` bytes | `hd` |\r\n\r\n## Limits\r\n\r\n| Item | Limit |\r\n| --- | --- |\r\n| Token | Single use, valid 60 seconds |\r\n| Connections per call | 1 |\r\n| Concurrent calls | Your Pearl's [agents](/pages/agents), shared with its phone calls. One is taken when the token is created. |\r\n| Call duration | Your Pearl's **Max Call Duration**, from 1 to 120 minutes; 30 minutes when it is not set. Reaching it ends the call with `max_duration`. |\r\n| Silence from your side | 60 seconds without any message ends the call with `no_audio` |\r\n| Message size | 16 KB |\r\n\r\n## Versioning\r\n\r\n`ready.protocol` is `1`. New event types may be added within a version, which is why unknown types are ignored. A change to the frame format would raise the version." parameters: - name: token in: query description: >- The `token` from Create Stream Token, starting with `wst_`. It is single use and valid for 60 seconds. From a server you can send it in the `Authorization: Bearer wst_...` header instead. required: true schema: type: string responses: '101': description: Connected. Pearl sends `ready` at once. '400': description: Not a WebSocket upgrade request. '401': description: >- The token is missing, unknown, expired or already used. Create a new token. '410': description: >- The call was ended before anyone connected, with [Hang Up Call](/api-reference/v2/call/hangup-call) or from the NLPearl platform. Create a new token. security: - {} servers: - url: wss://agents.nlpearl.ai description: The `url` that Create Stream Token returns x-codeSamples: - lang: javascript label: JavaScript source: "const ws = new WebSocket(`${call.url}?token=${call.token}`);\r\nws.binaryType = \"arraybuffer\";\r\n\r\nws.onmessage = (message) => {\r\n if (typeof message.data !== \"string\") {\r\n play(message.data); // 20 ms of Pearl's voice\r\n return;\r\n }\r\n const event = JSON.parse(message.data); // ready, pong, transfer, ended\r\n};\r\n\r\n// After ready, every 20 ms: ws.send(frame) with 20 ms of the end user's audio.\r\n// To hang up: ws.send(JSON.stringify({ type: \"hangup\" }));" - lang: python label: Python source: "async with websockets.connect(f\"{call['url']}?token={call['token']}\") as ws:\r\n ready = json.loads(await ws.recv()) # always the first message\r\n # every 20 ms: await ws.send(frame) with 20 ms of the end user's audio\r\n async for message in ws:\r\n if isinstance(message, bytes):\r\n play(message) # 20 ms of Pearl's voice\r\n else:\r\n print(json.loads(message)) # pong, transfer, ended" components: securitySchemes: bearer_auth: type: http description: >- Specify the authorization token. For more information, check out the [Authorization Guide](/api-reference/authorization). scheme: bearer oauth2_client_credentials: type: oauth2 description: >- OAuth2 client_credentials grant. Use the Id of an OAuth API key as `client_id` and its one-time secret as `client_secret`. flows: clientCredentials: tokenUrl: /oauth/token scopes: pearls:read: List and view Pearls pearls:write: >- Create and update Pearls (settings, conversation flows, voices); implies pearls:read pearls:moderate: Run/pause Pearls and reset customer memory; implies pearls:read pearls:analytics: Read Pearl analytics; implies pearls:read pearls:leads: Manage and read outbound leads; implies pearls:read pearls:calls: >- Read calls and recordings, make and delete calls; implies pearls:read pearls:stream: >- Create stream tokens and run calls over a WebSocket; implies pearls:read account:phone_numbers: >- Read account phone numbers and text channels, manage custom VoIP numbers, import Twilio numbers and delete phone numbers account:users: Read account users account:advanced: Manage advanced account settings (blacklist) account:billing: Read account information and credit balance account:audit_log: Read the audit log tts: Synthesise speech and clone voices ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.