Stream Calls

Stream WebSocket

The second step of a stream call. Create Stream Token returns the url and a token: open url?token=<token>, for example wss://agents.nlpearl.ai/Stream/WebSocket?token=wst_..., within 60 seconds, and the conversation runs on this connection until the call ends.

Always open the url that Create Stream Token returns rather than a fixed address. No subprotocol is needed. From a server you can send the token in the Authorization: Bearer wst_... header instead of the query. A browser does not show the status of a refused connection: the socket closes at once with code 1006.

Send

Your app sends two kinds of messages: binary messages with the end user’s audio, and text messages with JSON events.

Message	Kind	When
Audio frame	Binary	Every 20 ms, from `ready` until the call ends
`hangup`	Text (JSON)	To end the call
`ping`	Text (JSON)	Optional, to time the round trip

Audio frames

Send 20 ms of the end user’s audio in each binary message, one message every 20 ms, from ready until the call ends. Keep sending while the end user is quiet or muted: send silence, not nothing.

What one message holds depends on the call’s audioFormat:

`audioFormat`	One message
`opus_24000`, `opus_8000`	One Opus packet, mono, encoded at the call’s sample rate. Turn discontinuous transmission (DTX) off, so silence is encoded too. Packets of up to 120 ms are accepted.
`ulaw_8000`	160 bytes of μ-law: 160 samples.
`pcm_24000`	960 bytes of 16-bit little-endian PCM: 480 samples.

The raw formats also accept other frame lengths, as long as they hold whole samples. A message is at most 16 KB. Audio formats, below, gives each format’s encoding, sample rate and silence.

`hangup`

Ends the call. Pearl answers with ended (client_hangup), then closes the connection. From your backend, Hang Up Call ends it too.

{ "type": "hangup" }

`ping`

Optional. Pearl answers with pong, which echoes t, so you can time the round trip. t is any number you choose, such as your clock in milliseconds, and may be left out.

{ "type": "ping", "t": 1730624400123 }

You never need ping to keep the call alive: any message, audio included, resets the 60-second no_audio timer.

Pearl ignores unknown event types, and any text that is not a JSON object with a string type.

Receive

What Pearl sends: ready first, then audio and a pong for each of your pings, then at the end an optional transfer and one ended.

`ready`

Sent once, right after you connect.

{
  "type": "ready",
  "protocol": 1,
  "callId": "6720f3c1a8b4e5d6f7a8b9c0",
  "audio": { "encoding": "opus", "sampleRate": 24000, "channels": 1, "frameMs": 20 }
}

protocol: the protocol version, 1.
callId: the call’s id, as in the Create Stream Token response.
audio: the format of every binary message, in both directions: the same object as in the Create Stream Token response.

Audio

20 ms of Pearl’s voice, as a binary message, every 20 ms for the whole call, with silence while it listens. On an Opus call, each message is one 20 ms Opus packet.

Play the frames through a small jitter buffer of 60 to 100 ms. If it grows past about 300 ms, after a network stall, drop the oldest audio.
There is no “clear” or interruption event, and you need none: when the end user talks over Pearl, it stops speaking and the next frames are already silence.

`pong`

The answer to a ping, sent between two audio frames, within about 20 ms. t is the t of your ping, or null when it had none. A ping sent before the call’s audio starts gets no answer.

{ "type": "pong", "t": 1730624400123 }

`transfer`

The flow’s Transfer Call node transferred the call. It is sent once Pearl has finished its transfer announcement, and is followed at once by ended (transfer). Carrying the call on, to a person, a queue or a phone number, is up to you.

{
  "type": "transfer",
  "target": "+14155550123",
  "triggerDescription": "The caller asks for a human agent",
  "warmTransferMessage": "Dana is calling about order A-1234",
  "sipHeaders": {},
  "variables": { "firstName": "Dana", "orderId": "A-1234" }
}

target: the destination set on your Pearl’s transfer, as entered: usually a phone number or a SIP address.
triggerDescription: why the flow transferred.
warmTransferMessage: the warm-transfer message set for the receiving side, or null.
sipHeaders: SIP headers, only when the transfer is set to forward them.
variables: the data Pearl has collected so far, by variable name.

`ended`

The call is over. Pearl then closes the connection with code 1000 and the reason as the close description. durationSec is the whole seconds the call ran, from connection to end.

{ "type": "ended", "reason": "agent_hangup", "durationSec": 94 }

`reason`	Meaning
`client_hangup`	You sent `hangup`, closed the connection, or it dropped.
`agent_hangup`	Pearl ended the call: end of the flow, its silence handling, and so on.
`operator_hangup`	The call was hung up with Hang Up Call or from the NLPearl platform.
`transfer`	The flow transferred the call, right after `transfer`.
`max_duration`	Your Pearl’s maximum call duration was reached.
`no_audio`	No message from you for 60 seconds.
`error`	The call failed on NLPearl’s side.

If you close the connection yourself, or it drops, nothing more arrives: the call is recorded as client_hangup.

Audio formats

audioFormat, chosen in Create Stream Token, sets the format of every frame, the same in both directions. The same format has a name in each place it appears:

`audioFormat` (Create Stream Token)	`audio.encoding` (response and `ready`)	Sample rate	Samples	One 20 ms frame	Silence	`audioQuality` (call history)
`opus_24000`	`opus`	24000 Hz	Opus, mono	One Opus packet, its size varies	Encoded silence	`hd`
`opus_8000`	`opus`	8000 Hz	Opus, mono	One Opus packet, its size varies	Encoded silence	`phone`
`ulaw_8000` (default)	`mulaw`	8000 Hz	G.711 μ-law, 8-bit, mono	160 bytes	`0xFF` bytes	`phone`
`pcm_24000`	`pcm_s16le`	24000 Hz	Signed 16-bit PCM, little-endian, mono	960 bytes (480 samples)	`0x00` bytes	`hd`

Limits

Item	Limit
Token	Single use, valid 60 seconds
Connections per call	1
Concurrent calls	Your Pearl’s agents, shared with its phone calls. One is taken when the token is created.
Call duration	Your Pearl’s Max Call Duration, from 1 to 120 minutes; 30 minutes when it is not set. Reaching it ends the call with `max_duration`.
Silence from your side	60 seconds without any message ends the call with `no_audio`
Message size	16 KB

Versioning

ready.protocol is 1. New event types may be added within a version, which is why unknown types are ignored. A change to the frame format would raise the version.

JavaScript

const ws = new WebSocket(`${call.url}?token=${call.token}`);
ws.binaryType = "arraybuffer";

ws.onmessage = (message) => {
  if (typeof message.data !== "string") {
    play(message.data);                      // 20 ms of Pearl's voice
    return;
  }
  const event = JSON.parse(message.data);    // ready, pong, transfer, ended
};

// After ready, every 20 ms: ws.send(frame) with 20 ms of the end user's audio.
// To hang up: ws.send(JSON.stringify({ type: "hangup" }));

async with websockets.connect(f"{call['url']}?token={call['token']}") as ws:
    ready = json.loads(await ws.recv())      # always the first message
    # every 20 ms: await ws.send(frame) with 20 ms of the end user's audio
    async for message in ws:
        if isinstance(message, bytes):
            play(message)                    # 20 ms of Pearl's voice
        else:
            print(json.loads(message))       # pong, transfer, ended

Query Parameters

token

string

required

The token from Create Stream Token, starting with wst_. It is single use and valid for 60 seconds. From a server you can send it in the Authorization: Bearer wst_... header instead.

Response

Connected. Pearl sends ready at once.