OpenAIEndpointRealtime

Wisej.AI.Endpoints.OpenAIEndpointRealtime

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

• SmartEndpoint

  • SmartHttpEndpoint

    • OpenAIEndpointRealtime

Represents a component that manages communication with OpenAI's real-time API endpoints.

C#

public class OpenAIEndpointRealtime : SmartHttpEndpoint

VB.NET

Public Class OpenAIEndpointRealtime
    Inherits SmartHttpEndpoint

OpenAIEndpointRealtime provides a convenient way to connect to and interact with OpenAI's real-time endpoints within a Wisej application. It extends SmartHttpEndpoint, inheriting HTTP communication capabilities and adding specialized logic for real-time data exchange with OpenAI services. This component can be used to send prompts, receive streaming responses, and manage session state with OpenAI's real-time APIs.

Constructors

OpenAIEndpointRealtime()

Initializes a new instance of the OpenAIEndpoint class with default settings.

Properties

Model

String: Gets or sets the model used for chat completions. (Default: "gpt-4o-realtime-preview")

TranscriptionModel

String: Gets or sets the model used for the audio transcription. (Default: "gpt-4o-transcribe")

URL

String: Gets or sets the base URL for the OpenAI API. (Default: "https://api.openai.com/v1")

UseNativeTools

Boolean: (Default: True)

Methods

AddOptions(message, session)

Adds options to the message payload.

Parameter	Type	Description
message	Object	The message payload to modify.
session	SmartSession	The current session context.

Sets default values for temperature and max tokens in the message payload.

BuildInstructions(session)

Constructs the instruction string for the OpenAI real-time API endpoint based on the current session context.

Parameter	Type	Description
session	SmartSession	The SmartSession instance containing the session context and parameters used to build the instruction string.

Returns: String. A string containing the fully constructed instructions for the OpenAI real-time API endpoint.

This method generates the instruction prompt by extracting the system message from the session, incorporating tool prompts, and replacing template placeholders such as {{tools}}, {{today}}, and {{language}} with their corresponding values. The resulting instruction string is then processed to replace any additional session parameters. Override this method to customize how instructions are built for the OpenAI endpoint.

BuildToolsPayload(session)

Builds the tools payload based on the session context.

Parameter	Type	Description
session	SmartSession	The session context.

Returns: Object[]. A JSON object representing the tools payload.

Throws:

• ArgumentNullException Thrown when the session is null.

CreateConfiguration(session)

Creates the configuration payload for the OpenAI real-time API session.

Parameter	Type	Description
session	SmartSession	The SmartSession instance containing session-specific options and settings.

Returns: Object. A dynamic object representing the configuration payload for the session.

This method builds a dynamic configuration object that includes options, tools, instructions, and modalities required by the OpenAI realtime API. If audio transcription is enabled, the configuration will also include the transcription model. The transcription_enabled property is removed from the final configuration payload.

Disable()

Disables the OpenAIEndpointRealtime component, stopping it from processing or sending further requests.

This method deactivates the component, preventing any further communication with the OpenAI Realtime API endpoint until Enable is called.

Dispose(disposing)

Releases the unmanaged resources used by the component and optionally releases the managed resources.

Parameter	Type	Description
disposing	Boolean	true to release both managed and unmanaged resources; false to release only unmanaged resources.

Enable()

Enables the OpenAIEndpointRealtime component, allowing it to process and send requests.

This method activates the component, allowing it to resume communication with the OpenAI Realtime API endpoint after being disabled.

GetApiURL()

Constructs the API URL for the real time api.

Returns: String. The full API URL for the real time api..

GetEphemeralTokenAsync(session)

Asynchronously retrieves an ephemeral token for the specified session from the OpenAI realtime API endpoint.

Parameter	Type	Description
session	SmartSession	The SmartSession instance representing the current user session for which the ephemeral token is requested.

Returns: Task<Object>. A task that represents the asynchronous operation. The task result contains a dynamic object representing the ephemeral token response from the API.

This method constructs a request to the OpenAI real-time API endpoint to obtain a temporary (ephemeral) token associated with the provided session. The token is typically used for short-lived authentication or authorization purposes during real-time communication. The method sends a POST request with the session configuration and parses the response as a dynamic object.

Mute()

Mutes the OpenAIEndpointRealtime component, suppressing any audio or notification output.

Use this method to temporarily silence the component without disabling its core functionality. To restore audio or notifications, call Unmute.

OnFunctionCall(e)

Raises the FunctionCall event.

Parameter	Type	Description
e	FunctionCallEventArgs	A FunctionCallEventArgs instance containing the event data.

This method is called to notify subscribers that a function call has been detected in the AI's response. Override this method to provide custom handling for function calls.

OnResponseCreated(e)

Raises the ResponseCreated event.

Parameter	Type	Description
e	EventArgs	An EventArgs instance containing the event data.

OnResponseDone(e)

Raises the ResponseDone event.

Parameter	Type	Description
e	ResponseDoneEventArgs	A ResponseDoneEventArgs instance containing the event data for the completed response.

OnTranscriptionCompleted(e)

Raises the TranscriptionCompleted event.

Parameter	Type	Description
e	TranscriptionCompletedEventArgs	A TranscriptionCompletedEventArgs instance containing the event data for the completed transcription.

ReadUsage(message, reply)

Reads and updates the usage statistics from the OpenAI real-time API reply.

Parameter	Type	Description
message	Message	The message object to update with usage data.
reply	Object	The dynamic object containing the API response, expected to include input_tokens and output_tokens properties.

SendMessages(messages)

Sends an array of messages to the OpenAI Realtime API endpoint for processing.

Parameter	Type	Description
messages	Object[]	An array of dynamic message objects to be sent to the OpenAI endpoint.

Use this method to transmit one or more messages to the OpenAI Realtime API endpoint. Each message should conform to the expected structure required by the endpoint.

StartListening(session)

Initiates listening for real-time events from the OpenAI endpoint using the specified session.

Parameter	Type	Description
session	SmartSession	The SmartSession instance representing the current user session. This parameter cannot be null.

This method establishes a connection to the OpenAI real-time API endpoint by obtaining an ephemeral token for the provided session and starting the listening process. It also triggers the OnStart event for the session and updates the associated component in the Wisej application. If session is null, an ArgumentNullException is thrown.

StopListening(session)

Stops listening for real-time events from the OpenAI endpoint for the specified session.

Parameter	Type	Description
session	SmartSession	The SmartSession instance representing the current user session. This parameter cannot be null.

This method terminates the connection to the OpenAI real-time API endpoint for the given session. It also triggers the OnDone event for the session to indicate that listening has stopped. If session is null, an ArgumentNullException is thrown.

Unmute()

Unmutes the OpenAIEndpointRealtime component, restoring audio or notification output.

This method re-enables audio or notifications that were previously suppressed by Mute.

Events

FunctionCall

FunctionCallEventHandler Occurs when a function call is detected in the response from the OpenAI Realtime API.

This event is triggered when the SmartRealtimeAdapter identifies a function call in the AI's response, allowing the application to handle or execute the function as needed.

ResponseCreated

EventHandler Occurs when a new response is created by the OpenAI real-time endpoint.

ResponseDone

ResponseDoneEventHandler Occurs when the response from the OpenAI real-time endpoint has been fully received and processing is complete.

TranscriptionCompleted

TranscriptionCompletedEventHandler Occurs when a transcription operation has completed.