OpenAIEndpointRealtime
Wisej.AI.Endpoints.OpenAIEndpointRealtime
Last updated
Wisej.AI.Endpoints.OpenAIEndpointRealtime
Last updated
Namespace: Wisej.AI.Endpoints
Assembly: Wisej.AI (3.5.0.0)
Represents a component that manages communication with OpenAI's real-time API endpoints.
provides a convenient way to connect to and interact with OpenAI's real-time endpoints within a Wisej application. It extends , inheriting HTTP communication capabilities and adding specialized logic for real-time data exchange with OpenAI services. This component can be used to send prompts, receive streaming responses, and manage session state with OpenAI's real-time APIs.
Initializes a new instance of the class with default settings.
: Gets or sets the model used for chat completions. (Default: "gpt-4o-realtime-preview"
)
Adds options to the message payload.
message
The message payload to modify.
session
The current session context.
Sets default values for temperature and max tokens in the message payload.
Constructs the instruction string for the OpenAI real-time API endpoint based on the current session context.
session
This method generates the instruction prompt by extracting the system message from the session, incorporating tool prompts, and replacing template placeholders such as {{tools}}
, {{today}}
, and {{language}}
with their corresponding values. The resulting instruction string is then processed to replace any additional session parameters.
Override this method to customize how instructions are built for the OpenAI endpoint.
Builds the tools payload based on the session context.
session
The session context.
Throws:
Creates the configuration payload for the OpenAI real-time API session.
session
This method builds a dynamic configuration object that includes options, tools, instructions, and modalities required by the OpenAI realtime API. If audio transcription is enabled, the configuration will also include the transcription model. The transcription_enabled
property is removed from the final configuration payload.
Disables the OpenAIEndpointRealtime component, stopping it from processing or sending further requests.
Releases the unmanaged resources used by the component and optionally releases the managed resources.
disposing
true to release both managed and unmanaged resources; false to release only unmanaged resources.
Enables the OpenAIEndpointRealtime component, allowing it to process and send requests.
This method activates the component, allowing it to resume communication with the OpenAI Realtime API endpoint after being disabled.
Constructs the API URL for the real time api.
Asynchronously retrieves an ephemeral token for the specified session from the OpenAI realtime API endpoint.
session
This method constructs a request to the OpenAI real-time API endpoint to obtain a temporary (ephemeral) token associated with the provided session. The token is typically used for short-lived authentication or authorization purposes during real-time communication. The method sends a POST request with the session configuration and parses the response as a dynamic object.
Mutes the OpenAIEndpointRealtime component, suppressing any audio or notification output.
e
This method is called to notify subscribers that a function call has been detected in the AI's response. Override this method to provide custom handling for function calls.
e
e
e
Reads and updates the usage statistics from the OpenAI real-time API reply.
message
The message object to update with usage data.
reply
The dynamic object containing the API response, expected to include input_tokens
and output_tokens
properties.
Sends an array of messages to the OpenAI Realtime API endpoint for processing.
messages
An array of dynamic message objects to be sent to the OpenAI endpoint.
Use this method to transmit one or more messages to the OpenAI Realtime API endpoint. Each message should conform to the expected structure required by the endpoint.
Initiates listening for real-time events from the OpenAI endpoint using the specified session.
session
Stops listening for real-time events from the OpenAI endpoint for the specified session.
session
Unmutes the OpenAIEndpointRealtime component, restoring audio or notification output.
This event is triggered when the SmartRealtimeAdapter identifies a function call in the AI's response, allowing the application to handle or execute the function as needed.
: Gets or sets the model used for the audio transcription. (Default: "gpt-4o-transcribe"
)
: Gets or sets the base URL for the OpenAI API. (Default: "https://api.openai.com/v1"
)
: (Default: True
)
The instance containing the session context and parameters used to build the instruction string.
Returns: . A string containing the fully constructed instructions for the OpenAI real-time API endpoint.
Returns: . A JSON object representing the tools payload.
Thrown when the session is null.
The instance containing session-specific options and settings.
Returns: . A dynamic object representing the configuration payload for the session.
This method deactivates the component, preventing any further communication with the OpenAI Realtime API endpoint until is called.
Returns: . The full API URL for the real time api..
The instance representing the current user session for which the ephemeral token is requested.
Returns: . A task that represents the asynchronous operation. The task result contains a dynamic object representing the ephemeral token response from the API.
Use this method to temporarily silence the component without disabling its core functionality. To restore audio or notifications, call .
Raises the event.
A instance containing the event data.
Raises the event.
An instance containing the event data.
Raises the event.
A instance containing the event data for the completed response.
Raises the event.
A instance containing the event data for the completed transcription.
The instance representing the current user session. This parameter cannot be null
.
This method establishes a connection to the OpenAI real-time API endpoint by obtaining an ephemeral token for the provided session and starting the listening process. It also triggers the OnStart
event for the session and updates the associated component in the Wisej application.
If session is null
, an is thrown.
The instance representing the current user session. This parameter cannot be null
.
This method terminates the connection to the OpenAI real-time API endpoint for the given session. It also triggers the OnDone
event for the session to indicate that listening has stopped.
If session is null
, an is thrown.
This method re-enables audio or notifications that were previously suppressed by .
Occurs when a function call is detected in the response from the OpenAI Realtime API.
Occurs when a new response is created by the OpenAI real-time endpoint.
Occurs when the response from the OpenAI real-time endpoint has been fully received and processing is complete.
Occurs when a transcription operation has completed.