SmartEndpoint

Wisej.AI.SmartEndpoint

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents an abstract base class for a smart endpoint component that interacts with AI models.

C#

public class SmartEndpoint : Component

VB.NET

This class provides properties and methods to configure and interact with AI models, including handling messages, building payloads, and managing tool invocations.

// Example usage:
SmartEndpoint endpoint = new CustomSmartEndpoint();
var response = await endpoint.AskAsync(session, messages);

Constructors

SmartEndpoint()

Initializes a new instance of the SmartEndpoint class.

Properties

ContextWindow

Int32: Gets or sets the size of the context window in tokens. (Default: 32000)

EmbeddingModel

String: Gets or sets the embedding model used by the endpoint. (Default: null)

MaxOutputTokens

Int32: Gets or sets the maximum number of output tokens. (Default: 4096)

Model

String: Gets or sets the model used by the endpoint. (Default: null)

ModelOptions

Object: Gets the model options for the endpoint.

Name

String: Gets or sets the name of the endpoint.

SystemPrompt

String: Gets or sets the system prompt for the endpoint.

Warning, setting this property entirely overrides the internal system prompt.

ToolsPrompt

String: Gets or sets the tools prompt for the endpoint.

Warning, setting this property entirely overrides the internal tools prompt.

Usage

Metrics: Gets the usage metrics for the endpoint.

UseNativeTools

Boolean: Gets or sets a value indicating that the endpoint should use the native tools payload when adding tools to the request. (Default: False)

Methods

AddMessages(payload, session, messages)

Adds messages to the payload object.

Parameter	Type	Description
payload	Object	The payload object to update.
session	SmartSession	The session context.
messages	IList<Message>	The list of messages to add.

Throws:

• ArgumentNullException Thrown when the session, payload, or messages are null.

AddOptions(message, session)

Adds model options to the message object.

Parameter	Type	Description
message	Object	The message object to update.
session	SmartSession	The session context.

Throws:

• ArgumentNullException Thrown when the message is null.

AskAsync(session, messages)

Asynchronously sends a message to the AI model and returns the response.

Parameter	Type	Description
session	SmartSession	The session context.
messages	IList<Message>	The list of messages to send.

Returns: Task<Message>. A task representing the asynchronous operation, with a Message as the result.

This method must be implemented by derived classes to handle the specific logic for sending messages to the AI model.

// Example usage:
var response = await endpoint.AskAsync(session, messages);

The implementation of this method should call session.Usage.Add(assistantMessage.Usage) to keep track if the input and output tokens utilized at the session level.

AskEmbeddingsAsync(inputs)

Asynchronously requests embeddings for the given inputs.

Parameter	Type	Description
inputs	String[]	The array of input strings.

Returns: Task<Embedding>. A task representing the asynchronous operation, with an Embedding as the result.

Throws:

• NotImplementedException Thrown when the method is not implemented.

BuildAssistantMessage(response)

Builds an assistant Message from the given response.

Parameter	Type	Description
response	Response	The response to build from.

Returns: Message. A Message representing the assistant's response.

Throws:

• ArgumentNullException Thrown when the response is null.

BuildEmbeddingsPayload(inputs)

Builds a payload for requesting embeddings.

Parameter	Type	Description
inputs	String[]	The array of input strings.

Returns: Object. An object representing the embeddings payload.

Throws:

• NotImplementedException Thrown when the method is not implemented.

BuildMessage(message)

Builds a message object from the given Message.

Parameter	Type	Description
message	Message	The message to build from.

Returns: Object. An object representing the built message.

Throws:

• ArgumentNullException Thrown when the message is null.

BuildNativeToolsPrompt(session)

Builds the tools namespace prompt based on the session context.

Parameter	Type	Description
session	SmartSession	The session context.

Returns: String. A string representing the tools prompt.

Throws:

• ArgumentNullException Thrown when the session is null.

BuildParameters(parameters)

Builds a string representation of tool parameters.

Parameter	Type	Description
parameters	Parameter[]	The array of parameters to build from.

Returns: String. A string representing the tool parameters.

BuildPayload(session, messages)

Builds a payload object for the given Session and Message.

Parameter	Type	Description
session	SmartSession	The session context.
messages	IList<Message>	The list of messages to include in the payload.

Returns: Object. An object representing the built payload.

Throws:

• ArgumentNullException Thrown when the session or messages are null.

BuildToolResults(toolResults)

Builds the messages containing tool results.

Parameter	Type	Description
toolResults	ToolContext[]

Returns: Message[]. An array of Message representing the tool results.

BuildToolResultsMessage(toolResults)

Builds a message containing tool results.

Parameter	Type	Description
toolResults	ToolContext[]	The array of tools with results.

Returns: Message. A Message representing the tool results.

BuildToolsPayload(session)

Builds the tools payload based on the session context.

Parameter	Type	Description
session	SmartSession	The session context.

Returns: Object[]. A JSON object representing the tools payload.

Throws:

• ArgumentNullException Thrown when the session is null.

BuildToolsPrompt(session)

Builds the tools prompt based on the session context.

Parameter	Type	Description
session	SmartSession	The session context.

Returns: String. A string representing the tools prompt.

Throws:

• ArgumentNullException Thrown when the session is null.

GetCurrentLanguage(session)

Gets the current language based on the session context.

Parameter	Type	Description
session	SmartSession	The session context.

Returns: String. A string representing the current language.

GetTodayDescription()

Gets a description of the current date.

Returns: String. A string representing the current date.

GetToolsToInvoke(session, message)

Gets the tools to invoke based on the session and message context.

Parameter	Type	Description
session	SmartSession	The session context.
message	Message	The message context.

Returns: ToolContext[]. An array of ToolContext representing the tools to invoke.

Throws:

• ArgumentNullException Thrown when the session or message is null.

ReadAssistantMessage(response, message)

Reads the assistant message from the response and updates the message object.

Parameter	Type	Description
response	Response	The response containing the assistant message.
message	Message	The message object to update.

The implementation of this method should call ReadUsage to keep track if the input and output tokens utilized.

ReadEmbeddingsResponse(response)

Reads the embeddings response and returns the embeddings data.

Parameter	Type	Description
response	Response	The response containing embeddings data.

Returns: Single[][]. A jagged array of floats representing the embeddings.

Throws:

• NotImplementedException Thrown when the method is not implemented.

ReadUsage(message, reply)

Reads the usage information from reply.

Parameter	Type	Description
message	Message	The message containing usage information.
reply	Object	The reply object to update with usage information.

Inherited By

Name	Description
AmazonBedrockEndpoint	Represents an endpoint for connecting to Amazon Bedrock services.
AnthropicEndpoint	Represents an endpoint for connecting to Anthropic services.
AzureAIEndpoint	Represents an endpoint for connecting to Azure AI services, specifically designed to interact with OpenAI models.
CerebrasEndpoint	Represents a connection to Cerebras endpoints, inheriting from OpenAIEndpoint.
DeepSeekEndpoint	Represents a connection to DeepSeek endpoints, inheriting from OpenAIEndpoint.
GoogleAIEndpoint	Represents a connection to Google AI endpoints for generating content and embeddings.
GroqCloudEndpoint	https://console.groq.com/docs/openai
GroqCloudEndpointWhisper	Represents a connection to GroqCloud speech endpoints for audio transcription.
HuggingFaceEndpoint	Represents a connection to HuggingFace serverless endpoints for model inference and embeddings.
HuggingFaceJavaScriptEndpoint	Represents an endpoint that uses the transformers.js module in the user's browser to provide AI services to Wisej.AI components.
LocalAIEndpoint	Represents a connection to LocalAI endpoints, providing access to various AI models and services.
LocalAIEndpointImageGen	Represents an endpoint for connecting to LocalAI Image Generation services.
LocalAIEndpointTTS	Represents an endpoint for connecting to LocalAI's speech services.
LocalAIEndpointWhisper	Represents an endpoint for connecting to LocalAI's Whisper model for speech-to-text transcriptions.
NvidiaAIEndpoint	Represents a connection to NVIDIA AI endpoints, providing access to various AI models and services.
OllamaEndpoint	Represents an endpoint that connects to Ollama services, providing functionalities for chat and embeddings.
OpenAIEndpoint	Represents an endpoint for connecting to OpenAI services.
OpenAIEndpointDallE	Represents an endpoint for connecting to OpenAI DallE services.
OpenAIEndpointRealtime	Represents a component that manages communication with OpenAI's real-time API endpoints.
OpenAIEndpointTTS	Represents an endpoint for connecting to OpenAI's speech services.
OpenAIEndpointWhisper	Represents an endpoint for connecting to OpenAI's Whisper model for speech-to-text transcriptions.
SambaNovaEndpoint	Represents a connection to SambaNova endpoints, inheriting from OpenAIEndpoint.
SmartHttpEndpoint	Represents an abstract base class for a smart HTTP endpoint, providing common functionality for HTTP-based communication.
TogetherAIEndpoint	Represents a connection to TogetherAI endpoints, providing access to various AI models and services.
XAIEndpoint	Represents a connection to X.AI endpoints, extending the functionality of OpenAIEndpoint.