SmartEndpoint

Wisej.AI.SmartEndpoint

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents an abstract base class for a smart endpoint component that interacts with AI models.

public class SmartEndpoint : Component

This class provides properties and methods to configure and interact with AI models, including handling messages, building payloads, and managing tool invocations.


// Example usage:
SmartEndpoint endpoint = new CustomSmartEndpoint();
var response = await endpoint.AskAsync(session, messages);

Constructors

SmartEndpoint()

Initializes a new instance of the SmartEndpoint class.

Properties

ContextWindow

Int32: Gets or sets the size of the context window in tokens. (Default: 32000)

EmbeddingModel

String: Gets or sets the embedding model used by the endpoint. (Default: null)

MaxOutputTokens

Int32: Gets or sets the maximum number of output tokens. (Default: 4096)

Model

String: Gets or sets the model used by the endpoint. (Default: null)

ModelOptions

Object: Gets the model options for the endpoint.

Name

String: Gets or sets the name of the endpoint.

SystemPrompt

String: Gets or sets the system prompt for the endpoint.

Warning, setting this property entirely overrides the internal system prompt.

ToolsPrompt

String: Gets or sets the tools prompt for the endpoint.

Warning, setting this property entirely overrides the internal tools prompt.

Usage

Metrics: Gets the usage metrics for the endpoint.

UseNativeTools

Boolean: Gets or sets a value indicating that the endpoint should use the native tools payload when adding tools to the request. (Default: False)

Methods

AddMessages(payload, session, messages)

Adds messages to the payload object.

Parameter
Type
Description

payload

The payload object to update.

session

The session context.

messages

The list of messages to add.

Throws:

AddOptions(message, session)

Adds model options to the message object.

Parameter
Type
Description

message

The message object to update.

session

The session context.

Throws:

AskAsync(session, messages)

Asynchronously sends a message to the AI model and returns the response.

Parameter
Type
Description

session

The session context.

messages

The list of messages to send.

Returns: Task<Message>. A task representing the asynchronous operation, with a Message as the result.

This method must be implemented by derived classes to handle the specific logic for sending messages to the AI model.


// Example usage:
var response = await endpoint.AskAsync(session, messages);

The implementation of this method should call session.Usage.Add(assistantMessage.Usage) to keep track if the input and output tokens utilized at the session level.

AskEmbeddingsAsync(inputs)

Asynchronously requests embeddings for the given inputs.

Parameter
Type
Description

inputs

The array of input strings.

Returns: Task<Embedding>. A task representing the asynchronous operation, with an Embedding as the result.

Throws:

BuildAssistantMessage(response)

Builds an assistant Message from the given response.

Parameter
Type
Description

response

The response to build from.

Returns: Message. A Message representing the assistant's response.

Throws:

BuildEmbeddingsPayload(inputs)

Builds a payload for requesting embeddings.

Parameter
Type
Description

inputs

The array of input strings.

Returns: Object. An object representing the embeddings payload.

Throws:

BuildMessage(message)

Builds a message object from the given Message.

Parameter
Type
Description

message

The message to build from.

Returns: Object. An object representing the built message.

Throws:

BuildNativeToolsPrompt(session)

Builds the tools namespace prompt based on the session context.

Parameter
Type
Description

session

The session context.

Returns: String. A string representing the tools prompt.

Throws:

BuildParameters(parameters)

Builds a string representation of tool parameters.

Parameter
Type
Description

parameters

The array of parameters to build from.

Returns: String. A string representing the tool parameters.

BuildPayload(session, messages)

Builds a payload object for the given Session and Message.

Parameter
Type
Description

session

The session context.

messages

The list of messages to include in the payload.

Returns: Object. An object representing the built payload.

Throws:

BuildToolResults(toolResults)

Builds the messages containing tool results.

Parameter
Type
Description

toolResults

Returns: Message[]. An array of Message representing the tool results.

BuildToolResultsMessage(toolResults)

Builds a message containing tool results.

Parameter
Type
Description

toolResults

The array of tools with results.

Returns: Message. A Message representing the tool results.

BuildToolsPayload(session)

Builds the tools payload based on the session context.

Parameter
Type
Description

session

The session context.

Returns: Object[]. A JSON object representing the tools payload.

Throws:

BuildToolsPrompt(session)

Builds the tools prompt based on the session context.

Parameter
Type
Description

session

The session context.

Returns: String. A string representing the tools prompt.

Throws:

GetCurrentLanguage(session)

Gets the current language based on the session context.

Parameter
Type
Description

session

The session context.

Returns: String. A string representing the current language.

GetTodayDescription()

Gets a description of the current date.

Returns: String. A string representing the current date.

GetToolsToInvoke(session, message)

Gets the tools to invoke based on the session and message context.

Parameter
Type
Description

session

The session context.

message

The message context.

Returns: ToolContext[]. An array of ToolContext representing the tools to invoke.

Throws:

ReadAssistantMessage(response, message)

Reads the assistant message from the response and updates the message object.

Parameter
Type
Description

response

The response containing the assistant message.

message

The message object to update.

The implementation of this method should call ReadUsage to keep track if the input and output tokens utilized.

ReadEmbeddingsResponse(response)

Reads the embeddings response and returns the embeddings data.

Parameter
Type
Description

response

The response containing embeddings data.

Returns: Single[][]. A jagged array of floats representing the embeddings.

Throws:

ReadUsage(message, reply)

Reads the usage information from reply.

Parameter
Type
Description

message

The message containing usage information.

reply

The reply object to update with usage information.

Inherited By

Name
Description

Represents an endpoint for connecting to Amazon Bedrock services.

Represents an endpoint for connecting to Anthropic services.

Represents an endpoint for connecting to Azure AI services, specifically designed to interact with OpenAI models.

Represents a connection to Cerebras endpoints, inheriting from OpenAIEndpoint.

Represents a connection to DeepSeek endpoints, inheriting from OpenAIEndpoint.

Represents a connection to Google AI endpoints for generating content and embeddings.

https://console.groq.com/docs/openai

Represents a connection to GroqCloud speech endpoints for audio transcription.

Represents a connection to HuggingFace serverless endpoints for model inference and embeddings.

Represents an endpoint that uses the transformers.js module in the user's browser to provide AI services to Wisej.AI components.

Represents a connection to LocalAI endpoints, providing access to various AI models and services.

Represents an endpoint for connecting to LocalAI Image Generation services.

Represents an endpoint for connecting to LocalAI's speech services.

Represents an endpoint for connecting to LocalAI's Whisper model for speech-to-text transcriptions.

Represents a connection to NVIDIA AI endpoints, providing access to various AI models and services.

Represents an endpoint that connects to Ollama services, providing functionalities for chat and embeddings.

Represents an endpoint for connecting to OpenAI services.

Represents an endpoint for connecting to OpenAI DallE services.

Represents a component that manages communication with OpenAI's real-time API endpoints.

Represents an endpoint for connecting to OpenAI's speech services.

Represents an endpoint for connecting to OpenAI's Whisper model for speech-to-text transcriptions.

Represents a connection to SambaNova endpoints, inheriting from OpenAIEndpoint.

Represents an abstract base class for a smart HTTP endpoint, providing common functionality for HTTP-based communication.

Represents a connection to TogetherAI endpoints, providing access to various AI models and services.

Represents a connection to X.AI endpoints, extending the functionality of OpenAIEndpoint.

Last updated