1 of 100

Wisej.AI

Welcome

Wisej.AI: Connecting Business Applications to AI.

Welcome to Wisej.AI. This guide is designed to help you seamlessly integrate AI-driven features into your Wisej.NET applications. Whether you are starting from scratch or enhancing existing projects, this documentation will help you maximize the potential of your applications with Wisej.AI.

What is Wisej.AI

Wisej.AI is an advanced system that enhances Wisej.NET components, including third-party widgets, by incorporating AI features to create "Smart" applications. It is designed to be comprehensive, cohesive, standalone, open, and flexible. Wisej.AI operates above infrastructure libraries such as Microsoft Semantic Kernel (SK) or LangChain .NET.

Architecture

Wisej.AI offers a range of ready-made components to enhance your application with AI features, allowing you to concentrate on desired outcomes instead of complex implementations. As an open system, Wisej.AI supports integration with other libraries and functionalities, including those built with SK, giving you flexibility in your development choices.

How to Use It

It's quite simple, add the Wisej-3-AI or Wisej-4-AI NuGet package, and start by dropping the SmartHub component on any design surface. Then click "Create Endpoint" to pick your AI provider, then click "Create Adapter" to pick the AI feature you would like to add.

For a comprehensive understanding of Large Language Models (LLMs) and to learn what they can do to enhance your application, please review all concepts in the Concepts section of this documentation. This section contains all the necessary information.

Concepts

Getting Started

Step-by-step instructions to start using Wisej.AI.

Installing Wisej.AI

Wisej.AI is distributed through two distinct NuGet packages: Wisej-4-AI.4.0.x.x.nupkg and Wisej-3-AI.3.5.x.x.nupkg. These packages include the necessary designer assemblies for both the .NET Framework and .NET Core, eliminating the need for separate designer packages.

The NuGet packages will not be available on NuGet.org, instead you will need to deploy them locally on your file system. To utilize these local NuGet packages, you'll need to add the file path to Visual Studio's list of NuGet sources.

As an alternative you can simply add a NuGet.config file to your solution (at the same level as the .sln file) to include the configuration with the project. See nuget.config for details.

Adding AI Features to Your Application

After you've created a Wisej.NET application or opened an existing one, and added the Wisej-AI NuGet packages, you're ready to start using AI in your apps!

The fastest way is to drop a SmartHub component (you'll find it in the toolbox under the Wisej.AI tab) onto a Page (or any open designer).

Once you have the Hub (smartHub1) you have to select an endpoint (this is your AI provider, can be a public one or your own AI server, see for more information).

Selecting the endpoint first is important because it determines which adapters are compatible. Not all adapters work with every endpoint. For instance, some endpoints might support vision, while others require TTS or voice recognition.

Here are the initial steps:

Add a SmartHub

The SmartHub serves as the core unit that links and manages communication between the AI endpoint and adapters.

Once everything is in place, the specific control or set of controls that have been connected to the SmartAdapter will have a new set of AI properties.

Not all SmartAdapters are designed to connect directly to controls. Some, like the SmartObjectAdapter, offer functionality that can be utilized within the code itself.

How to Create Endpoints

After adding a SmartHub component, you have two options to create an Endpoint. You can either click the quick actions arrow and select "Create Endpoint," or you can select the SmartHub component and click the "Create Endpoint" link located at the bottom of the property grid.

You will see the SmartEndpoint picker dialog:

Select the provider you want to use, verify that the default properties are set correctly, and click OK. See for more information.

Wisej.AI will create the selected SmartEndpoint component and assign it to the Endpoint property of the SmartHub component. It is important to always check and make sure the SmartHub has an endpoint assigned.

All the designer dialogs are under development and will change before the release.

How to Select Adapters

After the SmartHub is associated with an endpoint, it can filter which adapters can work with that specific AI provider. Click the quick actions arrow and select "Create Adapter", or click the link "Create Adapter" in the property grid.

The SmartAdapter Picker shows all the built-in adapters that can use the endpoint plus any custom adapter that you have created in your application. Please note the Documentation link provided in the description field of each adapter. This link directs you straight to the documentation for the adapter, ensuring you have easy access to the relevant materials.

Click OK to create the adapter component.

Once the SmartAdapter is created, you have to connect it to the components to augment. Some adapters may support augmenting multiple controls, while others may allow extending only one control at a time, in which case you can add the same adater multiple times, if needed.

The screenshot above displays the SmartDataEntry adapter linked to the splitContainer1.Panel1 control. Once connected, the adapter will enhance all child controls, at every level, by integrating the new AI properties required for it to perform its functions effectively.

Configuring Api Keys

Most Endpoints, Vector Databases, and Web Search Engines require an Api Key to get authenticated. You can set Api Key for each component that requires one in three ways:

Set the ApiKey property on the component
Save the api key in the ~/AI/ApiKeys.json file
Set the api key in an environment variable

ApiKeys.json File

To set up your Wisej.NET project properly, create a new folder named "AI" at the root level of your project. This folder will serve as an organized space for various AI features. Once the "AI" folder is in place, create a file named ApiKeys.json within it. This file will allow you to manage your api keys in one central location.

To ensure the security of your ApiKeys.json file when deploying with .NET Core (ASP.NET Core), you should prevent the server from allowing its download. A simple way to do this is by modifying the code in Startup.cs.

When deploying using .NET Framework, all json files are blocked in web.config.

You can store various types of keys in the ApiKeys.json file. Wisej.AI utilizes these keys for its Endpoints, Vector Databases, and Web Search Engines by identifying the service by its name without suffixes. For instance, if the service is named "BingWebService", Wisej.AI will use "BingWeb" to locate the key within the ApiKeys.json file.

Api Keys in Environment Variables

You can manage API keys securely by storing them in environment variables through your cloud provider. For example, Azure App Services lets you manage environment variables for each service efficiently.

The name of the environment variable is:

WISEJ_AI_APIKEY_ + Endpoint Name without the "Endpoint" suffix.

For the OpenAIEndpoint the api key environment variable is WISEJ_AI_APIKEY_OPENAI.

Environment variables override the settings in the ApiKeys.json file.

Custom Api Keys

If you choose to store custom keys in ApiKeys.json or as environment variables, you can retrieve these keys using the class. Here’s how you can do it:

In the example above, the method first attempts to read the environment variable named "KEY_MYCOOLSERVICE" with the second parameter (the name postfix) left empty. If this environment variable does not exist, it then checks for the key "MyCoolService" in the ApiKeys.json file.

If the final parameter, the envPrefix, is not specified, it defaults to "WISEJ_AI_APIKEY_" as the prefix for environment variables.

Your First AI Application

Start by creating a Wisej.NET new application with either a main page or a main form. And add a few fields and a button, like this:

Use the NuGet Package Manager to add Wisej-AI and Wisej-AI-Design to the project.

Add a new folder "AI" to the root of the project.

From the Toolbox, under "Wisej.AI Test" select SmartHub and drop it on the page being designed.

Now click the "Create Endpoint" link in the property grid or click the quick action arrow and select "Create Endpoint". See above.

Pick your AI provider. In this case we use OpenAI. Make sure you have an api key from Open AI. Put the api key in the ApiKey property (see also ).

Now create the SmartDataEntryAdapter (see ).

Leave all the default properties unchanged. Select the Page and you should see this in the properties panel.

Select the smartDataEntryAdapter1 component to assign it to the Adapter property.

Once the adapter is connected to the Page, all its child controls will gain new AI properties. In this case you will find the FieldName, FieldPrompt and FieldRectangleproperties under the group "AI Features"

For each field from top down, set the FieldName property to "First Name", "Last Name", "Email Address", "Summary". Then double click on the button and add this code to the button1_Click event handler. Notice that the handler is async.

Run the app. Then go to an email you received from someone, select the text including the signature and copy to the clipboard. Go back to the app, make sure to click somewhere on the page first or the browser will block clipboard access, and click the Smart Paste button.

This is already something! Now add a checkbox, set the text to "SPAM" but now set the FieldPrompt to "Detect if the text is likely to be a SPAM email."

Run it again and try now.

You can also try by select and copying an image to the clipboard. The SmartDataEntryAdapter supports just about anything.

General Concepts

Introduction to LLM Concepts in Wisej.AI

Overview

In this section, we will explore the foundational concepts of Large Language Models (LLMs) as they are utilized within Wisej.AI. This primer is designed to provide software developers with a comprehensive understanding of how LLMs work. We will cover the basic principles, architectural frameworks, and applications of LLMs, ensuring that you have the necessary background to effectively leverage these advanced models in your development projects through Wisej.AI and Wisej.NET.

LLMs are built upon the architecture. This document assumes that the reader has at least a basic understanding of how neural networks function.

Large Language Models (LLMs)

Large Language Models are a type of artificial intelligence designed to understand, generate, and manipulate human language. These models are trained on vast datasets and use deep learning techniques to achieve natural language processing tasks with impressive accuracy and fluency. Some key attributes of LLMs include:

Contextual Understanding: LLMs comprehend the context of the information they process, enabling them to generate responses that are relevant and coherent within the given framework.

Adaptability: These models can be fine-tuned for a plethora of applications, ranging from content generation to automated customer support, and, in the case of Wisej.AI, augmenting user interface components and widgets.

Scalability: The architecture of LLMs allows them to handle concurrent requests efficiently, which is crucial for real-time interaction in browser-based applications.

LLMs (Large Language Models) are a component of Generative AI, which is built upon the foundational principles of traditional Machine Learning (ML) architecture.

Fundamentally, LLMs are straightforward to use. They operate by predicting the next token in a sequence and continue this process until they reach a specified stopping point or condition, such as the end of a sentence or a predefined token limit.

For example, if the prompt is "Hello, how are you?" the likelihood of the next token being "banana" is very low. In contrast, the probability of the sequence continuing with "Very," followed by "good," then ", thank," "you," "for," and "asking" is much higher.

The complexity of the prompt that a model can handle effectively typically depends on the size of its Artificial Neural Network (ANN). When reviewing a model's description, you might encounter a number like 90B next to its name. This number represents the model's parameters, which are the weights of the ANN, expressed as single-precision floating-point numbers (float32). Consequently, a model with 90 billion parameters consists of 90 billion float32 numbers, resulting in an approximate size of 360 GB.

The "Hello" example is quite trivial. To better illustrate how Wisej.AI can leverage LLMs to deliver valuable functionality to applications—primarily without engaging in chat—consider the following simplified example:

System Prompt is always the first part of the request sent to the LLM. After that the "User" prompt is the message from the application, and the "Assistant" message is the response from the LLM.

The example above demonstrates how Wisej.AI dynamically generates a prompt by analyzing the code to identify the fields required by a .NET class. The agent message, which is the LLM's response, is formatted as valid JSON. This approach enables the system to read the response, parse it, and populate the appropriate fields in the application. This process is accomplished without engaging in chat and without handling "conversational" responses.

In other words, we are utilizing the LLM in a manner similar to how one would use a SQL database or a cloud service.

This is just the first crucial step in understanding the Wisej.AI architecture. Once this concept is clear, you can appreciate how Wisej.AI leverages this highly structured request-response approach to seamlessly integrate AI features into virtually any aspect of a Wisej.NET application.

Building on the concept of "harnessing" the AI model, Wisej.AI develops agents and super-agents, along with tools that enhance the model's capabilities. For instance, while a model is inherently limited to appending words (or , to be precise) to a prompt, Wisej.AI constructs structured prompts with specific instructions for obtaining additional information. By parsing these responses and invoking code iteratively (), Wisej.AI creates agents that deliver significantly more value to the system.

Empowering the AI through Tools

Tools are essentially functions that the LLM can "invoke" within its response. See for details on how Wisej.AI defines tools.

The simplest way to understand tools is through an example: if you ask any model, "What time is it?" the model will typically respond with something like, "I'm sorry, but I don't have the capability to provide real-time information such as the current time."

If you ask ChatGPT "what time is it" it will respond correctly - that is because it utilized a tool! But when you ask directly any model anywhere it will respond that it doesn't know.

If we simulate the request-response cycle managed by the Wisej.AI internal agent and have provided the get_current_time tool to the AI, the query would look like this (the content is very simplified):

Now imagine equipping the LLM with a variety of tools: one for web searches, another for querying your company's database, another for scanning through documents, one for building charts, and yet another for sending emails, among others. These tools effectively expand the AI's capabilities. Consequently, when an application uses Wisej.AI to extract data from a PDF document, it can also fill in missing information, check for anomalies, send alerts via email, and correlate internal documents with public information—all through a single message to the AI provider via Wisej.AI.

Retrieval-Augmented Generation (RAG)

LLMs cannot learn or retain information; they have no memory or state. Each request is independent and must include all the necessary information for the LLM to generate the correct subsequent tokens (the response) to the prompt.

If you've read that LLMs can process vast amounts of data in a single request, that's a misconception. In reality, the opposite is true. The combined size of requests and responses is quite limited. While the model itself is likely trained on a vast amount of information, its capacity to generate responses is constrained by what is known as the . This is the maximum number of tokens (for simplicity, you can think of a token as a word) that can be submitted to the ANN at once. It encompasses everything from the system prompt to the final question being submitted. You can think of it as a large string, similar to a lengthy SQL statement. A context window of 128,000 tokens is considered large with the current technology.

A common use case might involve a request to create a table listing all legal deadlines, along with summaries, from 8,000 documents in a folder. It is entirely impractical to concatenate the text from all these documents into a single prompt. Assuming an average of 20 pages per document and 400 words per page, you would end up with approximately 64 million words, which far exceeds the typical limit of 128K tokens.

Indeed, just as an AI (or a person, for that matter) cannot feasibly read billions of web pages but can submit a query to a search engine to receive a list of relevant pages, RAG is used to provide pertinent context to the AI. It does this by extracting the few chunks of text that are relevant to a query from a large collection of data.

Even when using RAG, the process may yield too many chunks, necessitating the truncation of relevant data, which can result in inaccurate responses. There are advanced techniques that can summarize, extract, and aggregate data to process more information effectively. However, there is always a limit to how much data can be handled in this manner.

For instance, if we want to create a chart of the "5-year cumulative returns" from an 80-page PDF 10-K filing from Apple, we would need to extract and process the relevant data from the document, as it cannot be included in its entirety within the prompt.

We accomplish this by splitting the document into chunks and generating an vector for each chunk. Then, we generate an embedding vector for the query (or a version of the query optimized by the LLM), select the top N chunks based on relevance, and submit only those chunks for processing.

Another example would be asking the AI to classify a line item in an invoice according to a list of codes in a chart of accounts stored in a database. Since we cannot submit the entire database, we need to extract the most relevant data to include with the prompt, providing the AI with sufficient information to complete the task accurately.

In essence, RAG (Retrieval-Augmented Generation) leverages traditional coding methods, which are capable of processing unlimited amounts of data, to extract a small, relevant subset of information for submission to the LLM (Large Language Model). Utilizing is just one of the many techniques available to accomplish this task.

Embeddings (Vectors)

Embeddings are vector representations of text that position the specific text in relation to other texts within a multidimensional space.

A vector representation is an array of floating-point numbers that indicates the text's position within the space, with the array's length corresponding to the number of dimensions. For instance, a vector with 1536 values places the text within a 1536-dimensional space.

This representation enables applications to efficiently filter large volumes of data by performing similarity searches, which involve calculating the distance between two vectors in the multidimensional space.

Before the introduction of embeddings, a developer would typically split the query into individual words, then iterate through each text chunk, perhaps counting how many keywords appeared in each. Additionally, they might have used traditional letter-swapping algorithms to account for similar words and possibly the relative positions of the keywords. This approach was highly inaccurate, as it only considered character matches and sequences, ignoring conceptual similarity. For instance, the character "5" and the word "five" would never match in such a search.

Using embeddings and various methods to calculate the distance between vectors is a straightforward mathematical operation. It allows applications to process vast amounts of data efficiently and quickly.

Wisej.AI offers a range of services and methods to generate embeddings, with options to store them in memory, on the file system, or in efficient vector databases such as Azure AI Search, Pinecone, or Chroma.

Embeddings stored in memory or the file system should be used solely for development purposes. For deployment, it is recommended to use a vector database for optimal performance and scalability.

Context Window and Tokens

All models have a limited context window, which is measured in tokens. Tokens are not the same as words, but for simplicity, they can be considered as such. To gain a better understanding of tokens in the context of LLMs, you can try using this tokenizer from OpenAI: .

When you type "Hello, how are you?" (which is 4 words, 3 spaces and 2 punctuations) you get 6 tokens. But if you type "Dearest, how are you?" you get 7 tokens.

There are various types of tokenizers, and different models may be trained using distinct tokenizers. On average, a token typically consists of about 3.5 characters.

Wisej.AI includes the ITokenizerService and offers a default implementation based on OpenAI's . This service can be accessed directly by the Wisej.NET application to count tokens, truncate strings, or split a string into tokens.

When a request to an LLM exceeds the maximum allowed tokens in the context window, the request will typically fail, resulting in an error. To resolve this, you may need to shorten the input or split it into multiple requests that fit within the context token limit.

Wisej.AI automatically manages content overflow before any errors occur by utilizing the SmartEndpoint.ContextWindow property as the token limit. When this limit is exceeded, the SmartSession object optimizes the history using various techniques to ensure smooth operation.

Semantic Kernel does it like . Wisej.AI does it automatically. See and for more information.

For a more detailed description of the automatic trimming employed by Wisej.AI, refer to the "" section under Usage Metrics.

Reranking

Reranking refers to the process of reordering a list of items, typically search results or recommendations, to improve their relevance or quality with respect to a specific criterion. This is often achieved by applying machine learning algorithms that re-evaluate and adjust the initial ranking, utilizing additional information or more sophisticated models.

When utilizing one of the built-in implementations of the IEmbeddingStorageService service, the order of relevance is initially determined by the similarity score of the embedding vectors. However, there may be instances where a more precise ordering is desired. Reranking is a common technique used to achieve this. It involves submitting the results of the vector search to a more advanced model, which then reorders the text chunks based on their relevancy to the query. This approach leverages the enhanced capabilities of the model to provide a more accurate order of results.

Wisej.AI offers the and an overridable method named RerankAsync within the DocumentTools and DocumentSearchTools classes. The default implementation uses the IRerankingService to reorganize the RAG documents.

What is an Agent and a Super-Agent

Utilizing a model typically involves sending a question and receiving a response. There are numerous examples available that demonstrate this process using Semantic Kernel or LangChain. Additionally, you will find many samples of workflows or pipelines that take a response and use it as a new request.

The process can be visualized as follows: Start with the question "What is the capital of China?" which results in the response "Beijing." This response is then used in a follow-up question, "How is the weather in Beijing?" leading to the response "Rainy."

All of this is based on the concept of "conversational AI."

However, Wisej.AI does not employ LLMs in this manner. Instead, it consistently supports context management, tools, and adapters through a composition pattern. Therefore, the smallest unit of AI utilization in Wisej.AI is always an agent.

Subsequently, since the Wisej.AI agents are utilized by adapters operating at the next layer, and these adapters have specific tasks and can incorporate multiple decisions and interactions, we can refer to them as super-agents.

Refer to the section in the documentation to see straightforward examples of this approach.

Usage Metrics

How to keep track of the tokens

Overview

Most AI cloud providers charge based on the . They differentiate between input tokens (the tokens you send) and output tokens (the tokens generated).

Wisej.AI tracks InputTokens, OutputTokens, and CallCount in the Usage property across multiple levels, providing detailed insights into how resources are being utilized:

Logging & Error Handling

How to handle errors and log AI activity

Logging

All logging within Wisej.AI is managed via the . The default implementation is provided by , which forwards the log entries to according to the specified TraceLevel.

You can set up your own logging implementation by registering a service that implements the ILoggerService interface. The optimal location for this registration is within the static constructor of the Program class.

Markup

Overview

Wisej.AI's core components—including endpoint, prompt, session, and hub—utilize the fluent markup syntax, enabling developers to seamlessly chain property settings and event handling. This streamlined approach enhances code readability and efficiency by allowing developers to configure components in a more intuitive and concise manner.

Components

API

Welcome

This book covers the full API for the Wisej.AI assembly only. For an in-depth guide of Wisej.AI features, please refer to the section.

Legend

SmartCalendarAdapter

Wisej.AI.Adapters.SmartCalendarAdapter

Namespace: Wisej.AI.Adapters

Assembly: Wisej.AI (3.5.0.0)

SmartAdapter
- SmartCalendarAdapter

Represents a smart calendar adapter that extends the functionality of a SmartAdapter.

public class SmartCalendarAdapter : SmartAdapter

The SmartCalendarAdapter class provides methods to process text or clipboard content asynchronously.

Constructors

SmartCalendarAdapter()

Initializes a new instance of the class.

This constructor initializes the SmartPrompt with a default prompt message.

Methods

FromClipboardAsync()

Processes the content from the clipboard asynchronously.

Returns: . A task that represents the asynchronous operation.

This method reads text or image data from the clipboard and processes it asynchronously.

Throws:

Thrown when the adapter is busy.

FromTextAsync(text)

Processes the specified text asynchronously.

Parameter

Type

Description

Returns: . A task that represents the asynchronous operation.

This method sets the internal text and initiates the asynchronous processing.

Throws:

Thrown when the text is null.
Thrown when the adapter is busy.

RunAsyncCore(control)

Executes the core logic of the adapter asynchronously.

Parameter

Type

Description

Returns: . A task that represents the asynchronous operation, containing the message response.

This method uses a session to ask a question based on the provided text and tracks the usage.

Inherited By

Name

Description

Implements

Name

Description

SmartChartJS3Adapter

Wisej.AI.Adapters.SmartChartJS3Adapter

Namespace: Wisej.AI.Adapters

Assembly: Wisej.AI (3.5.0.0)

SmartAdapter
- SmartChartAdapter

Represents an adapter that enhances a ChartJS control with several AI features.

This class extends the functionality of the ChartJS3 control by integrating AI capabilities. It is part of the SmartAdapter API category and does not allow multiple extensions.

Constructors

SmartChartJS3Adapter()

Initializes a new instance of .

Properties

ChartJS3

: Gets the ChartJS3 control associated with this adapter.

This property is not browsable in the property grid and is hidden from designer serialization.

Methods

OnControlCreated(control)

Called when a control is created.

Parameter

Type

Description

This method initializes the ChartJS3Tools with the associated ChartJS3 control. It ensures that the base class's OnControlCreated method is also called.

Implements

Name

Description

SmartFullCalendarAdapter

Wisej.AI.Adapters.SmartFullCalendarAdapter

Namespace: Wisej.AI.Adapters

Assembly: Wisej.AI (3.5.0.0)

SmartAdapter.FieldNameAttribute

Wisej.AI.SmartAdapter FieldNameAttribute

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents an attribute that can be used to specify a custom field name for a property or control.

public class FieldNameAttribute : Attribute

Public Class FieldNameAttribute
    Inherits Attribute

Constructors

FieldNameAttribute(name)

Initializes a new instance of the class with the specified field name.

Name

Type

Description

Properties

Name

: Gets or sets the custom field name associated with the property or control.

SmartAdapter.FieldPromptAttribute

Wisej.AI.SmartAdapter FieldPromptAttribute

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents an attribute used to define a prompt for a property or control.

Constructors

SmartAdapter.FieldRectangleAttribute

Wisej.AI.SmartAdapter FieldRectangleAttribute

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents an attribute that defines a rectangular area using a Rectangle associated to a property or a control.

public class FieldRectangleAttribute : Attribute

Public Class FieldRectangleAttribute
    Inherits Attribute

This attribute can be used to specify a rectangle by providing either its dimensions or a string representation.

Constructors

FieldRectangleAttribute(x, y, width, height)

Initializes a new instance of the class with specified dimensions.

Name

Type

Description

FieldRectangleAttribute(rectangle)

Initializes a new instance of the class with a rectangle specified as a string.

Name

Type

Description

The string should be in a format that can be converted to a using the type converter.

Properties

Rectangle

: Gets or sets the associated with this attribute.

CerebrasEndpoint

Wisej.AI.Endpoints.CerebrasEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

SmartEndpoint
- SmartHttpEndpoint

Represents a connection to Cerebras endpoints, inheriting from OpenAIEndpoint.

This class is specifically designed to connect to Cerebras AI endpoints, providing default values for the model and URL.

Constructors

CerebrasEndpoint()

Initializes a new instance of the class with default settings.

The default authentication method is set to "Bearer", the model to "llama3.1-70b", and the URL to "https://api.cerebras.ai/v1".

Properties

Model

: Gets or sets the model used by the Cerebras endpoint. (Default: "llama-3.3-70b")

URL

: Gets or sets the URL of the Cerebras endpoint. (Default: "https://api.cerebras.ai/v1")

UseNativeTools

: (Default: True)

DeepSeekEndpoint

Wisej.AI.Endpoints.DeepSeekEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

GroqCloudEndpoint

Wisej.AI.Endpoints.GroqCloudEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

LocalAIEndpoint

Wisej.AI.Endpoints.LocalAIEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

NvidiaAIEndpoint

Wisej.AI.Endpoints.NvidiaAIEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

SambaNovaEndpoint

Wisej.AI.Endpoints.SambaNovaEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

TogetherAIEndpoint

Wisej.AI.Endpoints.TogetherAIEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

SmartEndpoint
- SmartHttpEndpoint

Represents a connection to TogetherAI endpoints, providing access to various AI models and services.

This class extends the to connect specifically to TogetherAI's API. It initializes with default models and a URL for the TogetherAI service.

Constructors

TogetherAIEndpoint()

Initializes a new instance of the class with default settings.

The default authentication method is set to "Bearer". The default models and URL are pre-configured for TogetherAI services.

Properties

EmbeddingModel

: Gets or sets the embedding model used for AI operations. (Default: "togethercomputer/m2-bert-80M-8k-retrieval")

Model

: Gets or sets the model used for AI operations. (Default: "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo")

URL

: Gets or sets the URL for the TogetherAI API endpoint. (Default: "https://api.together.xyz/v1")

XAIEndpoint

Wisej.AI.Endpoints.XAIEndpoint

Namespace: Wisej.AI.Endpoints

Assembly: Wisej.AI (3.5.0.0)

SmartEndpoint
- SmartHttpEndpoint

Represents a connection to X.AI endpoints, extending the functionality of OpenAIEndpoint.

This class is used to interact with the X.AI API, providing properties and methods specific to X.AI's capabilities.

Constructors

XAIEndpoint()

Initializes a new instance of the class with default settings.

The default authentication method is set to "Bearer", the model to "grok-beta", and the URL to "https://api.x.ai/v1".

Properties

Model

: Gets or sets the model used by the X.AI endpoint. (Default: "grok-beta")

URL

: Gets or sets the URL of the X.AI API endpoint. (Default: "https://api.x.ai/v1")

UseNativeTools

: (Default: True)

SmartSession.ConvertParameterEventHandler

Wisej.AI.SmartSession ConvertParameterEventHandler

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Parameters

Name

SmartSession.ErrorEventArgs

Wisej.AI.SmartSession ErrorEventArgs

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Provides data for the event in a .

Constructors

SmartSession.ErrorEventHandler

Wisej.AI.SmartSession ErrorEventHandler

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents the method that will handle an error event in a SmartSession.

public delegate void ErrorEventHandler(Object sender, ErrorEventArgs e)

Public Delegate Sub ErrorEventHandler(ByVal sender As [Object], ByVal e As ErrorEventArgs)

Parameters

Name

Type

Description

Fired By

Name

Description

SmartSession.InvokeToolEventArgs

Wisej.AI.SmartSession InvokeToolEventArgs

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Constructors

SmartSession.InvokeToolEventHandler

Wisej.AI.SmartSession InvokeToolEventHandler

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

public delegate void InvokeToolEventHandler(Object sender, InvokeToolEventArgs e)

Public Delegate Sub InvokeToolEventHandler(ByVal sender As [Object], ByVal e As InvokeToolEventArgs)

Parameters

Name

Type

Description

Fired By

Name

Description

SmartSession.MessagesEventArgs

Wisej.AI.SmartSession MessagesEventArgs

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Provides data for events that involve a collection of messages in the SmartSession API.

public class MessagesEventArgs : EventArgs

Public Class MessagesEventArgs
    Inherits EventArgs

Constructors

MessagesEventArgs(session, messages)

Initializes a new instance of the class with the specified list of messages.

Name

Type

Description

Properties

Messages

: Gets the list of messages sent and received.

Session

: Gets the session associated with the event.

Used By

Name

Description

SmartSession.MessagesEventHandler

Wisej.AI.SmartSession MessagesEventHandler

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents the method that will handle message-related events for the SmartSession object.

public delegate void MessagesEventHandler(Object sender, MessagesEventArgs e)

Public Delegate Sub MessagesEventHandler(ByVal sender As [Object], ByVal e As MessagesEventArgs)

Parameters

Name

Type

Description

Fired By

Name

Description

SmartPrompt.Parameter

Wisej.AI.SmartPrompt Parameter

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Parameters are typically used in scenarios where dynamic content needs to be inserted into a prompt with placeholders.

Constructors

SmartRealtimeSession

Wisej.AI.SmartRealtimeSession

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

SmartObject
- SmartSession

Constructors

SmartRealtimeSession(hub, systemPrompt)

Initializes a new instance of the class with the specified hub and optional system prompt.

Name

Type

Description

Throws:

Thrown when the hub is null.

SmartRealtimeSession(endpoint, systemPrompt)

Initializes a new instance of the class with the specified endpoint and optional system prompt.

Name

Type

Description

Throws:

Thrown when the endpoint is null.

SmartSession.Message

Wisej.AI.SmartSession Message

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents a message within a SmartSession, containing various types of content such as text, image, and binary data.

The Message class is used to encapsulate different types of content that can be part of a session. It includes properties for text, image, and binary content, as well as methods to retrieve role and image data in specific formats.

SmartSession.MessageRole

Wisej.AI.SmartSession MessageRole

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents the role of a message within a smart session context.

Fields

SmartSession.TrimmingStrategy

Wisej.AI.SmartSession TrimmingStrategy

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Specifies the strategy for handling context overflow in a session.

public enum TrimmingStrategy : Enum

Public Enum TrimmingStrategy As [Enum]

Fields

Name

Description

Used By

Name

Description

SmartTool

Wisej.AI.SmartTool

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Represents a tool that can be used to perform operations on a target object using a specified method.

Constructors

SmartTool.ToolAttribute

Wisej.AI.SmartTool ToolAttribute

Namespace: Wisej.AI

Assembly: Wisej.AI (3.5.0.0)

Attribute to categorize methods as part of the SmartTool API.

Constructors

Markup

UVLightOverlay

Wisej.AI.Controls.UVLightOverlay

Namespace: Wisej.AI.Controls

Assembly: Wisej.AI (3.5.0.0)

Control
- UVLightOverlay

Represents a control that displays an animated UV light overlay to simulate an optical scanner.

public class UVLightOverlay : Control

Constructors

UVLightOverlay()

Initializes a new instance of .

Methods

OnWebRender(config)

Renders the UV light overlay on the web client.

Parameter

Type

Description

Embeddings

Embedding

Wisej.AI.Embeddings.Embedding

Namespace: Wisej.AI.Embeddings

Assembly: Wisej.AI (3.5.0.0)

Represents an embedding which contains chunks of data, their associated vectors, and the model used for embedding.

Constructors

Matches

Wisej.AI.Embeddings.Matches

Namespace: Wisej.AI.Embeddings

Assembly: Wisej.AI (3.5.0.0)

public class Matches

Public Class Matches

Constructors

Matches(chunks, similarity)

Initializes a new instance of .

Name

Type

Description

Properties

Chunks

Similarity

Helpers

ApiKeys

Wisej.AI.Helpers.ApiKeys

Namespace: Wisej.AI.Helpers

Assembly: Wisej.AI (3.5.0.0)

Provides methods to retrieve API keys from environment variables or a configuration file.

Methods

General Concepts

Introduction to LLM Concepts in Wisej.AI

Overview

LLMs are built upon the architecture. This document assumes that the reader has at least a basic understanding of how neural networks function.

Large Language Models (LLMs)

Contextual Understanding: LLMs comprehend the context of the information they process, enabling them to generate responses that are relevant and coherent within the given framework.

Scalability: The architecture of LLMs allows them to handle concurrent requests efficiently, which is crucial for real-time interaction in browser-based applications.

LLMs (Large Language Models) are a component of Generative AI, which is built upon the foundational principles of traditional Machine Learning (ML) architecture.

System Prompt is always the first part of the request sent to the LLM. After that the "User" prompt is the message from the application, and the "Assistant" message is the response from the LLM.

In other words, we are utilizing the LLM in a manner similar to how one would use a SQL database or a cloud service.

Empowering the AI through Tools

Tools are essentially functions that the LLM can "invoke" within its response. See for details on how Wisej.AI defines tools.

If you ask ChatGPT "what time is it" it will respond correctly - that is because it utilized a tool! But when you ask directly any model anywhere it will respond that it doesn't know.

Retrieval-Augmented Generation (RAG)

Embeddings (Vectors)

Embeddings are vector representations of text that position the specific text in relation to other texts within a multidimensional space.

Embeddings stored in memory or the file system should be used solely for development purposes. For deployment, it is recommended to use a vector database for optimal performance and scalability.

Context Window and Tokens

When you type "Hello, how are you?" (which is 4 words, 3 spaces and 2 punctuations) you get 6 tokens. But if you type "Dearest, how are you?" you get 7 tokens.

There are various types of tokenizers, and different models may be trained using distinct tokenizers. On average, a token typically consists of about 3.5 characters.

Semantic Kernel does it like . Wisej.AI does it automatically. See and for more information.

For a more detailed description of the automatic trimming employed by Wisej.AI, refer to the "" section under Usage Metrics.

Reranking

What is an Agent and a Super-Agent

All of this is based on the concept of "conversational AI."

Refer to the section in the documentation to see straightforward examples of this approach.

Extensibility

How to Extend and Integrate Wisej.AI

Overview

Wisej.AI offers unparalleled flexibility, ensuring that you are not limited by its predefined features. The system is designed to be open and highly extensible. In fact, the majority of the components have virtual members, allowing you to override nearly any functionality. You have the ability to customize all built-in prompts and connect to a wide range of events at various levels, providing a seamless integration experience.

Considering the architecture of Wisej.AI, you have several options for overriding its built-in functionality and integrating any other AI library of your choice.

SmartEndpoint

You can override any of the built-in SmartEndpoints to reuse the existing functionality as needed. Alternatively, you can create a brand-new endpoint by extending either the base or . For instance, if you want to integrate a third-party client for OpenAI (or any other provider) into the Wisej.AI system, you can do so by extending and implementing at least the abstract methods. This approach allows you to customize and tailor the integration according to your specific requirements.

Many AI providers have standardized their APIs based on the OpenAI API. In such cases, you can use the OpenAIEndpoint within Wisej.AI and modify parameters like the URL, model, and authentication as needed. Alternatively, you can extend the to integrate different properties and adjust any variances specific to your provider. This flexibility allows for seamless adaptation to various AI service requirements.

SmartAdapter

To enhance any control with AI features in Wisej.AI, you can create custom adapters. Start by deriving your adapter from SmartAdapter and place your implementation in the RunAsyncCore() method. Within this method, you are responsible for interacting with the AI service and parsing its response. Notably, you are not required to use any of the SmartEndpoints within this process.

For example, if you want to create an adapter that populates a ComboBox with individual text items generated from an AI prompt, for simplicity you can use the property of the enhanced control to store the prompt. This approach provides a straightforward way to integrate AI-driven functionality into your applications.

If you wish to enhance the functionality of a control by adding a new property, you can make your adapter implement the interface. This allows the adapter to introduce new properties to the controls it augments. By implementing this interface, you can dynamically extend the capabilities of the associated controls, offering a more sophisticated level of customization.

After creating the MyComboBoxAdapter as outlined above, follow these steps to integrate it into your project:

Add a SmartHub and SmartEndpoint to your project.
Select your MyComboBoxAdapter.
Drag and drop your adapter onto the open form designer.
Associate the adapter with an existing ComboBox in your interface.

Once the adapter is linked to the ComboBox, it will automatically add the ItemsPrompt property to the ComboBox. Set this property to something like "European countries" or "US states" and run the application to see it in action. This will allow the ComboBox to populate dynamically based on the AI prompt specified in the ItemsPrompt property, or the Tag property if you didn't extend the IExtenderProvider interface.

SmartTool

One of the most powerful features in Wisej.AI is the system. You have the flexibility to utilize or extend any built-in tool, or you can create a new tool from scratch. The system is designed to make this process extremely straightforward and user-friendly, allowing developers to easily customize and extend their application's capabilities.

You can provide tools to the AI in multiple ways and at different levels within the Wisej.AI framework. For instance, if you annotate a method on the control being designed with the [SmartTool.Tool] attribute and have a component in place, the SmartHub will automatically include this method in the tools available to the AI. This inclusion is independent of the adapter, prompt, or endpoint being used. In essence, once a tool is added to a SmartHub, it is consistently made available to the AI model, ensuring seamless integration and accessibility.

Ways to provide tools to the AI:

To turn a method into a tool within the Wisej.AI system, annotate it with the [SmartTool.Tool] attribute. This method will be available as a tool if it is declared within a container that's associated with a SmartHub. Alternatively, the method can become a tool if it is part of an object that is passed to the UseTools() method of a , , , or . This flexible integration ensures that your tools can be widely accessible across different components of your application.
You can directly provide a specific method to the SmartHub, SmartPrompt, SmartAdapter, or SmartSession components by using the UseTool() method. For example, to pass a method such as get_customer_id to a SmartHub, you would use the following code: smartHub1.UseTool(this.get_customer_id);. This approach allows you to easily leverage specific methods as tools without needing to encapsulate them within a class or container.

For instance, if you want to enable the AI to send an email while performing other tasks, you can create a tools class with two methods. One method can be designed to send the email, while the other can be developed to find the recipient's email address, adding an intelligent layer to the process. By integrating this tools class into the Wisej.AI framework, you can provide the AI with advanced capabilities for handling emails efficiently as part of its operations.

You can enhance the usability and clarity of the tools within Wisej.AI by using the Description attribute to annotate various components. You can annotate the tools container class, each method, and each parameter with this attribute. These additional annotations provide the AI with detailed information, allowing it to utilize the tools more effectively and accurately, ensuring that each tool is used correctly within your application.

This is an example of the built-in annotation for the DatabaseTools bulilt-in tools container:

Using an INI file offers several advantages when working with prompts in Wisej.AI. It allows you to craft clearer and more complex prompts and gives you the flexibility to fine-tune these prompts externally, without needing to modify your application's code. This external configuration capability enhances maintainability and adaptability, enabling you to adjust prompts as needed without redeploying your application.

In Wisej.AI, a tool can modify its own prompt by programmatically replacing placeholders like {{server-type}} and {{database-schema}} with specific values relevant to the tool itself. This can be achieved by implementing logic within the tool's initialization or execution process to dynamically insert the appropriate values into the prompt.

For instance, you can define a method within the tool that retrieves the required information—for example, the server type or database schema—and injects these values into the prompt before it is processed. This approach ensures that the prompt is precisely tailored based on the tool's configuration or current state, maintaining the flexibility and relevance of the information provided to the AI.

To enable a tool to modify its own prompt with specific values for placeholders, your tools container class should either implement the interface or extend the class. By leveraging these options, you can incorporate the necessary logic to dynamically replace placeholders in prompts with context-specific information. This allows for greater customization and flexibility when configuring how tools interact with the AI in Wisej.AI.

If you modify the MyEmailTools example to include a placeholder in the tool's prompt, it might look something like this:

By using placeholders in prompts and implementing the logic to replace them with actual data dynamically, you can create flexible and context-aware tools in Wisej.AI.

Here are some functionalities you can use as inspiration for developing your tools:

Search the database
Search the file system
Search a document storage
Search the web

When augmented with sophisticated tools, there is virtually no limit to what an AI system can achieve.

Services

Wisej.AI's internal implementation leverages a variety of for its adapters, tools, embeddings, and other AI functionalities. As a developer, you have the flexibility to override these built-in implementations or replace them with your own, including the use of alternative AI frameworks.

Using Semantic Kernel

If your code relies on Semantic Kernel (SK), you can seamlessly integrate it into the Wisej.AI system at various points and execution levels. This flexibility allows you to enhance functionality and tailor the system to your specific needs.

As mentioned earlier, Wisej.AI functions at a higher level compared to foundational libraries like Semantic Kernel. While both Wisej.AI and SK offer similar basic functionalities, Wisej.AI provides a more integrated and advanced set of features designed for complex application development. To illustrate this, here is a comparison using a simple :

If you wish to integrate Semantic Kernel with Wisej.AI, you certainly can:

You can implement a new SKEndpoint derived from SmartEndpoint and use the SK's clients.
You can override the RunAsyncCore() in any of the existing adapters and use SK's plugin system, clients, embedding system, etc. Or you can implement it like this from the start in new adapters.
You can re-implement any of the that Wisej.AI uses in various places to use SK's vector storage support, document conversion, tokenization, chunking, etc.

Using Custom Libraries

While there may not be other directly comparable .NET libraries in the space that Wisej.AI occupies, the concept of integrating other libraries, like Semantic Kernel, applies broadly. You can seamlessly incorporate any other compatible library into your Wisej.AI applications. This approach allows you to extend the capabilities of your applications by combining the specific features and strengths of other libraries with the robust, high-level functionalities provided by Wisej.AI.

Architecture

Detailed description of the Wisej.AI architecture and all its components.

Overview

Wisej.AI is specifically designed with developers in mind to seamlessly integrate AI systems into business applications.

The system fundamentally operates by utilizing the application's controls and code to construct sophisticated prompts. These prompts are then sent to the AI provider, and the response is parsed in accordance with the requirements of the controls and code.

Wisej.AI is composed of these key components:

SmartEndpoint
SmartPrompt
SmartAdapter
SmartTool
SmartSession
SmartHub

In developing Wisej.AI, several fundamental principles were adopted: maintaining simplicity, avoiding over-engineering, delivering real value to developers, and ensuring the system is self-contained with minimal dependencies.

Wisej.AI supports .NET Core, .NET Framework, C#, and VB.NET.

To maintain simplicity and minimize dependencies, Wisej.AI does not utilize libraries such as Newtonsoft, Semantic Kernel, or any REST clients.

SmartEndpoint

As the name implies, the components are responsible for managing all communications between an endpoint and the rest of the system. An endpoint refers to the URL of the AI engine where the Large Language Model (LLM) is running. This can be hosted on a local server on-premises, a private server in a data center, at any public AI providers, or even in your browser as a JavaScript worker.

SmartEndpoint components are designed to support any AI provider and handle various payload types, including binary data and images. They define numerous properties and methods specifically tailored to handle inference and embedding requests efficiently.

SmartPrompt

The component manages both system and user prompts. It resolves prompt text from INI files, replaces placeholders with parameter values, and offers a straightforward method for interacting with the AI.

For instance, using Wisej.AI in its simplest form requires just an endpoint. Here's an example:

An even simpler approach, requiring fewer lines of code, is available by using the . However, regardless of the method chosen, an endpoint is always necessary.

Prompts can either be written directly in code or referenced using a key string enclosed in square brackets. SmartPrompt will locate the key and read the corresponding prompt from any .ini file stored in the /AI folder. See for more information on INI files.

Prompts can also include placeholders, or parameters, which are replaced at runtime. These placeholders must be enclosed within {{ }}. However, when setting a parameter value, you should specify the parameter name without the curly brackets.

Parameters are always replaced before submitting a question but only within the initial System Prompt and the current user prompt. The earlier example using parameters might look like this:

SmartAdapter

act as mediators between the application and the AI, adapting data and processes to fit specific needs. Each adapter has a distinct role and set of features. For instance, the SmartDataEntryAdapter extracts structured data from unstructured sources and updates Wisej.NET controls with the relevant information. Similarly, the SmartObjectAdapter performs this function by setting properties of .NET objects instead of updating controls. Another example is the SmartChartAdapter, which creates charts from unstructured data.

A SmartAdapter operates by leveraging an instance of the SmartHub, which in turn requires an instance of the SmartEndpoint. When using a SmartAdapter, you're essentially working within the structure outlined in the diagram at the top of this page.

Once the adapter is properly configured, the following example demonstrates the powerful capabilities of the SmartObjectAdapter:

With just one line of code, we accomplish a task that would be nearly impossible to achieve using standard programming methods.

SmartTool

are integral to the Wisej.AI architecture. By utilizing SmartTools, you can "compose" powerful AI systems that deliver significant value to your applications. A SmartTool, at its core, is a function—code that executes, accepts parameters, and returns a result.

In Wisej.AI, having an exceptionally powerful tools system is a core feature. SmartTools in Wisej.AI go beyond simple method callbacks or the plugin systems found in Semantic Kernel. They are designed to offer rich, extensible functionalities, providing developers with advanced capabilities to enhance and augment their applications seamlessly.

These functions (tools) operate within the context in which they were created, allowing them to:

Keep state information
Update controls on the screen
Work within the user session
Access any resource available to the application

Wisej.AI offers a unique capability to interact seamlessly with the user interface (UI) without limitations. Unlike Wisej.AI, Microsoft Semantic Kernel plugins are not integrated with any specific UI framework, which restricts their ability to perform such interactions. In any event, achieving this level of UI interaction would be extremely challenging with templating frameworks like Blazor, Angular, React, or other templating frameworks.

Since code often speaks louder than words, here is a very simple example to illustrate how SmartTools function:

Initially, the AI has responded that it cannot provide the current time. This is accurate, as AI models do not have the capability to gather real-time information outside of what is encoded in their neural network weights.

After incorporating prompt.UseTool(GetCurrentTime), the AI became capable of returning the current time. Essentially, we provided the AI with a tool, enabling it with a new capability. By programming such tools—be they small functions or more complex ones—we extend the AI's functionality.

In Wisej.AI, the interaction between a "new feature" (the tools) and the AI is handled effectively by the system. Essentially, every interaction is managed by an agent, ensuring seamless integration and functionality.

Wisej.AI employs its own tool definition prompting, rather than using the default defined by the AI model in use. This approach provides greater control over tool usage, reduces token consumption, and enables the use of tools even with models that were not originally trained with such capabilities.

Note that the AddTool() and AddTools() methods support call chaining:

Keep in mind that tools can also be implemented as asynchronous methods. This allows the code within these tools to interact with the user through the Wisej.NET user interface. Additionally, it enables the invocation of other services. Through the use of asynchronous methods, your application can handle long-running operations without blocking the user interface.

In addition to async tools, Wisej.AI also supports anonymous tools:

Being a lamba method, it runs in context allowing the code in the lamba to reference local variables declared in the containing code.

SmartSession

As explained on the General Concepts page, LLMs do not possess any memory, do not remember past interactions, and are inherently stateless. Each time you ask a question, it is akin to talking to .

Dory is a fictional blue tang fish and a major character of Pixar's animated film series Finding Nemo. She suffers from short-term memory loss.

The Wisej.AI agent system operates within the SmartSession. This is where Wisej.AI constructs the prompts, invokes the tools, and repeats the process until the LLM has completed the requested task. Additionally, the SmartSession in Wisej.AI is capable of managing message length when the context window is exceeded by either trimming or summarizing previous messages.

Every interaction with the LLM involves an instance of the SmartSession component. Even for a single prompt, Wisej.AI always creates and disposes of a session internally. This ensures that every request can utilize tools and operate within the boundaries set by Wisej.AI.

You can use the SmartSession, or a custom-derived class, to maintain the state of the interaction with the LLM and preserve the context effectively.

The example above demonstrates a session with two interactions with the LLM. In the first interaction, an image and a prompt are submitted. In the second interaction, you can refer to the image without resubmitting it, as it is part of the session and will be automatically included.

Interactions that include images are only compatible with models that have vision capabilities. Adapters that work with images have the UseOCR property allowing the usage the and extract text from the image (without relying on the model's vision) before sending it to the AI.

SmartHub

The final component is the . As the name suggests, it functions as a central hub, connecting an endpoint, multiple adapters, custom tools, and your Wisej.NET application.

With the SmartHub, you can manage messages, errors, and parameters related to all the adapters connected to both the SmartHub and the endpoint. A SmartHub always requires one endpoint, but it can be associated with multiple adapters.

In addition to centralizing events and connecting adapters to an endpoint, the SmartHub offers a variety of higher-level functions that leverage all the built-in services in Wisej.AI. Essentially, it provides developers with a high-level interface to interact with the AI.

When the SmartHub is placed onto a designer surface of a component, it automatically binds to the top-level component and registers all methods marked with the [SmartTool.Tool] attribute as tools.

In all code snippets when an endpoint is created without an api key, the assumption is that you have the ApiKeys.json file in /AI, see .

Events

In Wisej.AI, the majority of objects are designed to trigger various events, enabling applications to interact with AI processes at multiple stages. While individual SmartAdapters may offer their own unique events, three key components—, , and —uniformly raise a standardized set of events. This consistent event pattern facilitates seamless integration and enhances the interactivity between the application and Wisej.AI functionalities.

Name

Description

Events are typically triggered in the following sequence:

SmartSession
SmartPrompt
SmartHub

When event arguments are derived from HandledEventArgs, listeners have the option to set the Handled property to true. This signals that the event has already been addressed by the listener. For instance, when handling the BeforeInvokeTool event, if you set e.Handled to true, the system will not proceed to invoke the tool, as it will presume that your code has already taken care of the necessary actions.

This feature allows you to intercept a tool call and carry out additional operations either before, after, or instead of executing the tool. This capability provides flexibility in customizing the behavior of your application, enabling you to implement complex logic or alternative workflows as needed.