Extensibility

How to Extend and Integrate Wisej.AI

Overview

Wisej.AI offers unparalleled flexibility, ensuring that you are not limited by its predefined features. The system is designed to be open and highly extensible. In fact, the majority of the components have virtual members, allowing you to override nearly any functionality. You have the ability to customize all built-in prompts and connect to a wide range of events at various levels, providing a seamless integration experience.

Considering the architecture of Wisej.AI, you have several options for overriding its built-in functionality and integrating any other AI library of your choice.

SmartEndpoint

You can override any of the built-in SmartEndpoints to reuse the existing functionality as needed. Alternatively, you can create a brand-new endpoint by extending either the base SmartEndpoint or SmartHttpEndpoint. For instance, if you want to integrate a third-party client for OpenAI (or any other provider) into the Wisej.AI system, you can do so by extending SmartEndpoint and implementing at least the abstract methods. This approach allows you to customize and tailor the integration according to your specific requirements.

Many AI providers have standardized their APIs based on the OpenAI API. In such cases, you can use the OpenAIEndpoint within Wisej.AI and modify parameters like the URL, model, and authentication as needed. Alternatively, you can extend the OpenAIEndpoint to integrate different properties and adjust any variances specific to your provider. This flexibility allows for seamless adaptation to various AI service requirements.

SmartAdapter

To enhance any control with AI features in Wisej.AI, you can create custom adapters. Start by deriving your adapter from SmartAdapter and place your implementation in the RunAsyncCore() method. Within this method, you are responsible for interacting with the AI service and parsing its response. Notably, you are not required to use any of the SmartEndpoints within this process.

For example, if you want to create an adapter that populates a ComboBox with individual text items generated from an AI prompt, for simplicity you can use the Tag property of the enhanced Wisej.Web.ComboBox control to store the prompt. This approach provides a straightforward way to integrate AI-driven functionality into your applications.

[Extends(typeof(ComboBox), allowMultiple: true)]
public class MyComboBoxAdapter : SmartAdapter
{
   SmartPrompt _prompt;
   public MyComboBoxAdapter()
   {
       _prompt = new SmartPrompt(
         "Your job is to create a comma separated list " + 
         "of the items described by the user.");
   }

   protected override async Task<SmartSession.Message> RunAsyncCore(
       Control control)
   {
       var comboBox = control as ComboBox;
       if (comboBox == null)
           return null;

       var question = control.Tag as string;
       if (String.IsNullOrEmpty(question))
           return null;

       var response = await _prompt.AskAsync(this.Hub, question);
       comboBox.Items.Clear();
       comboBox.Items.AddRange(
           response.Text.Split(',')
                        .Select(t => t.Trim())
                        .ToArray());

       return response;
   }
}

<Extends(GetType(ComboBox), AllowMultiple:=True)>
Public Class MyComboBoxAdapter
    Inherits SmartAdapter

    Private _prompt As SmartPrompt

    Public Sub New()
        _prompt = New SmartPrompt( _
            "Your job is to create a comma " & _
            "separated list of the items described by the user.")
    End Sub

    Protected Overrides Async Function RunAsyncCore(control As Control) 
    As Task(Of SmartSession.Message)
        Dim comboBox = TryCast(control, ComboBox)
        If comboBox Is Nothing Then
            Return Nothing
        End If

        Dim question = TryCast(control.Tag, String)
        If String.IsNullOrEmpty(question) Then
            Return Nothing
        End If

        Dim response = Await _prompt.AskAsync(Me.Hub, question)
        comboBox.Items.Clear()
        comboBox.Items.AddRange(response.Text.Split(","c).
                                Select(Function(t) t.Trim()).
                                ToArray())
        Return response
    End Function
End Class

If you wish to enhance the functionality of a control by adding a new property, you can make your adapter implement the IExtenderProvider interface. This allows the adapter to introduce new properties to the controls it augments. By implementing this interface, you can dynamically extend the capabilities of the associated controls, offering a more sophisticated level of customization.

[ProvideProperty("ItemsPrompt", typeof(ComboBox))]
[Extends(typeof(ComboBox), allowMultiple: true)]
public class MyComboBoxAdapter : SmartAdapter, IExtenderProvider
{
	// Same as above

	[Category("AI Features")]
	public string GetItemsPrompt(Control control)
	{
		return control.UserData.ItemsPrompt as string;
	}

	public void SetItemsPrompt(Control control, string text)
	{
		control.UserData.ItemsPrompt = text;
	}

	bool IExtenderProvider.CanExtend(object extendee)
	{
		var control = extendee as Control;
		if (IsAssociatedWith(control))
			return control is ComboBox;

		return false;
	}
}

<ProvideProperty("ItemsPrompt", GetType(ComboBox))>
<Extends(GetType(ComboBox), allowMultiple:=True)>
Public Class MyComboBoxAdapter
    Inherits SmartAdapter
    Implements IExtenderProvider
    
    '' Same as above

    <Category("AI Features")>
    Public Function GetItemsPrompt(control As Control) As String
        Return TryCast(control.UserData.ItemsPrompt, String)
    End Function

    Public Sub SetItemsPrompt(control As Control, text As String)
        control.UserData.ItemsPrompt = text
    End Sub

    Private Function CanExtend(extendee As Object) As Boolean 
    Implements IExtenderProvider.CanExtend
        Dim control As Control = TryCast(extendee, Control)
        If IsAssociatedWith(control) Then
            Return TypeOf control Is ComboBox
        End If
        Return False
    End Function

End Class

After creating the MyComboBoxAdapter as outlined above, follow these steps to integrate it into your project:

Add a SmartHub and SmartEndpoint to your project.
Select your MyComboBoxAdapter.
Drag and drop your adapter onto the open form designer.
Associate the adapter with an existing ComboBox in your interface.

Once the adapter is linked to the ComboBox, it will automatically add the ItemsPrompt property to the ComboBox. Set this property to something like "European countries" or "US states" and run the application to see it in action. This will allow the ComboBox to populate dynamically based on the AI prompt specified in the ItemsPrompt property, or the Tag property if you didn't extend the IExtenderProvider interface.

SmartTool

One of the most powerful features in Wisej.AI is the SmartTool system. You have the flexibility to utilize or extend any built-in tool, or you can create a new tool from scratch. The system is designed to make this process extremely straightforward and user-friendly, allowing developers to easily customize and extend their application's capabilities.

You can provide tools to the AI in multiple ways and at different levels within the Wisej.AI framework. For instance, if you annotate a method on the control being designed with the [SmartTool.Tool] attribute and have a SmartHub component in place, the SmartHub will automatically include this method in the tools available to the AI. This inclusion is independent of the adapter, prompt, or endpoint being used. In essence, once a tool is added to a SmartHub, it is consistently made available to the AI model, ensuring seamless integration and accessibility.

Ways to provide tools to the AI:

To turn a method into a tool within the Wisej.AI system, annotate it with the [SmartTool.Tool] attribute. This method will be available as a tool if it is declared within a container that's associated with a SmartHub. Alternatively, the method can become a tool if it is part of an object that is passed to the UseTools() method of a SmartHub, SmartPrompt, SmartAdapter, or SmartSession. This flexible integration ensures that your tools can be widely accessible across different components of your application.
You can directly provide a specific method to the SmartHub, SmartPrompt, SmartAdapter, or SmartSession components by using the UseTool() method. For example, to pass a method such as get_customer_id to a SmartHub, you would use the following code: smartHub1.UseTool(this.get_customer_id);. This approach allows you to easily leverage specific methods as tools without needing to encapsulate them within a class or container.

For instance, if you want to enable the AI to send an email while performing other tasks, you can create a tools class with two methods. One method can be designed to send the email, while the other can be developed to find the recipient's email address, adding an intelligent layer to the process. By integrating this tools class into the Wisej.AI framework, you can provide the AI with advanced capabilities for handling emails efficiently as part of its operations.

[Description("[MyEmailTools]")]
public class MyEmailTools
{
 [SmartTool.Tool]
 private string find_email_address(string name)
 {
     var emailAddress = "[email protected]";
     return emailAddress;
 }

 [SmartTool.Tool]
 private bool send_email(string emailAddress, string subject, string body)
 {
     var email = new Emailer();
     return email.SendEmail(emailAddress, subject, body);
 }
}

...

this.smartDataEntryAdapter1.UseTools(new MyEmailsTools());

MyPrompts.ini file

# Prompt for MyEmailTools
[MyEmailTools]
Use this tool to send an email to the account manager assigned 
to the new case entered into the system. The subject should be the 
case number followed by a - and a short summary of the case. The body
should contain all the data of the case in a table.

<Description("[MyEmailTools]")>
Public Class MyEmailTools
    <SmartTool.Tool>
    Private Function find_email_address(name As String) As String
        Dim emailAddress = "[email protected]"
        Return emailAddress
    End Function

    <SmartTool.Tool>
    Private Function send_email(
    emailAddress As String, subject As String, body As String) As Boolean
        Dim email = New Emailer()
        Return email.SendEmail(emailAddress, subject, body)
    End Function
End Class

...

Me.smartAdapter1.UseTools(New MyEmailTools())

MyPrompts.ini file

# Prompt for MyEmailTools
[MyEmailTools]
Use this tool to send an email to the account manager assigned 
to the new case entered into the system. The subject should be the 
case number followed by a - and a short summary of the case. The body
should contain all the data of the case in a table.

You can enhance the usability and clarity of the tools within Wisej.AI by using the Description attribute to annotate various components. You can annotate the tools container class, each method, and each parameter with this attribute. These additional annotations provide the AI with detailed information, allowing it to utilize the tools more effectively and accurately, ensuring that each tool is used correctly within your application.

This is an example of the built-in annotation for the DatabaseTools bulilt-in tools container:

#
# DatabaseTools
#
[DatabaseTools]
Provides tools to access the database.
Unless instructed otherwise, use it before the web search tools.

Instructions:
- Use only the tables defined in the Database Schema.
- The generated SQL statement MUST be valid for "{{server-type}}".
- Define column aliases within single quotes.
- Enclose column names in [].

```Database Schema
{{database-schema}}
```

[DatabaseTools.select]
Executes a SQL SELECT command on the database to retrieve the specified data.

[DatabaseTools.select.sql]
A well-formed SQL SELECT statement using the tables and 
columns defined in the Database Schema.

Using an INI file offers several advantages when working with prompts in Wisej.AI. It allows you to craft clearer and more complex prompts and gives you the flexibility to fine-tune these prompts externally, without needing to modify your application's code. This external configuration capability enhances maintainability and adaptability, enabling you to adjust prompts as needed without redeploying your application.

In Wisej.AI, a tool can modify its own prompt by programmatically replacing placeholders like {{server-type}} and {{database-schema}} with specific values relevant to the tool itself. This can be achieved by implementing logic within the tool's initialization or execution process to dynamically insert the appropriate values into the prompt.

For instance, you can define a method within the tool that retrieves the required information—for example, the server type or database schema—and injects these values into the prompt before it is processed. This approach ensures that the prompt is precisely tailored based on the tool's configuration or current state, maintaining the flexibility and relevance of the information provided to the AI.

To enable a tool to modify its own prompt with specific values for placeholders, your tools container class should either implement the IToolsContainer interface or extend the ToolsContainer class. By leveraging these options, you can incorporate the necessary logic to dynamically replace placeholders in prompts with context-specific information. This allows for greater customization and flexibility when configuring how tools interact with the AI in Wisej.AI.

If you modify the MyEmailTools example to include a placeholder in the tool's prompt, it might look something like this:

MyPrompts.ini file

# Prompt for MyEmailTools
[MyEmailTools]
Use this tool to send an email to the {{recipient_types}} assigned 
to the new case entered into the system. The subject should be the 
case number followed by a - and a short summary of the case. The body
should contain all the data of the case in a table.

public MyEmailTools(bool notifySupervisor)
{
 if (!notifySupervisor)
     this.Parameters.Add("recipient_types", "Account Manager");
 else
     this.Parameters.Add("recipient_types", "Account Manager and Supervisor");
}

...

this.smartDataEntryAdapter1.UseTools(new MyEmailsTools(true));

MyPrompts.ini file

# Prompt for MyEmailTools
[MyEmailTools]
Use this tool to send an email to the {{recipient_types}} assigned 
to the new case entered into the system. The subject should be the 
case number followed by a - and a short summary of the case. The body
should contain all the data of the case in a table.

Public Sub New(notifySupervisor As Boolean)
    If Not notifySupervisor Then
        Me.Parameters.Add("recipient_types", "Account Manager")
    Else
        Me.Parameters.Add("recipient_types", "Account Manager and Supervisor")
    End If
End Sub

...

Me.smartDataEntryAdapter1.UseTools(New MyEmailTools(True))

By using placeholders in prompts and implementing the logic to replace them with actual data dynamically, you can create flexible and context-aware tools in Wisej.AI.

Here are some functionalities you can use as inspiration for developing your tools:

Search the database
Search the file system
Search a document storage
Search the web
Interact with Office 365 using Microsoft Graph
Read incoming emails
Send emails
Read signals in a SCADA system

When augmented with sophisticated tools, there is virtually no limit to what an AI system can achieve.

Services

Wisej.AI's internal implementation leverages a variety of services for its adapters, tools, embeddings, and other AI functionalities. As a developer, you have the flexibility to override these built-in implementations or replace them with your own, including the use of alternative AI frameworks.

Using Semantic Kernel

If your code relies on Semantic Kernel (SK), you can seamlessly integrate it into the Wisej.AI system at various points and execution levels. This flexibility allows you to enhance functionality and tailor the system to your specific needs.

As mentioned earlier, Wisej.AI functions at a higher level compared to foundational libraries like Semantic Kernel. While both Wisej.AI and SK offer similar basic functionalities, Wisej.AI provides a more integrated and advanced set of features designed for complex application development. To illustrate this, here is a comparison using a simple tool example:

using Wisej.AI;
using Wisej.AI.Endpoints;

// Create a smart prompt with a plugin (defined elsewhere)
var prompt = new SmartPrompt().UseTool(new LigthsPlugin());

// Create the AzureAI endpoint
var endpoint = new AzureAIEndpoint { URL = url };

// Ask the question and get the answer from the AI
var answer = prompt.AskAsync(endpoint, "Please turn on the lamp");

// Print the results
Console.WriteLine("Assistant > " + answer);

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;

// Create a kernel with Azure OpenAI chat completion
var builder = Kernel.CreateBuilder().AddAzureOpenAIChatCompletion(modelId, endpoint, apiKey);

// Build the kernel
Kernel kernel = builder.Build();
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

// Add a plugin (the LightsPlugin class is defined elsewhere)
kernel.Plugins.AddFromType<LightsPlugin>("Lights");

// Enable planning
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new() 
{
    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
};

// Create a history store the conversation
var history = new ChatHistory();
history.AddUserMessage("Please turn on the lamp");

// Get the response from the AI
var result = await chatCompletionService.GetChatMessageContentAsync(
   history,
   executionSettings: openAIPromptExecutionSettings,
   kernel: kernel);

// Print the results
Console.WriteLine("Assistant > " + result);

// Add the message from the agent to the chat history
history.AddAssistantMessage(result);

If you wish to integrate Semantic Kernel with Wisej.AI, you certainly can:

You can implement a new SKEndpoint derived from SmartEndpoint and use the SK's clients.
You can override the RunAsyncCore() in any of the existing adapters and use SK's plugin system, clients, embedding system, etc. Or you can implement it like this from the start in new adapters.
You can re-implement any of the services that Wisej.AI uses in various places to use SK's vector storage support, document conversion, tokenization, chunking, etc.

Using Custom Libraries

While there may not be other directly comparable .NET libraries in the space that Wisej.AI occupies, the concept of integrating other libraries, like Semantic Kernel, applies broadly. You can seamlessly incorporate any other compatible library into your Wisej.AI applications. This approach allows you to extend the capabilities of your applications by combining the specific features and strengths of other libraries with the robust, high-level functionalities provided by Wisej.AI.

PreviousArchitecture NextAI Providers

Last updated 6 months ago