Built-in SmartTools

Composable built-in tools

Overview

The integration of tools with AI and Wisej.NET applications and controls is one of the most powerful features of Wisej.AI. This integration offers unmatched capabilities, enabling developers to implement features that would be extremely challenging to achieve with other frameworks.

While using tools (or plugins) alongside LLMs is a common practice, Wisej.AI enables these tools to interact with the application and the browser in ways that are uniquely possible within the Wisej.NET architecture. For example, a tool can execute code directly in the user's browser and return the result to the AI. It can also display complex dialogs while the AI waits for a response or authenticate with services like Azure Entra ID using standard authentication flows, among other capabilities.

Some adapters rely entirely on a tool to specialize their functionality. For example, the SmartChartAdapter is a generic adapter designed for any chart control. However, the SmartChartJS3Adapter, which is derived from SmartChartAdapter, uses ChartJS3Tools specifically to populate the ChartJS3 control. This design means that by simply replacing the tool, the SmartChartAdapter can be adapted to work with chart controls from DevExpress, Syncfusion, Telerik, and others.

Wisej.AI tools have the capability to interact seamlessly with the surrounding application and code, external APIs, the user's browser, the user interface, and other large language models (LLMs).

Using Tools

You can integrate smart tools into various components, such as a SmartHub, SmartAdapter, SmartSession, or SmartPrompt. Tools added to a SmartHub are accessible to all connected adapters. Tools added to a SmartAdapter are limited to that specific adapter. Tools added to a SmartSession remain available throughout the entire session. Tools added to a SmartPrompt are accessible only within the context of that particular prompt.

Code Interaction

Considering how tools operate within an AI system, interacting with the code inherently involves interacting with the surrounding Wisej.NET application. For instance, if you want the SmartObjectAdapter to populate an object of type IncidentReport, including a field ReportID that cannot be part of the incident report itself but represents the ID of the active report currently open in the application, this what the tool would look like in C#:

// Tool to read the current ReportID
[SmartTool.Tool]
[Description("Returns the ReportID of the current incident report.")]
private string get_current_reportId() {
    return ((IncidentReportPage)Application.MainPage).GetReportID();
}

// Read the data from the clipboard
LoadIncidentReport(
    await this.smartObjectAdapter.FromClipboardAsync<IncidentReport>());

In the example above, no code was written to associate the tool with the adapter. This is because the container is linked to a SmartHub, which in turn is connected to the adapter. The SmartHub automatically registers all methods in the container that are marked with [SmartTool.Tool] across all adapters, streamlining the integration process.

External APIs Interaction

In Wisej.AI, tools can operate both synchronously and asynchronously. This flexibility allows a method to query any external API, or even multiple APIs, while the Wisej.AI agent "waits" without encountering timeouts or blocking any threads.

For example, the code snippet below demonstrates a simple weather tool that uses two HTTP REST calls to retrieve the weather forecast for a specific location.

[SmartTool.Tool]
[Description("[SmartChat.GetWeatherTool]")]
public async Task<string> GetWeather(

	[Description("[SmartChat.GetWeatherTool.latitude]")]
	float latitude,
	
	[Description("[SmartChat.GetWeatherTool.longitude]")]
	float longitude
)
{
	var client = new HttpClient();
	client.DefaultRequestHeaders.Add("User-Agent", "Wisej.AI");

	// first API call to the next endpoint
	var response = await client.GetAsync(Invariant($"https://api.weather.gov/points/{latitude},{longitude}"));
	var json = await response.Content.ReadAsStringAsync();
	var data = JSON.Parse(json);

	// second API call to get the forecast
	response = await client.GetAsync(data.properties.forecast);
	var weather = JSON.Parse(await response.Content.ReadAsStringAsync());
	return weather.properties.periods[0].detailedForecast;
}

Note that the sample code above does not directly include descriptions of the method and its arguments. Instead, it uses the "[Description]" attribute to reference text stored in an INI file located in the /AI folder.

User Browser Interaction

Wisej.AI tools have the capability to invoke code in the user's browser while the agent "waits." The sample C# code below demonstrates how to provide expression execution capabilities to the AI by requesting JavaScript in the browser to evaluate an expression.

[SmartTool.Tool]
[Description("[MathTools.evaluate_expression]")]
protected virtual Task<dynamic> evaluate_expression(

	[Description("[MathTools.evaluate_expression.expression]")]
	string expression)
{
	var result = Application.EvalAsync(expression);
	Application.Update(Application.Current);
	return result;
}

It's important to note the line Application.Update(Application.Current) in the sample above. This line is necessary to push a browser update between the evaluation and returning the value. Since the code operates within the current Wisej.NET application and browser session, it needs to send back the code for execution in the browser.

A tool interacting with the browser can also utilize native JavaScript popups to request information from the user:

[SmartTool.Tool]
[Description("Asks the user to answer a question from the assistant.")]
public Task<string> GetInformationFromUser(
	[Description("Question to ask the user")]
	string question)
{
	var response = Application.PromptAsync(question);
	Application.Update(this);
	return response;
}

User Interface Interaction

Interacting with the UI is one of the most powerful features of Wisej.AI tools. Consider a scenario where the AI is loading data, searching a database, or generating a report and needs to request user permission or prompt the user to select a record from the application, or enter additional data, among other interactions.

[SmartTool.Tool]
[Description("Returns the customer sales records required to calculate the ROI")]
public async Task<string[]> GetCustomerSalesAsync()
{
	var dlg = new PickCustomerDialog();
	var result = await dlg.ShowDialogAsync();
	if (result == DialogResult.OK)
	{
		var id = dlg.CustomerId;
		return LoadSalesRecords(id).ToArray();
	}

	return null;
}

Other LLMs Interaction

Last but not least, a tool can also invoke another model, AI provider, or even another AI framework, such as Semantic Kernel.