SmartObjectAdapter

Overview

The SmartObjectAdapter is a component that seamlessly integrates an AI model with an application's objects. Its function is to transform unstructured data into a structured instance of a .NET class, thereby facilitating easier manipulation and analysis within the application.

The sources of this unstructured data are diverse and can include files, streams, images, text, or clipboard content, which may contain either textual data or images. This capability allows for a more versatile and efficient handling of data, making the SmartObjectAdapter an essential tool in the Wisej.AI suite.

Configuration

[WorksWith(typeof(SmartHttpEndpoint))]

Properties

In addition to the properties inherited from the SmartAdapter, the SmartObjectAdapter exposes the following additional properties.

UseOCR

The UseOCR property determines whether Optical Character Recognition (OCR) service should be used for processing images. When this property is set to true, Wisej.AI utilizes the IOCRService to carry out OCR operations, with the DefaultOCRService implementation leveraging Wisej.NET's Tesseract extension to perform OCR directly in the user's browser.

Conversely, when UseOcr is set to false, which is the default setting, the image data is forwarded to the AI model, which is expected to possess vision capabilities to handle the image processing tasks.

MaxIterations

The MaxIterations property specifies the maximum number of attempts the adapter will make to utilize the available tools in order to fill in any missing values. This setting ensures that the adapter will make multiple efforts to complete the data, thus increasing the likelihood of success in providing a comprehensive output. Configuring this property allows developers to balance between performance and accuracy, by governing how persistent the adapter should be in resolving incomplete data scenarios.

Methods

The most relevant methods of this adapter are prefixed with "From," such as FromClipboardAsync(), FromStreamAsync(), and others. All methods are available in their generic forms and also accept a Type reference as a parameter.

FromTextAsync

This function converts the given text into an object of the specified type.

var invoice = await adapter.FromTextAsync<invoice>(text);

// or

var invoice = (Invoice)(await adapter.FromTextAsync(text, typeof(Invoice));)

FromImageAsync

This function converts the specified image into an object of a given type.

var invoice = await adapter.FromImageAsync<invoice>(image);

// or

var invoice = (Invoice)(await adapter.FromImageAsync(image, typeof(Invoice));)

FromStreamAsync

This function converts the specified stream into an object of a given type.

The final parameter is the content type, and it is optional. If not specified, Wisej.AI will attempt to autodetect the content type from the following options: PDF, PNG, JPG, BMP, GIF, and DOCX. If autodetection fails, it will default to TXT.

var invoice = await adapter.FromStreamAsync<invoice>(File.OpenRead("Invoice.pdf"));
var invoice = await adapter.FromStreamAsync<invoice>(File.OpenRead("Invoice.pdf"), ".pdf");

// or

var invoice = (Invoice)(await adapter.FromStreamAsync(image, typeof(Invoice), File.OpenRead("Invoice.pdf"));)

FromClipboardAsync

This function converts the content of the clipboard into an object of a specified type. It supports both text and image conversions.

var invoice = await adapter.FromClipboardAsync<invoice>();

// or

var invoice = (Invoice)(await adapter.FromClipboardAsync(typeof(Invoice));)

Please be aware that the web page must be in focus for the browser to access and read the clipboard contents.

Attributes

The adapter utilizes several attributes to collect information about the target class. Both the class itself and its members should be decorated with these attributes, as they provide instructions to the AI model.

Description

The Description attribute supplies extra context to the AI model, enhancing the information available from the member names in the code.

The Description attribute can be applied to describe either a class or a property. When used for a property, it serves as an alternative to the FieldPrompt attribute, although the FieldPrompt attribute takes precedence if both are present.

[Description("Represents an employee in the HR system.")]
public class Employee {
    [Description("First name. Capitalize it.")]
    public string Name{ get; set; }
    
    [Description("Extract a list of phone numbers. Use the format (999)-999-9999.")]
    public string[] ContactNumbers { get; set; }
}

FieldName

The FieldName attribute is used to specify a more descriptive name for a field in the AI model. For instance, if your class includes a property with an ambiguous name, such as DOB, you can use the FieldName attribute to clarify its meaning: [FieldName("Birthday")].

FieldPrompt

The FieldPrompt attribute can be utilized in conjunction with both the FieldName and FieldRectangle attributes to provide additional instructions and context. This combination aids the AI model in accurately extracting or generating the required value by offering more clarity and guidance on how the data should be interpreted.

[Description("Extract list of personal interests from the text, summarize in 1-3 words maximum and change to title case")]
public string[] Interests { get; set; }

FieldRectangle

FieldRectangle is applicable only when extracting data from an image. It defines an optional rectangle used to crop the image, assisting the AI model in concentrating on a specific area.

You can use the same rectangle for multiple fields. Wisej.AI internally groups controls sharing the same rectangle and optimizes the requests to the AI model.

Events

ObjectParsed

The event is triggered once the adapter has successfully completed parsing an object, but prior to the value being returned from one of the From methods. By handling this event, you have the opportunity to modify the parsed object as needed.

Name	Description
e.Value	The parsed object.