LogoLogo
HomeNewsSupportVideos
  • Welcome
  • Wisej.NET
  • Concepts
    • Getting Started
    • General Concepts
    • Architecture
    • Extensibility
    • AI Providers
    • Vector Databases
    • Usage Metrics
    • Logging & Error Handling
  • Markup
  • Components
    • API
      • SmartAdapter
        • SmartAudioTTSAdapter
        • SmartAudioWhisperAdapter
        • SmartCalendarAdapter
        • SmartChartAdapter
        • SmartChartJS3Adapter
        • SmartChatBoxAdapter
        • SmartComboBoxAdapter
        • SmartCopilotAdapter
        • SmartDataEntryAdapter
        • SmartDocumentAdapter
        • SmartFullCalendarAdapter
        • SmartObjectAdapter
        • SmartPictureBoxAdapter
        • SmartQueryAdapter
        • SmartRealtimeAdapter
        • SmartReportAdapter
        • SmartTextBoxAdapter
        • SmartAdapter.ExtendsAttribute
        • SmartAdapter.FieldNameAttribute
        • SmartAdapter.FieldPromptAttribute
        • SmartAdapter.FieldRectangleAttribute
        • SmartAdapter.WorksWithAttribute
      • SmartEndpoint
        • AmazonBedrockEndpoint
        • AnthropicEndpoint
        • AzureAIEndpoint
        • CerebrasEndpoint
        • DeepSeekEndpoint
        • GoogleAIEndpoint
        • GroqCloudEndpoint
        • GroqCloudEndpointWhisper
        • HuggingFaceEndpoint
        • HuggingFaceJavaScriptEndpoint
        • LocalAIEndpoint
        • LocalAIEndpointImageGen
        • LocalAIEndpointTTS
        • LocalAIEndpointWhisper
        • NvidiaAIEndpoint
        • OllamaEndpoint
        • OpenAIEndpoint
        • OpenAIEndpointDallE
        • OpenAIEndpointRealtime
        • OpenAIEndpointTTS
        • OpenAIEndpointWhisper
        • SambaNovaEndpoint
        • SmartHttpEndpoint
        • TogetherAIEndpoint
        • XAIEndpoint
        • SmartEndpoint.Metrics
        • SmartEndpoint.Response
      • SmartExtensions
      • SmartHub
        • SmartSession.ConvertParameterEventArgs
        • SmartSession.ConvertParameterEventHandler
        • SmartSession.ErrorEventArgs
        • SmartSession.ErrorEventHandler
        • SmartSession.InvokeToolEventArgs
        • SmartSession.InvokeToolEventHandler
        • SmartSession.MessagesEventArgs
        • SmartSession.MessagesEventHandler
      • SmartObject
      • SmartPrompt
        • SmartAgentPrompt
        • SmartParallelPrompt
        • SmartPrompt.Parameter
        • SmartSession.ConvertParameterEventArgs
        • SmartSession.ConvertParameterEventHandler
        • SmartSession.ErrorEventArgs
        • SmartSession.ErrorEventHandler
        • SmartSession.InvokeToolEventArgs
        • SmartSession.InvokeToolEventHandler
        • SmartSession.MessagesEventArgs
        • SmartSession.MessagesEventHandler
      • SmartRealtimeSession
      • SmartSession
        • SmartSession.ConvertParameterEventArgs
        • SmartSession.ConvertParameterEventHandler
        • SmartSession.ErrorEventArgs
        • SmartSession.ErrorEventHandler
        • SmartSession.InvokeToolEventArgs
        • SmartSession.InvokeToolEventHandler
        • SmartSession.Message
        • SmartSession.MessageCollection
        • SmartSession.MessageRole
        • SmartSession.MessagesEventArgs
        • SmartSession.MessagesEventHandler
        • SmartSession.TrimmingStrategy
      • SmartTool
        • SmartTool.IToolProvider
        • SmartTool.ToolAttribute
        • SmartTool.ToolContext
      • Markup
        • MarkupExtensions
      • Controls
        • UVLightOverlay
      • Embeddings
        • EmbeddedDocument
        • Embedding
        • Matches
        • Metadata
      • Helpers
        • ApiKeys
        • Markdown
        • TextTokenizer
      • Services
        • DefaultSessionTrimmingService
        • IDocumentConversionService
          • DefaultDocumentConversionService
        • IEmbeddingGenerationService
          • DefaultEmbeddingGenerationService
          • HuggingFaceEmbeddingGenerationService
        • IEmbeddingStorageService
          • AzureAISearchEmbeddingStorageService
          • ChromaEmbeddingStorageService
          • FileSystemEmbeddingStorageService
          • MemoryEmbeddingStorageService
          • PineconeEmbeddingStorageService
          • QdrantEmbeddingStorageService
        • IHttpClientService
          • DefaultHttpClientService
        • ILoggerService
          • DefaultLoggerService
        • IOCRService
          • DefaultOCRService
        • IRerankingService
          • DefaultRerankingService
          • LocalAIRerankingService
          • PineconeRerankingService
        • ISessionTrimmingService
          • DefaultSessionTrimmingService
        • ITextSplitterService
          • RecursiveCharacterTextSplitterService
          • TextSplitterServiceBase
        • ITokenizerService
          • DefaultTokenizerService
        • IWebSearchService
          • BingWebSearchService
          • BraveWebSearchService
          • GoogleWebSearchService
      • Tools
        • ArxivTools
        • ChartJS3Tools
        • DatabaseTools
        • DataTableFilterTools
        • DocumentSearchTools
        • DocumentTools
        • FullCalendarTools
        • IToolsContainer
        • MathTools
        • ToolsContainer
        • UtilityTools
        • WebSearchTools
    • Built-in Services
      • IOCRService
      • ILoggerService
      • ITextSplitterService
      • ITokenizerService
      • IHttpClientService
      • IWebSearchService
      • IRerankingService
      • ISessionTrimmingService
      • IDocumentConversionService
      • IEmbeddingStorageService
      • IEmbeddingGenerationService
    • Built-in SmartTools
      • ToolsContainer
      • MathTools
      • UtilityTools
      • DatabaseTools
      • DocumentTools
      • DocumentSearchTools
      • WebSearchTools
      • ChartJS3Tools
      • FullCalendarTools
    • Built-in SmartAdapters
      • SmartAdapter
      • SmartAudioTTSAdapter
      • SmartAudioWhisperAdapter
      • SmartCalendarAdapter
      • SmartChartAdapter
      • SmartChartJS3Adapter
      • SmartChatBoxAdapter
      • SmartComboBoxAdapter
      • SmartCopilotAdapter
      • SmartDataEntryAdapter
      • SmartDocumentAdapter
      • SmartFullCalendarAdapter
      • SmartObjectAdapter
      • SmartPictureBoxAdapter
      • SmartQueryAdapter
      • SmartRealtimeAdapter
      • SmartReportAdapter
      • SmartTextBoxAdapter
    • Configure Services
    • Using SmartHub
    • Using SmartTools
    • Using SmartPrompt
    • Using SmartSession
    • Using SmartRealTimeAdapter
    • UVLightOverlay Control
Powered by GitBook
On this page
  • Constructors
  • RecursiveCharacterTextSplitterService(chunkSize, chunkOverlap, separators, lengthFunction)
  • Methods
  • Split(text)
  • Implements
Export as PDF
  1. Components
  2. API
  3. Services
  4. ITextSplitterService

RecursiveCharacterTextSplitterService

Wisej.AI.Services.RecursiveCharacterTextSplitterService

PreviousITextSplitterServiceNextTextSplitterServiceBase

Last updated 5 days ago

Namespace: Wisej.AI.Services

Assembly: Wisej.AI (3.5.0.0)

A service for recursively splitting text into chunks based on specified separators and chunk size constraints. This service attempts to split text by different characters to find an optimal separation strategy.

public class RecursiveCharacterTextSplitterService : TextSplitterServiceBase
Public Class RecursiveCharacterTextSplitterService
    Inherits TextSplitterServiceBase

Constructors

RecursiveCharacterTextSplitterService(chunkSize, chunkOverlap, separators, lengthFunction)

Initializes a new instance of the class with specified separators, chunk size, overlap, and length function.

Name
Type
Description

chunkSize

The maximum size of each text chunk. Defaults to 1000 if not specified.

chunkOverlap

The allowed overlap size between chunks. Defaults to 200 if not specified.

separators

The array of string separators to be used for splitting the text. Default is an array containing "\n\n", "\n", " ", and "".

lengthFunction

A function to determine the length of the text, which will be used to comply with the chunk size constraint.

Methods

Split(text)

Splits the given text into chunks using the defined separators and chunk size constraints.

Parameter
Type
Description

text

The text to be split into chunks.

This method seeks to split the provided text based on the list of separators, starting with more significant separators and moving to less significant ones. If none of the separators are found, it will treat the text as a sequence of individual characters. The process ensures that each chunk does not exceed the specified chunk size. If a text segment is larger than the specified chunk size, it will be recursively split further. Usage example:


  var separators = new[] { ",", " ", "\n" };
  var splitterService = new RecursiveCharacterTextSplitterService(separators, 500, 100, text => text.Length);
  string textToSplit = "This is a sample text that will be split into smaller chunks.";
  string[] chunks = splitterService.Split(textToSplit);
  foreach (var chunk in chunks)
  {
   Console.WriteLine(chunk);
  }

Throws:

Implements

Name
Description

Represents a service for splitting text into an array of substrings.

Returns: . An array of strings, where each string represents a chunk of the original text.

Thrown when the text is null.

String[]
ArgumentNullException
Int32
Int32
String[]
Func<String, Int32>
String
ITextSplitterService
TextSplitterServiceBase
RecursiveCharacterTextSplitterService
RecursiveCharacterTextSplitterService