Skip to main content

File Content Search

The File Search tab in the Settings section manages the text search within files uploaded to the project: which files to process, how much text to extract from them, and how to determine the language of documents and queries. It also shows which part of the files has already been indexed and allows you to initiate reindexing.

info

There is no general "Save" button on the tab — each change is applied immediately after entering a value, selecting from a list, or toggling a checkbox. Only administrators with permission to change global settings can modify the parameters; without this permission, the fields are view-only.

Project capabilities​

The Project capabilities block shows what is available to the project. It is for informational purposes only — these values cannot be changed on the tab. Each line is marked with an icon: ✓ — capability available, ✕ — not available (with an explanation of the reason).

  • Whether the feature is included in the project plan.
  • Whether the text extraction service from files is available.
  • Whether OCR (Optical Character Recognition) is available.
  • Whether vector search is available.

If a capability is not available, the related settings below remain visible but are locked: for example, without the feature in the plan, file processing cannot be enabled, and without OCR, recognition cannot be configured.

Processing file contents​

The checkbox in the header of the Processing file contents block enables the processing of file contents. Disabling it stops text extraction from new files, but the already built index is preserved, and searching can still be performed on it.

In this block, you can configure:

  • Formats for extraction — file formats from which text is extracted: txt, md, html, pdf, docx, odt, epub, rtf. All formats are selected by default.
  • Sections whose files are indexed — sections whose files are included in the index: Products, Pages, Blocks, Slides, Templates, Discounts. All sections are selected by default.
  • Maximum characters per file — how many characters of text are extracted from a single file: from 10000 to 2000000, with a default of 500000. Text exceeding the limit is not included in the index, and the file is considered truncated. Below the field, an estimate of how many files the index budget can accommodate at the current limit is displayed.

OCR​

The OCR block is responsible for text recognition in scans. The checkbox in the header of the block enables recognition, and the Maximum pages per file field limits the number of pages recognized from a single file: from 1 to 500, with a default of 50.

If OCR is not deployed in the project, the checkbox and field are locked, and a warning is displayed in the block.

Language of documents and queries​

The Language of documents and queries block determines the language in which documents and search queries are processed.

  • Document language detection — how the language of the document is determined: automatically (default) or fixed.
  • Default language — the language of documents: russian, english (default), german, french, or simple.
  • Query language resolution — how the language of the search query is determined: automatically (default), fixed, or searching across all languages present among the indexed files.

Below are two checkboxes, both enabled by default:

  • allowing manual correction of the language of an individual file;
  • searching within parts of words — an additional pass over found documents that locates occurrences within a word but significantly slows down the search.

Index coverage​

The Index coverage block shows the status of the index:

  • how many files are available for search out of the total number;
  • how many files are waiting for the text extraction service;
  • how many files are in an unsupported format;
  • how many files were processed with an error;
  • how many files were truncated due to the character limit;
  • how many files were manually excluded from the search.

Each number is a link to a list of indexed files with the filter already applied. If a document is not found in the search, you can immediately see the reason through these links: the format is unsupported, extraction failed, or the text was truncated.

Below the counters, the size of the index and its share of the budget are displayed, in megabytes and percentages.

Reindexing​

To reprocess files, select from the What to reindex list which files to process:

  • only unprocessed (default value);
  • only those that ended with an error;
  • truncated and outdated;
  • all files.

Then click Rebuild index. After starting, the number of files queued will appear under the button, and in case of failure, an error message will be displayed.

warning

If file content processing is disabled in the Processing file contents block, reindexing tasks will be queued but will not be executed — the block warns about this. The Rebuild index button is only available to administrators with permission to manage the file index.