Skip to main content

File search settings

Open the settings with the Settings button on the search page. Here you choose which files to process, how much text to extract from them, and how to detect the language of documents and queries. The page also shows how much of your files is already indexed and lets you start reindexing.

info

The page has no general "Save" button. Each change applies as soon as you enter a value, select an item from a list, or toggle a checkbox.

The page and the Settings button are available to administrators with the right to manage the file index. To change values, you also need the right to change general settings; without it, the fields are read-only. For details, see Access to the section.

Project capabilities​

The Project capabilities block shows what is available to the project. It is informational only: you can't change these values here. Each row has an icon: ✓ means the capability is available, ✕ means it isn't.

  • Feature included in tariff: whether the feature is included in the project's tariff.
  • Text extraction service available: whether text extraction from files is available. If it isn't, the reason appears next to it: the service address isn't set, the service rejected the request, or the service isn't responding.
  • OCR available: whether scan recognition is available.
  • Vector search available: whether vector search is available.

If a capability is unavailable, the related settings below stay visible but are locked. For example, without the feature in the tariff you can't enable file processing, and without OCR you can't configure recognition.

Processing file contents​

The checkbox in the header of the Processing file contents block turns on processing of file contents. Processing is off by default. Turning it off stops text extraction from new files, but the existing index is kept and you can still search it.

The block has these settings:

  • Formats for extraction: file formats to extract text from: txt, md, html, pdf, docx, odt, epub, rtf. All of them are selected by default.
  • Sections whose files are indexed: sections whose files go into the index: Products, Pages, Blocks, Slides, Templates, Discounts, Forms, Events, User Groups, Users, Admins.
  • Maximum characters per file: how many characters of text to extract from one file, from 10000 to 2000000 (default 500000). Text beyond the limit isn't indexed, and the file is considered truncated. Below the field, you see roughly how many files fit in the allowed index size at the current limit.
Personal documents

The Users, User Groups, and Admins sections contain customer and administrator documents. If any of them is selected, the block warns that the text of these files becomes searchable by anyone who has the search right and access to the section.

OCR (scan recognition)​

The OCR (scan recognition) block controls text recognition in scans. The checkbox in the block header turns recognition on, and the Maximum pages per file field limits how many pages of one file are recognized: from 1 to 500, default 50.

If recognition is unavailable for the project, the checkbox and the field are locked and the block shows a warning. You can also recognize an individual scan manually in the list of indexed files.

Language of documents and queries​

The Language of documents and queries block sets the language used to process documents and search queries.

  • Document language detection: how a document's language is determined: Automatic (default) or Fixed.
  • Default language: the language of documents: russian, english (default), german, french, or simple.
  • Query language resolution: how a search query's language is determined: Automatic (default), Fixed, or All corpus languages, which searches all languages present in the indexed files at once.

Below are two checkboxes, both on by default:

  • Allow manual file language override: lets you manually correct the language of an individual file.
  • Search by part of a word: finds matches inside words, but is noticeably slower.

Index coverage​

The Index coverage block shows the state of the index:

  • how many files are searchable out of the total;
  • how many files are waiting for the text extraction service;
  • how many files are in an unsupported format;
  • how many files failed processing;
  • how many files were truncated at the character limit;
  • how many files are corrupted;
  • how many files were manually excluded from search;
  • how many files have questionable text (this row appears only if there are such files).

Each number links to the list of indexed files with the matching filter already applied. If search doesn't find a document, these links show the reason right away: unsupported format, failed extraction, or truncated text.

Below the counters you see the index size in megabytes and the percentage of the allowed size it takes up.

warning

If the allowed index size is used up, the block warns that new documents aren't accepted for processing, while the existing index remains searchable. Reindexing won't help in this case: free up space or increase the size.

If the platform operator has paused extraction, the block shows that too. The Processing file contents checkbox doesn't lift this pause; queued jobs aren't lost and resume once the pause is lifted.

Reindexing​

To reprocess files, choose which ones in the What to reindex list:

  • Only unprocessed: unprocessed files only (default);
  • Only failed: files whose processing failed;
  • Truncated and stale: truncated and outdated files;
  • Entire corpus: all files.

Then click Rebuild index. A progress bar appears below the button: first the files are queued, then it shows the percentage and the number of processed files out of the total. The button is disabled while reindexing runs. If the selected set has no files, the message "Nothing to reindex" appears instead of the progress bar. After reindexing finishes, the result of the last run (the percentage and the number of processed files out of the total) stays below the button.

warning

If processing of file contents is turned off in the Processing file contents block, files are queued but aren't processed until processing is turned on. The Rebuild index button is available only to administrators with the right to manage the file index.