Step 1 / 5
Acquire
Collect from many sources
Sources arrive through uploads or scheduled connectors. Before processing, files are scanned for malware.
- File uploads and connectors for web, feeds, email, archives and datasets
- Only new content is re-read, detected by comparing content
- Malware scan before processing
Step 2 / 5
Structure
Extract, normalize, classify
Text is extracted from the original format and put in order: headings, paragraphs, tables, attachments. Then the metadata is derived.
- Extraction from PDF, Office, HTML, email and ZIP archives, with OCR for scans
- Cleanup of hyphenation, repeated headers and corrupted text
- Metadata derived from the rules of the domain profile
Step 3 / 5
Index
Prepare for retrieval
Text becomes chunks of a useful size. Each one is indexed both by words and by meaning.
- Splitting driven by the structure of the document
- Full-text index and semantic vectors (embeddings)
- Processing run by workers, on the server or on remote nodes
Step 4 / 5
Contextualize
Provenance, versions, time
Every chunk remembers where it comes from. Versions are never overwritten, and validity over time is data, not a guess.
- Reference to document, version and section
- Validity over time and the reference date of the question
- Document relationships: “supersedes” and “amends”
Step 5 / 5
Activate
Make knowledge usable
With authorization already checked, knowledge is read by search, assistants and applications.
- Hybrid search with filters and explicit references
- Answers with citations verified against the passages used
- Web interface, Telegram bot, export to DOCX and Markdown
- Agents with toolsOn the roadmap