Tables

As stated previously, the database structure is defined in the database.models module. Currently, it is composed of the following tables.

Document

database.models.Document (document)

Documents used for RAG system.

Columns:

user_uuid	VARCHAR?
id*	INTEGER
text	TEXT
language	VARCHAR(3)?
url	TEXT
tags	ARRAY?
subtopics	ARRAY?
hyq	ARRAY?
hyq_declarative	ARRAY?
summary	TEXT?
organizations	ARRAY?
doctype	VARCHAR?
→ source_id	INTEGER
created_at	DATETIME
modified_at	DATETIME
tags_embedding	VECTOR(1536)?
subtopics_embedding	VECTOR(1536)?
summary_embedding	VECTOR(1536)?
hyq_embedding	VECTOR(1536)?
hyq_declarative_embedding	VECTOR(1536)?
text_embedding	VECTOR(1536)?

Constraints:

• FOREIGN KEY (source.id → source_id)

• PRIMARY KEY (id)

Indexes:

• idx_document_language (language)

• idx_document_language_user (language, user_uuid)

• idx_document_organizations (organizations)

• idx_document_subtopics_embedding (subtopics_embedding)

• idx_document_tags (tags)

• idx_document_tags_embedding (tags_embedding)

• idx_document_text_embedding (text_embedding)

• idx_document_text_gin (text)

Question

database.models.Question (question)

Questions for FAQ system, with answers stored in Document table.

Columns:

→ answer_id	INTEGER
user_uuid	VARCHAR?
id*	INTEGER
text	TEXT
language	VARCHAR(3)?
url	TEXT
tags	ARRAY?
subtopics	ARRAY?
hyq	ARRAY?
hyq_declarative	ARRAY?
summary	TEXT?
organizations	ARRAY?
doctype	VARCHAR?
→ source_id	INTEGER
created_at	DATETIME
modified_at	DATETIME
tags_embedding	VECTOR(1536)?
subtopics_embedding	VECTOR(1536)?
summary_embedding	VECTOR(1536)?
hyq_embedding	VECTOR(1536)?
hyq_declarative_embedding	VECTOR(1536)?
text_embedding	VECTOR(1536)?

Constraints:

• FOREIGN KEY (document.id → answer_id)

• FOREIGN KEY (source.id → source_id)

• PRIMARY KEY (id)

Indexes:

• idx_question_answer (answer_id)

• idx_question_language (language)

• idx_question_tags (tags)

• idx_question_text_embedding (text_embedding)

• idx_question_text_gin (text)

Source

database.models.Source (source)

Source of the data stored in Document and Question tables. It can be URL, file path, user ID, etc.

Columns:

id*	INTEGER
url*	TEXT
description	TEXT?
created_at	DATETIME
modified_at	DATETIME

Constraints:

• PRIMARY KEY (id)

• UNIQUE (url)

Indexes:

• idx_source_url (url)