> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hydradb.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Knowledge
> What Knowledge is, how it differs from Memories, and how to ingest it correctly.
## 1. What it is
Knowledge is shared, tenant-wide context that all users and agents in a tenant can recall - product documents, internal wikis, policy PDFs, Slack threads, Notion pages, CSVs, emails, and any other source content.
Knowledge is the counterpart to [Memories](/essentials/memories). Both are retrievable through Recall, but they live in separate stores and are retrieved through separate endpoints:
| | Knowledge | Memories |
| ------------------------ | ------------------------------------------ | ------------------------------------------------ |
| **Content** | Documents, files, app-generated content | User preferences, conversations, inferred traits |
| **Scope** | Shared across all users in a tenant | Per-user, scoped by `sub_tenant_id` |
| **Mutability** | Versioned - replaced or deleted explicitly | Dynamic - evolves with every interaction |
| **Recall endpoint** | `full_recall` | `recall_preferences` |
| **`vectorstore_status`** | Index `1` | Index `0` |
**Knowledge and Memories are separate stores.** `full_recall` only searches Knowledge. `recall_preferences` only searches Memories. For personalized answers grounded in shared documents, call both in parallel and merge the results. See [How to Use API Results](/essentials/api-results).
***
## 2. What it does
When you upload Knowledge, HydraDB:
1. Parses the source (PDF, DOCX, Markdown, CSV, plain text, or app-source JSON).
2. Chunks the content into semantically coherent segments.
3. Runs entity and relationship extraction - building graph nodes and edges.
4. Embeds chunks into the dense and sparse vector stores.
5. Makes the content searchable through [`/recall/full_recall`](/api-reference/endpoint/full-recall).
This pipeline runs **asynchronously**. Content is not searchable until `indexing_status` is `completed`. Poll [`/ingestion/verify_processing`](/api-reference/endpoint/verify-processing) with the returned `source_id`.
***
## 3. When to use it
Use Knowledge ingestion when the content is:
* Shared across all users in your tenant (product docs, policy PDFs, runbooks, wikis)
* Static or infrequently updated (versioned by replacing the source)
* Document-structured rather than conversational (files, threads, pages)
Use [Memories](/essentials/memories) instead for:
* User preferences and behavioral signals
* Per-user conversation history
* Content that should personalize a single user's experience
| If the content is... | Store it as... |
| ------------------------------- | -------------- |
| Shared across all users | Knowledge |
| Specific to one user or session | Memory |
***
## 4. Two ingestion paths
The `POST /ingestion/upload_knowledge` endpoint accepts two content formats in one request. Send either or both.
### Path A - Files
Binary files HydraDB should parse and chunk: PDF, DOCX, Markdown, CSV, TXT.
```bash cURL theme={"dark"}
curl -X POST 'https://api.hydradb.com/ingestion/upload_knowledge' \
-H "Authorization: Bearer $HYDRA_DB_API_KEY" \
-F "tenant_id=acme_corp" \
-F "files=@/path/to/policy.pdf" \
-F "files=@/path/to/runbook.pdf" \
-F 'file_metadata=[
{ "file_id": "policy_v2", "metadata": { "document_type": "policy", "status": "approved" } },
{ "file_id": "runbook_deploy", "metadata": { "document_type": "runbook" } }
]'
```
```typescript TypeScript SDK theme={"dark"}
const result = await client.upload.knowledge({
tenant_id: "acme_corp",
files: [
{ path: "/path/to/policy.pdf", filename: "policy.pdf", contentType: "application/pdf" },
{ path: "/path/to/runbook.pdf", filename: "runbook.pdf", contentType: "application/pdf" },
],
file_metadata: JSON.stringify([
{ file_id: "policy_v2", metadata: { document_type: "policy", status: "approved" } },
{ file_id: "runbook_deploy", metadata: { document_type: "runbook" } },
]),
});
```
```python Python SDK theme={"dark"}
import json
with open("/path/to/policy.pdf", "rb") as f1, open("/path/to/runbook.pdf", "rb") as f2:
result = client.upload.knowledge(
tenant_id="acme_corp",
files=[
("policy.pdf", f1, "application/pdf"),
("runbook.pdf", f2, "application/pdf"),
],
file_metadata=json.dumps([
{"file_id": "policy_v2", "metadata": {"document_type": "policy", "status": "approved"}},
{"file_id": "runbook_deploy", "metadata": {"document_type": "runbook"}},
]),
)
```
### Path B - App sources
Pre-parsed records from connected apps. Put the app item's searchable text in `fields` so app-aware recall can use IDs, actors, threads, and relations.
```bash cURL theme={"dark"}
curl -X POST 'https://api.hydradb.com/ingestion/upload_knowledge' \
-H "Authorization: Bearer $HYDRA_DB_API_KEY" \
-F "tenant_id=acme_corp" \
-F 'app_knowledge=[
{
"tenant_id": "acme_corp",
"sub_tenant_id": "acme_corp",
"id": "slack_thread_001",
"title": "Pricing discussion - Slack #product",
"type": "slack",
"kind": "message",
"provider": "slack",
"external_id": "1715012345.000100",
"fields": {
"kind": "message",
"body": "We agreed on three tiers: Starter at $29, Pro at $79, Enterprise at $199.",
"author": "alex",
"thread_id": "1715012345.000100"
},
"metadata": { "channel": "product", "status": "finalized" },
"additional_metadata": { "slack_ts": "1715012345.000100" }
}
]'
```
```typescript TypeScript SDK theme={"dark"}
const result = await client.upload.knowledge({
tenant_id: "acme_corp",
app_knowledge: JSON.stringify([
{
tenant_id: "acme_corp",
sub_tenant_id: "acme_corp",
id: "slack_thread_001",
title: "Pricing discussion - Slack #product",
type: "slack",
kind: "message",
provider: "slack",
external_id: "1715012345.000100",
fields: {
kind: "message",
body: "We agreed on three tiers: Starter at $29, Pro at $79, Enterprise at $199.",
author: "alex",
thread_id: "1715012345.000100",
},
metadata: { channel: "product", status: "finalized" },
additional_metadata: { slack_ts: "1715012345.000100" },
},
]),
});
```
```python Python SDK theme={"dark"}
import json
result = client.upload.knowledge(
tenant_id="acme_corp",
app_knowledge=json.dumps([
{
"tenant_id": "acme_corp",
"sub_tenant_id": "acme_corp",
"id": "slack_thread_001",
"title": "Pricing discussion - Slack #product",
"type": "slack",
"kind": "message",
"provider": "slack",
"external_id": "1715012345.000100",
"fields": {
"kind": "message",
"body": "We agreed on three tiers: Starter at $29, Pro at $79, Enterprise at $199.",
"author": "alex",
"thread_id": "1715012345.000100",
},
"metadata": {"channel": "product", "status": "finalized"},
"additional_metadata": {"slack_ts": "1715012345.000100"},
}
]),
)
```
Though app\_knowledge is an available option. We *recommend* storing the content as a markdown file and using path A i.e. file upload using `upload_knowledge` for better results.
**Response (both paths):**
```json theme={"dark"}
{
"success": true,
"message": "Knowledge uploaded successfully",
"results": [
{ "source_id": "policy_v2", "filename": "policy.pdf", "status": "queued", "error": null }
],
"success_count": 1,
"failed_count": 0
}
```
***
## 5. Verifying processing
```bash cURL theme={"dark"}
curl -X POST \
'https://api.hydradb.com/ingestion/verify_processing?file_ids=policy_v2&tenant_id=acme_corp&sub_tenant_id=acme_corp' \
-H "Authorization: Bearer $HYDRA_DB_API_KEY"
```
```typescript TypeScript SDK theme={"dark"}
const status = await client.upload.verifyProcessing({
tenant_id: "acme_corp",
sub_tenant_id: "acme_corp",
file_ids: ["policy_v2"],
});
```
```python Python SDK theme={"dark"}
status = client.upload.verify_processing(
tenant_id="acme_corp",
sub_tenant_id="acme_corp",
file_ids=["policy_v2"],
)
```
```json theme={"dark"}
{
"statuses": [
{
"file_id": "policy_v2",
"indexing_status": "completed",
"success": true
}
]
}
```
Status values: `queued` → `processing` → `graph_creation` → `completed` (or `error`).
Poll every few seconds. Most documents complete in 1–5 minutes.
***
## 6. Key parameters
### File metadata (`file_metadata` array items)
| Field | Type | Description |
| --------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `file_id` | string | Optional stable identifier. Used as upsert key when `upsert: true`. |
| `metadata` | object | Tenant-level fields, pre-filtered in the vector store. Keys must match the tenant metadata schema (`enable_match: true`). Filter via top-level keys in `metadata_filters`. |
| `additional_metadata` | object | Document-level fields. Filterable at recall by nesting under the **`additional_metadata`** key in `metadata_filters` (canonical) or `document_metadata` (legacy alias). Applied post-retrieval (slower than tenant-level; engine over-fetches \~3x). See [Full recall → Metadata filters](/api-reference/endpoint/full-recall#metadata-filters). |
### App source model (`app_knowledge` array items)
| Field | Type | Description |
| --------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | string | Required unique identifier for the source. Upsert key. |
| `tenant_id` | string | Required tenant that owns the source. |
| `sub_tenant_id` | string | Required sub-tenant scope. If you do not partition knowledge, use the same value as `tenant_id` - the default sub-tenant an omitted query scope resolves to. Do not use the literal string `"default"`. |
| `title` | string | Short title or subject. |
| `type` | string | Source category (e.g., `slack`, `notion`, `gmail`, `webpage`). |
| `description` | string | Optional long-form description. |
| `url` | string | Canonical URL or reference link. |
| `timestamp` | string | ISO-8601 timestamp (creation or last-updated). |
| `kind` | string | App object kind: `email`, `message`, `ticket`, `knowledge_base`, or `custom`. |
| `provider` | string | Provider namespace such as `slack`, `gmail`, `jira`, or `notion`. |
| `external_id` | string | Stable provider ID for this exact item. |
| `fields` | object | Primary app-source content and structure, such as message `body` or ticket `description`. |
| `content` | object | Legacy/generic content payload. Prefer `fields` for typed app sources. |
| `metadata` | object | Tenant-level fields matching the tenant schema. Filtered via top-level keys in `metadata_filters` (pre-filtered, fast). |
| `additional_metadata` | object | Document-level fields. Filterable by nesting under `additional_metadata` (canonical) or `document_metadata` (legacy alias) in `metadata_filters` (post-retrieval). See [Full recall → Metadata filters](/api-reference/endpoint/full-recall#metadata-filters). |
| `attachments` | array | Optional related attachments. Include `content.text` or `content.markdown` for extracted attachment text. |
| `relations` | object | Forcefully connect this source to others. |
### Request-level fields
| Field | Type | Default | Description |
| --------------- | ------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tenant_id` | string | - *(required)* | Target tenant. |
| `sub_tenant_id` | string | tenant's default sub-tenant | Scope this knowledge to a sub-tenant. If omitted, HydraDB uses the tenant's default sub-tenant (which shares the tenant's identifier). Knowledge is **not** shared across sub-tenants by default -- to share, ingest under each sub-tenant explicitly or query from the default sub-tenant. See [Multi-Tenant](/essentials/multi-tenant). |
| `upsert` | boolean | `true` | Replace existing items with the same `source_id` / `file_id`. |
***
## 7. Forceful relations
Forceful relations let you explicitly link Knowledge sources to each other at ingestion time. When a query matches a source that has forceful relations declared, HydraDB pulls the linked sources into `additional_context` alongside the primary results.
This is separate from the entity and relationship graph HydraDB builds automatically during ingestion. Graph extraction is derived from content; forceful relations are declared by you.
Set the `relations` field on any item in `file_metadata` or `app_knowledge`:
```json theme={"dark"}
{
"relations": {
"cortex_source_ids": ["source-id-1", "source-id-2"]
}
}
```
```bash cURL theme={"dark"}
curl -X POST 'https://api.hydradb.com/ingestion/upload_knowledge' \
-H "Authorization: Bearer $HYDRA_DB_API_KEY" \
-F "tenant_id=acme_corp" \
-F 'app_knowledge=[
{
"id": "auth-guide",
"title": "Authentication Guide",
"content": { "text": "..." },
"relations": { "cortex_source_ids": ["auth-troubleshooting", "auth-faq"] }
}
]'
```
```typescript TypeScript SDK theme={"dark"}
await client.upload.knowledge({
tenant_id: "acme_corp",
app_knowledge: JSON.stringify([
{
id: "auth-guide",
title: "Authentication Guide",
content: { text: "..." },
relations: { cortex_source_ids: ["auth-troubleshooting", "auth-faq"] },
},
]),
});
```
```python Python SDK theme={"dark"}
client.upload.knowledge(
tenant_id="acme_corp",
app_knowledge=json.dumps([
{
"tenant_id": "acme_corp",
"sub_tenant_id": "acme_corp",
"id": "auth-guide",
"title": "Authentication Guide",
"content": {"text": "..."},
"relations": {"cortex_source_ids": ["auth-troubleshooting", "auth-faq"]},
}
]),
)
```
At recall, linked sources are returned in the `additional_context` field. Set `search_forceful_relations: true` (the default) on the request to include them.
**Knowledge sources only.** Forceful relations work within the same store. Pointing a Knowledge source at a Memory source ID (or vice versa) will silently return nothing at recall time.
**`thinking` mode only.** Forceful-relation context is fetched only when `mode: "thinking"` is set on the recall request. In `fast` mode, `search_forceful_relations` is ignored even if set to `true`.
## 8. Metadata on knowledge
Attach `metadata` at ingestion to enable deterministic filtering at recall time:
```json theme={"dark"}
{
"metadata": { "document_type": "runbook", "environment": "production" }
}
```
Then at recall time:
```json theme={"dark"}
{
"query": "How do I rotate API keys?",
"metadata_filters": { "document_type": "runbook", "environment": "production" }
}
```
`metadata` keys must be defined in the tenant metadata schema with `enable_match: true` to be filterable via top-level keys. `additional_metadata` is also filterable -- nest it under the **`additional_metadata`** key in `metadata_filters` (canonical, matches the ingestion field name). `document_metadata` is also accepted as a legacy alias:
```json theme={"dark"}
{
"query": "How do I rotate API keys?",
"metadata_filters": {
"environment": "production",
"additional_metadata": { "author": "platform-team" }
}
}
```
Tenant-level filters are pre-applied in the vector store; nested `additional_metadata` filters are post-retrieval, so the engine over-fetches (\~3x) to compensate.
See [Essentials → Metadata](/essentials/metadata) for schema design and filtering patterns.
***
## 9. Common mistakes
| Mistake | What goes wrong | Fix |
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- |
| Querying Knowledge via `recall_preferences` | Nothing is returned - wrong store | Use `full_recall` for Knowledge |
| Filtering `additional_metadata` via a top-level `metadata_filters` key | The engine treats it as a tenant-schema field, can't find one, returns nothing | Nest under `"additional_metadata"`: `{"metadata_filters": {"additional_metadata": {"author": "..."}}}` |
| Putting hot-path filter fields in `additional_metadata` instead of `metadata` | Filter still works but slower (post-retrieval, over-fetches \~3x) | Move filterable fields to `metadata`, declare them in the tenant schema with `enable_match: true` for pre-filtering |
| Not polling `verify_processing` before recall | Chunks are not yet indexed - empty recall results | Wait for `indexing_status: completed` |
| Sending `file_metadata` as a JSON object instead of a JSON string | `multipart/form-data` validation fails | Stringify `file_metadata` before sending in the `-F` field |
| Omitting `tenant_id` | `422` validation error | Required on every call |
***
## Related
* [Memories](/essentials/memories) - user-scoped dynamic context
* [Recall](/essentials/recall) - how `full_recall` retrieves Knowledge
* [Metadata](/essentials/metadata) - designing filterable schemas
* [Upload Knowledge API Reference](/api-reference/endpoint/upload-knowledge)
* [Verify Processing API Reference](/api-reference/endpoint/verify-processing)