> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hydradb.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> AGENTS.md is HydraDB's maintained agent guide. Read https://docs.hydradb.com/AGENTS.md once, then use its content. /llms.txt and /llms-full.txt redirect to the same guide; they are not separate documentation indexes. If the guide is already loaded, continue with it instead of fetching either alias or reloading the guide.

# Build your own Glean

> Company search across Slack, Jira, and Notion that respects each person's permissions and cites its sources.

Company search answers a question from everything the company has written, and shows each person only what they are allowed to see. This cookbook builds it with three ideas: one collection for all company knowledge, an ACL on each source, and the asker's identity on every query.

| Benchmark | Result | What it shows here |
| - | - | - |
| [EnterpriseRAG-Bench](https://benchmarks.hydradb.com/enterprise-rag-bench) | 88.7 overall, 96.7% document recall\@10 | Retrieval across fragmented systems where knowledge is duplicated or outdated |
| [KyroBench](https://benchmarks.hydradb.com/kyrobench) | 0.776 nDCG | The current source ranked above the stale one |

## Prerequisites

* A HydraDB API key from [dashboard.hydradb.com](https://dashboard.hydradb.com), set as `HYDRADB_API_KEY`.
* An OpenAI API key, set as `OPENAI_API_KEY`. HydraDB returns context, not answers, so any model works; swap the `answer` function below for your own.
* Python 3.10 or later, with the SDKs: `pip install "hydradb-sdk>=2.1.7,<3" openai`.

Save this as `hydra.py`. Every cookbook imports it. It holds what every HydraDB application does around its own logic: wait for the database, check what an ingest accepted, wait for indexing, and pass query results to a model.

```python hydra.py theme={"dark"}
import os
import time

from hydra_db import HydraDB
from hydra_db.errors import ConflictError
from hydra_db.helpers import build_string
from openai import OpenAI

client = HydraDB(token=os.environ["HYDRADB_API_KEY"])
llm = OpenAI()


def create_database(database, schema=None):
    """Create the database if it is new, then wait until it accepts context."""
    try:
        if schema:
            client.databases.create(database=database, database_metadata_schema=schema)
        else:
            client.databases.create(database=database)
    except ConflictError:
        pass  # The database already exists.
    while not client.databases.status(database=database).data.infra.ready_for_ingestion:
        time.sleep(5)


def source_ids(result):
    """Return the IDs an ingest accepted. Raise if it rejected any item."""
    for item in result.data.results:
        if item.error_code:
            raise RuntimeError(f"{item.filename or item.id}: {item.error_code} {item.error}")
    return [item.id for item in result.data.results]


def wait_indexed(database, collection, ids):
    """Block until every source has finished indexing. Indexing runs in the background."""
    while True:
        statuses = client.context.status(
            database=database, collection=collection, ids=ids
        ).data.statuses
        for status in statuses:
            if status.indexing_status == "errored":
                raise RuntimeError(f"{status.error_code}: {status.error_message}")
        if all(s.indexing_status == "completed" for s in statuses):
            return
        time.sleep(2)


def answer(question, result, instructions=""):
    """Send a query result to the model and return its answer."""
    completion = llm.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "Answer using only the context provided. If the answer "
                "is not in the context, say you don't know. " + instructions,
            },
            {
                "role": "user",
                "content": f"Context:\n{build_string(result)}\n\nQuestion: {question}",
            },
        ],
    )
    return completion.choices[0].message.content
```

## 1. Create one database and one collection

Put everything in one collection so a single query searches all of it. Separate collections are for context that must never meet in one query; permissions are handled by ACLs, not by splitting collections. See [Databases and Collections](https://docs.hydradb.com/essentials/v2/databases-and-collections.md).

```python glean.py theme={"dark"}
import json

from hydra import answer, client, create_database, source_ids, wait_indexed

DATABASE, COLLECTION = "acme_corp", "company"

create_database(DATABASE)
```

## 2. Ingest sources from each app

Send records your code already holds as [app sources](https://docs.hydradb.com/essentials/v2/context.md#send-app-sources). `kind` tells HydraDB what the record is, `fields` carries its content, and `acl` lists who may see it. The samples include an outdated wiki page and the Slack message that replaced it, and a ticket only the finance lead can see.

```python glean.py theme={"dark"}
sources = [
    {
        "id": "notion_pricing_2025",
        "title": "Pricing (2025)",
        "kind": "knowledge_base",
        "provider": "notion",
        "external_id": "page_pricing_2025",
        "url": "https://notion.so/acme/pricing-2025",
        "timestamp": "2025-03-01T09:00:00Z",
        "fields": {
            "kind": "knowledge_base",
            "title": "Pricing (2025)",
            "body": "Pro is $79 per seat per month. Enterprise is $199.",
            "updated_at": "2025-03-01T09:00:00Z",
        },
        "acl": ["domain:acme.com"],
    },
    {
        "id": "slack_C01_1788000000_000100",
        "title": "Pricing change - Slack #product",
        "kind": "message",
        "provider": "slack",
        "external_id": "1788000000.000100",
        "timestamp": "2026-09-01T10:00:00Z",
        "fields": {
            "kind": "message",
            "body": "Decision: from October 1, 2026, Pro moves from $79 to $89 per seat.",
            "author": "alice@acme.com",
            "thread_id": "1788000000.000100",
            "created_at": "2026-09-01T10:00:00Z",
        },
        "acl": ["domain:acme.com"],
    },
    {
        "id": "jira_FIN_42",
        "title": "FIN-42: Enterprise discount floor",
        "kind": "ticket",
        "provider": "jira",
        "external_id": "FIN-42",
        "timestamp": "2026-09-03T15:00:00Z",
        "fields": {
            "kind": "ticket",
            "title": "Enterprise discount floor",
            "description": "Do not discount Enterprise below $150 per seat without CFO approval.",
            "status": "open",
            "assignee": "grace@acme.com",
        },
        "acl": ["grace@acme.com"],
    },
]

result = client.context.ingest(
    type="knowledge",
    database=DATABASE,
    collection=COLLECTION,
    app_knowledge=json.dumps(sources),
)
wait_indexed(DATABASE, COLLECTION, source_ids(result))
```

## 3. Query as the person asking

Send the signed-in user's identity as `acl`. Take it from your session on the server, never from the client: a query without `acl` returns everything in scope. The other settings are the ones the EnterpriseRAG-Bench run used: hybrid matching, `thinking` mode, and 12 results.

```python glean.py theme={"dark"}
def search(question, user_email):
    return client.query(
        database=DATABASE,
        collection=COLLECTION,
        query=question,
        type="knowledge",
        query_by="hybrid",
        mode="thinking",
        max_results=12,
        acl=[user_email],
    )
```

## 4. Answer with citations

Each chunk in the query result carries the ID and title of the source it came from. Print each source once under the answer.

```python glean.py theme={"dark"}
def ask(question, user_email):
    result = search(question, user_email)
    print(answer(question, result, "Prefer the most recent source when sources disagree."))
    cited = {chunk.id: chunk.source_title for chunk in result.data.chunks}
    for source_id, title in cited.items():
        print(f"  - {title} ({source_id})")


ask("What does Pro cost per seat?", "bob@acme.com")
ask("How far can we discount Enterprise?", "bob@acme.com")
ask("How far can we discount Enterprise?", "grace@acme.com")
```

## Run it

```bash theme={"dark"}
python glean.py
```

The first answer should give $89 and cite the Slack decision alongside the older page. The second should say it does not know, because the ticket is restricted to Grace. The third should give the $150 floor and cite FIN-42.

## Use real data

Replace step 2 with [connectors](https://docs.hydradb.com/essentials/v2/connectors.md): HydraDB then syncs each app on a schedule and copies its permissions into each source's ACL. Create one connector per app, all pointing at the same collection.

```python connect.py theme={"dark"}
from hydra import client

connector = client.connectors.create(
    provider="slack",
    database="acme_corp",
    collection="company",
    provider_account_scope="T12345ACME",
    credentials={"access_token": "xoxp-..."},
)
found = client.connectors.discover(connector.connector_id)
client.connectors.configure(
    connector.connector_id,
    resources=[
        {"resource_id": r.id, "resource_type": r.resource_type, "name": r.name}
        for r in found.resources
    ],
)
```

Keep sending `acl` on every query, and leave [`enforce_acl`](https://docs.hydradb.com/essentials/v2/connectors#enforce-acl) off for this application. With `enforce_acl` on, HydraDB ignores the request's `acl` and uses the identity of whoever created the API key, so every user of your application would see what the key's creator can see. Enforcement fits a caller who authenticates as themselves, such as a user signed in through MCP OAuth.

## Common mistakes

* **Turning on `enforce_acl` behind one API key:** every query then runs with the key creator's permissions, not the asker's.
* **Querying without `acl`:** the query returns restricted sources to everyone. Send the user's identity on every query.
* **One collection per app or team:** a query that names one collection does not read the others. Keep one collection and narrow with [metadata filters](https://docs.hydradb.com/essentials/v2/metadata.md).
* **Group permissions:** an email matches its `domain:` principal automatically, but not groups. Send `group:` principals yourself. See [Query](https://docs.hydradb.com/essentials/v2/query.md#6-query-on-behalf-of-a-user).

## Next steps

* [Query](https://docs.hydradb.com/essentials/v2/query.md): tune `mode`, `alpha`, and `recency_bias`.
* [Build your own customer support agent](https://docs.hydradb.com/cookbooks/v2/support-agent.md): add per-user memory to shared knowledge.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.