Build your own Glean

Company search answers a question from everything the company has written, and shows each person only what they are allowed to see. This cookbook builds it with three ideas: one collection for all company knowledge, an ACL on each source, and the asker’s identity on every query.

Benchmark	Result	What it shows here
EnterpriseRAG-Bench	88.7 overall, 96.7% document recall@10	Retrieval across fragmented systems where knowledge is duplicated or outdated
KyroBench	0.776 nDCG	The current source ranked above the stale one

Prerequisites

A HydraDB API key from dashboard.hydradb.com, set as HYDRADB_API_KEY.
An OpenAI API key, set as OPENAI_API_KEY. HydraDB returns context, not answers, so any model works; swap the answer function below for your own.
Python 3.10 or later, with the SDKs: pip install "hydradb-sdk>=2.1.7,<3" openai.

Save this as hydra.py. Every cookbook imports it. It holds what every HydraDB application does around its own logic: wait for the database, check what an ingest accepted, wait for indexing, and pass query results to a model.

hydra.py

import os
import time

from hydra_db import HydraDB
from hydra_db.errors import ConflictError
from hydra_db.helpers import build_string
from openai import OpenAI

client = HydraDB(token=os.environ["HYDRADB_API_KEY"])
llm = OpenAI()


def create_database(database, schema=None):
    """Create the database if it is new, then wait until it accepts context."""
    try:
        if schema:
            client.databases.create(database=database, database_metadata_schema=schema)
        else:
            client.databases.create(database=database)
    except ConflictError:
        pass  # The database already exists.
    while not client.databases.status(database=database).data.infra.ready_for_ingestion:
        time.sleep(5)


def source_ids(result):
    """Return the IDs an ingest accepted. Raise if it rejected any item."""
    for item in result.data.results:
        if item.error_code:
            raise RuntimeError(f"{item.filename or item.id}: {item.error_code} {item.error}")
    return [item.id for item in result.data.results]


def wait_indexed(database, collection, ids):
    """Block until every source has finished indexing. Indexing runs in the background."""
    while True:
        statuses = client.context.status(
            database=database, collection=collection, ids=ids
        ).data.statuses
        for status in statuses:
            if status.indexing_status == "errored":
                raise RuntimeError(f"{status.error_code}: {status.error_message}")
        if all(s.indexing_status == "completed" for s in statuses):
            return
        time.sleep(2)


def answer(question, result, instructions=""):
    """Send a query result to the model and return its answer."""
    completion = llm.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "Answer using only the context provided. If the answer "
                "is not in the context, say you don't know. " + instructions,
            },
            {
                "role": "user",
                "content": f"Context:\n{build_string(result)}\n\nQuestion: {question}",
            },
        ],
    )
    return completion.choices[0].message.content

1. Create one database and one collection

Put everything in one collection so a single query searches all of it. Separate collections are for context that must never meet in one query; permissions are handled by ACLs, not by splitting collections. See Databases and Collections.

glean.py

import json

from hydra import answer, client, create_database, source_ids, wait_indexed

DATABASE, COLLECTION = "acme_corp", "company"

create_database(DATABASE)

2. Ingest sources from each app

Send records your code already holds as app sources. kind tells HydraDB what the record is, fields carries its content, and acl lists who may see it. The samples include an outdated wiki page and the Slack message that replaced it, and a ticket only the finance lead can see.

glean.py

sources = [
    {
        "id": "notion_pricing_2025",
        "title": "Pricing (2025)",
        "kind": "knowledge_base",
        "provider": "notion",
        "external_id": "page_pricing_2025",
        "url": "https://notion.so/acme/pricing-2025",
        "timestamp": "2025-03-01T09:00:00Z",
        "fields": {
            "kind": "knowledge_base",
            "title": "Pricing (2025)",
            "body": "Pro is $79 per seat per month. Enterprise is $199.",
            "updated_at": "2025-03-01T09:00:00Z",
        },
        "acl": ["domain:acme.com"],
    },
    {
        "id": "slack_C01_1788000000_000100",
        "title": "Pricing change - Slack #product",
        "kind": "message",
        "provider": "slack",
        "external_id": "1788000000.000100",
        "timestamp": "2026-09-01T10:00:00Z",
        "fields": {
            "kind": "message",
            "body": "Decision: from October 1, 2026, Pro moves from $79 to $89 per seat.",
            "author": "[email protected]",
            "thread_id": "1788000000.000100",
            "created_at": "2026-09-01T10:00:00Z",
        },
        "acl": ["domain:acme.com"],
    },
    {
        "id": "jira_FIN_42",
        "title": "FIN-42: Enterprise discount floor",
        "kind": "ticket",
        "provider": "jira",
        "external_id": "FIN-42",
        "timestamp": "2026-09-03T15:00:00Z",
        "fields": {
            "kind": "ticket",
            "title": "Enterprise discount floor",
            "description": "Do not discount Enterprise below $150 per seat without CFO approval.",
            "status": "open",
            "assignee": "[email protected]",
        },
        "acl": ["[email protected]"],
    },
]

result = client.context.ingest(
    type="knowledge",
    database=DATABASE,
    collection=COLLECTION,
    app_knowledge=json.dumps(sources),
)
wait_indexed(DATABASE, COLLECTION, source_ids(result))

3. Query as the person asking

Send the signed-in user’s identity as acl. Take it from your session on the server, never from the client: a query without acl returns everything in scope. The other settings are the ones the EnterpriseRAG-Bench run used: hybrid matching, thinking mode, and 12 results.

glean.py

def search(question, user_email):
    return client.query(
        database=DATABASE,
        collection=COLLECTION,
        query=question,
        type="knowledge",
        query_by="hybrid",
        mode="thinking",
        max_results=12,
        acl=[user_email],
    )

4. Answer with citations

Each chunk in the query result carries the ID and title of the source it came from. Print each source once under the answer.

glean.py

def ask(question, user_email):
    result = search(question, user_email)
    print(answer(question, result, "Prefer the most recent source when sources disagree."))
    cited = {chunk.id: chunk.source_title for chunk in result.data.chunks}
    for source_id, title in cited.items():
        print(f"  - {title} ({source_id})")


ask("What does Pro cost per seat?", "[email protected]")
ask("How far can we discount Enterprise?", "[email protected]")
ask("How far can we discount Enterprise?", "[email protected]")

Run it

python glean.py

The first answer should give $89 and cite the Slack decision alongside the older page. The second should say it does not know, because the ticket is restricted to Grace. The third should give the $150 floor and cite FIN-42.

Use real data

Replace step 2 with connectors: HydraDB then syncs each app on a schedule and copies its permissions into each source’s ACL. Create one connector per app, all pointing at the same collection.

connect.py

from hydra import client

connector = client.connectors.create(
    provider="slack",
    database="acme_corp",
    collection="company",
    provider_account_scope="T12345ACME",
    credentials={"access_token": "xoxp-..."},
)
found = client.connectors.discover(connector.connector_id)
client.connectors.configure(
    connector.connector_id,
    resources=[
        {"resource_id": r.id, "resource_type": r.resource_type, "name": r.name}
        for r in found.resources
    ],
)

Keep sending acl on every query, and leave enforce_acl off for this application. With enforce_acl on, HydraDB ignores the request’s acl and uses the identity of whoever created the API key, so every user of your application would see what the key’s creator can see. Enforcement fits a caller who authenticates as themselves, such as a user signed in through MCP OAuth.

Common mistakes

Turning on enforce_acl behind one API key: every query then runs with the key creator’s permissions, not the asker’s.
Querying without acl: the query returns restricted sources to everyone. Send the user’s identity on every query.
One collection per app or team: a query that names one collection does not read the others. Keep one collection and narrow with metadata filters.
Group permissions: an email matches its domain: principal automatically, but not groups. Send group: principals yourself. See Query.

Next steps

Query: tune mode, alpha, and recency_bias.
Build your own customer support agent: add per-user memory to shared knowledge.

​Prerequisites

​1. Create one database and one collection

​2. Ingest sources from each app

​3. Query as the person asking

​4. Answer with citations

​Run it

​Use real data

​Common mistakes

​Next steps

Prerequisites

1. Create one database and one collection

2. Ingest sources from each app

3. Query as the person asking

4. Answer with citations

Run it

Use real data

Common mistakes

Next steps