Collections

Collections organize your documents, define their schema, and enable fast vector search, filtering, keyword search, semantic search, and multi-vector search.

Create

In order to create a collection, call the create() method on the client.collections() object:

from topk_sdk.schema import int, text, semantic_index

client.collections().create(
    "books",
    schema={
        "title": text().required().index(semantic_index()),
        "published_year": int().required(),
    },
)

import { int, text, semanticIndex } from "topk-js/schema";

await client.collections().create("books", {
  title: text().required().index(semanticIndex()),
  published_year: int().required(),
});

CREATE TABLE books (
  title          TEXT NOT NULL   INDEX semantic_index(),
  published_year INTEGER NOT NULL
);

Field names starting with _ are reserved for internal use.

Schema

Opt-in schema

TopK is schemaless-by-default. Fields without types can store any value. When types are specified, data is validated during upsert.

Indexed fields require explicit types.

Field types

Type	Use case
`text()`	Strings, descriptions, content, IDs
`bytes()`	Binary data, images, files
`int()`	Integers, counts, IDs
`float()`	Decimal numbers, prices
`bool()`	true/false values
`list(value_type)`	Arrays of text, integer, or float elements
`struct(fields)`	Nested objects with named fields
`f8_vector(dim)`	8-bit float embeddings
`f16_vector(dim)`	16-bit float embeddings
`f32_vector(dim)`	Dense embeddings (most common)
`u8_vector(dim)`	Quantized embeddings
`i8_vector(dim)`	Signed quantized embeddings
`binary_vector(dim)`	Binary embeddings
`f32_sparse_vector()`	Sparse embeddings
`u8_sparse_vector()`	Quantized sparse embeddings
`matrix(dim, value_type)`	Multi-vector embeddings

Nested objects are automatically treated as structs — no need to use struct() explicitly.

from topk_sdk.schema import struct, text

schema = {
    "meta": {
        "name": text(),
        "source": {
            "url": text(),
        },
    },
    # Implicit struct, equivalent to:
    # "meta": struct({
    #     "name": text(),
    #     "source": struct({
    #         "url": text(),
    #     }),
    # })
}

import { struct, text } from "topk-js/schema";

const schema = {
  meta: {
    name: text(),
    source: {
      url: text(),
    },
  },
  // Implicit struct, equivalent to:
  // meta: struct({
  //   name: text(),
  //   source: struct({
  //     url: text(),
  //   }),
  // })
};

Required fields

Fields are optional by default. Add required() to make them mandatory—required fields must be present in every document during upsert. Documents missing a required field are rejected with a validation error.

from topk_sdk.schema import int, text

schema = {
    "name": text().required(),   # Must be present in all documents
    "price": int(),              # Can be omitted (null)
}

import { int, text } from "topk-js/schema";

const schema = {
  name: text().required(),   // Must be present in all documents
  price: int(),              // Can be omitted (null)
};

CREATE TABLE books (
  name  TEXT NOT NULL,  -- NOT NULL marks the field as required
  price INTEGER         -- nullable by default (can be omitted)
);

Indexes

Only indexed fields can be searched. Non-indexed fields support exact-match filters only.

Vector Index

Used for vector search. Supports dimensions up to 2^14. Enabled by vector_index().

from topk_sdk.schema import f32_vector, vector_index

schema = {
    "embedding": f32_vector(dimension=1536).index(vector_index(metric="cosine")),
}

import { f32Vector, vectorIndex } from "topk-js/schema";

const schema = {
  embedding: f32Vector({ dimension: 1536 }).index(vectorIndex({ metric: "cosine" })),
};

CREATE TABLE books (
  embedding f32_vector(1536) INDEX vector_index(metric = 'cosine')
);

Similarity metrics compatibility:

Vector Type	`cosine`	`euclidean`	`dot_product`	`hamming`
`f8_vector`	✅	✅	✅	—
`f16_vector`	✅	✅	✅	—
`f32_vector`	✅	✅	✅	—
`u8_vector`	✅	✅	✅	—
`i8_vector`	✅	✅	✅	—
`binary_vector`	—	—	—	✅
`f32_sparse_vector`	—	—	✅	—
`u8_sparse_vector`	—	—	✅	—

Multi-Vector Index

Enables multi-vector search on matrix() fields using the maxsim metric for late-interaction scoring. Enabled by multi_vector_index(). See multi-vector search for more information.

from topk_sdk.schema import matrix, multi_vector_index

schema = {
    "token_embeddings": matrix(
        dimension=1536,
        value_type="f32"
    ).index(
        multi_vector_index(metric="maxsim")
    ),
}

import { matrix, multiVectorIndex } from "topk-js/schema";

const schema = {
  token_embeddings: matrix({
    dimension: 1536,
    valueType: "f32",
  }).index(
    multiVectorIndex({ metric: "maxsim" })
  ),
};

CREATE TABLE passages (
  token_embeddings f16_matrix(128) INDEX multi_vector_index(metric = 'maxsim')
);

Keyword Index

Traditional text search with BM25 relevance scoring. Fast keyword matching with no embedding overhead. Enabled by keyword_index().

from topk_sdk.schema import keyword_index, text

schema = {
    "title": text().index(keyword_index()),
}

import { keywordIndex, text } from "topk-js/schema";

const schema = {
  title: text().index(keywordIndex()),
};

CREATE TABLE books (
  title TEXT INDEX keyword_index()
);

Semantic Index

Convenience method for automatic embeddings. Enabled by semantic_index().

from topk_sdk.schema import semantic_index, text

schema = {
    "title": text().index(semantic_index()),
}

import { semanticIndex, text } from "topk-js/schema";

const schema = {
  title: text().index(semanticIndex()),
};

CREATE TABLE books (
  title TEXT INDEX semantic_index()
);

See semantic_index() for details.

List

You can list all collections in a project by calling client.collections().list():

collections = client.collections().list()

const collections = await client.collections().list();

SELECT table_name FROM information_schema.tables;

The list() function returns a list of Collection objects:

for collection in collections:
    print(f"Collection name: {collection.name}")
    print(f"Organization ID: {collection.org_id}")
    print(f"Project ID: {collection.project_id}")
    print(f"Region: {collection.region}")
    print(f"Schema: {collection.schema}")

for (const collection of collections) {
  console.log(`Collection name: ${collection.name}`);
  console.log(`Organization ID: ${collection.orgId}`);
  console.log(`Project ID: ${collection.projectId}`);
  console.log(`Region: ${collection.region}`);
  console.log(`Schema: ${collection.schema}`);
}

Get

You can get a specific collection in a project by calling client.collections().get(name):

collection = client.collections().get("books")

const collection = await client.collections().get("books")

The get() function takes the name of a collection and returns a single Collection object.

print(f"Collection name: {collection.name}")
print(f"Organization ID: {collection.org_id}")
print(f"Project ID: {collection.project_id}")
print(f"Region: {collection.region}")
print(f"Schema: {collection.schema}")

console.log(`Collection name: ${collection.name}`);
console.log(`Organization ID: ${collection.orgId}`);
console.log(`Project ID: ${collection.projectId}`);
console.log(`Region: ${collection.region}`);
console.log(`Schema: ${collection.schema}`);

Delete

Once you decide that you no longer need a collection or that you want to start over, you can delete it. Deleting a collection will remove all the documents and indexes associated with it. To delete a collection, call the client.collections().delete(name) method:

client.collections().delete("my-collection")

await client.collections().delete("my-collection")

DROP TABLE "my-collection";
-- Use IF EXISTS to suppress the error if the collection does not exist:
DROP TABLE IF EXISTS "my-collection";

If your collection is too large it can take a moment to delete it. You can check the status of the deletion by listing the collections again.

This operation is irreversible and will permanently delete all data in the collection.

Overview

Dataset API

Collection API

Management API

Collections

Create

Schema

Opt-in schema

Field types

Required fields

Indexes

Vector Index

Multi-Vector Index

Keyword Index

Semantic Index

List

Get

Delete

​Create

​Schema

​Opt-in schema

​Field types

​Required fields

​Indexes

​Vector Index

​Multi-Vector Index

​Keyword Index

​Semantic Index

​List

​Get

​Delete

Create

Schema

Opt-in schema

Field types

Required fields

Indexes

Vector Index

Multi-Vector Index

Keyword Index

Semantic Index

List

Get

Delete