Get all documents from ChromaDb using Python and langchain – Langchain

The Problem:

You have a collection of documents stored in a Mongo database. You want to load these documents into a ChromaDb vector storage using Langchain for further processing. After storing the data, you need to retrieve all the documents and their embeddings, along with their IDs, to store them back into MongoDB and perform topic categorization using Bertopic. The challenge is to write Python code using Langchain to extract all documents from the ChromaDb vector storage and obtain their associated metadata.

The Solutions:

Solution 1: Get documents from ChromaDb using Python and langchain

To retrieve all documents from a ChromaDb vector storage created using Langchain, you can use the following code:

“`python
import langchain
db = langchain.vectorstores.Chroma.from_path(‘db’)
documents = db.get()
“`

The `get()` method will return a list of JSON objects, with the following structure for each object:

• `id`: The unique identifier of the document
• `embedding`: The document’s embedding vector
• `doc`: The original document text

Solution 2: Get all documents from ChromaDB using langchain

Once the ChromaDB database is initialized, create a client using the DB directory:

import chromadb
client = chromadb.Client(Settings(is_persistent=True, db_path=<PERSIST_DIR_NAME>))

Then, get the specified collection:

collection = client.get_collection("collection_name")

Finally, retrieve all the data:

data = collection.get()

The retrieved JSON will include all the data, metadata, source, and embeddings.

Video Explanation: