When RAGFlow parses a document, it does more than split text into smaller pieces. A chunk task must transform a source document into retrieval-ready records: it creates chunks, may generate questions and tags, produces vector embeddings, and finally stores the results in Elasticsearch.

The important detail is that these responsibilities are separated. In the chunk-task execution path, do_handle_task() coordinates the work, but build_chunks() does not persist data. It prepares the chunk records. Embedding and storage happen in later stages.

This post explains that division of responsibility and provides a useful mental model for debugging document parsing in RAGFlow.

The Chunk Task Entry Point

do_handle_task() is responsible for executing a queued task. When the task is a chunk-processing task, the high-level flow is:

do_handle_task()
	-> build_chunks()
	-> embedding()
	-> insert_chunks()

Each stage has a distinct output and side effect:

This sequence matters because it prevents parsing, model inference, and backend persistence from being treated as one opaque operation.

build_chunks() Produces Chunk Data

For a chunk task, do_handle_task() first calls build_chunks(). Its job is to construct the chunk records that describe the parsed document.

Conceptually, the input and output look like this:

document content + parsing configuration
	-> text segmentation
	-> chunk records + metadata

A chunk record can include the chunk text and metadata such as the document ID, dataset ID, chunk position, page information, and parser-specific fields.

Depending on the parsing configuration, this stage can also generate additional retrieval aids, including:

• Questions derived from the chunk content.
• Tags or keywords associated with the chunk.
• Other enriched fields used later by search or ranking.

These generated values become part of the chunk data that will later be embedded and stored. They do not mean the chunk has already been indexed.

Building Is Not Storing

The most common misunderstanding is to assume that build_chunks() completes document ingestion. It does not.

After build_chunks() returns, RAGFlow has prepared chunk objects, but those objects have not yet been written to Elasticsearch. At this point, they are intermediate results in the task pipeline.

That distinction is useful when diagnosing failures:

• If chunk text, generated questions, or tags are incorrect, investigate parsing and build_chunks().
• If chunk content is correct but semantic retrieval is weak, investigate embedding.
• If chunks are generated and embedded but cannot be found in search, investigate insertion and Elasticsearch.

embedding() Generates Vectors

The next stage is embedding(). It converts the relevant chunk content into numerical vectors using the configured embedding model.

chunk text and enriched fields
	-> embedding model
	-> vector fields attached to chunk records

An embedding is a dense numeric representation of text meaning. Elasticsearch can later use it for vector or hybrid retrieval, allowing a query to match chunks by semantic similarity rather than only literal keywords.

The embedding stage is separate from chunk creation for practical reasons:

• Parsing logic can focus on text structure and metadata.
• Embedding can be batched for model throughput.
• Embedding failures can be retried without changing the parser.
• Different embedding models can be used without rewriting chunking logic.

embedding() enriches the prepared records with vectors. It is not the stage responsible for making those records searchable in Elasticsearch.

insert_chunks() Persists to Elasticsearch

Once chunk records contain their text, metadata, generated fields, and embeddings, insert_chunks() stores them in Elasticsearch.

completed chunk records
	-> insert_chunks()
	-> Elasticsearch index
	-> searchable RAG knowledge

This is the persistence boundary of the chunk-processing pipeline. Before insert_chunks() succeeds, a chunk may exist as data produced during task execution, but it is not yet available through the Elasticsearch-backed retrieval path.

The method is therefore responsible for turning an in-progress parsing result into indexed knowledge. Elasticsearch receives the fields needed for filtering, keyword search, vector search, and result presentation.

Complete Mental Model

The whole chunk-task path can be understood as a three-stage transformation:

Source document
	-> build_chunks()
	-> chunk text, metadata, questions, and tags
	-> embedding()
	-> chunk records with vectors
	-> insert_chunks()
	-> Elasticsearch documents available for retrieval

Or, expressed by responsibility:

do_handle_task()  : orchestrates task execution
build_chunks()    : produces and enriches chunks
embedding()       : calculates embeddings for chunks
insert_chunks()   : stores chunks in Elasticsearch

Why the Separation Matters

This design makes each part of the ingestion pipeline easier to reason about and operate.

For example, a document may parse successfully but still fail before indexing because the embedding provider is unavailable. Conversely, embeddings may be generated correctly while Elasticsearch rejects the final write because of an index, mapping, or connection issue.

Treating these as distinct stages gives a clearer failure model:

Summary

In RAGFlow, do_handle_task() executes document-processing tasks. For a chunk task, it calls build_chunks() to create chunk records and enrich them with information such as generated questions and tags.

build_chunks() does not write those records to Elasticsearch. embedding() creates the vector representations, and insert_chunks() performs the Elasticsearch persistence step that makes the completed chunks available to retrieval.

Keeping these responsibilities separate is the key to understanding the parsing pipeline and to locating problems in chunk generation, vectorization, or indexing.