How RAGFlow Builds the Document Filter Panel

RAGFlow’s dataset document page shows filters for file type, processing status, and metadata. These controls are not calculated from the currently visible page of documents. When the page loads, the frontend sends a dedicated type=filter request, and the backend aggregates filter data across every document matching the request.

This post traces the complete path: the frontend trigger, the type=filter route, the construction of the three filter groups, and the backend aggregation that supplies their counts.

The Frontend Filter Request

The document API endpoint is defined by getDatasetFilter:

getDatasetFilter: (datasetId: string) =>
	`${restAPIv1}/datasets/${datasetId}/documents?type=filter`,

knowledge-service.ts exposes this endpoint through documentFilter():

export const documentFilter = (kb_id: string) =>
	request.get(api.getDatasetFilter(kb_id), { params: {} });

On the dataset document page, useGetDocumentFilter() in use-document-request.ts calls documentFilter(datasetId) through React Query. The response contains three counter maps:

{
	run_status: {},
	suffix: {},
	metadata: {},
}

The separate request is important. A normal document-list request is paginated, but the filter request needs counts that represent the complete result set.

Creating the Three Filter Groups

use-select-filters.ts converts the response into the filter controls used by the document page:

{ field: 'type', label: t('fileType'), list: fileTypes }
{ field: 'run', label: t('status'), list: fileStatus }
{ field: 'metadata', label: t('metadataField'), list: metaDataList }

The UI field names do not exactly match the backend field names. In particular, the UI calls the first group type, but the server calculates it from the document’s file extension, suffix.

Therefore, the file-type filter is an extension filter. It is not a count of the database’s Document.type category field.

The type=filter Route

The documents route in document_api.py checks the request query parameter before taking the normal list path:

if request.args.get("type") == "filter":
		err_code, err_msg, payload, total = _get_doc_filters_with_request(
				request, dataset_id
		)
		return get_json_result(data={"total": total, "filter": payload})

_get_doc_filters_with_request() parses the incoming search and filter arguments, then delegates to the document service:

DocumentService.get_filter_by_kb_id(
		dataset_id, keywords, run_status_converted, types, suffix
)

This means the filter endpoint can respect the same request constraints as the document list, including keyword, status, type, and suffix constraints. It returns a total count alongside the filter payload.

Dataset document page
		-> useGetDocumentFilter()
		-> GET /datasets/{dataset_id}/documents?type=filter
		-> _get_doc_filters_with_request()
		-> DocumentService.get_filter_by_kb_id()
		-> suffix, status, and metadata counters
		-> use-select-filters.ts
		-> filter panel

Backend Aggregation

DocumentService.get_filter_by_kb_id() is responsible for generating the counters. It selects the run state, suffix, and ID of every matching document and calculates the total:

rows = query.select(cls.model.run, cls.model.suffix, cls.model.id)
total = rows.count()

doc_ids = [row.id for row in rows]
metadata = DocMetadataService.get_metadata_for_documents(doc_ids, kb_id)

The selected document IDs are then used to retrieve metadata for the same dataset. The service iterates over the document rows and metadata values to build three counter maps:

suffix_counter[row.suffix] += 1
run_status_counter[str(row.run)] += 1
metadata_counter[key][value] += 1
metadata_counter["empty_metadata"] = {"true": empty_metadata_count}

The final payload has this shape:

{
		"suffix": suffix_counter,
		"run_status": run_status_counter,
		"metadata": metadata_counter,
}

empty_metadata is a special metadata bucket rather than a normal metadata field. It counts documents for which metadata is absent, empty, or does not contain an effective field value.

Performance Implication

The filter panel is based on a full aggregation, not the current paginated list. Each request to the filter endpoint does the following:

1. Finds all documents in the dataset that match the request constraints.
2. Reads the run state, suffix, and ID of those documents.
3. Retrieves metadata for all matching document IDs.
4. Aggregates file extensions, processing states, metadata fields, metadata values, and empty-metadata documents in Python.

For a small dataset, this makes the UI straightforward and gives accurate counts. For a large dataset, it also makes opening or refreshing the filter panel more expensive than fetching one list page, because the endpoint scans the whole matching set and fetches metadata for it.

This behavior explains why a document page can display quickly for the first page while the filter request takes noticeably longer. The two paths have different costs: one is page-oriented, while the other is dataset-oriented.

Summary

RAGFlow builds its document filter panel from a dedicated frontend request to GET /datasets/{dataset_id}/documents?type=filter. The frontend maps the response to file-type, status, and metadata groups, although the displayed file type is actually the document suffix.

The backend route delegates to DocumentService.get_filter_by_kb_id(), which reads every matching document and its metadata, then aggregates counts in Python. As a result, the filter panel reflects the full matching dataset rather than the current page, with a corresponding cost for large datasets.