Create and Use a Knowledge Base in Jinba Flow

Create a knowledge base, add files, search them, and use the results in a RAG workflow.

Create a knowledge base in workspace storage, add files, and use vector search to retrieve relevant content for a flow.

The knowledge base list. Create and manage vector databases for semantic search

Store and search documents in a knowledge base

Jinba Flow’s Knowledge feature provides powerful vector database capabilities for building RAG (Retrieval-Augmented Generation) systems. You can create knowledge bases, upload documents, perform semantic search, and integrate AI-powered question-answering into your workflows.

What are Knowledge Bases?

Knowledge Bases are vector databases that store documents with automatic chunking and vectorization. They enable semantic search capabilities, allowing you to find information based on meaning rather than exact text matches.

Key Features

• Document Storage: Upload PDF, TXT, MD, CSV, DOCX, XLSX and JSON files of up to 100 MB each. A DOCX or XLSX file must be a genuine Office document (password-protected, encrypted or malformed files are rejected); an archive that would expand far beyond its size (more than 10,000 entries, more than 256 MiB in one entry or 1 GiB in total, or — once it expands past 8 MiB — more than 200× its compressed size) is rejected at upload
• Automatic Processing: Documents are automatically chunked and vectorized
• Semantic Search: Find relevant information using natural language queries
• RAG Support: Perfect for building AI-powered question-answering systems
• Vector Embeddings: Uses OpenAI’s text-embedding-3-large for high-quality embeddings

Creating a Knowledge Base

1. Navigate to Storage in the workspace sidebar
2. Click on the Knowledge Bases tab
3. Click Create Knowledge Base
4. Enter a name and description for your knowledge base
5. The knowledge base is created and ready for file uploads

Adding Files to Knowledge Base

Using the UI

1. Open your knowledge base from the Storage page
2. Click Upload File or Add File
3. Select files from your computer, drop files or whole folders onto the dialog (up to 1,000 files per folder), or provide a URL. Files in an unsupported format or over the size limit are listed and skipped
4. Files are automatically processed:
   • Parsing: Extract text content from documents; PDF pages without a text layer are read with OCR
   • Chunking: Split documents into manageable chunks (configurable)
   • Embedding: Convert chunks to vector embeddings
   • Indexing: Store vectors for fast similarity search

Processing a file consumes Jinba Credits of the workspace’s organization, priced from the embedding tokens of its chunks and, for PDF pages read with OCR, the OCR tokens — see Knowledge Base Uploads. While the organization has no Jinba Credits left, a new file is not processed and is marked failed with an insufficient-credits message.

Using Workflows

You can also add files to knowledge bases programmatically using the JINBA_KNOWLEDGE_BASE_FILE_ADD tool:

- id: add_file_to_kb
  name: add_file_to_kb
  tool: JINBA_KNOWLEDGE_BASE_FILE_ADD
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: file
      value: "https://example.com/document.pdf"
    - name: filename
      value: "document.pdf"
    - name: executionMode
      value: "SYNCHRONOUS"
    - name: chunkerSettings
      value:
        chunkSize: 512
        chunkOverlap: 128
        chunkingIdentifier: "\\n\\n"

File Processing Status:

• pending: File uploaded, waiting for processing
• processing: File is being chunked and vectorized
• completed: File is ready for search
• failed: Processing encountered an error

Vector Search

Vector search enables semantic search across your knowledge base using natural language queries.

How It Works

1. Query Vectorization: Your search query is converted to a vector using OpenAI’s text-embedding-3-large
2. Similarity Search: The system finds chunks with similar vectors
3. Ranking: Results are ranked by similarity score
4. Filtering: Results below the threshold are filtered out

Using Vector Search

- id: search_kb
  name: search_kb
  tool: JINBA_VECTOR_SEARCH
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: query
      value: "How do I configure authentication?"
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: topK
      value: 5
    - name: threshold
      value: 0.3

Parameters:

Parameter	Type	Required	Default	Description
query	string	Yes	-	Natural language search query
knowledgeBaseId	string	Yes	-	ID of the knowledge base to search
topK	number	No	3	Number of results to return (1-50)
threshold	number	No	0.3	Similarity threshold (0-1)

Building RAG (Retrieval-Augmented Generation) Systems

RAG combines retrieval of relevant information from knowledge bases with AI generation to create accurate, context-aware responses.

What is RAG?

RAG (Retrieval-Augmented Generation) is a technique that:

1. Retrieves relevant information from a knowledge base using semantic search
2. Augments the AI prompt with retrieved context
3. Generates responses based on both the query and retrieved context

This approach allows AI models to answer questions using your own documents and data, rather than relying solely on their training data.

Step-by-Step: Building a RAG System

Step 1: Create a Knowledge Base

1. Go to Storage → Knowledge Bases
2. Create a new knowledge base (e.g., “Company Documentation”)
3. Note the knowledge base ID for later use

Step 2: Upload Documents

1. Open your knowledge base
2. Upload relevant documents (PDFs, DOCX, text files)
3. Wait for processing to complete (files show “completed” status)

Step 3: Build the RAG Workflow

Create a workflow that combines vector search with AI generation:

- id: user_question
  name: user_question
  tool: INPUT_TEXT
  input:
    - name: description
      value: "Enter your question"

- id: search_knowledge_base
  name: search_knowledge_base
  tool: JINBA_VECTOR_SEARCH
  needs: [user_question]
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: query
      value: "{{steps.user_question.result}}"
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: topK
      value: 5
    - name: threshold
      value: 0.3

- id: generate_answer
  name: generate_answer
  tool: ANTHROPIC_INVOKE
  needs: [user_question, search_knowledge_base]
  config:
    - name: version
      value: claude-sonnet-5-5
  input:
    - name: prompt
      value: |
        Based on the following information from our knowledge base, answer the user's question.
        
        Question: {{steps.user_question.result}}
        
        Relevant Information:
        {%- for r in steps.search_knowledge_base.results %}
        **Source: {{ r.file.filename }}** (Relevance: {{ r.score }})
        {{ r.chunk.content }}
        
        ---
        {%- endfor %}
        
        Please provide a comprehensive and accurate answer based on the information above.
        If the information doesn't contain enough details to answer the question, say so.

Step 4: Configure Secrets

1. The token config of the Jinba tools defaults to {{ secrets.JINBAFLOW_WS_API_KEY }}, the workspace API key that Jinba Flow provides to every run, so you can omit it; the examples spell it out
2. Store your Knowledge Base ID as a secret under Secrets (or use it directly in the workflow)

Advanced RAG Patterns

Multi-Step RAG with Refinement

- id: initial_search
  name: initial_search
  tool: JINBA_VECTOR_SEARCH
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: query
      value: "{{steps.user_question.result}}"
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: topK
      value: 10

- id: refine_query
  name: refine_query
  tool: ANTHROPIC_INVOKE
  needs: [initial_search]
  config:
    - name: version
      value: claude-sonnet-5-5
  input:
    - name: prompt
      value: |
        Based on the initial search results, generate a more specific search query.
        
        Original Query: {{steps.user_question.result}}
        
        Initial Results:
        {%- for r in steps.initial_search.results %}
        - {{ r.file.filename }}: {{ r.chunk.content }}
        {%- endfor %}
        
        Generate a refined search query that will find more specific information.

- id: refined_search
  name: refined_search
  tool: JINBA_VECTOR_SEARCH
  needs: [refine_query]
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: query
      value: "{{steps.refine_query.result.content}}"
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: topK
      value: 5
    - name: threshold
      value: 0.4

- id: final_answer
  name: final_answer
  tool: ANTHROPIC_INVOKE
  needs: [refined_search]
  config:
    - name: version
      value: claude-sonnet-5-5
  input:
    - name: prompt
      value: |
        Answer the user's question using the refined search results.
        
        Question: {{steps.user_question.result}}
        
        Refined Search Results:
        {%- for r in steps.refined_search.results %}
        **{{ r.file.filename }}** (Score: {{ r.score }})
        {{ r.chunk.content }}
        {%- endfor %}
        
        Provide a detailed, accurate answer.

Managing Knowledge Base Files

Updating Files

You can update existing files in a knowledge base using the JINBA_KNOWLEDGE_BASE_UPDATE tool:

- id: update_kb_file
  name: update_kb_file
  tool: JINBA_KNOWLEDGE_BASE_UPDATE
  config:
    - name: token
      value: "{{secrets.JINBAFLOW_WS_API_KEY}}"
  input:
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: knowledgeBaseFileId
      value: "{{secrets.FILE_ID}}"
    - name: fileUrl
      value: "https://example.com/updated-document.pdf"
    - name: updateType
      value: "FULL_REFRESH"
    - name: executionMode
      value: "SYNCHRONOUS"

Update Types:

• FULL_REFRESH: Replaces the entire file with new content
• Other update modes may be available depending on your configuration

Chunking Configuration

When adding files to knowledge bases, you can configure how documents are chunked:

• chunkSize: Size of each chunk in tokens (default: 512, max: 8192)
• chunkOverlap: Overlap between chunks in tokens (default: 128, max: 2048)
• chunkingIdentifier: String used to identify chunk boundaries (default: “\n\n”)

Best Practices:

• Smaller chunks (256-512): Better for precise information retrieval
• Larger chunks (1024-2048): Better for maintaining context
• Overlap: Helps maintain context across chunk boundaries

RAG Best Practices

1. Quality Knowledge Base: Upload high-quality, relevant documents
2. Appropriate Chunking: Use appropriate chunk sizes (typically 512-1024 tokens)
3. Threshold Tuning: Adjust similarity thresholds based on your use case
4. TopK Selection: Retrieve enough context (typically 3-10 chunks) for comprehensive answers
5. Prompt Engineering: Craft prompts that clearly instruct the AI to use retrieved context
6. Source Attribution: Always cite sources for transparency and verification
7. Error Handling: Handle cases where no relevant information is found

Use Cases for RAG

• Customer Support: Answer questions using product documentation
• Internal Knowledge: Access company policies and procedures
• Research Assistant: Search through research papers and documents
• Legal Document Q&A: Answer questions about contracts and legal documents
• Technical Documentation: Help developers find information in technical docs
• Product Information: Answer questions about product specifications

Related Features

• Workspace - Learn about workspace management and file storage
• Vector Search Tool - Detailed vector search documentation
• Knowledge Base Tool - Knowledge base management tools