Run Semantic Search on Knowledge Bases in Jinba Flow

Use JINBA_VECTOR_SEARCH to run semantic queries against a knowledge base using vector embeddings in a Jinba Flow workflow.

Jinba Vector Search enables semantic search capabilities for knowledge bases using vector embeddings. This tool uses OpenAI’s text-embedding-3-large to vectorize queries and retrieve relevant data from your knowledge base based on similarity scores, providing powerful RAG (Retrieval-Augmented Generation) functionality.

• Semantic Search: Find conceptually similar content, not just exact matches
• Knowledge Base Integration: Search across uploaded documents and files
• Similarity Scoring: Filter results by similarity threshold
• Configurable Results: Control the number of results returned
• RAG Support: Perfect for building question-answering and information retrieval workflows

This tool authenticates with a workspace secret key via the token config, which defaults to {{ secrets.JINBAFLOW_WS_API_KEY }} — you usually do not need to set it explicitly.

Configuration:

• token (string): Jinba Flow workspace API key (stored as a secret; defaults to {{ secrets.JINBAFLOW_WS_API_KEY }})

Input Parameters

Parameter	Type	Required	Default	Description
query	string	Yes	-	Search query for semantic search
knowledgeBaseId	string	Yes	-	ID of the knowledge base to search
topK	number	No	3	Number of top results to return (1-50)
threshold	number	No	0.3	Similarity threshold for filtering results (0-1)
filter	object	No	-	Filter options; filter.metadata is a key-value map that restricts results by chunk metadata

Output Structure

The output has the following top-level fields (steps.<id>.results, steps.<id>.query, steps.<id>.totalResults — not under result):

Field	Type	Description
results	array	Search results; each entry has chunk (id, fileId, content, contentType, metadata, createdAt), score (0-1, higher is more similar), and file (id, filename, contentType, metadata)
query	string	The original search query
totalResults	number	Total number of results found

Build a document Q&A system

- id: search_knowledge_base
  name: search_knowledge_base
  tool: JINBA_VECTOR_SEARCH
  config:
    - name: token
      value: "{{secrets.JINBA_API_TOKEN}}"
  input:
    - name: query
      value: "What is our company's return policy?"
    - name: knowledgeBaseId
      value: "{{secrets.KNOWLEDGE_BASE_ID}}"
    - name: topK
      value: 5
    - name: threshold
      value: 0.4

- id: generate_answer
  name: generate_answer
  tool: OPENAI_INVOKE
  needs: [search_knowledge_base]
  config:
    - name: version
      value: gpt-5.2
  input:
    - name: prompt
      value: |
        Based on the following search results from our knowledge base, answer the user's question about our return policy.
        
        Search Results:
        {% for r in steps.search_knowledge_base.results %}
        **From: {{r.file.filename}}**
        Content: {{r.chunk.content}}
        Similarity: {{r.score}}
        
        {% endfor %}
        
        Question: {{steps.search_knowledge_base.query}}
        
        Please provide a comprehensive answer based on the search results above.

Example: Research Assistant

- id: research_query
  name: research_query
  tool: INPUT_TEXT
  input:
    - name: description
      value: "Enter your research question"

- id: search_research_base
  name: search_research_base
  tool: JINBA_VECTOR_SEARCH
  needs: [research_query]
  config:
    - name: token
      value: "{{secrets.JINBA_API_TOKEN}}"
  input:
    - name: query
      value: "{{steps.research_query.result}}"
    - name: knowledgeBaseId
      value: "{{secrets.RESEARCH_KB_ID}}"
    - name: topK
      value: 10
    - name: threshold
      value: 0.3

- id: analyze_findings
  name: analyze_findings
  tool: PYTHON_SANDBOX_RUN
  needs: [search_research_base]
  input:
    - name: code
      value: |
        import json
        
        # Previous step outputs are injected as the `steps` object
        results = steps.search_research_base.results
        query = steps.search_research_base.query
        total_results = steps.search_research_base.totalResults
        
        print("=== Research Findings ===")
        print(f"Query: {query}")
        print(f"Total Results: {total_results}")
        print()
        
        # Group results by source file
        sources = {}
        for result in results:
            filename = result['file']['filename']
            if filename not in sources:
                sources[filename] = []
            sources[filename].append({
                'content': result['chunk']['content'][:200] + "...",
                'score': result['score']
            })
        
        print("=== Sources Found ===")
        for filename, chunks in sources.items():
            print(f"\n📄 {filename}")
            for i, chunk in enumerate(chunks[:3]):  # Top 3 chunks per file
                print(f"  {i+1}. Score: {chunk['score']:.3f}")
                print(f"     {chunk['content']}")
        
        # Calculate average similarity
        avg_score = sum(r['score'] for r in results) / len(results)
        print(f"\n📊 Average Similarity: {avg_score:.3f}")

- id: synthesize_response
  name: synthesize_response
  tool: ANTHROPIC_INVOKE
  needs: [search_research_base]
  config:
    - name: token
      value: "{{secrets.ANTHROPIC_API_KEY}}"
  input:
    - name: prompt
      value: |
        You are a research assistant. Synthesize the following search results into a comprehensive response.
        
        Original Query: {{steps.search_research_base.query}}
        
        Search Results:
        {% for r in steps.search_research_base.results %}
        
        **Source: {{r.file.filename}}** (Similarity: {{r.score}})
        {{r.chunk.content}}
        
        ---
        {% endfor %}
        
        Please provide:
        1. A direct answer to the question
        2. Key insights from multiple sources
        3. Any conflicting information found
        4. Suggestions for further research

Example: Content Recommendation

- id: get_user_interests
  name: get_user_interests
  tool: INPUT_TEXT
  input:
    - name: description
      value: "Describe topics or content you're interested in"

- id: find_related_content
  name: find_related_content
  tool: JINBA_VECTOR_SEARCH
  needs: [get_user_interests]
  config:
    - name: token
      value: "{{secrets.JINBA_API_TOKEN}}"
  input:
    - name: query
      value: "{{steps.get_user_interests.result}}"
    - name: knowledgeBaseId
      value: "{{secrets.CONTENT_KB_ID}}"
    - name: topK
      value: 8
    - name: threshold
      value: 0.25

- id: format_recommendations
  name: format_recommendations
  tool: PYTHON_SANDBOX_RUN
  needs: [find_related_content]
  input:
    - name: code
      value: |
        # Previous step outputs are injected as the `steps` object
        results = steps.find_related_content.results
        query = steps.find_related_content.query
        total_results = steps.find_related_content.totalResults
        
        print("🔍 **Content Recommendations**")
        print(f"Based on: *{query}*\n")
        
        recommendations = []
        for i, result in enumerate(results[:5], 1):
            chunk = result['chunk']
            file_info = result['file']
            score = result['score']
            
            # Extract first sentence or 100 characters
            content_preview = chunk['content'][:100].split('.')[0] + "..."
            
            recommendations.append({
                'rank': i,
                'title': file_info['filename'],
                'preview': content_preview,
                'relevance': f"{score:.1%}",
                'content_type': file_info['contentType']
            })
        
        for rec in recommendations:
            print(f"**{rec['rank']}. {rec['title']}**")
            print(f"   📋 {rec['preview']}")
            print(f"   🎯 Relevance: {rec['relevance']}")
            print(f"   📄 Type: {rec['content_type']}")
            print()
        
        if len(results) == 0:
            print("No relevant content found. Try adjusting your search terms or lowering the threshold.")

Best Practices

Query Optimization

• Use natural language: Write queries as you would ask a human
• Be specific: More specific queries often yield better results
• Include context: Add relevant keywords and context terms

Threshold Selection

• 0.7-1.0: Very high similarity, exact or near-exact matches
• 0.4-0.7: High similarity, closely related content
• 0.2-0.4: Moderate similarity, potentially relevant content
• 0.0-0.2: Low similarity, may include tangentially related content

Performance Tips

• Limit topK: Don’t retrieve more results than needed
• Adjust threshold: Higher thresholds = fewer, more relevant results
• Use metadata: Leverage chunk metadata for additional filtering

Set up a knowledge base

Before using vector search, ensure your knowledge base contains relevant documents:

1. Upload Documents: Add PDFs, text files, or other supported formats
2. Processing: Allow time for document chunking and vectorization
3. Test Queries: Start with simple queries to understand your data
4. Iterate: Refine queries and thresholds based on results

Use Cases

• Customer Support: Find relevant documentation for user questions
• Research Assistant: Discover related research papers and documents
• Content Discovery: Recommend similar articles or resources
• FAQ Automation: Automatically answer common questions
• Document Analysis: Find specific information across large document sets
• Knowledge Management: Quick access to institutional knowledge
• Legal Research: Search through contracts and legal documents
• Product Information: Find technical specifications and manuals