Dify AI Platform Setup: Advanced RAG Pipelines, Agents, and Production Workflows for Real Apps
Getting Dify running takes under an hour. Getting it to power a real production AI feature — a customer-facing chatbot that answers from your documentation, an internal agent that queries your database and takes actions, or a multi-step workflow that classifies and routes incoming requests — takes a deeper understanding of how Dify's components fit together. This guide covers advanced Dify AI platform setup: production RAG configuration, knowledge base management, agent tool design, workflow orchestration, and API integration patterns your backend team can actually build on.
If you haven't deployed Dify yet, start with our Dify deployment guide which covers Docker Compose setup, LLM provider configuration, and first-app creation. This guide picks up from a running instance with at least one LLM provider connected.
Prerequisites
- A running Dify instance with HTTPS configured — see our Dify deployment guide
- At least one LLM provider connected (OpenAI, Anthropic, or Ollama for local models)
- An embedding model configured — this is required for RAG;
text-embedding-3-small(OpenAI) ornomic-embed-text(Ollama) work well - At least 4GB RAM available on the host — Weaviate (Dify's default vector store) needs room
- Docker and Docker Compose access for configuration changes
Verify your instance is healthy and models are configured:
# Check all Dify services are running
cd dify/docker
docker compose ps
# Verify Weaviate (vector DB) is healthy
curl http://localhost:8080/v1/.well-known/ready
# Should return: {}
# Check the Dify API is responding
curl https://dify.yourdomain.com/console/api/version
# Should return version info JSONProduction RAG: Knowledge Bases That Actually Work
The default RAG settings in Dify are fine for demos. For production — where retrieval quality directly affects whether your app is useful — you need to tune every step of the pipeline.
Choosing the Right Chunking Strategy
Chunking determines how your documents are split before embedding. The wrong strategy is the single biggest cause of poor RAG quality. In Dify's knowledge base settings, go to Chunk Settings and consider:
- Automatic chunking — Dify splits on paragraph boundaries. Good default for prose documents like FAQs and documentation.
- Fixed-length chunking — splits at a fixed token count with configurable overlap. Better for structured content like product specs, API docs, and tables where paragraph boundaries don't exist.
- Custom separator — split on a specific delimiter you define. Best for structured documents that use consistent section markers like
###,---, or custom tags.
A practical starting point for most technical documentation:
# Recommended chunk settings for technical docs:
Chunk size: 512 tokens
Chunk overlap: 50 tokens
Maximum chunk length: 512
# For long-form content (legal docs, manuals):
Chunk size: 1024 tokens
Chunk overlap: 100 tokens
# For short Q&A content (FAQs, support articles):
Chunk size: 256 tokens
Chunk overlap: 20 tokens
# Rule of thumb: overlap should be ~10% of chunk size
# Test retrieval quality with real queries before going to productionRetrieval Mode: Full-Text vs. Vector vs. Hybrid
Dify supports three retrieval modes, configurable per knowledge base in Retrieval Settings:
- Vector search — semantic similarity. Finds conceptually related content even with different wording. Best for natural language queries about concepts.
- Full-text search — keyword matching. Finds exact terms. Best when users query specific product names, error codes, or identifiers.
- Hybrid search — combines both with a configurable weight. Best for production where you don't know exactly how users will phrase queries. Start here.
For hybrid search, also enable Reranking if you have a reranker model configured. Reranking runs the top-K retrieved chunks through a cross-encoder model that re-scores them for relevance — significantly improves precision without changing your chunk count.
Managing Knowledge Base Documents via API
For production setups where documentation updates automatically (from Git, CMS, or database), use Dify's Knowledge Base API to push documents programmatically:
# Create a document in a knowledge base via API
curl -X POST https://dify.yourdomain.com/v1/datasets/YOUR_DATASET_ID/document/create_by_text \
-H 'Authorization: Bearer YOUR_DATASET_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"name": "Refund Policy v2.1",
"text": "## Refund Policy\n\nWe offer a 30-day money-back guarantee...",
"indexing_technique": "high_quality",
"process_rule": {
"mode": "automatic"
}
}'
# Upload a file to a knowledge base
curl -X POST https://dify.yourdomain.com/v1/datasets/YOUR_DATASET_ID/document/create_by_file \
-H 'Authorization: Bearer YOUR_DATASET_API_KEY' \
-F 'file=@/path/to/documentation.pdf' \
-F 'data={"indexing_technique":"high_quality","process_rule":{"mode":"automatic"}}'
# Check indexing status
curl https://dify.yourdomain.com/v1/datasets/YOUR_DATASET_ID/documents \
-H 'Authorization: Bearer YOUR_DATASET_API_KEY' | \
jq '[.data[] | {name: .name, status: .indexing_status, word_count: .word_count}]'Sync Documentation from GitHub on Push
Combine Dify's Knowledge Base API with a GitHub Actions workflow to automatically re-index documentation when it changes:
# .github/workflows/sync-docs.yml
name: Sync Docs to Dify
on:
push:
branches: [main]
paths:
- 'docs/**'
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Sync changed docs to Dify knowledge base
env:
DIFY_API_KEY: ${{ secrets.DIFY_DATASET_API_KEY }}
DIFY_URL: ${{ secrets.DIFY_URL }}
DATASET_ID: ${{ secrets.DIFY_DATASET_ID }}
run: |
for file in docs/*.md; do
echo "Uploading $file..."
curl -X POST "${DIFY_URL}/v1/datasets/${DATASET_ID}/document/create_by_file" \
-H "Authorization: Bearer ${DIFY_API_KEY}" \
-F "file=@${file}" \
-F 'data={"indexing_technique":"high_quality","process_rule":{"mode":"automatic"}}'
doneBuilding Tool-Using Agents
What Makes a Good Agent Tool
Dify's Agent apps let the LLM choose which tools to call based on the user's query. The quality of your tool definitions determines how reliably the agent uses them. Three things matter:
- Name — short, verb-noun:
search_orders,get_weather,create_ticket - Description — one or two sentences telling the LLM exactly when to use this tool and what it returns. Be specific about the input format expected.
- Parameters — well-typed with clear descriptions. The LLM fills in parameter values from the conversation context.
Creating a Custom API Tool
In Tools → Custom Tools → Create Tool, define your API tool using OpenAPI schema format. Here's a complete example for an order lookup tool:
# OpenAPI schema for a custom Dify tool
openapi: 3.0.0
info:
title: Order Management API
description: Tools for looking up and managing customer orders
version: 1.0.0
servers:
- url: https://api.yourdomain.com
paths:
/orders/lookup:
get:
operationId: lookup_order
summary: Look up an order by order ID or customer email
description: >
Use this tool when a customer asks about their order status,
shipping information, or order details. Returns order status,
items, tracking number, and estimated delivery date.
parameters:
- name: order_id
in: query
description: The order ID (format: ORD-XXXXXXXX)
required: false
schema:
type: string
- name: email
in: query
description: Customer email address to look up recent orders
required: false
schema:
type: string
format: email
responses:
'200':
description: Order details
content:
application/json:
schema:
type: objectIn the Dify tool editor, also set the Authentication header your API requires. Dify stores this as an encrypted credential referenced by name — the actual key never appears in the tool definition that the LLM sees.
Agent System Prompt Design
The agent's system prompt controls its personality, scope, and how it uses tools. A well-structured system prompt for a customer support agent:
# Agent system prompt template for a customer support agent:
You are a helpful customer support assistant for Acme Corp.
## Your capabilities
- Look up order status and shipping information using the lookup_order tool
- Search the help documentation using the search_docs tool
- Create support tickets using the create_ticket tool when issues need human follow-up
## How to handle requests
1. Always look up the customer's order before discussing order-specific issues
2. Search documentation before answering product questions
3. If you cannot resolve an issue, create a support ticket and give the customer the ticket ID
4. Never make up order information — always use the lookup_order tool
## Tone
- Friendly but professional
- Concise — don't over-explain
- Acknowledge frustration when customers are upset
## Boundaries
- Do not discuss competitor products
- Do not make promises about refunds or policy exceptions
- Escalate billing disputes via create_ticket immediatelyWorkflow Apps for Multi-Step AI Pipelines
When to Use Workflows vs. Chatbots
Chatbot apps handle conversation. Workflow apps handle processing. Use workflows when you need:
- Deterministic multi-step pipelines (classify → route → generate)
- Parallel processing (run multiple LLM calls simultaneously)
- Conditional branching based on LLM output
- Document processing pipelines (ingest → extract → transform → store)
- Batch operations triggered by webhook
Building a Document Classification Workflow
A common enterprise use case: inbound documents (emails, forms, uploads) need to be classified and routed to the right team. Here's how to build it in Dify's workflow canvas:
- Start node — input variable:
document_text(string, required) - LLM node — classify the document into categories. Prompt: "Classify the following document into exactly one of: [billing, technical_support, sales, legal, other]. Return only the category name." Output variable:
category - IF/ELSE node — branch on
categoryvalue - Multiple LLM nodes (one per branch) — generate category-specific responses or extract category-specific fields
- HTTP Request node — POST the classification result and extracted data to your backend webhook
- End node — return the routing decision and response
Calling Workflow Apps via API
# Trigger a workflow app and get the result
curl -X POST https://dify.yourdomain.com/v1/workflows/run \
-H 'Authorization: Bearer YOUR_WORKFLOW_APP_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"inputs": {
"document_text": "Hi, I was charged twice for my subscription this month..."
},
"response_mode": "blocking",
"user": "system-pipeline"
}' | jq '.data.outputs'
# Returns:
# {
# "category": "billing",
# "response": "I can see this is a billing inquiry...",
# "confidence": "high"
# }
# For long-running workflows, use streaming mode:
curl -X POST https://dify.yourdomain.com/v1/workflows/run \
-H 'Authorization: Bearer YOUR_WORKFLOW_APP_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"inputs": {"document_text": "..."},
"response_mode": "streaming",
"user": "system-pipeline"
}'Integrating Dify Apps Into Your Backend
Conversation Management with Session IDs
When integrating a Dify chatbot into your product, maintain conversation continuity by passing a consistent session ID per user. Dify uses this to look up conversation history:
import requests
class DifyClient:
def __init__(self, api_key: str, base_url: str):
self.api_key = api_key
self.base_url = base_url
self.headers = {
'Authorization': f'Bearer {api_key}',
'Content-Type': 'application/json'
}
def chat(self, user_id: str, message: str, conversation_id: str = '') -> dict:
response = requests.post(
f'{self.base_url}/v1/chat-messages',
headers=self.headers,
json={
'inputs': {},
'query': message,
'response_mode': 'blocking',
'conversation_id': conversation_id, # Empty string = new conversation
'user': user_id # Your internal user identifier
}
)
response.raise_for_status()
data = response.json()
return {
'answer': data['answer'],
'conversation_id': data['conversation_id'], # Save this for next message
'message_id': data['message_id']
}
# Usage:
client = DifyClient('YOUR_APP_API_KEY', 'https://dify.yourdomain.com')
# First message — no conversation_id
result = client.chat('user-123', 'What is your refund policy?')
conversation_id = result['conversation_id'] # Store per user session
# Follow-up — pass conversation_id for context
result = client.chat('user-123', 'What if I paid with crypto?', conversation_id)Streaming Responses for Chat UIs
For a responsive chat interface, use streaming mode. Each token arrives as a Server-Sent Event:
import requests
import json
def stream_chat(api_key: str, base_url: str, message: str, conversation_id: str = ''):
"""
Stream a Dify chat response, yielding text tokens as they arrive.
"""
with requests.post(
f'{base_url}/v1/chat-messages',
headers={
'Authorization': f'Bearer {api_key}',
'Content-Type': 'application/json',
'Accept': 'text/event-stream'
},
json={
'inputs': {},
'query': message,
'response_mode': 'streaming',
'conversation_id': conversation_id,
'user': 'streaming-user'
},
stream=True
) as response:
for line in response.iter_lines():
if line and line.startswith(b'data: '):
data = json.loads(line[6:])
if data.get('event') == 'message':
yield data.get('answer', '') # Incremental token
elif data.get('event') == 'message_end':
return # Stream complete
# Usage:
for token in stream_chat('YOUR_API_KEY', 'https://dify.yourdomain.com', 'Hello'):
print(token, end='', flush=True)Tips, Gotchas, and Troubleshooting
RAG Returns Irrelevant Results
If your chatbot is hallucinating or ignoring the knowledge base, retrieval quality is the issue. Diagnose it:
# Test retrieval quality directly via API
curl -X POST https://dify.yourdomain.com/v1/datasets/YOUR_DATASET_ID/retrieve \
-H 'Authorization: Bearer YOUR_DATASET_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"query": "what is the return policy for digital products",
"retrieval_model": {
"search_method": "hybrid_search",
"top_k": 5,
"score_threshold_enabled": true,
"score_threshold": 0.5
}
}' | jq '[.records[] | {content: .segment.content, score: .score}]'
# If top results are irrelevant:
# 1. Check your chunks are the right size for the content
# 2. Switch to hybrid search if using vector-only
# 3. Lower score_threshold to see more candidates
# 4. Check your embedding model is the same for indexing and retrievalWorker Service Crashing on Document Upload
# Check worker logs for the actual error
docker compose logs worker --tail 50
docker compose logs api --tail 30
# Common causes:
# 1. Weaviate is down or unreachable
curl http://localhost:8080/v1/.well-known/ready
docker compose restart weaviate
# 2. Worker OOM — increase memory limit
# In docker-compose.yml under worker service:
# deploy:
# resources:
# limits:
# memory: 2G
# 3. Embedding API rate limit or key issue
docker compose logs worker | grep -i embedding
docker compose logs worker | grep -i error | tail -20Workflow Stuck in Running State
# Check workflow run status via API
curl https://dify.yourdomain.com/v1/workflows/runs/WORKFLOW_RUN_ID \
-H 'Authorization: Bearer YOUR_APP_API_KEY' | jq '{status: .status, error: .error}'
# Check for LLM timeout (default is 60s per node)
# Increase in app settings: LLM node → Advanced → Timeout
# Check Celery worker queue depth
docker exec dify-redis redis-cli llen celery
# If this is very high, the worker is backed up — scale workers:
# docker compose up -d --scale worker=3Updating Dify
cd dify/docker
# Check release notes for breaking changes first:
# https://github.com/langgenius/dify/releases
# Pull latest
git pull origin main
docker compose pull
# Restart — migrations run automatically
docker compose up -d
# Watch for migration completion
docker compose logs api --tail 30 | grep -E 'migration|error|started'
# Verify all apps still work after update
curl https://dify.yourdomain.com/console/api/versionPro Tips
- Use Dify's annotation feature for RAG quality improvement — in the app's Annotations tab, mark responses as good or bad and add corrected answers. Dify uses these as high-priority retrieval results, effectively overriding vector search for common queries.
- Set LLM node output variables explicitly — in workflow LLM nodes, always define what the node should extract as a structured output. Unstructured string outputs are hard to use downstream and cause brittle workflows.
- Use conversation variables for stateful agents — workflow apps support conversation variables that persist across turns. Use these to track user context (their account ID, preferences, current issue) rather than re-extracting it from every message.
- Monitor token usage per app — in Monitoring → Overview, review token consumption per application weekly. A single misconfigured prompt can consume 10x expected tokens and spike your LLM bill before you notice.
- Test with Dify's built-in debugger before publishing — every app type has a debug panel. Use it with real production-representative inputs before exposing any app via API. The debugger shows retrieved chunks, token counts, and intermediate node outputs — essential for diagnosing quality issues.
Wrapping Up
A production Dify AI platform setup is a layered system: well-chunked knowledge bases with hybrid retrieval, clearly defined agent tools with precise descriptions, workflow pipelines with deterministic routing, and a backend integration that maintains conversation context and handles streaming gracefully. Each layer compounds on the others — good retrieval makes agents smarter, good agent tool design makes workflows more reliable, and proper API integration makes the whole thing usable by real users.
If you're still getting Dify deployed for the first time, start with our Dify deployment guide and return here once your first app is running. The advanced configuration in this guide is what separates a prototype from something you'd actually put in front of customers.
Need Production AI Apps Built and Maintained?
Building a solid RAG chatbot or workflow pipeline in Dify takes more than configuration — it takes careful prompt engineering, retrieval tuning, backend integration work, and ongoing quality monitoring. The sysbrix team builds and maintains production AI apps on Dify for teams that need results, not prototypes.
📚 More Dify Guides on Sysbrix