Are you tired of repeatedly searching for information across multiple sources, then spending extra time compiling and summarizing your findings? You're not alone. Many individuals and teams face the challenge of sifting through various web pages or internal documents for a comprehensive overview of a topic. Fortunately, Dify, a low-code, open-source platform for LLM application development, solves this problem by automating workflows for multi-step searches and efficient summarization, requiring only minimal coding.
In this article, we'll create "DeepResearch," a tool that orchestrates searches, generates keywords, and summarizes results into a unified report. By following these steps, you'll understand how Dify streamlines information gathering, letting you focus on insights instead of repetitive tasks.
Prerequisites
Before proceeding, please ensure you have the following:
Dify Account: You can register for an online account or, alternatively, deploy Dify on-premise for internal organizational use.
LLM API Key: DeepResearch uses Large Language Models (LLMs) for generating search queries and summaries. You'll need an API key from a provider (like OpenAI, Google, Anthropic), or deploy open-source models locally using tools like Ollama and integrate them with Dify. This article uses the DeepSeek API.
To get a DeepSeek API key, visit the DeepSeek API Open Platform or use a third-party model platform.
See also: Dify x DeepSeek: Deploy a Private AI Assistant & Build a Local DeepSeek R1 + Web Search App
External Search Tool or API: To search the web, you'll need a search provider API key. This article uses the Tavily Search API. Obtain a Tavily API key from Tavily. While this article assumes a search API, Dify also supports other data retrieval methods.
Feel free to adjust these prerequisites based on your desired scope and data sources.
Creating a Workflow from Template
Navigate to Explore: In the Dify dashboard, click "Explore."
Choose "DeepResearch" Template: In "Explore," search for and click on the "DeepResearch" template.
Name Your Application: The default name is "DeepResearch," but you can customize it.
Building the Multi-Search Workflow
1. Previewing the Application
Click "Preview" to try the application. You'll be prompted for a topic and depth.
topic (Text): The subject you want to investigate.
depth (Number): Number of search rounds.
By providing these inputs, we give our workflow the flexibility to handle various research contexts.
2. Mechanism for Repeating the Search Steps
To avoid manually duplicating each search step, weʼll place an Iteration node in the workflow. This node processes each element in an array, effectively looping through your search sequence.
The Iteration node will initiate the next search round at the conclusion of each iteration or terminate the process if it determines that sufficient information has been gathered.
3. Explaining the Key Nodes
Common nodes that we may consider placing within the Iteration container:
a) LLM Node
Purpose: The AI suggests or refines subsequent search keywords or determines if the search process should conclude.
Prompt Tip: Consider using a prompt such as, “Based on the current findings and the user’s topic, suggest the next angle to explore. If further searches are unnecessary, set
shouldContinueto false.”
b) Search/Extraction Node
Purpose: Executes a web search or queries an external API using the “next topic” from the LLM node.
Implementation: Configure the node by inputting your search keyword variable.
c) Assigner Node (Variable Update)
Purpose: Combines each new set of results into a cumulative list (
findings) or updates the conversation state.Key Setting: Select “append” when adding new data to avoid overwriting previous results.
d) IF-ELSE Node
Purpose: Evaluates a
shouldContinueboolean or a similar flag to decide whether to proceed with another iteration or exit the loop.
e) Final LLM Node
Purpose: Summarizes or synthesizes the complete
findingsarray to produce a cohesive narrative or a bullet-point summary.Prompt Tip: A system prompt like, “Combine all search insights into a concise technical overview. Format as Markdown,” can be effective.
f) Answer Node
Purpose: Delivers the final output from the LLM back to the user.
This node represents the concluding “response” step of your application.
These nodes, connected correctly, form the basis for repeated searching and data collection. With these nodes connected, users input a topic and depth to receive a thorough summary from multiple searches. The workflow visually demonstrates the looping process until sufficient data is collected or stopped.
Future Possibilities
Building "DeepResearch" is just the beginning. Dify's low-code environment supports many add-ons and variations:
Custom Reporting: Generate tailored reports based on the gathered data, allowing for deeper insights and strategic decision-making.
Link Internal Data: Connect your companyʼs proprietary knowledge base, making the search process even more comprehensive for internal use.
Integration with Other Tools: Automate notifications via Slack. The Dify ecosystem makes it straightforward to extend.
By incorporating these enhancements, you can elevate DeepResearch into a comprehensive, end-to-end system for information gathering and consolidation. This can all be achieved without delving into the intricacies of traditional coding or complex infrastructure.
This tutorial aimed to demonstrate how to automate and streamline in-depth research tasks using Dify. By leveraging Dify's capabilities, you can significantly reduce the time and effort spent on manual research while easily adapting your multi-search workflow to various topics, APIs, or data sources. We encourage you to experiment with Dify and discover how quickly you can build an iterative research solution tailored to your specific needs.
