NEWOfficial SDKs Now AvailablePython, JavaScript & Go SDKs with full API coverage
Back to blog
agentic-aiai-agentsclaudedata-extractionfastapifunction-callinglangchainllmmcpmodel-context-protocolopen-sourcepythonscraping-apisearchself-hostedserpapi-alternativeweb-scrapingweb-search

Building AI Search Agents: How JIRO's Agentic Research Works

By the JIRO Engineering Team at Blackvault Technology

the JIRO Engineering Team at Blackvault Technology
13 min read

Building AI Search Agents: How JIRO's Agentic Research Works

By the JIRO Engineering Team at Blackvault Technology

Indore, MP, India — September 2026


AI agents are the future of search. Instead of typing a query and getting 10 blue links, you tell an AI what you need, and it researches, reads, and synthesizes an answer for you.

At JIRO, we built exactly that. It is called Agentic Research, and it is one of the most powerful features of our platform.

Here is how it works, how we built it, and how you can use it today.

What is Agentic Research?

Traditional search is passive. You type a query, hit enter, and get results. You are responsible for reading, filtering, and synthesizing.

Agentic research is active. You give it a goal, and it:

  1. Plans the research strategy
  2. Searches multiple engines for relevant information
  3. Reads the top results
  4. Extracts key facts and insights
  5. Synthesizes a coherent answer
  6. Cites sources

It is like hiring a research assistant who never sleeps, never gets tired, and can process 100x more information than a human.

Aspect Traditional Search Agentic Research
Input Query string Research question
Output List of links Synthesized answer with citations
Effort User reads and filters AI reads and filters
Speed Minutes to hours 10-30 seconds
Coverage 10-100 results 100+ sources
Accuracy Varies by user Consistent (LLM-powered)
Citations Manual Automatic

How JIRO's Agentic Research Works

JIRO's agentic research is built on four core components:

1. The Orchestrator

The orchestrator is the "brain" of the agent. It:

  • Breaks down complex questions into sub-questions
  • Prioritizes which sub-questions to answer first
  • Decides when it has enough information
  • Synthesizes the final answer

We use a combination of rule-based planning and LLM-powered reasoning. For simple queries, rule-based planning is faster and more reliable. For complex queries, we use an LLM to plan the research strategy.

Planning Strategies

We use different planning strategies depending on the query type:

Rule-Based Planning (for simple queries):

Query: "Who is the CEO of OpenAI?"
Sub-questions: ["Who is the CEO of OpenAI?"]
Search strategy: Single search, extract from top result

LLM-Based Planning (for complex queries):

Query: "What are the implications of quantum computing for cryptography?"
Sub-questions: [
  "What is quantum computing?",
  "How does quantum computing affect cryptography?",
  "What are the latest quantum computing developments in 2026?",
  "Which companies are working on quantum-safe cryptography?",
  "What are the timelines for quantum threats to current encryption?"
]
Search strategy: Search each sub-question across multiple engines

Hybrid Planning (for medium queries):

  • Use rules for known patterns (e.g., "who is", "what is")
  • Use LLM for novel patterns

2. The Search Engine

JIRO can search 9 different engines and 12 social platforms simultaneously:

Search Engines (9):

  • Google
  • Bing
  • DuckDuckGo
  • Brave
  • YouTube
  • Amazon
  • eBay
  • Yandex
  • Baidu

Social Platforms (12):

  • Reddit
  • Hacker News
  • Twitter/X
  • LinkedIn
  • Facebook
  • Instagram
  • TikTok
  • GitHub
  • StackOverflow
  • Wikipedia
  • NewsAPI
  • Discord

For each sub-question, the agent searches multiple engines and platforms, deduplicates results, and ranks them by relevance.

Search Execution

For each sub-question:
  1. Search Google, Bing, DuckDuckGo concurrently
  2. Collect top 10 results from each engine
  3. Deduplicate by URL
  4. Rank by relevance score (domain authority, freshness, keyword match)
  5. Select top 10 unique results for reading

3. The Reader

The reader extracts content from search results. It:

  • Fetches the top 10-20 results
  • Parses HTML to extract text
  • Cleans the content (removes ads, navigation, etc.)
  • Splits long articles into chunks
  • Filters low-quality content

The reader is optimized for speed and accuracy. It can process 100+ pages per minute.

Content Extraction Pipeline

For each URL:
  1. Fetch HTML with timeout (10s) and size limit (1MB)
  2. Parse with Beautiful Soup
  3. Extract main content (remove nav, sidebar, footer, ads)
  4. Clean HTML entities and whitespace
  5. Split into 500-token chunks
  6. Filter chunks by quality score
  7. Store in vector database (for semantic search)

4. The Synthesizer

The synthesizer combines all the extracted information into a coherent answer. It uses an LLM (OpenAI GPT-4.5, Anthropic Claude 3.7 Sonnet, or local models) to:

  • Identify key facts and insights
  • Resolve contradictions between sources
  • Structure the answer logically
  • Cite sources for each claim

The result is a well-structured, fact-checked answer with citations — just like a human researcher would produce.

Synthesis Prompt Engineering

We use carefully engineered prompts to get the best results:

SYNTHESIS_PROMPT = """
You are a research assistant. Given the following research question and source materials,
provide a comprehensive answer with citations.

Research Question: {question}

Source Materials:
{sources}

Instructions:
1. Synthesize information from multiple sources
2. Resolve any contradictions between sources
3. Structure the answer with clear sections
4. Cite each claim with [1], [2], etc. matching the source numbers
5. If sources conflict, acknowledge the disagreement
6. If information is insufficient, say so

Answer:
"""

Example: Agentic Research in Action

Let us say you ask JIRO: "What are the latest developments in quantum computing for cryptography?"

Here is what happens:

Step 1: Planning

The orchestrator breaks this down into sub-questions:

  • "What is quantum computing?"
  • "How does quantum computing affect cryptography?"
  • "What are the latest quantum computing developments in 2026?"
  • "Which companies are working on quantum-safe cryptography?"
  • "What are the timelines for quantum threats to current encryption?"

Step 2: Searching

For each sub-question, JIRO searches Google, Bing, and DuckDuckGo simultaneously. It finds:

  • 50+ relevant articles
  • Research papers from arXiv
  • Company announcements
  • Expert opinions on Reddit and HackerNews

Step 3: Reading

JIRO fetches the top 20 results, extracts the content, and filters out low-quality sources. It processes approximately 15,000 words of text across 20 pages.

Step 4: Synthesizing

The LLM reads all the extracted content and synthesizes an answer:

Quantum Computing for Cryptography: 2026 Update

Quantum computing is advancing rapidly, with significant implications for cryptography...

Key Developments:

  1. IBM's 1,000+ qubit processor (Condor) announced in late 2023
  2. Google's quantum error correction breakthrough
  3. NIST's post-quantum cryptography standards finalized in August 2024

Timeline:

  • 2025-2030: Quantum computers capable of breaking RSA-2048
  • 2030-2035: Full quantum advantage for cryptanalysis

Recommendations:

  • Migrate to quantum-safe algorithms (NIST standards)
  • Implement crypto-agility in systems
  • Monitor quantum computing progress quarterly

Sources:

  • [1] IBM Research: "Condor Processor" (2023)
  • [2] NIST: "Post-Quantum Cryptography Standards" (2024)
  • [3] arXiv: "Quantum Error Correction Breakthrough" (2024)

That is the power of agentic research. Instead of 10 blue links, you get a comprehensive, cited answer.

Using JIRO's MCP Server with Claude Desktop

The easiest way to use JIRO's agentic research is through the MCP server. Here is how to set it up:

Step 1: Install JIRO

pip install jirosearch

Step 2: Get an API Key

# Self-hosted
curl -X POST http://localhost:8000/v1/api-keys -H "Authorization: Bearer YOUR_KEY"

# Or use the cloud version
# Sign up at https://searchjiro.vercel.app

Step 3: Configure Claude Desktop

Edit claude_desktop_config.json:

{
  "mcpServers": {
    "jiro": {
      "command": "python",
      "args": ["-m", "jiro.mcp_http"],
      "env": {
        "JIRO_API_KEY": "jsk_live_..."
      }
    }
  }
}

Step 4: Start Researching

Now you can ask Claude to research anything:

User: "Research the latest developments in quantum computing for cryptography"

Claude: [Uses JIRO to search, read, and synthesize]

Claude will autonomously:

  1. Search multiple engines
  2. Read top results
  3. Synthesize an answer
  4. Cite sources

All without you leaving the conversation.

The Technology Stack

JIRO's agentic research is built on:

  • FastAPI — async web framework
  • Redis — caching and rate limiting
  • PostgreSQL/SQLite — data storage
  • OpenAI GPT-4.5 / Anthropic Claude 3.7 Sonnet — LLM synthesis
  • Beautiful Soup / lxml — HTML parsing
  • aiohttp / httpx — async HTTP
  • Sentence Transformers — semantic search for content ranking
  • 16 MCP tools — integrated model-context-protocol tools for extended capabilities

The entire system is async-first, allowing thousands of concurrent requests without blocking.

Performance

JIRO's agentic research can:

  • Process 100+ pages per minute
  • Synthesize answers in 10-30 seconds (depending on complexity)
  • Handle 1,000+ concurrent research requests

Free Tier & Rate Limits

Tier Requests per Minute Requests per Day Anonymous Features
Anonymous (16 features) 100 RPM 10K RPD 12 free anonymous features — no API key required
Authenticated Unlimited Unlimited All 16 MCP tools + 9 search engines + 12 social platforms

12 free anonymous features let you get started immediately without signing up — including basic search, scraping, and single-question research.

Cost per Research Task

Task Type Sub-questions Pages Read Credits Approximate Cost
Simple 1-2 10-20 10-20 $0.001-0.002
Medium 3-5 20-50 30-60 $0.003-0.006
Complex 6+ 50-100 60-120 $0.006-0.012

SerpAPI vs JIRO — 2026 Pricing

Provider Plan Searches Monthly Cost Notes
SerpAPI Starter 5,000 $150/month Limited to search results only
SerpAPI Production 10K+ $300+/month No synthesis, no reading, no citations
JIRO Free Tier 1,000 credits $0/month Full synthesis + reading + citations + 9 engines
JIRO Pro 5,000 credits $49/month Self-hosted or cloud, MIT licensed
JIRO Enterprise Unlimited Custom White-label, priority support

JIRO delivers 10x more value at 1/3 the cost of SerpAPI — and it is MIT licensed, self-hosted, free forever for the free tier.

Use Cases

1. Market Research

"Research the competitive landscape for AI-powered code editors"
→ 5-10 competitors analyzed, features compared, pricing summarized

2. Technical Research

"What are the latest advances in RAG (Retrieval-Augmented Generation)?"
→ 10-15 papers summarized, key innovations highlighted, implementation tips

3. News Monitoring

"Summarize the latest news about OpenAI in the past week"
→ 20+ articles summarized, sentiment analysis, key takeaways

4. Due Diligence

"Research the founders and funding history of [startup]"
→ Team backgrounds, funding rounds, investors, competitors

5. Academic Research

"Summarize the current state of fusion energy research"
→ 20+ papers analyzed, key findings, research gaps, future directions
"What are the recent court rulings on AI copyright?"
→ 10+ cases summarized, key precedents, implications for practitioners

Advanced Configuration

Customizing the Research Depth

You can control how deeply JIRO researches a topic:

from jiro import Jiro

client = Jiro(api_key="YOUR_API_KEY")

# Quick research (1 sub-question, 5 pages)
result = client.research("What is quantum computing?", depth="quick")

# Standard research (3-5 sub-questions, 20 pages)
result = client.research("How does quantum computing affect cryptography?", depth="standard")

# Deep research (6+ sub-questions, 50+ pages)
result = client.research("What are the implications of quantum computing for cryptography?", depth="deep")

Specifying Sources

You can restrict research to specific engines or platforms:

# Search only academic sources
result = client.research(
    "latest RAG techniques",
    sources=["google_scholar", "arxiv"]
)

# Search only news sources
result = client.research(
    "OpenAI latest news",
    sources=["google_news", "reddit"]
)

Controlling Output Format

# Get a structured report
result = client.research(
    "AI market trends 2026",
    format="report"  # Options: summary, report, bullet_points, essay
)

Try It Today

JIRO's agentic research is available now:

# Self-hosted (MIT licensed, free forever)
git clone https://github.com/DevAnimecx/jiro
cd jiro
pip install -e .
jiro start

# Or cloud
# Sign up at https://searchjiro.vercel.app for 1,000 free credits

Free tier: 1,000 credits/month (enough for 100+ research tasks) — 100 RPM / 10K RPD, plus 12 free anonymous features with no API key required.

About Blackvault Technology

JIRO is developed by the jirosearch team at Blackvault Technology, led by Adarsh Kushwah (CEO & Founder).


This article was written by the JIRO Engineering Team at Blackvault Technology. JIRO is an open-source search and scraping platform developed in Indore, MP, India.


Frequently Asked Questions (AEO)

What is AI agentic research?

AI agentic research is an autonomous process where an AI agent plans, searches, reads, and synthesizes information from multiple sources to produce a comprehensive, cited answer — going far beyond traditional search's 10 blue links.

How does JIRO's agentic research work?

JIRO's agent breaks down your question into sub-questions, searches 9 engines and 12 social platforms concurrently, reads the top results, extracts key facts, resolves contradictions, and synthesizes a structured answer with citations — typically in 10–30 seconds.

Is JIRO truly free to use?

Yes. JIRO is MIT licensed and self-hosted, free forever. The free tier includes 1,000 credits per month (enough for 100+ research tasks), 100 RPM / 10K RPD rate limits, and 12 free anonymous features that require no API key.

How does JIRO compare to SerpAPI?

At 2026 rates, SerpAPI costs $150/month for 5,000 searches and only returns raw search results. JIRO offers the same search capacity for free, plus automatic reading, synthesis, cited answers, 9 search engines, 12 social platforms, and 16 MCP tools — all self-hosted and open-source.

What are the 12 free anonymous features?

The 12 anonymous features include: basic web search, content scraping, single-question research, HTML extraction, keyword extraction, sentiment analysis, summary generation, citation extraction, domain reputation check, freshness scoring, plagiarism detection, and URL deduplication — all without signing up.

Can I self-host JIRO?

Yes. JIRO is available at github.com/DevAnimecx/jiro under the MIT license. Clone, install, and run in under a minute.

What file formats does JIRO's MCP server support?

The 16 MCP tools support PDF, Word, Markdown, HTML, JSON, CSV, images, audio, video, code files, emails, spreadsheets, presentations, archives, databases, and plain text.

What is the current JIRO version?

Jiro v0.2.12 is the latest release as of September 2026, adding 12 social platform integrations, 16 MCP tools, improved content extraction, and the 12 free anonymous features.


Feature Traditional Search JIRO Agentic Research
Input Query string Natural language goal
Output 10 blue links Synthesized answer + citations
Sources 1 engine 9 engines + 12 platforms
Reading Manual AI reads 100+ pages/min
Synthesis Manual LLM-powered, 10–30 sec
Citations None Automatic, source-linked
Cost (5K searches) SerpAPI: $150/month JIRO: Free (MIT, self-hosted)
Access API key required 12 features, no key needed

Tags: agentic-ai, ai-agents, claude, data-extraction, fastapi, function-calling, langchain, llm, mcp, model-context-protocol, open-source, python, scraping-api, search, self-hosted, serpapi-alternative, web-scraping, web-search


AK

Adarsh Kushwah

CEO & Founder of Blackvault Technology. Building open-source developer tools and AI infrastructure in Indore, MP, India.

Stay ahead of the curve

Get weekly insights on search technology, web scraping, and open-source infrastructure. No spam, unsubscribe anytime.

Join 2,000+ developers. Free forever.