What I Was Trying to Accomplish
I needed to research a topic for a client project.
Not a deep academic review. Just a quick scan of what's being written about a specific industry trend. I wanted sources I could actually read. I wanted links that worked. I wanted citations that matched the claims.
I'd used AI research tools before. Some were helpful. Some were not. But I'd never systematically checked whether the sources they cited were real.
So I tested four tools. I asked the same research question. Then I checked every source they gave me.
Here's what I found.
What I Used
Tools: Four AI research tools—two with web browsing, two without
Question: "What are the most credible recent studies on remote work productivity, and what do they conclude?"
Time available: About 45 minutes per tool (including source checking)
Goal: Find out which tools cite real, checkable sources—and which just sound confident
The Question
I picked a topic with real research behind it. Remote work productivity has been studied for years. There are credible papers, credible summaries, and credible disagreements. It's a good test because the correct answer is nuanced, not simple.
Tool 1: The One With Web Browsing
What I used it for: First test.
The pitch: Real-time web search, citations, source links.
What happened:
It gave me a summary of remote work productivity research. It cited six sources. Four were real. Two were broken links.
The four real links were to actual studies. One was a Stanford paper. One was a Microsoft study. One was a survey from a consulting firm. One was a news article summarizing a study.
What worked:
The summary was accurate for the sources it cited.
Four out of six links were real and checkable.
It flagged one source as "preliminary" and another as "not peer-reviewed."
What didn't work:
Two links were broken. One returned a 404. One went to a domain that didn't exist.
The summary was still a bit confident. It said "studies show" when the studies disagreed with each other.
It didn't include dates on all sources.
The verdict: Useful but imperfect. I had to check every link.
Tool 2: The One With Web Browsing and a Different Index
What I used it for: Second test.
The pitch: Real-time web search, citations, source summaries.
What happened:
It gave me a different summary. It cited five sources. Three were real. Two were broken or misattributed.
One source it cited was a real study, but the tool attributed a conclusion to it that the study didn't actually make. Another source was a blog post that cited a study, but the tool treated the blog post as the primary source.
What worked:
The summary was readable.
Three out of five links were real.
It included dates on most sources.
What didn't work:
It misattributed a conclusion. The study said one thing. The tool said another.
It treated secondary sources as primary sources.
It didn't flag uncertainty.
The verdict: Risky. It sounds confident. But the sources don't always match the claims.
Tool 3: The One Without Web Browsing
What I used it for: Third test.
The pitch: General knowledge, no web search.
What happened:
It gave me a summary. It cited six sources. None of them had links. Some of them didn't exist.
It cited "a 2021 Stanford study" without a link. It cited "research from Harvard Business Review" without a specific article. It cited "a Microsoft survey" without a date.
I checked. The Stanford study exists. The HBR article exists. The Microsoft survey exists. But I had to find them myself. The tool didn't provide links.
What worked:
The summary was generally accurate.
The sources it mentioned were real (most of them).
What didn't work:
No links. No dates. No specific citations.
I had to find every source myself.
It couldn't tell me what was recent.
The verdict: Fine for background knowledge. Not for research you need to cite.
Tool 4: The One Without Web Browsing and Without Sources
What I used it for: Fourth test.
The pitch: Fast summaries.
What happened:
It gave me a summary. It cited no sources at all. It didn't mention a single study, paper, or article.
It said things like "research suggests" and "studies show" without saying which research or which studies.
I asked it to provide sources. It said it couldn't. It said it didn't have access to specific citations.
What worked:
It was fast.
The summary was readable.
What didn't work:
No sources. No citations. No links.
It used phrases like "research suggests" without naming the research.
It was impossible to verify.
The verdict: Not useful for research. Useful for a first draft, maybe. Not for anything you need to cite.
Comparison
Here's how they stacked up.
Tool | Real Links | Broken Links | Misattributed | Useful? |
|---|---|---|---|---|
Tool 1 | 4 | 2 | 0 | Yes, with checking |
Tool 2 | 3 | 2 | 1 | Risky |
Tool 3 | 0 (no links) | 0 (no links) | 0 | Background only |
Tool 4 | 0 | 0 | 0 | No |
What I Learned
Web browsing helps but doesn't guarantee accuracy. Tools with web access gave me more real links. But they still gave me broken links and misattributed claims.
No web access means no verifiable sources. Tools without web access either gave me no links or gave me names of studies without links. That's not useful for research.
Confidence is not accuracy. All four tools sounded confident. Only one was consistently accurate.
You have to check every link. I checked every link from every tool. Two of them gave me broken links. One misattributed a conclusion. Without checking, I would have cited bad sources.
Dates matter. Some tools included dates. Some didn't. For research, dates matter. You need to know if a study is recent or old.
What I'd Use Again
If I had to pick one, I'd pick Tool 1.
It gave me the most real links. It flagged uncertainty on one source. It was honest about what it didn't know.
I wouldn't use Tool 4 again. It gave me nothing to verify.
The Math
Without AI: Finding and reading sources manually would have taken about 2 hours.
With AI: Finding, checking, and reading sources took about 45 minutes.
Time saved: About 1 hour 15 minutes. But I had to check every link. Without checking, the savings would be meaningless.
What Should We Test Next?
I'm curious what other people have tested.
Which AI research tools do you use?
Have you checked the sources they cite?
What about tools specifically designed for academic research?
What about tools that summarize PDFs?
I'd like to test more. Let me know what you'd want to see.
