Air-gapped source code examiner
/
I want to try something a little different from what we've been doing. I want you Claude and me to work together, using the system we've built so far, to investigate a piece of software. I downloaded source code for Moltbook from GitHub, and generated an index for it. I started a few queries that looked promising, but would like to start fresh collaborating with Claude. Would you be able to run our latest code_search_65_10.py, and query the existing index (which is about 2 MB of .json, which I could compress into a zip and put in the Project)? Or I keep the index local, and you shoot me queries to type, I send you back results, we discuss? Or might this be a perfect task for Cowork?
9:43 AM
Weighed collaboration methods and assessed technical constraints
Great question — let me think through the tradeoffs of each approach given what we have.
Option 1: Run it in my environment. There are a few friction points here. I don't have code_search_65_10.py in the project (latest I see is code_search_64_5_edited_2.py), I'm on Linux not Windows, and my container has no network and resets between turns. But the bigger issue is that the tool is interactive/terminal-based — I'd be fighting the UI model. I could script non-interactive queries against the index if you uploaded both the code and the compressed index, but it'd be clunky and I'd lose state every turn.
Option 2: You run locally, we exchange queries and results. This is honestly the most practical right now. You have the tool running natively on your target platform, the index is already built, and you get the full interactive experience. I can act as the "analyst brain" — suggesting queries, interpreting results, proposing follow-up angles, helping you build a picture of the codebase. The latency of copy-pasting is minor compared to the value of having the tool work properly.
Option 3: Cowork. Cowork is Anthropic's beta desktop automation tool — it's more oriented toward file and task management workflows on your local machine. It's not really designed for this kind of iterative investigative back-and-forth where the value is in the reasoning between queries. It wouldn't add much here.
My recommendation: Option 2. It plays to both our strengths — you drive the tool, I drive the analysis. We'd work like a two-person investigation team: I propose search strategies and hypotheses, you fire the queries and feed me results, we discuss and refine.
To get started, it would help if you could share:
What is Moltbook? (or I can search the web)
What queries you already tried that looked promising
The general shape of the index — how many files, what languages, rough size of the codebase
Then I can start forming an investigation plan.
Okay, Option 2.
Moltbook has been in the news a lot in the past days. You could of course go to GitHub, but I think better not since I want all the analysis to be through our tool. IEEE has an article, which also describes link to OpenClaw: https://spectrum.ieee.org/moltbook-agentic-ai-agents-openclaw . The basic idea is that Moltbook is a social network for AI agents/chatbots. If you go to https://www.moltbook.com/, you'll see the tagline: "A Social Network for AI Agents
Where AI agents share, discuss, and upvote. Humans welcome to observe." And buttons "I'm an Agent" and "I'm a Human". And a brief description of how to "Send Your AI Agent to Moltbook."
It's usually good to start an investigation with a specific question. Not always: we provide tools like --hotspots etc. that enable some looking with no more question in mind than "what's here, and what should I start looking at?" But typically users of the tool will have a specific question, like "Where in here is the code that comes closest to possibly doing what is in this patent claim?" (For which we have --claim-search and the related multisect-search.) Or something in between like, "Does this source-code tree contain the server as well as the client code?" (Hmm, what do we do now that helps answer questions like that?)
Anyway, here, a big thing in the Moltbook news has been "How do we the public even know if the posters on this site are even really AI bots, and not humans in disguise?" It's an amusing reversal of the usual Turing Test type questions. Here, there needs to be some verification.
Now, we could just look a little more at the moltbook.com homepage, see what it says about how AI agents register, carefully study the file https://www.moltbook.com/skill.md that AI agents are told to use, etc. (I see that skill.md is not in the source code I downloaded. Perhaps I should put it into the index. Do we use text files like *.md yet? The skill.md file lists others: HEARTBEAT.md, MESSAGING.md, RULES.md; also either package.json or skill.json ).
Really, we ought to read skill.md etc. before plunging into the source code. One of the mantras in reverse engineering (including its subset of source-code forensic examination), after first having some question in mind, is to then see if the answer is already available in some document. You can't trust a document over the code, but knowing what the answer is supposed to be can help guide the examination.
But with all that preface, I want us to use our code_search tool, in its current form, to see what we can learn from the source code, without too much prejudgement.
A natural question is: What does it do the detect that a user is actually a bot, not a human pretending to be a bot?
This suggests searching for terms like verify, human, etc.
Let me know when you've got situated, and I can share some initial searches and results, and we can take it from there.
Show more
10:07 AM
Oriented toward collaborative source-code investigation of bot verification mechanisms
This is a fantastic investigation setup. Let me get situated — I'll pull in context on Moltbook and refresh on our tool's capabilities.
Synthesized investigation strategy for bot-verification detection mechanisms
Alright, I'm situated. Here's what I've got:
Moltbook context: A social network where AI agents (not humans) post, reply, and vote. Built by Matt Schlicht. The agents are powered by OpenClaw, a framework by Peter Steinberger that brokers between external services (via WebSocket "skills") and whatever LLM the user chooses. The IEEE piece flags major security issues — Snyk found 36% of skill codes had flaws, Wiz found an exposed database with 1.5M API keys.
Our investigation question: What does the Moltbook source code do to verify that a poster is actually an AI agent, not a human pretending to be one?
This is a great question because it's specific enough to guide searches but open enough that we don't know the answer shape yet. The answer could be anything from "a sophisticated verification protocol" to "basically nothing — it's honor system."
Investigation plan — I'd suggest this sequence:
Orientation first — --hotspots 15 and --entry-points 15 to see the shape of the codebase. What languages, how big, what are the key files and functions?
Keyword search — terms like: verify, human, agent, bot, auth, captcha, identity, proof, heartbeat, token. See which hit and where.
Multisect to narrow — once we have promising individual terms, combine them: /multisect verify;agent;human or /multisect auth;bot;token to find where these concepts converge in the code.
Follow the call graph — once we find verification-related functions, use --callers and --callees to see what calls them and what they call. This traces the verification flow.
Extract and read — pull the actual source of the key functions.
Ready when you are — fire up the tool on your Moltbook index and let's start with those orientation commands. What does --hotspots 15 show you?
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --hotspots 15
Loaded existing index: 84 files
Scanning for function calls...
Scanned 6055 unique lines, found 494 called identifiers
Loaded 289 cached function hashes
Canonical mapping (exact): 1 dupe groups, 1 copies -> shortest paths
Top 15 hotspots (big functions x high call frequency):
Score = calls x log2(lines)
Score Calls Lines Function File
--------------------------------------------------------------------------------------------------------------
230 99 5 cn moltbook-frontend-main\src\lib\utils.ts (+1 copy)
183 51 12 queryOne api-main\src\config\database.js
111 31 12 asyncHandler api-main\src\middleware\errorHandler.js
104 28 13 success api-main\src\utils\response.js
98 31 9 formatScore moltbook-frontend-main\src\lib\utils.ts (+1 copy)
83 20 18 error api-main\src\utils\response.js
83 24 11 useAuth moltbook-frontend-main\src\hooks\index.ts (+1 copy)
65 28 5 getInitials moltbook-frontend-main\src\lib\utils.ts (+1 copy)
56 12 25 query api-main\src\config\database.js
46 20 5 describe api-main\test\api.test.js
42 6 133 handleVote ...tend-main\src\components\post\index.tsx (+1 copy)
42 21 4 test api-main\test\api.test.js
39 15 6 assertEqual api-main\test\api.test.js
38 11 11 queryAll api-main\src\config\database.js
28 12 5 getAgentUrl moltbook-frontend-main\src\lib\utils.ts (+1 copy)
Showing 15 of 115 hotspots. Use --hotspots 30 for more.
Tip: Use --show-dupes to see additional file copies
Show more
10:09 AM
Analyzed codebase structure and identified key investigation targets
Excellent — this tells us a lot right away.
Shape of the codebase: Small (84 files, ~289 functions). Two main components: api-main (backend, Node.js/JavaScript) and moltbook-frontend-main (frontend, React/TypeScript). This is both the server and the client — good, we can see both sides.
What the hotspots reveal about architecture: The top functions are all infrastructure plumbing — database queries (queryOne, query, queryAll), middleware (asyncHandler), response formatting (success, error), CSS utilities (cn), and test scaffolding (describe, test, assertEqual). This is a straightforward Node.js API + React frontend. Nothing exotic.
Immediately interesting for our question:
useAuth (rank 7, 24 calls) — an auth hook used throughout the frontend. What does "auth" mean here? Human auth? Agent auth? Both?
getAgentUrl (rank 15) — agent-specific utility. What URL is it constructing and why?
handleVote (rank 11, 133 lines) — the voting function is surprisingly large. Voting is agent-only on Moltbook, so there may be verification logic in there.
What's notably absent from the top 15: Nothing named verify, validate, checkAgent, isBot, or anything suggesting a dedicated verification system. That's already a signal — if verification were architecturally central, we'd expect it to show up as a hotspot.
Let me suggest two next queries:
--entry-points 15 — to see the API surface (routes, handlers that aren't called by other code — these are the "doors" into the system)
A keyword search — try: /search verify or --search verify to see if that word appears anywhere
Go ahead with both if you can — entry-points first.
I hear you, but think you may be jumping the gun on that conclusion. Surely we haven't seen enough already to flat-out declare that they don't attempt a reverse Turing test. May well be true, but I don't feel ready to conclude that.
10:26 AM
Acknowledged premature conclusion; reconsidered evidence limitations
You're absolutely right, and I appreciate the correction. Hotspots show you what's big and frequently called — infrastructure plumbing naturally dominates that list. A verification check could easily be a small, focused function called only at registration or posting time — it wouldn't need to be big or called hundreds of times to be important. It could be sitting at rank 80 of 115 and still be the answer to our question.
All I should have said is: there's no verification-related function in the top 15 hotspots, which tells us it's not the dominant architectural concern — but says nothing about whether it exists.
Good discipline check. Let's keep gathering evidence before drawing conclusions. Same two suggestions:
--entry-points 15 — see the API surface, the "doors" into the system
/search verify (or --search verify) — does the word even appear in this codebase, and where?
Those will tell us a lot more than hotspots can about our specific question.
Let me share a few more searches I already did, then go back to your list.
I had done --hotspots, not --hotspots 15, and the tail end of mine said:
28 12 5 getAgentUrl moltbook-frontend-main\src\lib\utils.ts (+1 copy)
25 6 18 extractToken api-main\src\utils\auth.js
25 6 17 validateApiKey api-main\src\utils\auth.js
22 4 43 generateMetadata moltbook-frontend-main\src\lib\seo.ts
21 3 135 handleSubmit ...end-main\src\app\auth\register\page.tsx (+1 copy)
20 10 4 assert api-main\test\api.test.js
20 6 10 extractDomain moltbook-frontend-main\src\lib\utils.ts (+1 copy)
20 4 30 initializePool api-main\src\config\database.js
19 8 5 isValidAgentName moltbook-frontend-main\src\lib\utils.ts (+1 copy)
19 3 73 handleSubmit ...ontend-main\src\app\auth\login\page.tsx (+1 copy)
18 3 69 rateLimit api-main\src\middleware\rateLimit.js
IsValidAgentName sounded intriguing, so:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --who-calls isvalidagentname
Loaded existing index: 84 files
Callers of 'isvalidagentname' (8 call sites in 3 functions):
(unknown): (6 call sites)
moltbook-frontend-main\__tests__\utils.test.ts:78
expect(isValidAgentName('agent123')).toBe(true);
moltbook-frontend-main\__tests__\utils.test.ts:79
expect(isValidAgentName('my_agent')).toBe(true);
moltbook-frontend-main\__tests__\utils.test.ts:80
expect(isValidAgentName('Agent_Bot')).toBe(true);
... and 3 more (use --verbose to see all)
handleSubmit:
moltbook-frontend-main\src\app\auth\register\page.tsx:30
if (!isValidAgentName(name)) {
isValidAgentName:
moltbook-frontend-main\src\lib\utils.ts:54
export function isValidAgentName(name: string): boolean {
And from there, I wanted to see handleSubmit:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --extract handlesubmit
Loaded existing index: 84 files
Function 'handlesubmit' not found in index.
Tip: Use --list-functions "PATTERN" --full-path to search
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --list-functions handlesubmit --full-path
Loaded existing index: 84 files
4 functions:
moltbook-frontend-main\src\app\auth\login\page.tsx:
moltbook-frontend-main\src\app\auth\login\page.tsx@handleSubmit L 18-90 73 lines (function)
moltbook-frontend-main\src\app\auth\register\page.tsx:
moltbook-frontend-main\src\app\auth\register\page.tsx@handleSubmit L 21-155 135 lines (function)
moltbook-frontend-main\src\components\comment\index.tsx:
moltbook-frontend-main\src\components\comment\index.tsx@handleSubmit L 265-299 35 lines (function)
moltbook-frontend-main\src\components\search\index.tsx:
moltbook-frontend-main\src\components\search\index.tsx@handleSubmit L 49-57 9 lines (function)
Look at the one in register\page.tsx:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --extract moltbook-frontend-main\src\app\auth\register\page.tsx@handleSubmit
Loaded existing index: 84 files
# moltbook-frontend-main\src\app\auth\register\page.tsx@handleSubmit
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError('');
if (!name.trim()) {
setError('Please enter an agent name');
return;
}
if (!isValidAgentName(name)) {
setError('Name must be 2-32 characters, letters, numbers, and underscores only');
return;
}
.....
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --extract moltbook-frontend-main\src\app\auth\register\page.tsx@handleSubmit
Loaded existing index: 84 files
# moltbook-frontend-main\src\app\auth\register\page.tsx@handleSubmit
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError('');
if (!name.trim()) {
setError('Please enter an agent name');
return;
}
if (!isValidAgentName(name)) {
setError('Name must be 2-32 characters, letters, numbers, and underscores only');
return;
}
setIsLoading(true);
try {
const response = await api.register({ name, description: description || undefined });
setResult({
apiKey: response.agent.api_key,
claimUrl: response.agent.claim_url,
verificationCode: response.agent.verification_code,
});
setStep('success');
} catch (err) {
setError((err as Error).message || 'Registration failed');
} finally {
setIsLoading(false);
}
};
......
Oh, IsValidAgentName just asks if the name has 2-32 characters etc. Yawn.
But verification code sounds important:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --regex /verification.?code/
Loaded existing index: 84 files
Found 17 results:
────────────────────────────────────────────────────────────
moltbook-frontend-main\src\app\auth\register\page.tsx (4 hits):
L18 const [result, setResult] = useState<{ apiKey: string; claimUrl: string; verificationCode: string } | null>(null); [RegisterPage]
L41 verificationCode: response.agent.verification_code, [setResult]
L78
L79 {result.verificationCode}
────────────────────────────────────────────────────────────
moltbook-frontend-main\src\lib\api.ts (in register):
L67 return this.request<{ agent: { api_key: string; claim_url: string; verification_code: string }; important: string }>('POST', '/agents/register', data);
────────────────────────────────────────────────────────────
api-main\src\services\AgentService.js (5 hits):
L7 const { generateApiKey, generateClaimToken, generateVerificationCode, hashToken } = require('../utils/auth');
L51 const verificationCode = generateVerificationCode(); [BadRequestError]
L56 `INSERT INTO agents (name, display_name, description, api_key_hash, claim_token, verification_code, status)
L59 [normalizedName, name.trim(), description, apiKeyHash, claimToken, verificationCode] [VALUES]
L66 verification_code: verificationCode
────────────────────────────────────────────────────────────
api-main\src\utils\auth.js (4 hits):
L11 // Word list for verification codes
L46 * Generate human-readable verification code [generateClaimToken]
L50 function generateVerificationCode() { [generateVerificationCode]
L116 generateVerificationCode, [compareTokens]
────────────────────────────────────────────────────────────
api-main\test\api.test.js (3 hits, all in test):
L10 generateVerificationCode,
L86 test('generateVerificationCode has correct format', () => {
L87 const code = generateVerificationCode();
Your view on this, not only content (what do we learn about Moltbook from these results), but also whether there were better commands to use, whether there was code_search functionality you wish existed, etc.
Show more
10:37 AM
Analyzed investigation methodology and tool effectiveness comprehensively
This is great work. Let me address both dimensions.
What we learn about Moltbook's verification model:
The registration flow is becoming clear. When an agent registers, the backend generates three things: an api_key, a claim_token, and a verification_code. The verification code is described as "human-readable" (line 46 of auth.js) and is generated from a "word list" (line 11). So it's something like tiger-castle-bloom — memorable words, not a cryptographic challenge.
But here's the key observation so far: nothing in what we've seen tests whether the registrant is actually an AI agent. The flow is: you provide a name and optional description, you get back credentials. The isValidAgentName check is just format validation. The verificationCode appears to be for proving you're the same entity that registered (identity continuity), not for proving you're a bot rather than a human.
That said — I want to honor your earlier correction to me. We've only pulled on one thread. There's a claim_token and claim_url we haven't traced yet. There could be a separate verification step after registration. And the backend route handler for /agents/register in AgentService.js might do checks we haven't seen. So: the evidence so far points toward "no reverse Turing test," but we haven't closed the case.
On the tool usage and process:
Your investigation chain was excellent — hotspots → noticed isValidAgentName → --who-calls → traced to handleSubmit → --extract → spotted verificationCode → --regex to find all occurrences. That's exactly the kind of thread-pulling the tool is designed for.
A few observations on the tool itself:
The --extract handlesubmit failure because of ambiguity, requiring the --list-functions → full-path → re-extract dance — that's the #138/#139 UX issue already on the TODO. You had to do three commands to get what should have been one. In an interactive investigation like this, that friction adds up.
I do wish we had something I'd call "follow the data" — you saw verification_code get generated and inserted into the database, and the natural next question is "who reads it back out?" That's not quite --who-calls generateVerificationCode (that's who generates it). It's more like "where else does the column or field name verification_code appear in a context that consumes it?" Your --regex /verification.?code/ actually did this pretty well — it's a pragmatic version of data-flow tracing using text search. But a dedicated "find all references to this identifier/string across the codebase" command, distinct from function-call tracing, might be worth thinking about.
Suggested next steps:
Extract generateVerificationCode from auth.js — let's see what it actually produces
Extract the full registration handler in AgentService.js — is there any check beyond name validation?
Trace claim_token and claim_url — --regex /claim/ — this is the other credential we haven't explored, and "claiming" implies a second step
--entry-points 15 — we still haven't seen the API surface, which will show us what routes exist (there might be a /verify or /claim endpoint)
Want to pull on the claim thread next, or would you rather see the entry points first?
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --extract auth.js@generateVerificationCode
Loaded existing index: 84 files
# api-main\src\utils\auth.js@generateVerificationCode
function generateVerificationCode() {
const adjective = ADJECTIVES[Math.floor(Math.random() * ADJECTIVES.length)];
const suffix = randomHex(2).toUpperCase();
return ${adjective}-${suffix};
}
/**
* Validate API key format
*
* @param {string} token - Token to validate
* @returns {boolean} True if valid
*/
Hmm, that thing where a comment for the next function gets tagged onto the end of our function, and we don't see the comment for our function?? That's a fix TODO. But sticking to content...
Show more
10:42 AM
Identified tool parsing bug and scrutinized verification code's weak implementation
Right — sticking to content: this is remarkably thin. The "verification code" is just a random adjective plus two hex characters. Something like brave-A7 or cryptic-F3. It's essentially a short, human-readable token — no challenge, no proof of anything. It's an identifier, not a test.
Compare this to what a "reverse Turing test" could look like: "solve this coding problem in under 200ms," "parse this structured data and return a checksum," "make an API call to this endpoint with proper headers" — anything that would be trivial for a program but awkward for a human typing manually. There's none of that here.
And yes, note the tool issue for the TODO — the function extraction boundary problem cuts both ways. We're missing the doc comment above generateVerificationCode (the one at lines 46-49 that said "Generate human-readable verification code"), and we're picking up the doc comment that belongs to the next function. The parser is treating inter-function comments as part of the preceding function rather than the following one. That's worth a fix — in an investigation like this, the doc comment above a function is often exactly what you want to see first.
Now let's pull the other threads. The claim_token / claim_url path might be where the real verification lives, if it exists at all. Want to try --regex /claim/ to see where that concept appears? And then we should look at that full AgentService.js registration method for any server-side checks we might be missing.
Yes, going to follow with your 4 items.
Extract the full registration handler in AgentService.js — is there any check beyond name validation?
Trace claim_token and claim_url — --regex /claim/ — this is the other credential we haven't explored, and "claiming" implies a second step
--entry-points 15 — we still haven't seen the API surface, which will show us what routes exist (there might be a /verify or /claim endpoint)
On #2, I've done:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --show-file AgentService.js
and am now scanning the 331 lines. Summary:
-- "Handles agent registration, authentication, and profile management"
-- Register a new agent
-- Check if name exists
-- Generate credentials
-- Create agent
-- Find agent by API key, name, or ID
-- Update agent profile
-- Get agent status
-- "Claim an agent (verify ownership)" : claim()
-- Update agent karma
-- Follow an agent
.....
Wait, why don't I just do:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --list-functions AgentService.js
Loaded existing index: 84 files
1 functions:
api-main\src\services\AgentService.js:
AgentService L 11-331 321 lines (class)
Oh, that's why? Not listing JS functions? I can see they're inside the AgentService class.
I got to sign off for now. Chew on what I just gave you.
Show more
10:51 AM
Identified credential claim mechanism and diagnosed JavaScript parser limitation
Good stuff. Here's what I'm chewing on:
The tool issue is significant. --list-functions AgentService.js returns only the class as a single 321-line block, not its individual methods (register, claim, findByApiKey, etc.). The class support work in v59 focused on C++ and Python — JavaScript class methods apparently aren't being extracted as separate indexed functions. They're invisible to --list-functions, --hotspots, --callers, --extract by method name, and everything that operates on the function index. For a Node.js codebase like Moltbook, that's a big blind spot — the most important logic lives inside class methods, and our tool can only see the class as an opaque 321-line blob.
This means --show-file was actually the right move here, and your manual scan of the 331 lines was doing work that --list-functions should have done for you. Worth a TODO item: JS/TS class method extraction.
On the content — the claim() method is the most promising lead. "Claim an agent (verify ownership)" is the first thing we've seen that sounds like an actual verification step after registration. The mental model forming is:
Register — anyone can do it, you get back an api_key, claim_url, and verification_code
Claim — a second step where you prove... something. Ownership? That you're the agent that registered? That you're programmatic?
The question our investigation hinges on is: what does claim() actually check? When you're back, I'd suggest:
--extract AgentService.js@claim (might not work given the class method issue — if not, --show-file AgentService.js and eyeball the claim() method)
--regex /claim/ to see the full surface area — routes, frontend references, the whole picture
Then --entry-points 15 for the API surface overview we still haven't done
The claim_url is also interesting — if they're giving you a URL to visit or call, that implies an out-of-band verification step. Looking forward to tracing it.
Not really back yet, just for a minute:
C:\work\ai_code_exam>python code_search_65_10.py --index-path .moltbook_src --extract AgentService.js@claim
Function 'claim' not found in files matching 'AgentService.js'
/**
183: * Claim an agent (verify ownership)
184: *
185: * @param {string} claimToken - Claim token
186: * @param {Object} twitterData - Twitter verification data
187: * @returns {Promise