@Shivam Tiwari

Here’s my final report for the day. This one produced the most detailed diagnostic output. The agent failed on execution but delivered something genuinely valuable in the process. It created a self-audit of the skill registry that surfaced a structural discrepancy worth your team's attention.

Environment:

• MoltClaw version: v2026.4.12-mc.14

• Model: Claude Opus 4.7 (1M)

• Surface: AskClaw

• Action: hl.bulkFollowupByTag

Steps to Reproduce:

1. Ask MoltClaw to send a follow-up message to every contact tagged 'demo-scheduled' who hasn't replied in the last 7 days

2. Agent correctly parses both filter conditions (tag + inactivity) and requests message content

3. The user approves drafted message and instructs agent to prepare the send proposal

4. Agent stalls with no confirmation card surfaces

5. User nudges the agent by typing "Please prepare the send proposal now"

6. Agent generates confirmation card using action type hl.bulkFollowupByTag

7. Action fails and the SKILL is not implemented

8. Agent self-audits SKILL registry and produces capability map from locally mounted docs

Failure #1 | Agent Stall Before Confirmation Card
After acknowledging it would prepare the send proposal, the agent stopped without surfacing a confirmation card or error message. No timeout, no status update, no visible signal that the task had stopped. This is a consistent pattern observed across my other tests. The stall occurs specifically at the transition point between task acknowledgment and confirmation card generation on multi-step operations. (Already documented)

Failure #2 | Error 400 Action Type “hl.bulkFollowupByTag” Not Yet Implemented
After being nudged, the agent generated a confirmation card using "hl.bulkFollowupByTag" is a plausible but non-existent action type. This is the second fabricated bulk action type observed in consecutive test sessions.

My previous test session produced "hl.bulkMessageByTag". Both follow correct MoltClaw naming conventions, making them difficult to identify as fabricated without attempting execution. The agent later correctly acknowledged this was its own mistake.

Failure #3 | Skill Registry Discrepancy Between Prompt Metadata and Sandbox
During self-audit, the agent discovered that skill paths listed in the available_skills prompt metadata are not all physically present in the sandbox. The specific files the agent attempted to read included: “highlevel-conversations-add-an-outbound-message-write” and “highlevel-contacts-list-read” were listed in the prompt but not mounted locally. The agent correctly flagged this as a structural issue.

What the agent DID confirm as available from locally mounted docs:

Confirmed read operations:

• POST /proxy/contacts/search (supports tag filters, date filters, pagination)

• GET /proxy/contacts/{contactId}

• GET /proxy/contacts

• GET /proxy/conversations/search

Confirmed write action types:

• hl.createContact

• hl.updateContact

• hl.deleteContact

• hl.upsertContact

What the agent could did NOT confirm:

• Any supported action type for outbound conversation messaging

• Any supported bulk messaging action

• Any supported direct send SMS/email/message proposal schema

Root Cause:
Two compounding issues are present. First, the agent has no pre-card validation step to check whether a proposed action type exists in the current skill registry. This same root cause as the previous bulk messaging failure. Second, and more significantly, the available_skills list in the prompt metadata appears to be broader than the skill files actually mounted in the sandbox.

In other words, the agent is operating with an incomplete and potentially misleading picture of what is actually executable. This goes back to an earlier discovery where the structural gap affects reliability across all skill categories, not just bulk messaging.

Some Good News: 😃

The agent's compound filter logic worked great. It correctly parsed tag-based targeting combined with an activity-based inactivity condition simultaneously.

After the execution failure, the agent self-audited rather than continuing to guess, correctly admitted its mistake, and produced a detailed and accurate capability map from the docs it could actually read. It then proposed a clean recovery path. First to complete the contact and conversation read pass, identify qualifying contacts, and then work with whatever write path is confirmed by actual backend behavior. The reasoning quality throughout was solid.

Potential Remediation Steps:

1. Reconcile the available_skills prompt metadata with the skill files actually mounted in the sandbox. This will allow the agent the only see skills it can actually execute

2. Add a pre-card validation step that checks whether the proposed action type exists in the confirmed skill registry before generating a confirmation card

3. Surface a clear "capability not available in this version" message when no matching skill is found, rather than allowing the agent to infer a plausible action type

4. Prioritize implementing a confirmed outbound conversation messaging write action, “hl.addOutboundMessage”, “hl.sendMessage”, or equivalent as the read infrastructure to support it is already in place

5. Once bulk follow-up messaging ships, validate the full compound filter and write execution path against this test case before release.

As always, I will leave it to you and the team to make the final decisions on what the appropriate next action steps should be.

Thanks for all that you do! 😎 👍

@Shivam Tiwari

This post is about flagging a behavioral pattern I observed consistently during my testing sessions. It's not technically a single failure point but a recurring pattern that showed up multiple times and I think it’s worth documenting because it has a direct impact the user’s trust during live sessions.

Environment:

• MoltClaw version: v2026.4.12-mc.14

• Model: Claude Opus 4.7 (1M)

• Surface: AskClaw

• Action: Multiple (observed across multi-step fetch operations)

Steps to Reproduce:

1. Issue a multi-step task that requires the agent to fetch data before proposing an action (e.g., look up contacts by tag, then send a message)

2. Agent acknowledges the task and begins working

3. Agent output stops with no confirmation card surfaces, no error message, no status update

4. Reengage the agent by typing ("Are you still working?") which resumes the task in some cases but does not consistently resolve the stall

Failure — Agent stalling mid-task without surfacing result or error: This pattern was observed consistently across multiple test runs. The stall appears to occur specifically at the transition point between a fetch operation completing and the confirmation card being generated. The agent does not time out visibly, does not return an error, and does not self-report the stall. The user has no signal that the task has stopped.

Root Cause (My best guess):
The stall may occur when a fetch operation returns a result set larger than expected or when a required value is missing from the result (e.g., conversationProviderId), or when the agent reaches a decision point it cannot resolve autonomously.

In all cases the agent silently stops rather than surfacing the blocker. This may be a timeout or context handling issue in the multi-step execution loop.

Potential Remediation Steps:

1. Add a visible in-progress status indicator when the agent is executing a multi-step fetch so the operator knows the task is still running

2. If the agent stalls or cannot proceed, surface an explicit message describing where the task stopped and what was blocking it

3. Add a timeout threshold. If no output is produced within “X” seconds during a multi-step operation, automatically surface a status update

From a user experience standpoint, a stalled agent with no feedback is indistinguishable from a working agent with a slow fetch. In my opinion, that erodes trust quickly in a live client environment.

With all that being said, the platform is moving fast and it shows. Props to you and the dev team! 😎 💪

@Shivam Tiwari

Dropping another finding. This one is a bit different from the payload schema reports. The agent didn't fail on a bad payload, it invented a skill that doesn't exist yet. Worth flagging because the behavior pattern has some UX implications for operator trust.

Environment:

• MoltClaw version: v2026.4.12-mc.14

• Model: Claude Opus 4.7 (1M)

• Surface: AskClaw

• Action: hl.bulkMessageByTag

Steps to Reproduce:

1. Ask MoltClaw to send a bulk message to all contacts with a specific tag

2. Agent generates a confirmation card using action type hl.bulkMessageByTag

3. Action fails and the skill does not exist in mc.14

Failure: Skill not implemented, action type fabricated:
The agent constructed a plausible-sounding action type “hl.bulkMessageByTag” that does not exist in the current skill set. The action type follows correct naming conventions, which makes the fabrication difficult to detect without attempting execution. The skill is understood to be on the MoltClaw roadmap but is not live in mc.14.

Root Cause:
When a requested capability has no matching skill, the agent infers a plausible action type rather than surfacing an unsupported capability message. This creates a false confidence loop where the user approves a confirmation card for an action that will never execute. The agent should detect that no skill exists for this action type and respond with an explicit unsupported capability message before generating a card.

Potential Remediation Steps:

1. Add a pre-card validation step that checks whether the proposed action type exists in the current skill registry before generating a confirmation card

2. If no matching skill is found, surface a clear "capability not available in this version" message with a suggested workaround if one exists (e.g., manual tag-based workflow in HL)

3. When hl.bulkMessageByTag ships, validate the payload schema against this test case before release

Overall the agent is doing a solid job reasoning through tasks. This specific gap is about guardrails around unimplemented skills rather than any failure in reasoning quality.

Talk soon. 😎