Whatsapp Receipt OCR Scanner AI Agent with Vision

What This Agent Does

This intelligent automation agent transforms your WhatsApp into a powerful receipt management system. When you snap a photo of any receipt or invoice and send it via WhatsApp, the agent automatically downloads the image, uses advanced AI to extract all relevant data (merchant name, date, amount, items, tax), and logs everything into a Google Sheet—all within seconds. You'll receive an instant WhatsApp confirmation once processing is complete.

Key benefits:

• Save 10+ hours monthly on manual data entry and expense tracking
• Eliminate human error with AI-powered data extraction achieving 95%+ accuracy
• Real-time expense tracking with immediate Google Sheets updates accessible anywhere
• Seamless mobile workflow requiring just a photo and a message

Perfect for: Small business owners tracking expenses, freelancers managing receipts for tax purposes, teams handling petty cash, sales representatives on the road, or anyone tired of shoebox accounting.

Who Is It For

This agent is ideal for:

• Small business owners and entrepreneurs who need to track expenses on-the-go without dedicated accounting software
• Freelancers and contractors collecting receipts throughout the day for tax deductions and client reimbursements
• Sales and field teams submitting expense reports from multiple locations
• Finance teams streamlining receipt collection from employees across departments
• Anyone managing personal finances who wants automated expense categorization and tracking

No technical expertise required—if you can send a WhatsApp message, you can use this agent.

Required Integrations

Anthropic (Claude AI)

Why it's needed: Anthropic's Claude AI powers the intelligent receipt analysis, extracting merchant names, dates, amounts, line items, tax, and other structured data from receipt images with remarkable accuracy.

Setup steps:

1. Visit console.anthropic.com and create an account or sign in
2. Navigate to API Keys in the left sidebar
3. Click Create Key and give it a descriptive name like "TaskAGI Receipt Scanner"
4. Copy the API key immediately (it won't be shown again)
5. Store it securely—you'll need it for TaskAGI configuration

Configuration in TaskAGI:

1. Go to Integrations in your TaskAGI dashboard
2. Search for Anthropic and click Connect
3. Paste your API key in the authentication field
4. Click Test Connection to verify
5. Save the integration

Cost consideration: Claude offers generous free tier credits. Receipt analysis typically costs $0.01-0.03 per image, making this extremely cost-effective for most users.

Google Sheets

Why it's needed: Google Sheets serves as your centralized receipt database, providing easy access, powerful filtering, and automatic calculations for expense tracking and reporting.

Setup steps:

1. Create a new Google Sheet or use an existing one for receipt tracking
2. Set up column headers in Row 1: Date, Merchant, Amount, Tax, Category, Items, Receipt URL, Timestamp
3. Copy the complete sheet URL from your browser
4. In Google Cloud Console (console.cloud.google.com), create a new project or select existing
5. Enable the Google Sheets API for your project
6. Create credentials (Service Account) and download the JSON key file
7. Share your Google Sheet with the service account email (found in the JSON file) with Editor permissions

Configuration in TaskAGI:

1. Navigate to Integrations → Google Sheets
2. Click Connect New Account
3. Upload your service account JSON key file
4. Click Authorize to complete the connection
5. Test by selecting your sheet from the dropdown

Configuration Steps

Step 1: Configure WhatsApp Webhook Trigger

The WhatsApp Message Received node listens for incoming messages with receipt images.

1. Set up a Twilio account at twilio.com and enable WhatsApp messaging
2. In TaskAGI, copy the webhook URL from the trigger node
3. In Twilio Console, navigate to your WhatsApp sender → Webhook Configuration
4. Paste the TaskAGI webhook URL in the "When a message comes in" field
5. Set method to POST
6. Save configuration

What gets captured: Sender number, message body, media URLs (receipt images), timestamp

Step 2: Download Receipt Image

The Download Receipt Image node retrieves the photo from WhatsApp's servers.

Configuration:

• URL field: {{nodes.3950.data.MediaUrl0}} (references the image URL from the WhatsApp message)
• Method: GET
• Headers: Add Twilio authentication if required (Account SID and Auth Token)

Pro tip: Enable the "Follow Redirects" option as media URLs often redirect to CDN locations.

Step 3: Extract Receipt Data with AI

The Extract Receipt Data with AI node uses Claude to analyze the image.

Configuration:

• Model: claude-sonnet-4-5-20250929 (optimal balance of speed and accuracy)
• Image input: {{nodes.3951.data.body}} (the downloaded image data)
• Prompt:

Analyze this receipt/invoice image and extract the following information in JSON format:
{
  "merchant_name": "",
  "date": "YYYY-MM-DD",
  "total_amount": 0.00,
  "tax_amount": 0.00,
  "currency": "",
  "category": "",
  "line_items": [],
  "payment_method": ""
}

Be precise with numbers. If any field is unclear, use null.

Temperature: Set to 0.1 for consistent, deterministic extraction

Step 4: Parse AI Response

The Parse AI Response node converts Claude's text output into structured data.

Configuration:

• Input: {{nodes.3952.analysis}}
• Parse method: JSON
• Error handling: Enable "Continue on error" to catch malformed responses

This creates individual data fields accessible as {{nodes.3953.data.merchant_name}}, {{nodes.3953.data.total_amount}}, etc.

Step 5: Save to Google Sheets

The Save to Google Sheets node appends a new row with extracted data.

Configuration:

• Sheet URL: Your Google Sheet URL (e.g., https://docs.google.com/spreadsheets/d/ABC123...)
• Row data mapping:
  • Column A (Date): {{nodes.3953.data.date}}
  • Column B (Merchant): {{nodes.3953.data.merchant_name}}
  • Column C (Amount): {{nodes.3953.data.total_amount}}
  • Column D (Tax): {{nodes.3953.data.tax_amount}}
  • Column E (Category): {{nodes.3953.data.category}}
  • Column F (Items): {{nodes.3953.data.line_items}}
  • Column G (Receipt URL): {{nodes.3950.data.MediaUrl0}}
  • Column H (Timestamp): {{$now}}

Important: Ensure column order matches your sheet headers exactly.

Step 6: Send WhatsApp Confirmation

The Send WhatsApp Confirmation node notifies you of successful processing.

Configuration:

• URL: https://api.twilio.com/2010-04-01/Accounts/YOUR_ACCOUNT_SID/Messages.json
• Method: POST
• Authentication: Basic Auth with Twilio Account SID and Auth Token
• Body (form-encoded):

From=whatsapp:+YOUR_TWILIO_NUMBER
To={{nodes.3950.data.From}}
Body=✅ Receipt processed! {{nodes.3953.data.merchant_name}} - ${{nodes.3953.data.total_amount}} saved to your expense sheet.

Personalization tip: Include the spreadsheet row number in the confirmation for easy reference.

Testing Your Agent

Running Your First Test

1. Prepare a test receipt: Use a clear, well-lit photo of a simple receipt
2. Send via WhatsApp: Message your configured Twilio WhatsApp number with the receipt image
3. Monitor execution: Open the TaskAGI dashboard and watch the workflow execute in real-time
4. Check each node: Click on individual nodes to inspect data passing through

Verification Checklist

✅ Trigger node: Confirms webhook received with MediaUrl0 populated
✅ Download node: Shows successful HTTP 200 response with image data
✅ AI extraction node: Returns structured JSON with all expected fields
✅ Parse node: Creates individual data variables without errors
✅ Google Sheets node: Returns spreadsheet_id and row number
✅ Confirmation node: Shows successful message delivery status

Expected Results

• Processing time: 5-15 seconds from photo to confirmation
• Google Sheet: New row appears with extracted data
• WhatsApp confirmation: Arrives within 20 seconds
• Data accuracy: 95%+ for clear, standard receipts

Troubleshooting

"Image download failed" Error

Cause: Twilio media URL expired or authentication issue
Solution:

• Verify Twilio credentials in the download node
• Check that webhook is configured with correct authentication
• Ensure media URLs haven't expired (process within 24 hours)

"AI extraction returned incomplete data"

Cause: Poor image quality, unusual receipt format, or prompt mismatch
Solution:

• Enhance the prompt with specific examples for your receipt types
• Increase image quality requirements (good lighting, focus)
• Add validation logic to flag incomplete extractions
• Consider using Claude Opus model for complex receipts

"Google Sheets permission denied"

Cause: Service account lacks access to the sheet
Solution:

• Re-share the Google Sheet with the service account email
• Verify Editor permissions (not just Viewer)
• Check that the sheet URL is correct and accessible
• Ensure Google Sheets API is enabled in Cloud Console

"No WhatsApp confirmation received"

Cause: Twilio configuration or rate limiting
Solution:

• Verify Twilio credentials are current
• Check Twilio account balance and messaging limits
• Confirm the "From" number is WhatsApp-enabled
• Review Twilio error logs for delivery failures

"Workflow doesn't trigger"

Cause: Webhook misconfiguration
Solution:

• Verify webhook URL is correctly pasted in Twilio
• Check that HTTP method is POST
• Test webhook directly using Twilio's test feature
• Ensure TaskAGI workflow is published and active

Next Steps

After Successful Setup

1. Create a receipt photo guide for your team with lighting and angle best practices
2. Set up Google Sheets formulas for automatic totals, categorization, and monthly summaries
3. Configure notifications for high-value expenses requiring approval
4. Build a dashboard using Google Sheets' built-in charting for expense visualization

Optimization Suggestions

• Add category auto-detection: Enhance the AI prompt to categorize expenses (meals, travel, supplies)
• Implement approval workflows: Route expenses over a threshold to managers via email or Slack
• Create duplicate detection: Check for similar amounts/merchants within 24 hours to prevent double-entry
• Add currency conversion: Integrate exchange rate APIs for international receipts
• Schedule weekly summaries: Send digest emails with expense totals and trends

Advanced Usage Tips

• Multi-user support: Create separate Google Sheets tabs per user based on WhatsApp sender number
• Receipt archival: Upload images to Google Drive with organized folder structure
• Accounting integration: Connect to QuickBooks or Xero for automatic expense import
• Mileage tracking: Extend the agent to process odometer photos for vehicle expenses
• Tax preparation: Generate IRS-compliant expense reports with one-click exports

Congratulations! You've built an intelligent receipt management system that saves hours of manual work. Start sending receipts and watch your expense tracking transform from tedious to effortless.