Web Scraping Terms and Conditions
Last Updated: 2 November 2025
These Web Scraping Terms govern your use of web scraping integrations and services available through TaskAGI. These terms supplement our Terms and Conditions and Acceptable Use Policy. By using any web scraping features, you agree to these terms.
1. Available Scraping Integrations
1.1. Third-Party Scraping Services: TaskAGI integrates 25+ web scraping services, including but not limited to:
1.2. Service Availability: Scraping services may be added, removed, or modified at any time. Some services may have usage quotas or rate limits.
1.3. Third-Party Provider Terms: When using third-party scraping services, you are also bound by the provider's terms of service. TaskAGI is NOT responsible for provider policy changes or service discontinuation.
- Lead Generation: Apollo.io (50,000+ business leads), Hunter.io (email finding), Clearbit (company enrichment)
- E-commerce Scraping: Amazon product data, eBay listings, Shopify stores
- Social Media Scraping: LinkedIn profiles, Instagram data, Twitter/X feeds, Facebook pages
- General Web Scraping: Apify actors, Bright Data, ScrapingBee, ParseHub
- Data Enrichment: ZoomInfo, Crunchbase, PeopleDataLabs
- Real Estate: Zillow, Realtor.com scrapers
- Job Boards: Indeed, LinkedIn Jobs, Glassdoor
1.2. Service Availability: Scraping services may be added, removed, or modified at any time. Some services may have usage quotas or rate limits.
1.3. Third-Party Provider Terms: When using third-party scraping services, you are also bound by the provider's terms of service. TaskAGI is NOT responsible for provider policy changes or service discontinuation.
2. Critical Legal Disclaimer - User Responsibility for Scraping Compliance
⚠️ READ THIS SECTION CAREFULLY - YOU ARE SOLELY RESPONSIBLE FOR THE LEGALITY OF YOUR SCRAPING ACTIVITIES
2.1. No Legal Advice:
2.2. User Responsibility for Compliance:
2.3. TaskAGI Does NOT Verify Legality:
2.4. Potential Legal Risks: Web scraping may violate:
2.1. No Legal Advice:
- TaskAGI does NOT provide legal advice about the legality of web scraping
- Web scraping legality varies by jurisdiction, website, and use case
- YOU are responsible for consulting legal counsel if uncertain about scraping legality
- TaskAGI makes NO representations about whether scraping specific websites is lawful
2.2. User Responsibility for Compliance:
- YOU ARE SOLELY RESPONSIBLE for ensuring your scraping activities comply with all applicable laws
- YOU must comply with target websites' Terms of Service and acceptable use policies
- YOU must respect robots.txt files and crawl-delay directives
- YOU must comply with rate limits to avoid overloading target servers
- YOU must obtain necessary permissions before scraping non-public data
2.3. TaskAGI Does NOT Verify Legality:
- TaskAGI does NOT verify whether your scraping activities are lawful
- TaskAGI does NOT monitor compliance with target websites' Terms of Service
- Availability of a scraping integration does NOT constitute endorsement or legal approval
- YOU assume ALL legal risk when using scraping services
2.4. Potential Legal Risks: Web scraping may violate:
- Terms of Service: Many websites prohibit scraping in their ToS (breach of contract)
- Computer Fraud and Abuse Act (USA): Unauthorized access to computer systems
- GDPR (EU/UK): Processing personal data without legal basis
- Copyright Law: Scraping copyrighted content without permission
- Database Rights (EU): Extracting substantial portions of databases
- Trespass to Chattels: Overburdening servers with scraping requests
- Anti-Circumvention Laws: Bypassing technical protection measures (paywalls, CAPTCHAs)
3. Prohibited Scraping Activities
You may NOT use TaskAGI scraping services for:
3.1. Illegal Scraping:
3.2. Personal Data Scraping Without Legal Basis:
3.3. Competitive Harm:
3.4. Content Theft:
3.5. Deceptive Practices:
3.1. Illegal Scraping:
- Scraping websites that explicitly prohibit scraping in their ToS
- Circumventing login screens, paywalls, or CAPTCHAs without authorization
- Scraping non-public data without permission (e.g., behind authentication)
- Exceeding rate limits or overloading target servers (DoS-like behavior)
- Ignoring robots.txt or crawl-delay directives
3.2. Personal Data Scraping Without Legal Basis:
- Scraping personal data without a valid GDPR legal basis (consent, legitimate interest, etc.)
- Collecting sensitive personal data (health, religion, political views, etc.) without explicit consent
- Scraping children's data without parental consent
- Building databases of personal data for spam, harassment, or identity theft
3.3. Competitive Harm:
- Scraping to create a competing product that mirrors the target website
- Real-time price scraping to undercut competitors (may violate antitrust laws)
- Scraping to misappropriate trade secrets or proprietary data
3.4. Content Theft:
- Scraping copyrighted content (articles, images, videos) for republication without permission
- Scraping product descriptions, reviews, or listings for use on competing platforms
- Aggregating content in ways that harm the original publisher's traffic/revenue
3.5. Deceptive Practices:
- Scraping to create fake reviews, ratings, or social proof
- Harvesting email addresses for spam or phishing
- Scraping social media profiles for impersonation or catfishing
4. Best Practices and Compliance Requirements
4.1. Check Target Website's Terms of Service:
4.2. Respect robots.txt:
4.3. Implement Rate Limiting:
4.4. GDPR Compliance for Personal Data:
4.5. Obtain Permission for Non-Public Data:
4.6. Identify Your Scraper:
- ALWAYS read the target website's ToS before scraping
- Look for clauses prohibiting "automated access," "scraping," "crawling," or "data mining"
- If ToS prohibits scraping, do NOT scrape the website (or obtain written permission)
4.2. Respect robots.txt:
- Check the target website's robots.txt file (e.g., https://example.com/robots.txt)
- Respect "Disallow" directives for scrapers
- Comply with "Crawl-delay" directives to avoid overloading servers
4.3. Implement Rate Limiting:
- Do NOT send excessive requests that could overload or disrupt the target server
- Use reasonable delays between requests (e.g., 1-2 seconds minimum)
- Reduce scraping frequency if you notice server slowdowns or errors
4.4. GDPR Compliance for Personal Data:
- Only scrape publicly available personal data (do NOT scrape behind login walls)
- Have a valid legal basis for processing (legitimate interest, consent, contract)
- Conduct a Legitimate Interest Assessment (LIA) if relying on legitimate interest
- Honor data subject rights (deletion, access, objection) - see GDPR Data Rights
- Implement appropriate security measures for scraped personal data
- Do NOT scrape data from EU/UK residents for marketing without explicit consent
4.5. Obtain Permission for Non-Public Data:
- If scraping requires login or bypassing access controls, obtain written permission from the website owner
- Do NOT share login credentials with scraping services without authorization
4.6. Identify Your Scraper:
- Use a descriptive User-Agent string that identifies your scraper and provides contact info
- Example: "TaskAGI-Scraper (+https://taskagi.net; contact@taskagi.net)"
- This allows website owners to contact you if there are issues
5. Notable Legal Precedents (For Informational Purposes Only)
This section is for educational purposes only and does NOT constitute legal advice. Consult a lawyer for your specific situation.
5.1. Cases Where Scraping Was Found Lawful (USA):
5.2. Cases Where Scraping Was Found Unlawful (USA):
5.3. GDPR Considerations (EU/UK):
5.4. Key Takeaway: The legality of web scraping depends on:
5.1. Cases Where Scraping Was Found Lawful (USA):
- hiQ Labs v. LinkedIn (9th Circuit, 2022): Scraping publicly accessible data may NOT violate CFAA if no login is required
- Field v. Google (D. Nev. 2006): Caching and indexing publicly available web content for search engines is generally lawful
5.2. Cases Where Scraping Was Found Unlawful (USA):
- Facebook v. Power Ventures (9th Cir. 2016): Scraping after receiving cease-and-desist may violate CFAA
- Craigslist v. 3Taps (N.D. Cal. 2013): Circumventing IP blocks to scrape may violate CFAA and ToS
- QVC v. Resultly (3rd Cir. 2020): Aggressive scraping that harms server performance may violate trespass to chattels
5.3. GDPR Considerations (EU/UK):
- Scraping personal data without legal basis violates GDPR (fines up to €20M or 4% of global revenue)
- Even publicly available personal data is protected under GDPR
- Data minimization: Only scrape data you actually need
5.4. Key Takeaway: The legality of web scraping depends on:
- Whether data is publicly accessible vs. behind login/paywall
- Whether website's ToS prohibits scraping
- Whether you respect robots.txt and rate limits
- Whether scraping harms the target website (server load, lost revenue)
- Jurisdiction (US, EU, UK have different laws)
- Nature of data (personal data has stricter rules)
6. Pricing and AI Credits
6.1. Credit-Based System: Web scraping services consume AI credits. Pricing models:
6.2. Non-Refundable Credits: AI credits are non-refundable once used, even if scraping fails or returns fewer results than expected. See our Refund Policy.
6.3. Rate Limits: We may impose rate limits or usage quotas to prevent abuse and comply with third-party provider limits.
6.4. Provider Cost Changes: If third-party scraping providers increase pricing, we reserve the right to adjust AI credit costs with 30 days' notice.
- Per-Run Pricing: Fixed cost per scraping operation
- Per-Result Pricing: Variable cost based on number of results returned (see Pricing)
6.2. Non-Refundable Credits: AI credits are non-refundable once used, even if scraping fails or returns fewer results than expected. See our Refund Policy.
6.3. Rate Limits: We may impose rate limits or usage quotas to prevent abuse and comply with third-party provider limits.
6.4. Provider Cost Changes: If third-party scraping providers increase pricing, we reserve the right to adjust AI credit costs with 30 days' notice.
7. Data Quality and Accuracy
7.1. No Data Accuracy Guarantees:
7.2. Scraper Maintenance:
7.3. User Verification Obligation: YOU must verify scraped data before using it for critical decisions (sales outreach, hiring, compliance, etc.).
- Scraped data may be incomplete, outdated, or inaccurate
- Websites may change structure, breaking scrapers
- Anti-scraping measures (CAPTCHAs, IP blocks) may prevent data collection
- TaskAGI makes NO warranties about scraped data quality
7.2. Scraper Maintenance:
- Third-party scraping providers maintain their scrapers
- Scrapers may break when target websites update their HTML structure
- We are NOT responsible for broken scrapers operated by third parties
7.3. User Verification Obligation: YOU must verify scraped data before using it for critical decisions (sales outreach, hiring, compliance, etc.).
8. Liability and Indemnification
8.1. No Liability for User Scraping Activities: TaskAGI is NOT liable for:
8.2. User Indemnification: YOU agree to indemnify and hold TaskAGI harmless from:
8.3. Maximum Liability: To the extent permitted by law, TaskAGI's total liability for scraping-related claims is limited to the amount you paid for scraping credits in the 3 months prior to the claim.
- Legal claims arising from your scraping activities (ToS violations, CFAA violations, GDPR fines, etc.)
- Cease-and-desist letters or lawsuits from websites you scrape
- Damages caused by inaccurate or incomplete scraped data
- IP blocks, account bans, or other consequences from target websites
- GDPR fines or data protection authority investigations
8.2. User Indemnification: YOU agree to indemnify and hold TaskAGI harmless from:
- Claims by websites or third parties regarding your scraping activities
- Violations of website Terms of Service
- GDPR or data protection law violations
- Copyright or database rights infringement claims
- Any prohibited scraping activities listed in Section 3
8.3. Maximum Liability: To the extent permitted by law, TaskAGI's total liability for scraping-related claims is limited to the amount you paid for scraping credits in the 3 months prior to the claim.
9. Service Limitations and Changes
9.1. Scraper Removal: We may remove scraping integrations if:
9.2. No Migration Guarantees: If a scraper is removed, we are NOT obligated to migrate your workflows to alternative scrapers.
9.3. Account Suspension: We may suspend your account without refund if:
- The target website sends a cease-and-desist or DMCA notice
- The scraper provider discontinues the service
- The scraper violates our policies or legal requirements
- The target website implements effective anti-scraping measures
9.2. No Migration Guarantees: If a scraper is removed, we are NOT obligated to migrate your workflows to alternative scrapers.
9.3. Account Suspension: We may suspend your account without refund if:
- You engage in illegal scraping activities
- You violate this policy or our Acceptable Use Policy
- We receive legal complaints about your scraping activities
10. Reporting Abuse
10.1. For Website Owners: If you believe a TaskAGI user is unlawfully scraping your website:
10.2. DMCA Takedown Requests: If a user is scraping copyrighted content, submit a DMCA notice to: dmca@taskagi.net
- Email: abuse@taskagi.net
- Include: Evidence of scraping (logs, User-Agent strings), link to your ToS prohibiting scraping
- We will investigate and may suspend the user's account if warranted
10.2. DMCA Takedown Requests: If a user is scraping copyrighted content, submit a DMCA notice to: dmca@taskagi.net
11. Updates to This Policy
We may update these Web Scraping Terms at any time. Material changes will be notified via email or platform notice. Continued use after changes constitutes acceptance of the updated terms.
12. Contact
Scraping Policy Questions: support@taskagi.net
Report Scraping Abuse: abuse@taskagi.net
Legal Inquiries: legal@taskagi.net
DMCA Notices: dmca@taskagi.net
Related Documents:
Report Scraping Abuse: abuse@taskagi.net
Legal Inquiries: legal@taskagi.net
DMCA Notices: dmca@taskagi.net
Related Documents: