Web Scraping for AI: Terms of Service as a Critical Line of Defense

Web Scraping for AI: Terms of Service as a Critical Line of Defense

In the digital space, we often encounter misleading headlines, but it’s rare for such an error to lead us directly to one of the most critical yet frequently neglected areas of AI security. A news item, ostensibly about “EU AI Act Readiness,” actually contained the Terms of Service (ToS) for a financial data provider called TipRanks. This coincidence provides a perfect case study to examine the legal and security pitfalls of data collection for training AI models.

The extracted facts are clear: the TipRanks service and its content are the company’s intellectual property, created with substantial effort and proprietary methods. The service is licensed for personal, non-commercial use only, and most importantly, it explicitly prohibits automated data collection.

Do you have a question about AI security? You can reach us here:

Intellectual Property and Contractual Limitations

The TipRanks ToS states that the service and its content are protected by “copyright laws, and other intellectual property laws and treaties,” and is also protected as a “collective work or compilation under U.S. copyright and other laws and treaties.” This is not merely a legal formality. The document is a contract between the provider and the user.

From an AIQ standpoint, this point fundamentally challenges the common developer practice of assuming that “if it’s publicly available on the internet, it’s free to use.” The reality is that most online content is governed by a user agreement, which we implicitly accept by visiting the website. If a company trains an AI model on data collected in breach of these agreements, it faces not just ethical issues but significant legal and financial risks. In the context of GDPR, data processing must be “lawful”; data obtained by breaching a contract hardly meets this criterion, even if it’s not personal data.

The Explicit Prohibition of Automated Data Collection

The TipRanks terms leave no room for ambiguity. They strictly forbid the use of any “robot, spider, scraper, web crawler, or other automated means or any similar manual process.” Furthermore, they prohibit violating the “restrictions in any robot exclusion headers” (typically instructions found in the `robots.txt` file) or bypassing other measures employed to limit access.

In a corporate context, this means that running data-scraping scripts on this site constitutes a breach of contract. From an AIQ standpoint, this directly relates to several items in the OWASP LLM Top 10, particularly LLM05: Supply Chain Vulnerabilities. The data for an AI model is the foundation of its supply chain. If this data comes from an illegitimate source, the entire model is built on a compromised foundation. During an audit, such a flaw would be an immediate red flag, as the model’s outputs and the company’s reputation could be jeopardized.

The terms explicitly forbid any use that competes with the service or affects their ability to earn money. Creating a competing financial AI model trained on their data would be the most severe violation of these clauses.

Corporate Risks in the Shadow of the EU AI Act

Although the original article’s title was misleading, the lessons are highly relevant to the EU AI Act. The new regulation places a strong emphasis on the transparency of AI systems and the documentation of data used. For high-risk AI systems, developers will need to prove that their training data is relevant, representative, error-free, and complete.

From an AIQ standpoint, the requirement for “lawfully obtained” data will implicitly become a key part of compliance audits. A company unable to demonstrate where its datasets came from and on what legal basis they were collected will face serious compliance issues. Web scraping in violation of a ToS undermines a company’s ability to meet the AI Act’s transparency and documentation requirements. This is not just a theoretical risk; it’s a potential source of multi-million euro fines and market exclusion.

Audit Takeaways: What Should a Responsible Company Do?

This case highlights that AI security audits and internal compliance processes must extend to the legal aspects of data acquisition. A thorough audit should examine the following:

  • Data Source Documentation: Is the source (URL), collection date, and collection method accurately documented for every single dataset?
  • Terms of Service Analysis: Have the source websites’ `robots.txt` files and Terms of Service been legally and technically analyzed? Do they contain explicit prohibitions on automated collection or commercial use?
  • Risk Assessment: Has a risk assessment been conducted on the use of data from “grey area” sources? What would be the business impact if a data provider took legal action?
  • Data Provenance: Is it ensured throughout the entire MLOps pipeline that you can track which model version was trained on which dataset, allowing for the tracing and removal of problematic data?

The lesson is clear: AI development is not just about algorithms and computing power. Data, the fuel for the AI engine, must be procured with due diligence and respect for the legal framework. The TipRanks Terms of Service, through a random but powerful coincidence, have provided a stark reminder of this fact.

Attila Rácz-Akácosi

Independent AI Security Specialist

Two decades of analytical and systems-oriented experience. I have been working with artificial intelligence since 2017. In recent years, I have specialized in AI/LLM security and AI Red Teaming. Systems-level thinking instead of endless vulnerability checklists.