In an era of opaque search algorithms and AI-generated answers, the Result Assessment Tool (RAT) enables professionals to collect, analyze, and evaluate search engine results with scientific rigor, free of charge.
What Is the Result Assessment Tool (RAT)?
The Result Assessment Tool (RAT) is a free, open-source, Python-based software toolkit developed by researchers at the Hamburg University of Applied Sciences, Germany, with continued funding from the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) through 2028. Its core purpose is to enable systematic research and analysis of results from commercial search engines, social media platforms, and library information systems.
First introduced as the Relevance Assessment Tool in 2012, the platform has evolved significantly through years of academic and professional use. Today, RAT offers a comprehensive environment for study design, automated search result collection, human evaluation, and machine-learning classification.
“Define. Collect. Analyze. Evaluate.” — The RAT Design Philosophy
How RAT Works: The Four-Stage Workflow
RAT uses a structured four-stage workflow, with each stage supported by a dedicated software module. This modular approach allows researchers and business teams to select only the components they require.
Stage 1 — Define
Researchers define the study scope using the Study Designer module in the RAT Researcher View. This involves selecting search engines to monitor (such as Google, Bing, DuckDuckGo, library catalogues, social platforms, or custom systems), uploading a list of queries, and configuring result types, including standard search snippets, full-page archives, screenshots, or AI-generated overviews from Google and ChatGPT. The integrated Query Sampler, connected to the Google Ads API, can automatically generate comprehensive query sets for any topic or domain.
Stage 2 — Collect
RAT’s Backend automatically runs all queries against selected search engines using Selenium WebDriver, ensuring results are collected as a real user would see them. This includes dynamic elements, JavaScript-rendered content, sponsored listings, featured snippets, and AI Overviews. The tool captures and stores result URLs, rankings, snippet text, and full-page source code. Screenshots of all documents are encrypted for archival accuracy. Importantly, RAT collects results at a specific point in time, addressing the temporal inconsistency found in manual methods.
Stage 3 — Analyse (Automated)
The RAT Classifier Framework supports automated analysis of collected results using machine learning models. The built-in SEO Classifier calculates a probability score for each URL, indicating the likelihood of artificial optimization for search placement. Researchers and analysts can extend the framework by developing custom classifiers for domain-specific analyses, such as health information quality, political bias, commercial intent, genre classification, sentiment analysis, or misinformation detection. All classifiers use the complete collected data, including URL, domain, position, snippet, source code, and original query.
Stage 4 — Evaluate (Human Assessment)
For assessments that require human judgment, RAT offers an Evaluation View—a streamlined web interface where evaluators review archived screenshots and answer pre-defined questions. Question formats include Likert scales, open-ended responses, sliders, and multiple-choice items. Evaluators access the system via a unique URL, allowing anonymous participation globally. The system removes duplicate results across search engines before presenting them, ensuring consistent evaluation. The Analyser module provides real-time statistics on collection progress, evaluator responses, and classifier outputs, all exportable to XLSX format.at.
Advantages for Businesses
Although RAT was developed for academic research, its features align closely with the needs of businesses involved in competitive intelligence, SEO auditing, brand monitoring, and content strategy. The following are the main advantages organizations gain by using RAT.
Scalability Beyond Human Capacity
Manual search result collection, even for small studies, is time-consuming and limits sampling. RAT removes this bottleneck. Automated data collection enables businesses to scale from a few manually checked queries to thousands across multiple search engines. Competitive analysis that previously took weeks can now be completed overnight.
Temporal Precision and Reproducibility
Search results are constantly changing. Manual monitoring captures only a single snapshot, influenced by timing, location, and search history. RAT collects results at a precise, configurable time with geo-location support, making findings reproducible and defensible. Businesses can document SERP landscapes at specific dates, such as before and after algorithm updates, competitor campaigns, or their own optimization efforts.
Elimination of Researcher Bias
Manual collection and classification by analysts introduces subjective variability. RAT’s standardized collection and assessment interfaces ensure all evaluators apply consistent criteria to identical content. This consistency is essential for reliable benchmarking, content audits, and brand safety reviews.
Multi-Engine and Multi-Platform Coverage
Businesses can monitor Google, Bing, DuckDuckGo, Brave, Yahoo, social media platforms, e-commerce search systems, and library catalogues within a single study. This cross-platform intelligence provides brand and marketing teams with a unified view of their own and competitors’ content across the information landscape.
SEO Manipulation Detection
The built-in SEO Classifier assigns a machine-learning probability score to each URL, indicating the likelihood of artificial search engine optimization. This enables businesses to determine whether competitors achieve rankings through content quality or aggressive SEO tactics, and to audit their own pages for over-optimization risks that could lead to ranking penalties.
Cost — Free and Open Source
RAT is available free of charge via its web interface at rat-software.org or as an open-source package for self-hosting on GitHub. Organizations can modify and extend the software to meet their specific needs without licensing fees. This levels the playing field for small and mid-sized businesses that cannot afford premium enterprise SEO intelligence platforms.
FAIR-Compliant Data Practices
RAT is designed according to FAIR principles: Findability, Accessibility, Interoperability, and Reusability. All research data generated in RAT studies is accessible through the Open Science Framework (OSF). For businesses conducting regulated research or subject to compliance requirements, this audit trail provides a significant governance advantage.
Why Businesses Must Use RAT
The business case for adopting RAT extends beyond convenience. In today’s information environment, characterized by opaque search algorithms, AI-generated answers, and increasing content competition, RAT addresses structural challenges that spreadsheets or manual monitoring cannot resolve.
Search Engines Are Black Boxes
Google, Bing, and other search platforms treat their ranking algorithms as proprietary. They do not provide public transparency about result order or changes. Businesses relying on intuition or anecdotal SERP observations make strategic decisions without sufficient data. RAT transforms the opaque outputs of commercial search engines into structured, analyzable datasets.
API Access Is Restricted and Unreliable
Microsoft Bing has discontinued public search API access. Google offers only limited API data, which does not reflect the actual user experience and omits featured snippets, AI Overviews, universal search integrations, and personalization effects. RAT’s Selenium-based scraping captures what real users see, not just what APIs provide.
AI-Generated Answers Are Transforming the SERP
Google AI Overviews and AI-powered chatbot answers now appear prominently in search results, often displacing traditional organic links. Businesses need to understand where their brand, products, and content appear or do not appear in these AI-generated summaries. RAT is among the few tools capable of capturing and archiving AI Overview content for systematic analysis.
Source Diversity and Algorithmic Fairness Matter
Regulators and researchers are increasingly examining whether search engines promote diverse sources or create echo chambers and unfair advantages. Businesses in regulated sectors such as health information, financial services, and news media face growing pressure to demonstrate that their search presence is legitimate and that their monitoring practices meet audit standards. RAT’s structured methodology provides this necessary evidence base.
Competitive Intelligence Requires Data, Not Impressions
Understanding competitor SERP performance requires systematic measurement across consistent time points, query sets, and geographic locations. Checking a few search results anecdotally is insufficient for strategic decisions. RAT provides the infrastructure for enterprise-grade competitive intelligence at no cost.
In 2026, professional search intelligence is essential. RAT is the only free, open-source, peer-reviewed tool that enables this capability.
How to Set Up and Use RAT
RAT is accessible in two ways: through the hosted web application at rat-software.org (no installation required) or as a self-hosted open-source deployment for organizations needing data control, customization, or integration with internal systems. The following guide covers both options.
Option A — Use the RAT Web Interface (Recommended for New Users)
No installation is needed. Follow these steps:
Visit rat-software.org: Navigate to https://rat-software.org in any modern browser. The platform is free to use and does not require a paid subscription.
Register an Account: Create a free researcher account. After logging in, you will be directed to the Researcher View, the central control panel for all study activities.
Create a New Study: Click ‘New Study’ and configure your study parameters: give it a descriptive name, select the search engines you wish to monitor (e.g., Google, Bing, DuckDuckGo), and define the result types to collect (snippets, full-page archival, AI-generated content, or screenshots).
Upload Your Query List: Prepare a plain-text or CSV file containing your research queriesUpload Your Query List: Prepare a plain-text or CSV file with your research queries, one per line. Upload this file to the Study Designer. Alternatively, use the integrated Query Sampler to auto-generate queries from a seed keyword using the Google Ads API.ies against selected search engines automatically using Selenium WebDriver. Results are stored with full metadata, including URL, position, snippet, HTML source code, and screenshots.
Configure Automated Analysis: Navigate to the Classifier module and enable the built-in SEO Classifier. Optionally, upload custom classifier scripts for domain-specific analysis (health, sentiment, bias detection, etc.).
Set Up Human Evaluation (Optional): If you require human judgment, create a questionnaireSet Up Human Evaluation (Optional): If human judgment is needed, create a questionnaire in the Assessment Interface. Define question types (Likert scales, multiple-choice, open-ended, sliders), then share the unique participant URL with your evaluators. RAT automatically deduplicates results and presents consistent archived screenshots to all evaluators.ress, result overlap across engines, source diversity, participant responses, and classifier outputs.
Export Data: Download the complete dataset, including all collected results, metadata, classifier scores, and evaluation responses, as an XLSX spreadsheet using the Data Exporter. Use this data in Excel, R, Python, or any BI tool for further analysis.
Option B — Self-Hosted Installation
For organisations requiring full data control, custom infrastructure, or code-level modifications, RAT can be self-hosted. The source code is available at github.com/rat-software.
Prerequisites:
Python 3.10 or higher
PostgreSQL database server
Selenium WebDriver with a compatible browser driver (Chrome or Firefox)
A Linux or macOS server environment (Ubuntu 22.04+ recommended)
Git for cloning the repository
Installation Steps:
Clone the Repository: Run: git clone https://github.com/rat-software/rat-software.git
Configure PostgreSQL: Create a dedicated database and user for RAT. Update the database connection settings in the RAT configuration file.
Install Python Dependencies: Navigate to the project root and run: pip install -r requirements.txt
Configure Selenium: Download and install the appropriate browser driver (ChromeDriver or GeckoDriver). Update the RAT scraper configuration to point to your driver binary.
Initialise the Database: Run the provided database initialisation script to create required tables and seed configuration data.
Launch RAT Frontend: Start the Flask-based web server: python app.py. Access the Researcher View at localhost on your configured port.
Launch RAT Backend Scheduler: In a separate terminal, start the APScheduler-based backend: python scheduler.py. This process manages all background collection and classification jobs.
Consult the Knowledge Base: Visit the RAT Knowledge Base at rat-software.org/knowledge-base or attend a monthly RAT webinar for guided tutorials. Free one-to-one consultations are available for new users.
RAT Technologies Available Today
RAT’s modular architecture allows businesses and researchers to deploy only the components relevant to their use case. The following table summarizes all major technologies currently available within the RAT ecosystem as of 2026.
| Technology / Module | Function | Platform | AI-Powered? |
|---|---|---|---|
| RAT Web Interface | Study design & management | Browser / Cloud | Partial |
| Search Engine Scraper | Auto-collects SERP data (Google, Bing, etc.) | Python / Selenium | No |
| Source Scraper | Full-page text & screenshot archival | Python / Selenium | No |
| AI Overview Scraper | Captures Google AI Overviews & ChatGPT answers | Python / Selenium | Yes |
| SEO Classifier | ML probability score of SEO manipulation per URL | Python / ML | Yes |
| Custom Classifiers | User-defined ML classifiers (health, sentiment, bias, etc.) | Python | Yes |
| Human Assessment Interface | Juror evaluation with questionnaires & Likert scales | Web / Flask | No |
| Analyzer Module | Real-time statistics, overlap, diversity reports | Web / Flask | Partial |
| Data Exporter | Downloads all collected data in XLSX format | Python / openpyxl | No |
| Query Sampler | Generates query sets via Google Ads API | Python / Google API | Partial |
| RAT Browser Extension | In-browser real-user SERP collection (in development) | Browser Extension | Partial |
RAT’s open extension framework means new classifiers and scrapers are being contributed continuously by the research community. Notable recent additions include classifiers for health domain document quality (2024), web genre classification using machine learning (2024), and commercial intent scoring (2024). An in-browser extension for collecting search results from real user sessions is currently in active development as part of RAT’s 2025–2028 DFG-funded research phase.
How RAT Uses Artificial Intelligence
Artificial intelligence is integrated into RAT at multiple levels, serving both as a subject of analysis and as a methodological tool for automating assessment tasks that would otherwise require human effort.
AI as a Subject: Analyzing AI-Generated Search Content
The emergence of generative AI in search, including Google AI Overviews, Bing Copilot, and chatbot-style answers, has introduced a new dimension of SERP analysis that traditional SEO tools cannot address. RAT includes dedicated scrapers to capture AI-generated content from Google’s AI Overview panels and chatbot systems such as ChatGPT. This enables researchers and businesses to compare how AI-generated answers represent brands, topics, or competitors relative to traditional organic results, and to track changes in AI content over time.
AI as a Tool: The Classifier Framework
RAT’s classifier module is an extensible machine-learning pipeline built on Python. Each classifier receives the full collected data for each search result — URL, domain, position, snippet text, complete HTML source code, and original query — and returns a structured analysis score or label.
The classifiers currently documented and available include:
SEO Classifier — Calculates the probability that a given URL was artificially optimised for search placement using ML pattern recognition across content and structural signals.
Health Domain Classifier — Assesses whether web documents meet quality standards for health information, drawing on training data from authoritative medical sources.
Web Genre Classifier — Categorises documents by type (news article, blog post, product page, academic paper, etc.) using supervised ML.
Commercial Intent Scorer — Rates the degree to which a URL’s content is commercially motivated rather than informational or editorial.
Misinformation Detection Classifier — Identifies list-based indicators of misinformation using NLP pattern matching.
Sentiment Analyser — Performs sentiment analysis on search result snippet text to detect tonality patterns in SERP coverage of a topic.
Custom classifiers can be developed using the templates provided in the RAT GitHub repository. Any Python-based ML model — whether trained with scikit-learn, TensorFlow, PyTorch, or a large language model API — can be integrated as a RAT extension.
AI-Driven Query Generation
The Query Sampler module uses the Google Ads Keyword Planner API to generate semantically related, real-world query sets from seed keywords. This AI-assisted query expansion ensures research studies cover the full range of user search behavior, rather than relying solely on an analyst’s intuition.
Human-AI Hybrid Assessment
RAT is designed to operate in a human-AI hybrid model. Automated classifiers handle large-scale pattern recognition tasks — processing thousands of URLs for SEO probability or content category — while the Human Assessment Interface routes ambiguous, high-stakes, or nuanced evaluation tasks to human jurors. The Analyzer module integrates both streams, allowing researchers to compare automated and human assessments and to quantify inter-rater agreement statistics.
Key insight: RAT does not replace human judgment — it amplifies it, by handling the scale and repetition of data collection and automated classification so that human expertise can focus where it matters most.
Conclusion
The Result Assessment Tool (RAT) is a significant advancement in infrastructure for researchers, SEO professionals, competitive intelligence analysts, and business strategists. By offering a free, open-source, peer-reviewed, and AI-augmented platform for systematic search result analysis, RAT removes technical and financial barriers that have hindered rigorous, evidence-based approaches to understanding commercial search engines.
For businesses, the implications are substantial. Search visibility is one of the most consequential and For businesses, the implications are substantial. Search visibility is a critical yet opaque competitive factor in the digital economy. Decisions about content strategy, brand positioning, SEO investment, and digital advertising are often based on anecdotal spot-checks and proprietary tools lacking methodological transparency. RAT fundamentally changes this dynamic.ducible research pipeline. Its AI-powered classifier framework enables automated detection of SEO manipulation, content quality assessment, sentiment analysis, and misinformation flagging at scales no human team could achieve manually. Its dedicated scrapers for Google AI Overviews and chatbot responses place it at the frontier of the field precisely when AI-generated search content is transforming how businesses are discovered — or not discovered — online.
Importantly, RAT is not a commercial product seeking to monetize search intelligence. It is a publicly funded, open-science tool developed by and for the research community, freely available to any organization willing to use it. Funding through 2028 by the German Research Foundation ensures ongoing development, stability, and community support.
For internet research professionals, especially those pursuing or maintaining CIRS™ certification, RAT exemplifies the systematic, evidence-based methodology that distinguishes professional research from informal search monitoring. In an environment marked by opacity, manipulation, and rapid AI-driven change, tools that restore transparency and rigor are essential.
RAT is free, open source, and peer-reviewed. It is the most capable search result analysis platform available to the public today. The question is not whether to use it, but when to start.
References
Sünkler, S., Lewandowski, D., Schultheiß, S., & Yagci, N. (2025). Result Assessment Tool (RAT): empowering search engine data analysis. PeerJ Computer Science, 11, e2962. https://doi.org/10.7717/peerj-cs.2962
Sünkler, S., Yagci, N., Schultheiß, S., von Mach, S., & Lewandowski, D. (2023). Result Assessment Tool (RAT): A Software Toolkit for Conducting Studies Based on Search Results. Proceedings of the Association for Information Science and Technology. https://doi.org/10.1002/pra2.972
Sünkler, S., Yagci, N., Schultheiß, S., von Mach, S., & Lewandowski, D. (2024). Result Assessment Tool: Software to Support Studies Based on Data from Search Engines. In Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14612. Springer. https://doi.org/10.1007/978-3-031-56069-9_19
Sünkler, S., Yagci, N., Schultheiß, S., von Mach, S., & Lewandowski, D. (2026). Result Assessment Tool (RAT): An Open-Source Toolkit for Conducting Studies based on Search Results. Proceedings of the 2026 Conference on Human Information Interaction and Retrieval (CHIIR 2026). ACM. https://doi.org/10.1145/3786304.3787925
RAT Software Project. (2025). rat-software.org — The Free Open-Source Research Toolbox for Collecting, Analysing, and Evaluating Search Results. Retrieved June 2026, from https://rat-software.org
Search Studies Research Group, Hamburg University of Applied Sciences. (2025). RAT Project Overview. Retrieved June 2026, from https://searchstudies.org/research/rat/
RAT Software GitHub. (2026). rat-software/rat-software. GitHub. https://github.com/rat-software
