# Devin AI Testing: Automated Unit Tests, End-to-End Verification, and Video Proof Devin AI testing is an autonomous verification workflow where Devin prepares test environments, generates test cases, executes test suites, and records video evidence to prove PR changes work. Operating in three distinct phases of setup, test planning, and recording, Devin delivers verifiable proof directly to reviewers. Connecting Devin to Fastio workspaces provides persistent storage for test videos, coverage metrics, and execution logs across ephemeral sandbox sessions. Source: https://fast.io/resources/devin-ai-test/ Last reviewed: 2026-09-04 ## Autonomous Verification: How Devin AI Tests Code and Proves Changes In an official verification walkthrough on an Express and Prisma codebase, Devin analyzed an existing test suite and increased function coverage from 28.57% to 57.14% of functions. This jump illustrates how autonomous engineering agents shift testing from an unfulfilled backlog aspiration into an automated post-PR verification routine. Writing code is only half the engineering equation. Without thorough automated verification, code generation tools simply pass the burden of testing, debugging, and edge-case discovery onto human reviewers. When an engineer writes a pull request, code review typically requires pulling down the branch, running containerized dependencies, executing test scripts, and clicking through the user interface to verify that changes did not break visual components. When an autonomous AI writes the code, this review burden multiplies. Without verifiable evidence that the software works, human engineers spend more time auditing agent code than they would have spent writing it by hand. Devin AI testing is an autonomous verification workflow where Devin prepares test environments, generates test cases, executes test suites, and records video evidence to prove PR changes work. Devin operates inside an isolated, cloud-hosted virtual machine sandbox. Unlike local text editors that only manipulate open file buffers, Devin has access to a full Linux shell, a desktop browser, a code editor, and network connectivity. This architecture allows Devin to run real local development servers, start background databases, interact with rendered web pages, and observe runtime behavior directly. The testing workflow operates in three distinct phases: setup, test planning, and recording. 1. Setup: Devin prepares the execution environment by reading the pull request diff, loading repository skills from `.agents/skills/`, starting local servers or container dependencies, and requesting missing credentials or environment variables. 2. Test planning: Devin analyzes the modified code paths to design a targeted, minimal end-to-end test plan. It writes explicit verification steps that trace exact user interface elements and identifies the primary user flow that validates the change. 3. Recording and execution: Devin executes the test plan inside its desktop browser environment, capturing a screen recording with automated cursor zooming and text annotations. Once verification finishes, Devin compresses idle time and attaches the processed video directly to the pull request for human review. Competitors focus only on unit test generation by prompting an LLM on isolated function signatures. Devin pairs unit test generation with complete end-to-end browser execution, testing both backend business logic and real browser interactions while providing video proof. ## How the Three-Phase Testing Workflow Prepares, Plans, and Records Execution When Devin enters testing mode after opening a pull request, it follows a structured sequence designed to minimize reviewer overhead. Reviewing code by inspecting text diffs alone leaves gaps in runtime confidence. By walking through environment preparation, structured test planning, and recorded execution, Devin provides tangible proof of functionality. ### Phase 1: Environment Setup and Dependency Resolution Before executing any test suites or opening a browser, Devin inspects the repository state. It analyzes the pull request diff against the existing codebase to determine which services, database schemas, API routes, or frontend components were affected. During this phase, Devin searches the repository for testing skills under `.agents/skills/`. If a relevant skill exists, such as instructions on how to start local services or seed test databases, Devin executes those steps automatically. If the application requires environment variables, API keys, or database credentials that are not present, Devin asks for them in the session interface. Once provided, Devin stores these values securely as secrets so they persist across future sessions without manual re-entry. Devin then verifies service connectivity across available local or staging environments. It launches background services using Docker Compose, runs database migrations, starts local development servers, and verifies that the application responds with healthy status codes on expected local ports before moving to the planning stage. ### Phase 2: Test Planning and Critical Path Selection Once the local environment is operational, Devin formulates a targeted test plan. Instead of attempting to test every possible permutation, Devin focuses on the single most important end-to-end user path that proves the pull request fulfills its objective. Devin traces through the frontend source code to map out concrete, unambiguous interactions. Rather than relying on vague directions, Devin identifies exact button labels, form fields, and expected DOM responses. For example, in an e-commerce pull request updating cart discounts, the test plan specifies adding an item to the cart, opening the checkout screen, applying a specific promotional code, and checking that the total reflects the discount. Devin posts this test plan into the chat before executing it. This gives human engineers an immediate opportunity to inspect the planned verification steps and suggest adjustments before browser execution begins. ### Phase 3: Desktop Browser Execution and Video Proof After unit tests pass and any review feedback is resolved, Devin begins execution inside its containerized desktop browser. Video recordings are generated directly within Devin's desktop browser environment, capturing every step of the verification sequence. The resulting screen recording incorporates several built-in post-processing features that assist human code review: * Annotations: Text labels appear on screen at key moments, indicating exactly what Devin is verifying, such as testing form validation or confirming successful record creation. The video slows down around these annotations to provide clear visibility. * Auto-zoom and cursor tracking: The recording automatically zooms into the areas where Devin clicks, types, or toggles controls, panning smoothly to follow user interactions without requiring the reviewer to decipher full-screen desktop captures. * Speed adjustments: Raw recordings are automatically processed to compress idle waiting periods, such as package compilation or network requests, delivering a concise 15- to 45-second verification video. * Pull request attachments: The finalized video is attached directly to the session chat and pull request, enabling reviewers to verify UI and functional behavior without pulling the branch locally. ## Automating Unit Test Generation and Coverage Expansion While browser recordings prove that user-facing workflows function correctly, maintainable engineering requires solid unit test suites to protect against silent regressions. Engineering teams frequently struggle with incomplete test coverage because writing unit tests for existing features competes with shipping new capabilities. Devin addresses this gap by analyzing existing codebases, assessing test coverage, and generating targeted unit tests to expand test suites autonomously. ### Establishing a Test Coverage Baseline To improve test coverage, Devin first determines where coverage gaps exist. When directed to assess an application, Devin clones the repository, checks project configuration files, and executes the existing test suite inside its shell environment. In the official Express and Prisma walkthrough, Devin inspected a project with incomplete coverage. By executing the test runner with coverage flags enabled, Devin established that existing tests covered 28.57% of functions. Devin distilled the raw terminal output into a readable summary in chat, pinpointing modules that completely lacked test coverage. ### Pattern Matching and Test Case Implementation Once Devin identifies uncovered modules, it reads the target source files and examines adjacent test files to match the team's testing conventions. It adopts the project's existing test framework (such as Jest, Vitest, Pytest, or Go testing utilities), mocking patterns, file naming structures, and assertion styles. In the test coverage demonstration, Devin targeted an uncovered utility module (`profileViewer.ts`). Devin analyzed the function signatures, return types, and potential error states, and then authored comprehensive test cases covering both expected inputs and edge cases. These included validating correct data transformations, handling unauthorized requests, and checking behavior when database queries returned null. Devin wrote the new tests directly using its built-in editor and re-executed the test runner in the shell. In this walkthrough, Devin verified that all new tests passed, elevating function coverage from 28.57% to 57.14% of functions. Engineers can review the new test files directly in Devin's web interface or instruct Devin to submit the tests as a GitHub pull request. ### Codifying Testing Procedures into Repository Skills A key advantage of Devin's testing workflow is its ability to learn project-specific routines. After completing a test run, Devin compiles the commands and environment steps that succeeded and proposes saving them as a reusable Skill under `.agents/skills/` via pull request. A testing skill codifies setup commands, required services, and verification routines into an actionable markdown file: ```markdown --- name: devin-test-suite description: Setup local dependencies, run unit tests, and verify application pages before opening a pull request. --- ## Setup 1. Install project dependencies: `npm install` 2. Start local services: `docker compose up -d postgres redis` 3. Apply database migrations: `npx prisma migrate dev` 4. Launch the dev server: `npm run dev` ## Verification 1. Execute the unit test suite: `npm test -- --coverage` 2. Verify that function coverage meets or exceeds repository baselines 3. Open modified application routes in the desktop browser to confirm layout stability 4. Inspect browser console logs for uncaught exceptions or network errors ``` Once committed to the repository, Devin references this skill on future runs, eliminating repetitive environment onboarding and ensuring consistent verification across every task. ## Why Ephemeral Sandboxes Require Persistent Workspace Storage Devin's autonomous testing workflow produces valuable verification assets: high-resolution screen recordings, full terminal execution logs, LCOV coverage summaries, visual diff snapshots, and mock database dumps. However, managing where these assets live introduces an architectural challenge for engineering teams. ### The Ephemeral Sandbox Problem Devin executes tasks inside containerized cloud virtual machines. While Devin attaches final video recordings to the session and pull request, the underlying virtual machine environment is ephemeral. When a session terminates or gets archived, uncommitted intermediate artifacts on the sandbox disk disappear. Committing large video files, binary traces, or detailed coverage dumps directly into a Git repository is bad practice. Binary files cause Git repository bloat, dramatically increasing clone times and storage costs for every team member. Furthermore, storing test proof across disconnected platforms like personal Google Drive folders or raw Amazon S3 buckets creates fragmented silos where test evidence cannot be easily searched, indexed, or reviewed in context. ### Fastio Workspaces as Persistent Storage for Agent Outputs [Fastio shared workspaces](/product/workspaces/) provide org-owned spaces designed for agentic teams, giving both human developers and autonomous agents a unified, persistent workspace layer. Instead of discarding intermediate test evidence or polluting source code repositories, engineering teams connect Devin to Fastio workspaces to persist test artifacts automatically. Teams exploring dedicated storage setups can reference [Fastio storage for agents](/storage-for-agents/) for configuration patterns. Fastio addresses the requirements of agent test storage through specific workspace capabilities: * Media Engine and adaptive bitrate streaming: Fastio includes adaptive bitrate streaming (HLS) that automatically transcodes uploaded MP4 test recordings. Team leads and QA reviewers can play back and scrub through Devin's video proof instantly in the browser without downloading massive video files locally. * Per-file version history: Every file uploaded to a Fastio workspace maintains a complete, restorable version history. As Devin refines pull requests across successive commits, updated test recordings and coverage reports are preserved in sequence, allowing reviewers to compare test outputs over time. * Append-only audit log: Fastio records every upload, access, and share operation in an immutable audit log. This provides engineering organizations with a clear, tamper-evident chain of custody documenting when tests ran and who verified the results. * Workspace Intelligence: With Intelligence Mode enabled on a workspace, Fastio automatically indexes uploaded test summaries, coverage files, and documentation for semantic search and citation-backed RAG chat. Engineers can query past verification runs using natural language, asking questions like "Which pull request test run identified a migration failure in August?" and receive answers backed by citations to specific files. ### Structured Test Metrics with Metadata Views Beyond semantic search across test logs, engineering teams need structured tracking to monitor test coverage, pass rates, and performance across agent runs. [Fastio Metadata Views](/product/document-data-extraction/) turn unstructured test outputs and logs into a live, queryable database. Users describe the fields they want extracted in natural language, and Fastio's AI designs a typed schema supporting Text, Integer, Decimal, Boolean, URL, JSON, and Date & Time fields. Fastio automatically classifies incoming files in the workspace and populates a filterable spreadsheet grid. For test management, a Metadata View can automatically extract pull request numbers, tested branches, execution runtimes, passed test counts, and function coverage percentages from test summaries. Engineers can filter, sort, and inspect metrics across hundreds of Devin runs, and agents can query extracted test data programmatically via the Fastio MCP server. ## Verification Pipeline Architecture: Connecting Devin to Team Review Integrating Devin's testing capabilities with persistent workspace storage creates a streamlined pipeline that connects autonomous code generation to human code review. ``` +-------------------------------------------------------------------------+ | Devin Cloud Sandbox | | | | 1. Inspect Diff & Skills --> 2. Generate Unit Tests & Run Coverage | | | | | 4. Attach Video to PR <-- 3. Execute Browser Test & Record Video | +-------------------------------------------------------------------------+ | (fastio CLI / MCP upload) v +-------------------------------------------------------------------------+ | Fastio Shared Workspace | | | | * Media Engine (HLS Video Streaming) * Per-File Version History | | * Metadata Views (Structured Metrics) * Append-Only Audit Log | | * Intelligence Mode (Semantic Search) * Scoped Branded Shares | +-------------------------------------------------------------------------+ | (Scoped Portal / Share) v +-------------------------------------------------------------------------+ | Human Reviewers & QA | | Review annotated video proof, inspect coverage metrics, and merge PR | +-------------------------------------------------------------------------+ ``` ### Step 1: Triggering Devin and Executing Test Suites The workflow begins when an engineer assigns an issue to Devin or asks Devin to implement a feature branch. Devin launches its cloud virtual machine, installs project dependencies, reads testing skills from `.agents/skills/`, and writes the necessary code. Devin runs the project's test suite to identify missing test coverage, writes new unit tests to cover untested functions, and verifies that all assertions pass cleanly. ### Step 2: Running End-to-End Verification and Capturing Video Proof Once unit tests pass, Devin starts the local development server and transitions to end-to-end testing. Devin creates a concise test plan and executes the interactions within its desktop browser environment, capturing an annotated screen recording that follows cursor movements and confirms visual rendering. ### Step 3: Exporting Test Artifacts to Fastio To ensure test evidence remains accessible beyond the lifespan of the ephemeral sandbox VM, Devin uploads the video recording, LCOV coverage report, and test execution log to a dedicated Fastio workspace. This can be accomplished inside Devin's shell using the official Fastio command-line interface: ```bash fastio upload file --workspace devin-verification ./artifacts/test-recording.mp4 fastio upload file --workspace devin-verification ./coverage/lcov-report.json ``` Alternatively, Devin can interact with Fastio through the official Fastio Model Context Protocol (MCP) server, as detailed on [Fastio storage for agents](/storage-for-agents/). The MCP server provides remote access via Streamable HTTP at `https://mcp.fast.io/mcp` (or `https://mcp.fast.io/mcp/key` with bearer token authentication) and legacy SSE at `https://mcp.fast.io/sse`. Devin can call consolidated MCP tools to upload files, retrieve workspace context, or query structured Metadata Views. ### Step 4: Review, Scoped Sharing, and Governance With artifacts persisted in Fastio, team leads and QA engineers can review the test video using instant HLS streaming without pulling down the branch. When sharing test evidence with external clients, audit teams, or cross-functional stakeholders, teams can create branded shares (Send, Receive, or Exchange) with custom expiration controls and per-recipient access grants. Collaborative Notes within the workspace allow humans and agents to co-edit test review checklists in real time. For agency and consulting workflows, Fastio supports ownership transfer: an agent account can build an organization workspace, populate it with project documentation and test evidence, and transfer administrative ownership to a human client or team lead while preserving developer access. Every organization starts with a 14-day free trial, which requires a credit card. Team subscriptions are available across three tiers on the [Fastio pricing page](/pricing/): | Subscription Plan | Monthly Pricing | Storage Allowance | Usage-Based Credits | | --- | --- | --- | --- | | Starter | $29/mo | 1 TB | 300,000 credits | | Business | $99/mo | 10 TB | 1,200,000 credits | | Growth | $299/mo | 50 TB | 4,500,000 credits | ## Frequently asked questions ### How does Devin AI test code? Devin AI tests code by entering an autonomous testing mode inside its cloud sandbox environment. It prepares dependencies and databases, plans a targeted test flow by analyzing the pull request diff, executes the steps using its built-in browser and terminal, and generates unit tests to expand code coverage. ### Can Devin AI record video proof of test execution? Yes, Devin AI records video proof directly inside its desktop browser environment. The recording automatically tracks cursor movements, zooms in on interactive elements, adds explanatory text annotations at key verification points, and compresses idle waiting time before attaching the processed video to the pull request. ### How do you increase code coverage with Devin AI? You increase code coverage with Devin AI by prompting it to inspect your repository's existing test suite and establish a coverage baseline. Devin identifies uncovered functions and branch logic, writes new unit test files that match your project's testing conventions, and re-executes the test runner to verify that coverage metrics improved without regressions. ### Where are Devin AI video recordings and test artifacts stored? Devin attaches test recordings directly to session messages and pull requests, while storing local run files inside its temporary sandbox virtual machine. To preserve test videos, coverage data, and logs permanently across sessions, teams export them to shared Fastio workspaces using the Fastio CLI or MCP server. ### How do you trigger Devin's automated testing workflow? You can trigger Devin's testing workflow by clicking the Test the app button after Devin creates a pull request, or by including explicit testing instructions in your prompt, such as asking Devin to verify changes and record a video demonstration before opening the PR. ## About Fast.io Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server at https://mcp.fast.io/mcp, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli.