Evaluation record · devin

Devin

v2.x

Cognition

Agentautonomoussoftware-engineeringcloud-agentproprietary

Adequate

About This Agent

Autonomous AI software engineer from Cognition that plans and executes multi-step engineering tasks in a sandboxed cloud workspace with its own editor, shell, and browser, and delivers work as pull requests. Now spans Devin Cloud agents, Devin Desktop (the rebranded Windsurf IDE, June 2026) with the Rust-based Devin Local agent, and Cognition's in-house SWE model line (SWE-1.7, July 2026).

Last Evaluated: July 9, 2026

Official Website

Trust Vector Analysis

Dimension Breakdown

🚀Performance & Reliability

task completion accuracy

Assessment of task completion on scoped engineering work based on vendor documentation, customer case studies, and independent user reports

Evidence

Cognition - Devin Product Page — Completes well-scoped engineering tasks end-to-end as PRs; reliability is strongest on bounded tasks like migrations, bug fixes, and test backfills

mediumVerified: 2026-07-09

tool use reliability

Review of integrated toolchain reliability across shell, browser, and VCS operations

Evidence

Devin Documentation — Operates a full cloud workspace (editor, shell, browser) with native GitHub, Slack, Jira, and Linear integrations

highVerified: 2026-07-09

multi step planning

Evaluation of plan generation, user-editable plans, and plan adherence on long-horizon tasks

Evidence

Devin Documentation - Planning — Devin 2.0 introduced interactive planning: drafts an editable plan before execution and revises it as the task evolves

highVerified: 2026-07-09

memory persistence

Review of cross-session memory features (Knowledge, Playbooks, Wiki) and session snapshot persistence

Evidence

Devin Documentation - Knowledge — Persistent Knowledge base, Playbooks for repeatable procedures, and Devin Wiki auto-generated codebase documentation carry context across sessions

mediumVerified: 2026-07-09

error recovery

Assessment of autonomous debugging behavior and failure-mode reports from production users

Evidence

Devin Documentation — Iterates on failing tests and build errors autonomously; can go down unproductive paths on ambiguous tasks, consuming ACUs until interrupted

mediumVerified: 2026-07-09

agent collaboration

Review of parallel session capabilities and multi-Devin task delegation

Evidence

Cognition - Devin Product Page — Supports running multiple parallel Devin sessions on independent tasks; orchestration across Devins is coarser than dedicated multi-agent frameworks

mediumVerified: 2026-07-09

🛡️Security

tool sandboxing

Security architecture review of isolated cloud workspace model

Evidence

Devin Security Documentation — Each session runs in an isolated, sandboxed cloud VM separate from user infrastructure; code execution never touches the local machine

highVerified: 2026-07-09

access control

Review of identity, repository scoping, and secrets handling controls

Evidence

Devin Security Documentation — Enterprise tier offers SSO/SAML, scoped repository access via GitHub app permissions, and secrets management for credentials

mediumVerified: 2026-07-09

prompt injection defense

Threat surface analysis of autonomous browsing and untrusted repo content; limited public disclosure of defenses

Evidence

Devin Documentation — Autonomous web browsing and repository content processing create injection surface; mitigations exist but are not publicly detailed

lowVerified: 2026-07-09

data isolation

Data architecture review of tenant and session isolation claims

Evidence

Devin Security Documentation — Per-session VM isolation and tenant separation; SOC 2 Type II attested infrastructure

mediumVerified: 2026-07-09

open source transparency

Source availability assessment

Evidence

Cognition — Fully proprietary product and models (in-house SWE model line, latest SWE-1.7, plus frontier models); no source code or model weights published

highVerified: 2026-07-09

🔒Privacy & Compliance

data retention

Review of published retention practices and enterprise data controls

Evidence

Devin Security Documentation — Session data and workspace snapshots retained in Cognition's cloud; enterprise contracts offer training opt-out and retention controls

mediumVerified: 2026-07-09

gdpr compliance

Compliance documentation assessment

Evidence

Devin Security Documentation — SOC 2 Type II attestation and DPA availability for enterprise customers; GDPR posture depends on contractual terms

mediumVerified: 2026-07-09

third party data sharing

Data flow analysis of model routing between in-house and third-party providers

Evidence

Cognition - Devin Product Page — Code is processed by Cognition's own SWE model line (latest SWE-1.7) and routed to third-party models (OpenAI, Claude, Gemini options on paid plans) for some tasks

mediumVerified: 2026-07-09

local deployment option

Deployment options assessment

Evidence

Devin Documentation — Cloud-only service; no self-hosted or on-premises deployment, with VPC options limited to enterprise arrangements

highVerified: 2026-07-09

👁️Trust & Transparency

documentation quality

Documentation completeness review

Evidence

Devin Documentation — Well-organized docs covering onboarding, ACU model, integrations, Playbooks, and enterprise administration

highVerified: 2026-07-09

execution traceability

Review of session visibility, live workspace observation, and replay features

Evidence

Cognition - Devin Product Page — Full visibility into Devin's plan, editor, shell, and browser in real time; complete session timeline is replayable

highVerified: 2026-07-09

decision explainability

Assessment of plan transparency and change justification quality

Evidence

Devin Documentation - Planning — Explicit upfront plans, step-by-step narration during execution, and PR descriptions explaining changes

mediumVerified: 2026-07-09

open source code

Open source assessment

Evidence

Cognition — Closed-source product; limited public benchmark reproducibility and no published model details beyond blog posts

highVerified: 2026-07-09

community activity

Community and ecosystem engagement analysis

Evidence

TechCrunch - Cognition raises $1B at $26B valuation — Large and growing commercial user base (~$492M ARR reported); community is customer-driven rather than open-source contributor-driven

mediumVerified: 2026-07-09

⚙️Operational Excellence

ease of integration

Integration surface assessment across team workflows

Evidence

Devin Documentation - Integrations — Assign tasks from Slack, GitHub issues, Jira, or Linear; Devin 2.0 added IDE-style interface and API access

highVerified: 2026-07-09

scalability

Scalability assessment of parallel cloud session model

Evidence

Cognition - Devin Product Page — Parallel cloud sessions allow teams to fan out many tasks simultaneously without local resource constraints

mediumVerified: 2026-07-09

cost predictability

Pricing model analysis; 2026 quota-based subscription tiers improve predictability over pure ACU metering (score raised 62 to 66), though heavy usage still burns variable quota per task

Evidence

Devin Pricing — Restructured plans: Free $0; Pro $20/mo (full model access, Devin Cloud agents); Max $200/mo (higher quotas); Teams $80/mo base + $40/mo per full developer seat; Enterprise custom. Quota-based usage refreshing daily/weekly replaces headline ACU pricing (ACUs remain the underlying meter, ~$2.00-2.25/ACU on team tiers)

highVerified: 2026-07-09

monitoring capabilities

Monitoring and usage governance features assessment

Evidence

Devin Documentation — Session dashboards, ACU usage tracking, and admin controls for team usage oversight

mediumVerified: 2026-07-09

production readiness

Vendor maturity and product stability assessment

Evidence

TechCrunch - Cognition raises $1B at $26B valuation — Closed $1B+ round at $26B valuation (2026-05-27) with ~$492M ARR; acquired Windsurf 2025-07-14, signaling strong vendor viability

mediumVerified: 2026-07-09

Strengths

+True end-to-end autonomy: plans, codes, tests, browses docs, and opens PRs in its own cloud workspace
+Sandboxed cloud VMs isolate execution from user infrastructure
+Interactive, editable plans and fully replayable session timelines provide strong traceability
+Accessible pricing: Free tier and $20/mo Pro plan (vs the original $500/mo), with quota-based subscriptions since mid-2026
+Persistent Knowledge, Playbooks, and auto-generated Devin Wiki retain organizational context
+Strong vendor trajectory: $26B valuation, ~$492M ARR, Windsurf acquisition (2025-07-14)

Limitations

!Usage-quota/ACU-metered consumption makes costs unpredictable, especially when the agent pursues unproductive paths
!Fully proprietary stack with no self-hosted option; code must be processed in Cognition's cloud
!Reliability drops on ambiguous or large unscoped tasks, requiring careful task decomposition
!Prompt injection defenses for autonomous browsing are not publicly documented
!Some workloads route to third-party frontier models, complicating data governance review
!Output still requires human code review; unsupervised merging is not advisable

Metadata

license: Proprietary

supported models

0: Cognition SWE model line (in-house; SWE-1.7 launched 2026-07-08, served at ~1,000 tokens/sec via Cerebras)

1: OpenAI, Anthropic Claude, and Google Gemini models on paid plans

programming languages

0: Most major languages (Python, TypeScript, Java, Go, etc.)

deployment type: Cloud (sandboxed VM workspaces)

tool support

0: Cloud editor and shell

1: Built-in browser

2: GitHub/GitLab integration

3: Slack, Jira, Linear

4: API access

first release: 2024 (limited), Devin 2.0 April 2025

pricing: Free $0; Pro $20/mo; Max $200/mo; Teams $80/mo base + $40/mo per full developer seat; Enterprise custom (quota-based, ACU-metered underneath)

company milestones: Acquired Windsurf 2025-07-14 (rebranded Devin Desktop 2026-06-02, with Rust-based Devin Local agent); raised $1B+ at $26B valuation (closed 2026-05-27); ~$492M ARR; AI Productivity Guarantee (up to $10M in usage credits) announced June 2026; SWE-1.7 model launched 2026-07-08

Use Case Ratings

code generation

Purpose-built autonomous software engineer; excels at scoped tasks like migrations, bug fixes, test coverage, and PR-sized features

data analysis

Can write and run analysis scripts in its workspace, but is optimized for software engineering rather than analytics workflows

research assistant

Browser access enables technical research and documentation digging, though it is not designed for general research synthesis

education

Replayable sessions showing plan and execution can teach engineering practice, but ACU costs make it expensive for learning

Similar Agents

OpenAI Codex

OpenAI

Claude Code

Anthropic

GitHub Copilot Coding Agent

GitHub (Microsoft)

Google Jules

Google

Cursor Agent

Anysphere (SpaceX acquisition announced 2026-06-16, expected to close Q3 2026)