Tools & Code

Open-source CLI for testing, evaluating, and red-teaming LLM prompts. YAML configs, CI/CD integration, adversarial testing. ~9K+ ⭐

Solve NLP Problems with LLM's & Easily generate different NLP Task prompts for popular generative models like GPT, PaLM, and more with Promptify

Open-source LLM developer platform for prompt management, evaluation, human feedback, and deployment.

Version, test, and monitor every prompt and agent with robust evals, tracing, and regression sets.

Production prompt monitoring and optimization platform.

Framework for structured and meta-prompt design. 10K+ ⭐

Visual toolkit for building, testing, and comparing LLM prompt responses without code.

A query language for LLMs making complex prompt logic programmable.

Platform for developing, testing, and managing structured LLM prompts.

AI-powered prompt management system for streamlining prompt workflows.

Browser extension to automatically improve user prompts for any AI model.

Web-based "Prompt Engineering IDE" for iteratively creating and running prompts.

Test suite for LLM prompts before pushing to production.

Open-source framework for prompt-learning research.

Toolkit for creating, sharing, and using natural language prompts.

NPM utility library for creating and maintaining prompts for LLMs (Microsoft).

Framework for quantitative analysis of LLM robustness to adversarial prompt attacks.

Self-hostable platform for managing AI IDE config files (.cursorrules, CLAUDE.md, copilot-instructions.md). Web UI, REST API, CLI, and federated blueprint marketplace for 30+ AI coding assistants.

Visual AI prompt builder that decomposes prompts into 12 semantic blocks (role, context, constraints, examples, etc.) and compiles them into optimized XML. Browser extension for ChatGPT/Claude/Gemini, and MCP server for Claude Code agents. Free, open-source.

Open-source evaluation framework covering RAG, agents, and conversations with CI/CD integration. ~7K+ ⭐

RAG evaluation with knowledge-graph-based test set generation and 30+ metrics. ~8K+ ⭐

LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications.

Open-source LLM observability with tracing, prompt management, and human annotation. ~7K+ ⭐

End-to-end AI evaluation platform, SOC2 Type II certified.

Real-time LLM monitoring with drift detection and tracing.

Evaluating and explaining LLM apps; tracks hallucinations, relevance, groundedness.

Purpose-built for evaluating agents against benchmarks (UK AISI).

Evaluate, test, and ship LLM applications across dev and production lifecycles.

CLI tool for testing multi-step AI agents with YAML test cases, regression detection, and production monitoring.

Most widely adopted LLM app framework; LangGraph adds graph-based multi-step agent workflows. ~100K+ / ~10K+ ⭐

Role-playing AI agent orchestration with 700+ integrations. ~44K+ ⭐

Microsoft's multi-agent conversational framework. ~40K+ ⭐

Stanford's framework for programming LLMs with automatic prompt/weight optimization. ~22K+ ⭐

Official agent framework with function calling, guardrails, and handoffs. ~10K+ ⭐

Microsoft's AI framework powering M365 Copilot; C#, Python, Java. ~24K+ ⭐

Data framework for RAG and agent capabilities. ~40K+ ⭐

Open-source NLP framework with pipeline architecture for RAG and agents. ~20K+ ⭐

Python agent framework with microsecond instantiation. ~20K+ ⭐

Hugging Face's minimalist code-centric agent framework (~1000 LOC). ~15K+ ⭐

Type-safe agent framework using Pydantic for structured validation. ~8K+ ⭐

TypeScript AI agent framework with assistants, RAG, and observability. ~20K+ ⭐

Agent Development Kit deeply integrated with Gemini and Google Cloud.

Model-agnostic framework with deep AWS integrations.

Node-based visual agent builder with drag-and-drop. ~50K+ ⭐

Workflow automation with AI agent capabilities and 400+ integrations. ~60K+ ⭐

All-in-one backend for agentic workflows with tool-using agents and RAG.

Multi-AI Agents framework with 100+ LLM support, MCP integration, and built-in memory.

Multi-provider AI agent framework unifying 12+ providers with workflow orchestration.

Connect 100+ tools to AI agents with zero setup.

Multiple optimizers (MIPROv2, BootstrapFewShot, COPRO) for automatic prompt tuning. ~22K+ ⭐

Automatic differentiation via text (Stanford). ~2K+ ⭐

Google DeepMind's optimization by prompting.

LLM vulnerability scanner for hallucination, injection, and jailbreaks — the "nmap for LLMs." ~3K+ ⭐

Python Risk Identification Tool for automated red-teaming. ~3K+ ⭐

40+ vulnerabilities, 10+ attack methods, OWASP Top 10 support.

Security toolkit for LLM I/O validation. ~2K+ ⭐

Programmable guardrails for conversational systems. ~5K+ ⭐

Define strict output formats (JSON schemas) to ensure system reliability.

AI security platform for real-time prompt injection detection.

Open-source LLM safety evaluation including CyberSecEval.

Automated jailbreak template generation achieving >90% success rates.

Open-source tool for detection and prevention of prompt injection.

"Open-source scanner that runs 150 attack probes to test AI agents for prompt injection and extraction vulnerabilities."

The core protocol specification and SDKs. ~15K+ ⭐

Official implementations: fetch, filesystem, GitHub, Slack, Postgres.

High-level Pythonic framework for building MCP servers. ~5K+ ⭐

GitHub's official MCP server for repo, issue, PR, and Actions interaction. ~15K+ ⭐

Curated list of 10,000+ community MCP servers. ~30K+ ⭐

MCP server providing version-specific documentation to reduce code hallucination.

Creates remote MCP servers for any GitHub repo by changing the domain.

Visual testing tool for MCP server development.

Anthropic's agentic coding CLI; understands full codebases and executes complex multi-step tasks via natural language.

Open-source terminal coding agent from OpenAI; lightweight, local-first, with sandboxed code execution. ~68K+ ⭐

Google's open-source terminal AI agent with 1M-token context window and Google Search grounding. ~96K+ ⭐

Open-source terminal AI agent optimized for Qwen3-Coder; multi-protocol support (OpenAI/Anthropic/Gemini APIs), 1,000 free requests/day. ~21K+ ⭐

AI pair programming in terminal with deep Git integration; maps entire codebases and auto-commits changes. ~42K+ ⭐

Powerful open-source AI coding agent with beautiful TUI; supports nearly all AI model providers. ~120K+ ⭐

Extensible open-source AI agent from Block (Square/Cash App); installs, executes, edits, and tests with any LLM. ~29K+ ⭐

Glamorous agentic coding agent from Charmbracelet with multi-model support, LSP integration, and beautiful terminal UI. ~9K+ ⭐

Agentic chat experience in terminal from AWS; transitioning to Kiro CLI.

Sourcegraph's agentic coding tool (Cody successor); works across CLI and IDE.

JetBrains' LLM-agnostic coding agent CLI (beta 2026); supports all major model providers.

Self-evolving autonomous terminal coding agent with multi-provider LLM support, 40+ tools, and modular skills system.

Description

:-----------

Leading AI-native code editor (VS Code fork); Composer generates entire apps from natural language, agentic multi-file edits.

AI-powered IDE (VS Code fork) with proprietary Cascade agent and SWE-1.5 model; acquired by Cognition AI.

High-performance editor in Rust with native AI features, Zeta edit prediction, and Agent Client Protocol support. ~77K+ ⭐

Free AI-powered IDE from ByteDance ("The Real AI Engineer") with Builder Mode; provides free access to Claude, GPT-4o, and DeepSeek.

Google's agent-first IDE (VS Code fork) with Manager view for orchestrating multiple agents in parallel; powered by Gemini.

AWS's spec-driven agentic AI IDE (VS Code fork); turns prompts into specs, then working code, docs, and tests.

Open-source AI code editor (VS Code fork) with Continue-based chat and completions. ~40K+ ⭐

Open-source Cursor alternative (VS Code fork); any model or local hosting with change visualization. ~28K+ ⭐

Open-source chat-first AI code editor with multi-file editing and deep Git integration. ~7K+ ⭐

Open-source agentic dev environment (YC W26) for running multiple coding agents in parallel in isolated Git worktrees.

Description

:-----------

Most widely adopted AI coding assistant; inline completions, chat, and agentic coding agent across VS Code, JetBrains, Neovim.

Autonomous coding agent in VS Code with human-in-the-loop approvals; file editing, terminal commands, and browser use. ~59K+ ⭐

Open-source VS Code and JetBrains extension for creating custom, modular AI dev systems; any model. ~32K+ ⭐

Sourcegraph-powered AI assistant that pulls context from local and remote codebases; VS Code, JetBrains, Visual Studio.

Free AI coding extension for 40+ IDEs with completions, chat, and search across 70+ languages.

AWS's AI coding assistant with completions, inline chat, and agent mode; deep AWS integration.

Google's IDE extension powered by Gemini with completions, Next Edit Predictions, and inline diffs; free for individuals.

Privacy-focused AI assistant trained on permissive-licensed OSS; supports all major IDEs with on-premises deployment.

Enterprise AI coding assistant with 200K-token Context Engine for deep codebase understanding.

AI code review and quality platform with multi-agent architecture; test generation, code review, CI/CD enforcement.

Open-source multilingual code generation model supporting 20+ languages with VS Code and JetBrains extensions. ~11K+ ⭐

Self-hosted open-source AI coding assistant (Copilot alternative); runs entirely on your infrastructure. ~25K+ ⭐

Description

:-----------

First fully autonomous cloud-based AI software engineer; plans, codes, tests, and opens PRs independently.

Cloud-native AI agent that autonomously builds, tests, and deploys full-stack apps in-browser; 50+ languages.

AI-powered web dev agent; prompt, run, edit, and deploy full-stack apps directly in the browser via WebContainers. ~15K+ ⭐

Community fork of bolt.new with extended features and broader LLM flexibility. ~12K+ ⭐

Full-stack apps from natural language with built-in Supabase, auth, and one-click deploy; fastest European startup to $20M ARR.

Vercel's AI platform for generating high-quality React/Next.js UI components from natural language.

Cloud-based coding environment with plan, brainstorm, and repair agents; included with paid Copilot plans.

Google's agentic cloud-based development environment.

Description

:-----------

Leading open-source platform for cloud coding agents; consistently top on SWE-bench. Formerly OpenDevin. ~69K+ ⭐

Takes a GitHub issue and automatically fixes it using a custom agent-computer interface. [NeurIPS 2024] ~19K+ ⭐

LangChain's async cloud-hosted coding agent framework built on LangGraph with Slack/Linear integration. ~8K+ ⭐

Open-source agentic software engineer; breaks down instructions, researches, and writes code. Devin alternative. ~18K+ ⭐

Autonomous program improvement combining LLMs with fault localization for GitHub issue resolution. ~2.8K+ ⭐

Simple three-phase approach (localize → repair → validate) to solving software development problems. ~2K+ ⭐

Open-source pair programmer SWE agent with code writing, planning, and research; supports Claude, GPT-4, Llama, Ollama. ~3.5K+ ⭐

The definitive open-source guide and resource hub. 3M+ learners. ~55K+ ⭐

World's largest open-source prompt library. 1000s of prompts for all major models.

Principles for building production-grade LLM-powered software. ~17K+ ⭐

22 hands-on Jupyter Notebook tutorials. ~3K+ ⭐

First-principles handbook for moving beyond prompt engineering to context design.

Collection of system prompts from production AI coding agents (Claude Code, Gemini CLI, Cline, Aider, Roo Code).

Curated list of 245+ tools and resources for building software through natural language prompts.

Official recipes for prompts, tools, RAG, and evaluations.

Framework to create ChatGPT-like bots over your dataset.

Framework for the science of machine thinking.

Extracts and formats code context for AI prompts with token counting.

Compare LLM API pricing across 200+ models.

CLI tool (npx pawmode) that turns Claude Code into a personal assistant by generating system prompts (CLAUDE.md + SOUL.md) with personality, memory, and 38 skill routers.

Open-source CLI that permanently injects 10 structured decision frameworks (MECE, Issue Trees, Pre-Mortems) and 12 cognitive bias detectors into AI assistant prompts. Go, MIT.