Open Training Corpus
PolyPatch-11k pairs a real GitHub issue with the test changes that define the fix, across Python, Rust, Go, C++, PHP, Java and JavaScript. Free and unmetered — query it over HTTP, connect an MCP client, or download the whole corpus in one 27.5 MB file. No key, no registration.
Corpus
PolyPatch-11k is a training and mining corpus. It is five times larger than the established multi-language issue-resolution benchmarks and draws on fifty times as many repositories — but unlike them, its test ids are derived statically rather than by execution. Pick it for volume and diversity; pick an execution-verified benchmark when you need to score a model.
| Dataset | Instances | Languages | Repositories | Execution-verified |
|---|---|---|---|---|
| Multi-SWE-bench | 2,132 | 8 | 76 | Yes, expert-annotated |
| SWE-PolyBench | 2,110 | 4 | 21 | Yes, execution-based |
| PolyPatch-11k | 11,611 | 7 | 3,886 | No — statically derived |
Splits are repository-disjoint: no repository appears in both train and test.
Task instances
Issue + test patch + test ids
Every gold instance carries the issue text, the unified diff of the test changes that define the task, and the test ids those changes touch. Instances whose test patch mechanically swept more than 20 tests were excluded, so the id lists stay meaningful.
Language mix
Seven languages, Python-led
Python 8,399 · Rust 1,619 · Go 1,405 · C++ 70 · PHP 63 · JavaScript 31 · Java 24. The distribution reflects what is actually available upstream with derivable tests — C++ in particular is thin, and we would rather say so than pad it.
Facet
CI workflow metadata
2,681 instances (23.1%) carry the CI workflow files for their repository — 6.19 workflows each across 15,077 distinct workflow paths. Useful if you are working on CI-aware agents. It covers less than a quarter of the corpus, so filter with ci_annotated rather than assuming it.
Method
Each changed line in a test patch is attributed to its enclosing test, seeded by the declaration name git records in the hunk header. A test is listed only when the patch adds or removes lines inside its body, or adds its signature.
Context is not evidence
A test that merely appears in a diff's context window is never listed. Matching over a flattened patch made 68% of ids spurious in an earlier revision; hunk-scoped attribution removed them.
Deleted tests are subtracted
If a patch removes a test's signature, that test cannot pass afterwards, so it is dropped from the id list rather than shipped as a target.
Mechanical sweeps excluded
Import and decorator refactors touch hundreds of tests without changing behaviour. Instances attributing more than 20 tests are held out of the gold tier entirely.
Nothing is executed
No test in this corpus has ever been run. test_ids_verified is false on every instance. Treat the ids as a static approximation and validate against your own harness before reporting any result.
Access
No API key, no registration, no payment. Requests to /v1/* are rate limited to 120 per minute per IP. If you want the whole corpus, take the bulk file rather than paging — it is one request instead of twelve thousand.
| Endpoint | Purpose |
|---|---|
POST /v1/instances/fetch | Sample instances, filtered by language, repo, ci_annotated, min_tests |
POST /v1/instances/search | BM25 full-text search across problem statements |
GET /v1/instances/{id} | Retrieve one instance |
GET /v1/schema | Field contract, withheld columns, caveats, licence |
GET /v1/count | Corpus totals and per-language breakdown |
GET /v1/bulk | Redirect to the full 27.5 MB compressed dump |
POST /mcp | MCP server — fetch_instances, search_instances, get_instance, get_corpus_stats |
Machine-readable contracts: OpenAPI · plugin manifest · llms.txt · MCP server.json · Croissant metadata
Quickstart
Search the corpus in one call
curl -s -X POST https://polypatch.lateos.ai/v1/instances/search \
-H 'content-type: application/json' \
-d '{"q": "race condition", "language": "go", "limit": 5}'Or point any MCP client at https://polypatch.lateos.ai/mcp — no auth headers needed.
What you get, what you don't
Every instance omits the gold patch column — the diff that actually fixes the issue. It is the answer key: serving it would contaminate any evaluation built on the corpus, and it accounted for 82% of the bytes. Reconstruct it from provenance.source_url if you genuinely need it.
Every instance carries content_hash (SHA-256 over the canonical task fields) and a source_url pointing at the originating GitHub commit. All 11,611 resolve. Your downstream audit trail inherits ours.
The compilation and the derived columns are MIT. Each instance embeds code from its upstream repository under that repository's own licence — 3,886 repositories, many licences. Check source_url before redistributing or training on any individual instance.
Derived from the SwingBench-data corpus released with SwingArena (arXiv:2505.23932). This repository contributes deduplication, structural validation, repository-disjoint splits, static test attribution, and quality tiering. Please cite the original work.
arXiv:2505.23932