Free Supervision from Video Games

Philipp Krähenbühl

[paper] [code]


The data provided is for research and educational use only. Commercial use is prohibited. If you use the data, please buy the game(s).

About the data

Both train and test data are split into roughly 30s continuous clips. For each frame (recorded or not), a *_state.json file contains from basic information about the frame (including the player position, heading, control signal, weather, …). We record all image modalities at 6 FPS. All modalities are stored as compressed images (png or webp):

Code to read those image will be released soon.

Overfitting to the test data

Please don’t train on the test data! Out of scientific curiosity I collected an additional test set, slated to be release in 2021 (or whenever the dataset becomes irrelevant). I hope to benchmark the top methods on that heldout dataset at that time. Feel free to overfit as you see fit…