Create initial games catalog from contrib/games

[edwardchalstrey1]

This feature request is for...

PyGambit

Requested enahancement or feature

This issue is meant to scope an initial PR of the games library/catalog idea discussed in #623 but not yet to move all tests & documentation/tutorials etc over to loading from the catalog.

To do:

• Create a new folder called /catalog and move the EFG/NFG files from contrib/games there
• Create a new catalog.py which contains the catalog of games.
• Decide whether to structure catalog and contained games as a class with functions, class with subclasses, or dataclasses. You should be able to get a game like g = pygambit.catalog.PrisonersDilemma()
• Should be a way to retrieve/list all games
• Each game needs a name that makes sense according to a consistent naming convention that can easily remain stable
  • Derive class name slugs from the filename for now
  • Restore getting game_type from game_file
  • Restore loading attributes from games - no metadata fields should be added that can already be part of a game object anyway
    • But we still want ability to add metadata e.g. is_perfect_recall
  • Get the description field from the gambit game "comment"
    • it should let you do arbitrary text
    • Citation should be included in comment
  • Decide a way to auto-generated from contrib files
  • Write a script that builds a "database" from contrib/games by updating catalog.py
    • Make it so this script ignores files that already have a class in the catalog, just adding anything not already added
    • Ensure comments are retained
    • Ted and Rahul can then go in and update the "database" for the slug/class, title & comment/description
• Think about whether catalog.py should be refactored to be a proper database, whether a yml/json file or sqlite etc
  • I think a json/yml file would be harder to maintain and look at diffs in PRs, db is overkill
  • Add catalog.yml which is generated by the database building script
  • Make it so we can add whatever custom metadata we want to each class - It could be that we update the Game class to have an attribute which is a dataclass for the catalog metadata
    • Another option could be a "catalog metadata" dataclass that is added to Game objects directly
    • Add catalog metadata to classes
    • Ignore any that have valid_game: false
    • You should be able to filter on all metadata fields, whether they come from the game object or the catalog metadata
• Each game has to have the following attributes:
  • title
  • description
  • citation
  • no. of players
    • should be able to filter on this
  • extensive or normal form
    • should be able to filter on this
• For the initial games (from contrib/games) the functions will simply call read_efg("game.efg") or read_nfg("game.nfg") and return the game object
  • Add a class for all the efg and nfg games
• Add tests
• Ensure catalog functions are properly documented in API reference
  • Make it so that all classes that are generated from catalog.yml have a docstring, which gets populated by game.comment or description field (not metadata) in the catalog entry
• Some of the game functions (or classes) you can pass in parameters to get variants of games
• New games can be added to the catalog as functions that create the game from the python code directly rather than loading from efg/nfg e.g. those currently in tests/games.py
• Ensure that update.py adds games to the rst doc from catalog.py as well as catalog.yml
• Refactor so that metadata extraction from game object to catalog entry is no longer needed and the games() function builds each game when it runs
• Refactor so that catalog description overwrites game description rather than the other way around
• Update games() to use all possible attributes of Game objects when filtering and to use subclass.game to get them
• Make it so the notebook tests don't fail with an ImportError when doing from pygambit import catalog
  • Should we implement a catalog.load() function which generates the classes, so we don't have to do so when importing pygambit to begin with?
• Consider whether to move manually coded files back to their own script or keep in catalog.py
• Ensure catalog_update.py works as expected now catalog location moved
• catalog_update.py should also update Makefile.am OR the latter should load from catalog.yml
• If the filename already exists in the catalog but the class has been renamed, then running catalog_update.py again should update rst
• Make it so that you can provide any custom metadata in catalog.yml with a docstring field that gets used to describe the field in the docs page - can we make each metadata field an optional attribute of CatalogGame with a type and possible values
• Rename contrib/games to catalog_game_files
• Update docs with links to contrib files
• Refactor how the _game func works in coded games such that args are displayed in the docs
  • Why is W_ex1 and similar catalog funcs not getting docs built with _game docstring?
    loaded from (could have an img field in catalog.yml)
• Move the catalog API reference to an easy place to find or link to it and update catalog_update.py
• Test the coded_games and make sure an error is thrown if you try to make one with the _game function not returning a game
• Add a test that validates that all entries in catalog.yml are valid
• Add documentation for how to update the catalog including
  • adding to catalog.yml
  • adding to catalog_games.py
  • how the catalog_update.py script works
  • that you also need to manually update links to that game in any documentation if you rename the file
  • pip install ruamel.yaml
  • GUI -> EFG -> catalog pipeline
  • How to set up a metadata filter, giving a default on CatalogGame
• Update tutorials with clear instructions on how to use the catalog, maybe just in one place rather than two and ensure all examples are using the catalog rather than loading from file
• Ensure game description does not have full docstring
• Can we build the docstring including parts derived from a game object when generating docs with make html?
  • Can we also include game title as part of the catalog game class docstring?

[edwardchalstrey1]

Subsequent PR/issues

• Open a new issue: Can you use instrospection to get game code from the catalog even with games from file?
• Open a new issue:
  • Can we draw all the EFG games in the catalog (and print the NFG tables) in the generated API docs?
  • One approach could be to run draw_tree when building the docs, could be slow
  • Another approach could be to have the catalog_update.py generate the images and put them in a folder that gets read from (implemented and commented out for now) - Need to fix docstring and game description handling in this case
  • Need to make improvements to draw_tree for this to work really well
• Add game files that are not EFG/NFG filetypes e.g. .agg to the catalog
• Add the ability to download EFG/NFG files from the Catalog, similar to the current Sample games page

^ added to #394

[edwardchalstrey1]

TODO 3rd Feb - New priorities for catalog

Refactor so that we no longer rely on creating classes for catalog games and the API is more like:

pygambit.catalog.load("stipped_down_poker")

• Close the current PR Add Catalog of games #735
• Start a new branch
• Keep contrib/games where they are but add a new catalog folder at the top level where items will gradually be moved to - the filenames should be the slug source of truth
• Implement the load function that can simply load the slug name using read_nfg or read_efg based on the extension
  • Figure out a way to load files which are outside pygambit higher in the dir structure of the repo. This may involve some build process copying the games into pygambit e.g. cp -r catalog src/pygambit
• Implement simple version of games() (search) function which outputs the slugs with game titles as a 2 column pandas df.
• Add a catalog docs page which has an rst two column table with the game slug and title
• Created a catalog_update.py script. This should be called after adding a new EFG/NFG file to /catalog. It should call games() and then update the docs page, as well as Makefile.am.
• Add an "Adding to the catalog" section of contributor docs based on this process. Should just involve adding a NFG/EFG to /catalog and then running the update script and making a PR.
• Ensure there are tests for how the catalog works on this branch
• Update all tutorials to load from catalog
• Make sure readthedocs build works with catalog.csv
• Create a separate PR with the change to writer.cc for Ted to review

Issues to open and work on later:

• Straight to PR - Include the change made in Add Catalog of games #735 of Game.comment becoming Game.description
• 1) Implement the option to have games from code with variant functions using random seeds and allow families of games
• 2) Catalog search feature, using both game attributes and metadata
  • Could still use the catalog.yml to get the metadata for a dataclass; the catalog could be refactored into a list of dataclass instances, where they either include a callable function or the file
  • 1 above (coded games issue) should be a pre-requisite
• 3) 3rd party game libraries inclusion:
  • We want Gamut to be a subset of the catalog
  • Can also load OpenSpiel/SageMath games with the same names as part of the catalog?
  • How can we keep external libraries optional dependencies?
  • Could call like pygambit.catalog.load("openspiel_tinyhanabi") which fails if OpenSpiel not installed or better still just have the EFGs from OpenSpiel in Gambit Catalog. Note: We may need to do a PR to OpenSpiel to ensure the EFG output works for all games
• 4) Add an extra field to the pandas df from games() and the table in the docs that includes the Game.description field
• 5) Should we implement a type argument for cases where we have nfg and efg versions of a game such as load("2x2x2", type="nfg"), or are these interoperable and therefore there should only be one saved file for the game?