Python modules

Generated from the backend’s own docstrings, so it cannot drift from the code.

The library

The game library: PGN files on disk plus a SQLite index over them.

Layout (DATA_DIR defaults to ./library, or /data on Railway with a volume):

library/
  library.db                  index + repertoires + settings
  collections/
    my-games/games.pgn        plain PGN, appended to; still valid PGN
    masters/games.pgn

Games stay in ordinary PGN files you can copy elsewhere or open in any other program; the database only remembers where each one starts and ends.

backend.store.slugify(name)

class backend.store.Library(data_dir)

Bases: object

static signature(text)

POOL_SIZE = 4

connect()

release()

Hand this thread’s connection back for the next request to reuse.

close()

Close this thread’s connection and every idle pooled one.

Callers rely on close() letting go of the file — Windows will not delete a library folder while any connection holds it — so the pool goes too. It refills on the next requests.

collections()

collection(ident)

ensure_collection(name, kind='games')

rename_collection(ident, name)

Give a collection a new name, taking its PGN folder with it.

Games record where their text lives as a path under the library, so the folder move and the paths have to travel together or the games stop being readable. If the destination folder is already taken the files stay where they are: the games still read correctly from the old folder, and only new imports land under the new name.

collect_into(name, filters, kind='games')

Put every game matching a search onto one shelf, without copying it.

The master collections are built this way. PGN Mentor ships one archive per player, so a collection for one opening of that player’s is the archive filtered: the games keep living in the collection they were imported into and are linked onto the new shelf, rather than being stored a second time.

collections_for(game_ids)

Every collection each of these games sits in, owner first then links.

link_game(game_id, collection)

Shelve an existing game in another collection. Returns False if it is already there.

unlink_game(game_id, collection)

Take a game off a shelf it was linked onto. The owning collection is not a link.

name_openings(collection=None)

Fill in opening names for games imported before, or without, an Opening tag.

delete_collection(ident, remove_files=False)

add_games(pgn_text, collection='My games', source='import', skip_duplicates=True, kind='games')

Append every game in pgn_text to a collection. Returns a summary dict.

kind only applies when the collection is being created: a collection that already exists keeps whatever kind it was given, so importing into it never moves somebody’s games out of the database behind their back.

game_pgn(game_id)

game(game_id)

SORTS = {'added': 'added_at DESC, id DESC', 'annotator': 'annotator COLLATE NOCASE ASC, id DESC', 'black': 'black COLLATE NOCASE ASC', 'date': 'date DESC, id DESC', 'date_asc': 'date ASC, id ASC', 'eco': 'eco ASC, date DESC', 'elo': 'top_elo DESC', 'event': 'event COLLATE NOCASE ASC, date DESC', 'event_date': "COALESCE(NULLIF(event_date,''), date) DESC, event COLLATE NOCASE ASC, round ASC", 'length': 'ply_count DESC', 'opening': 'opening COLLATE NOCASE ASC, eco ASC', 'result': 'result ASC, date DESC', 'round': 'event COLLATE NOCASE ASC, LENGTH(round) ASC, round ASC', 'white': 'white COLLATE NOCASE ASC'}

search(query=None, collection=None, player=None, white=None, black=None, eco=None, result=None, opening=None, min_elo=None, year=None, sort='date', limit=100, offset=0, event=None, min_length=None, max_length=None, tag=None, added_from=None, added_to=None, position=None, eco_to=None, max_elo=None, outcome=None, player_prefix=None, white_prefix=None, black_prefix=None, event_prefix=None, annotator=None, site=None, round=None, termination=None, annotated=None, date_from=None, date_to=None, source=None, category=None, white_min_elo=None, white_max_elo=None, black_min_elo=None, black_max_elo=None, kind=None, team=None, title=None, fide_id=None, source_title=None, variation=None, event_type=None, event_date_from=None, event_date_to=None)

REFERENCE_DIR = 'reference'

references()

The reference bases attached to this library, and their PGNs’ health.

A base is attached, not imported: its rows carry a path into a file that lives outside any collection folder. So the honest answer includes whether that file is still where the rows say it is — a base whose PGN has moved is a collection of games that cannot be opened, and the reader should be told which.

attachable(minimum_bytes=52428800)

Big PGNs in the library’s reference folder that nothing has attached yet.

This is what lets the app notice a gigabase and offer to use it, instead of making somebody find a file picker for something already sitting in place.

LARGE_LIBRARY = 200000

is_large()

Is this library big enough that a full scan is felt rather than measured?

Read from collection_counts, which the triggers keep exact.

game_count()

Every game in the library, without counting ten million rows to find out.

owned_count(collection_id)

OPENING_SUMMARY_SQL = "\n        INSERT INTO opening_summary (name, eco_from, eco_to, games)\n        SELECT opening,\n               MIN(CASE WHEN eco GLOB '[A-E][0-9][0-9]*' THEN substr(eco,1,3) END),\n               MAX(CASE WHEN eco GLOB '[A-E][0-9][0-9]*' THEN substr(eco,1,3) END),\n               COUNT(*)\n        FROM games WHERE opening IS NOT NULL AND opening <> ''\n        GROUP BY opening"

merge_openings(conn, added)

Fold newly imported games into the opening summary.

A rebuild is a full scan; an import knows exactly which openings it brought and how many of each, so it adds those counts rather than recounting the library. The ECO span widens to cover whatever arrived.

rebuild_opening_summary()

Recount the openings in the library. One full pass, run when games change.

openings(term='', limit=40)

Opening names for the picker, and whether the summary behind them is current.

Never rebuilds inline: the caller is a keystroke, and a rebuild is a full scan. A stale answer with stale: True beside it is worth more than a frozen box.

begin_import(label)

end_import(batch)

import_history()

undo_import(batch)

delete_game(game_id)

Drops the index entry. The PGN text stays in the file until compacted.

replace_game(game_id, pgn_text)

Save an edited/annotated game: appended fresh, index re-pointed at it.

stats(full=False)

The library’s headline numbers.

full adds three whole-table aggregates — commonest players, commonest openings, counts by source. Ranking players means grouping white and black together, which is two rows per game: seventeen million for a library with a reference base attached, and a minute of work for a panel that does not exist. The database page asks for the cheap answer; anything that actually wants the rankings asks for them.

setting(key, value=None)

repertoires()

repertoire(rep_id)

save_repertoire(name, color, data, rep_id=None)

delete_repertoire(rep_id)

cache_explorer(fen, db_name, body=None, max_age=604800)

PGN reading

PGN splitting and header extraction.

The server never needs to understand chess rules — the browser has a full engine for that. All it does here is cut a PGN stream into games, read the tag pairs, and pull out enough of the movetext to index and search on.

backend.pgnutil.split_games(text)

Cut a PGN stream into individual game texts.

backend.pgnutil.headers(game_text)

Tag pairs as a dict.

backend.pgnutil.movetext(game_text)

Everything after the tag pairs.

backend.pgnutil.moves(game_text, limit=None)

Main-line SAN moves, variations skipped.

backend.pgnutil.to_int(value)

backend.pgnutil.normalize_date(value)

PGN dates are ‘YYYY.MM.DD’ with ‘??’ for unknown parts; make them sortable.

backend.pgnutil.speed_of(time_control)

Bucket a PGN TimeControl tag. Unknown or correspondence returns ‘’.

backend.pgnutil.describe(game_text)

Everything the index wants to know about one game.

Importing

Streaming imports of portable chess files. Archives are read, never extracted.

backend.importers.sniff_encoding(sample, more_follows=True)

Pick the encoding for a PGN file from its opening bytes.

backend.importers.decoded(binary, encoding=None)

Wrap a binary stream as text, sniffing the encoding when the stream allows it.

backend.importers.streams(path)

backend.importers.game_stream(stream)

backend.importers.epd_stream(stream)

backend.importers.puzzle_csv(stream)

backend.importers.import_local(library, path, collection)

backend.importers.reference_games(handle, encoding, start_at=0)

Yield (offset, length, text) for every game in an open PGN, without loading it.

Offsets are what make attaching possible, so they are counted in bytes off the file itself rather than inferred from decoded text: a game is looked up later by seeking to exactly this many bytes and reading exactly this many more.

start_at is one of those offsets from an earlier pass, which is what lets a scan of an eight-gigabyte file be picked up where it stopped rather than begun again.

backend.importers.attach_reference(library, path, collection, progress=None, batch_size=20000, resume=False, deadline=None)

Register a reference PGN where it already lies, instead of copying it in.

The library keeps only headers and a byte offset in SQLite and reads move text back from the file on demand, so a multi-gigabyte base does not need to be ingested — it needs to be pointed at. The scan records where each game starts and how long it is; the game list, the analysis board and every export then open the original file at that offset. Nothing is duplicated and nothing is rewritten.

The trade is that the file has to stay where it is. It is stored relative to the library folder when it lives inside one, so moving the whole library is still safe.

Duplicate detection is deliberately skipped. It costs a hash and an index probe per game, and holds every signature seen in memory — affordable for an import of a few thousand games, not for ten million. A reference base is attached once, whole.

A scan of that size is worth being able to stop and pick up. deadline is a clock time to stop cleanly at, and resume carries on from the offset the last pass committed, so the work survives a closed app or a machine that needs its memory back. The result says whether the end of the file was reached.

backend.importers.reference_dependents(library, collection_id)

Collections that shelve games belonging to this base.

A carved collection links to the base’s rows rather than copying them, which is what makes carving free — and what makes detaching destructive: the link is a foreign key that cascades. So the dependents are counted before anything is deleted.

backend.importers.detach_reference(library, collection, force=False)

Forget an attached base. The PGN on disk is never touched.

Collections carved out of the base hold links to its rows, so detaching empties them too. That is a surprise worth refusing by default: the caller is told which collections would be emptied, and has to ask again to go through with it.

backend.importers.import_source(library, source, collection)

backend.importers.chesscom(library, user, collection, maximum=100, perf=None, since=None, until=None)

Import a chess.com account, newest game first.

maximum of 0 means every game the account has. perf narrows to one time control — bullet, blitz, rapid, classical or correspondence — and since/until are ISO dates: whole months outside the range are never fetched, which is what keeps “every rapid game this year” from downloading a decade.

Chess rules

Standard chess rules, 0x88 board, legal SAN and canonical position keys.

No runtime dependencies; used only for library indexing, never for UI latency.

backend.chess.square(name)

backend.chess.name(sq)

backend.chess.color(piece)

class backend.chess.Chess(fen='rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1')

Bases: object

fen()

attacked(target, by)

check(side=None)

pseudo()

push(move)

snapshot()

restore(state)

legal()

san(move, legal=None)

move(text)

key()

perft(depth)

The engine

The bundled Stockfish, spoken to over UCI.

One long-lived process, guarded by a lock, plus a background worker that can annotate a whole game and store the result. The browser no longer needs its own engine when the app runs on the desktop.

backend.engine.default_binary()

Where the bundled engine lives, in the source tree or inside a build.

exception backend.engine.EngineError

Bases: Exception

class backend.engine.Engine(path=None, threads=None, hash_mb=256)

Bases: object

A single UCI process. All public methods are safe to call from threads.

available()

start()

stop()

set_option(name, value)

info()

analyze(fen, movetime=None, depth=None, multipv=1, on_update=None, skill=20)

Blocking analysis of one position. Returns the best lines, best first.

class backend.engine.LiveAnalysis

Bases: object

Dedicated UCI process so batch annotation cannot block live updates.

stop()

start(fen, multipv)

status()

class backend.engine.AnnotationJob(engine, library)

Bases: object

Walks a game’s positions and scores every one of them.

JUDGMENTS = [(300, 'blunder'), (150, 'mistake'), (75, 'inaccuracy')]

status()

start(game_id, positions, movetime=300, depth=None)

positions is [{ply, fen, san}] worked out by the browser’s rules engine.

stop()

classmethod judge(loss)

Machine telemetry

Best-effort machine telemetry to show beside the engine.

Nothing here is required and nothing here is estimated. Core counts come from the OS; heat and power draw are read from whatever the firmware chooses to publish, which on Windows is a coin toss — plenty of desktops expose no thermal zone at all, and a machine on mains power reports no discharge rate because it is not discharging. Anything the machine will not tell us is reported as None so the interface can say “not available” instead of inventing a number.

Probing costs a subprocess on Windows, so readings are cached and refreshed on a background thread: callers always get the last known values immediately.

backend.hardware.cores()

Logical and physical core counts, physical only when psutil can tell us.

class backend.hardware.CpuLoad

Bases: object

Whole-machine CPU load, measured from our own reading of the CPU time counters.

psutil.cpu_percent(interval=None) reports the load since the last call made anywhere in the process, and keeps that “last call” in module-level state. The live-analysis panel polls several times a second across many server threads, so that shared state gets reset out from under us and a poll lands on a zero-length interval — reporting 0% while the engine has every core pinned.

Taking the cpu_times() snapshot ourselves removes the shared state: the window is the one between our own two samples, whoever else is asking. Readings are cached between samples so rapid polling is cheap as well as correct.

MIN_INTERVAL = 0.5

read()

class backend.hardware.Telemetry

Bases: object

Cached sensor readings, refreshed off the request thread.

snapshot()

backend.hardware.system()

Whole-machine load, heat and power, as far as this machine will say.

Repertoire lines from PGN

Turn an opening PGN — a lichess study export, or anything with variations — into repertoire lines.

A lichess repertoire study is a tree: one main line with alternatives branching off it, often many levels deep. The repertoire trainer stores flat lines instead, so this walks the tree and writes down every root-to-leaf path through it.

Two decisions are worth stating plainly. A line is trimmed so it ends on a move by the side the repertoire is for: finishing a drill on the opponent’s reply teaches nothing. And a line that is only the opening part of a longer line is dropped, because drilling it would be drilling the same moves twice.

backend.repertoire.from_pgn(pgn_text, color='w', max_plies=40)

Repertoire lines for one side, plus a report on what was read and skipped.

backend.repertoire.merge(existing, incoming)

Add only the lines a repertoire does not already hold, keeping review history.

Position index and study folders

Position knowledge and portable study folders layered on the PGN library.

class backend.study.Study(library)

Bases: object

index(collection)

position(fen)

pin(body)

folders()

create_folder(body)

delete_folder(folder_id)

Remove a folder and everything nested inside it.

Collections are only unassigned: their PGN files live in the library’s collections directory, not in the study folder. On disk we take back the manifest we wrote and the directories we created, but a directory the user has put their own files in is left alone and reported back.

shelve(collection_id, folder_name, category='Reference')

File a collection under a study folder, making the folder if it is missing.

A base that only exists in a drop-down is a base you forget you have. Study folders are where this library keeps the things it is working from, so an attached base is filed there on arrival like anything else.

assign(body)

write_manifest(folder_id)

categorize_folder(ident, category)

The opening tree

One player’s openings, judged by how they actually went.

A reference database answers “what is played here”. This answers a different and often more useful question: when this player played it, what happened to them. That player may be the reader, and may just as easily be Fischer or whoever a collection was imported for. Every number is from the named player’s side of the board, so a 38% score is 38% for them, not for White.

It reads the position index, so a collection has to be indexed before it has anything to say. Two things fall out of that design:

• Transpositions come for free. Positions are stored under a transposition key, so a line reached by a different move order is the same node.

• Depth costs nothing at query time. The index already holds every position of every game, so walking twenty moves deep is the same query as walking one.

The weakest-line scan is the part worth explaining. Ranking by score alone surfaces a 0% line played twice; ranking by volume surfaces the main line, which is no news. What a reader wants is where the points actually went, so lines are ranked by points dropped — games multiplied by the shortfall against an even score — and a line has to clear a minimum number of games before it is listed at all.

class backend.openingtree.Filters(player=None, color=None, collection=None, speed=None, since=None, until=None, min_opponent_elo=None, max_opponent_elo=None, rated=None, kind=None)

Bases: object

The set of games a report is about, and the SQL to select them.

property perspective

side_sql()

An expression naming the side the report is written from, and its params.

opponent_elo_sql()

where()

Clauses and params common to every query in this module.

backend.openingtree.position(library, fen, filters)

What happened from this position, move by move, from the named player’s side.

backend.openingtree.trend(rows, buckets=None)

Score by year over rows already read. Kept for callers working in memory.

backend.openingtree.weakest(library, filters, min_games=3, limit=15, max_ply=24)

The lines that cost the most points, ranked by points dropped rather than by score.

A single 0% game is noise; twelve games at 25% is a hole in the repertoire. Each entry names the line by replaying the game that reached it, so the answer reads as moves rather than as a hash.

backend.openingtree.players(library, collection=None, limit=40)

Who appears most often in a collection, so the report can be pointed at someone.

Automatic imports

Keep a lichess account’s games flowing into the library as they are played.

A background thread asks lichess, every so often, for the games played since it last looked. That is deliberately a poll rather than a live stream: streaming an account’s games needs board scopes this app has no business holding, and a game is worth analysing a few minutes after it ends just as much as a few seconds after.

Three decisions worth stating, because each one is a trap avoided:

• Windows overlap. Each poll asks for games since a little before the last one finished, not since exactly then. A game that ended during the gap is caught, and the repeat is free because duplicates are detected by lichess game id.

• Enabling does not backfill. Switching this on starts from now. Pulling in a decade of blitz because somebody ticked a box is not a welcome surprise; the manual importer is still there for history, and “Import now” fetches a window on request.

• Every run is an undoable batch, exactly like an import you started by hand, so a run that brings in something unwanted can be taken back.

class backend.autoimport.AutoImport(library, token, study=None)

Bases: object

Polls lichess for new games and files them in the library.

settings()

save(changes)

Validate and store the watcher’s settings, then start or stop it to match.

apply(values=None)

start()

stop(timeout=2.0)

trigger()

Ask for a check now, whether or not the watcher is running.

run_once(values=None)

Fetch and file whatever has been played since the last look.

save_cursor(values, stamp)

status()

lichess client

Server-side lichess client.

Doing the network work here rather than in the browser fixes the rate limiting: one process, one queue, one request at a time, with a shared cooldown. The browser can click “load” as often as it likes.

exception backend.lichess.RateLimited(retry_after)

Bases: Exception

class backend.lichess.Throttle(min_gap=1.0)

Bases: object

One request at a time, with a minimum gap and a cooldown after a 429.

wait()

penalize(seconds)

cooldown_left()

backend.lichess.user_games(user, max_games=100, color=None, rated=None, perf=None, since=None, until=None, token=None)

backend.lichess.import_all_user_games(library, user, collection, token=None, batch_size=200, progress=None, **selection)

Stream a whole lichess account into the library, however many games that is.

Asking for every game means not knowing how many are coming, which rules out holding the answer in memory: an active account is hundreds of megabytes of PGN. So the response is read as it arrives and written in batches, and the caller is told the running count rather than made to wait in silence.

selection takes the same narrowing arguments as user_games — color, rated, perf, since, until — so “every rated blitz game since 2023” is one call.

backend.lichess.explorer(db='masters', play=None, fen=None, moves=12, top_games=8, extra=None, token=None)

backend.lichess.masters_pgn(game_id, token=None)

class backend.lichess.MastersCrawler(library)

Bases: object

Walks the masters opening explorer and files away the games it names.

Breadth-first from the start position: every position it visits contributes its top games, and every move played often enough becomes a new position to visit. Slow by design (the explorer is a free service) but it can be left running, and it picks up where it left off.

status()

start(max_depth=8, max_positions=400, min_games=200, top_games=8, collection='Masters', opening_moves=None)

stop()

backend.lichess.import_dump(library, path, collection='Masters', limit=None, batch_size=500, progress=None)

Import a downloaded lichess PGN dump (plain .pgn, or .pgn.zst with zstandard).

Streamed in batches so a multi-gigabyte file never has to fit in memory.

backend.lichess.token_scopes(token)

What a personal access token is allowed to do, straight from lichess.

backend.lichess.account(token)

The signed-in account, plus the scopes the token carries.

backend.lichess.studies(username, token=None)

Every study lichess will show this token, newest first. Private ones need study:read.

backend.lichess.study_pgn(study_id, token=None, comments=True, variations=True)

One study exported as PGN, every chapter in one file.

The API surface

JSON API over the library.

Every handler returns (status, payload). Payloads are dicts (sent as JSON) or a (bytes, content_type) tuple for raw downloads.

exception backend.api.ApiError(message, status=400)

Bases: Exception

class backend.api.Api(data_dir)

Bases: object

handle(method, path, query, body)

OPENING_SUMMARY_INLINE = 50000

SECRET_SETTINGS = ('lichess_token',)