Guides

Extending Lore

Most contributions are small. This guide covers the common ones: teaching discovery a convention, adding a language or framework, widening the search vocabulary, adding a risk pattern, and adding a command or a tool.

Teach discovery a convention

A convention is one small object, a marker, added to a language or framework module. Pick the kind that fits:

Every marker also takes id (unique), glob (which files it looks at, such as **/*.py), rule (the proposed text, one imperative sentence), and optionally exclude_tests, min_ratio and min_applicable to override the defaults for that marker. The thresholds are explained under discover.py.

AbsentMarker(id="py-assert", glob="**/*.py", bad=r"^\s*assert\b",
             rule="Raise exceptions instead of using assert in application code."),

ChoiceMarker(id="django-urls", glob="**/urls.py", options=(
    Option("path()", r"\bpath\(", "Declare URLs with path()."),
    Option("re_path()", r"\bre_path\(", "Declare URLs with re_path().")))),

Write the pattern so it matches the thing precisely: it is a regular expression applied to whole files with the multiline flag. A marker can mis-fire on comments and strings, so test against a small fixture.

Add a language

A new language is one file in discovery/languages/ that exports LANGUAGE. Nothing needs registering: discovery/registry.py imports every module in the folder.

from ..markers import AbsentMarker
from .base import Language

LANGUAGE = Language(
    name="zig",                      # must equal the language name in core/mapper.py LANGS for these extensions
    extensions=("zig",),             # without dots
    markers=(
        AbsentMarker(id="zig-unreachable", glob="**/*.zig", bad=r"\bunreachable\b",
                     rule="Do not use unreachable in application code."),
    ),
)

Add a framework

A framework is a Framework(...) in a module of discovery/frameworks/ that exports a FRAMEWORKS list. Its markers are evaluated only when the framework is detected.

FLASK = Framework(
    name="Flask", languages=("python",),
    deps=("flask",),                               # dependency names (lower case) from any manifest
    markers=(
        AbsentMarker(id="flask-global-state", glob="**/*.py", bad=r"^\s*global\s+\w+",
                     rule="Do not keep request state in module globals."),
    ),
)
FRAMEWORKS = [DJANGO, FLASK]

Detection uses deps (a dependency in a manifest) and/or signatures, regular expressions searched in the source of the listed languages for frameworks that have no manifest entry; signature_files (default 3) is how many files must match. A framework with no marker and no detector fails a test.

Quick one-offs without touching the code

Pass a JSON list to discover.py --markers extra.json; each object has kind set to file, occ, absent or choice, plus the marker's fields. See your own markers.

Test a marker

Add a case to tests/test_discovery.py: write a few files into a temporary project that should trigger the marker and a few that should not, and check the proposed rules. The existing tests in TestFrameworkMarkers and TestLanguageMarkers are the pattern to copy. Two tests protect you: every marker id must be unique, and every regular expression must compile.

Search vocabulary

Rule selection (how rules are chosen) uses plain word lists in core/search.py:

_TOPICS

Areas of work. A word in a topic also matches every other word of that topic, so a task about a "column" finds a rule about "migrations". To add a word, put it in the right topic's string; to add an area, add a key. Keep topics small and distinct: a word in several topics widens matches.

_ALIASES

Short forms that mean something (py → python, db → database).

_STOP

Filler words that never carry meaning.

_SUFFIXES

The endings the rough stemmer removes.

Add a test to tests/test_agent_tools.py (TestSearch) showing a task that now finds a rule it missed.

Risk patterns

core/risk.py holds the patterns that keep a proposed rule from being activated automatically. Each entry in _CHECKS is a short reason and a compiled regular expression. Add one when you see a class of attack text that slips through, and add tests to tests/test_safety.py for two things: that the attack is flagged, and that ordinary conventions ("Never commit secrets", "Do not ignore failing tests") are not. Prefer precise patterns; a false alarm costs one extra confirmation, but a pattern that fires on common rule wording trains people to click through.

Add a command

1. Put the logic in a module in cli/ with a main(argv=None) decorated with @entry (it turns CliError into error: … and exit code 2).
2. Add a root script that only wraps it, like the existing ones (16 lines or fewer):

   #!/usr/bin/env python3
   """Entry point; the implementation (and --help text) lives in cli/mytool.py."""
   import sys
   
   from cli.mytool import main
   
   if __name__ == "__main__":
       sys.exit(main())

3. Add the script's name to ROOT_SCRIPTS in tests/test_architecture.py; that test checks that the set of root scripts is exactly the expected one and that each is a thin wrapper.
4. Build on core. If the web UI should offer it, add a handler in web/api.py that builds the script's argument list.

Add an MCP tool

1. Write a handler in mcpserver/tools.py: it takes the arguments dictionary and returns text. Decide the project with _target or _project_for; a tool that writes rules must use by_name=False so it cannot be aimed at another project by name.
2. Add a Tool(name, description, input_schema, handler) to TOOLS. The description is what the agent reads: keep it true. Include paths in the schema.
3. If the tool needs session state (what was already sent, the activation count), add its name to the own mapping in create_server() and accept a session argument.
4. Test it through the server (create_server().handle(...)) in tests/test_agent_tools.py, and update the tool list in tests/test_mcp.py.
5. Document it in MCP tools.

Before you open a pull request

• Fork the repository, create a branch in your fork, and open the pull request from there.
• Keep it to one change, run the whole suite, and update the docs if behaviour changed.
• The full contributor guide is CONTRIBUTING.md.