Repository / docs/CODE-MAP.md
Code and validation map
TgWitness / Documentation / Development
Start here to locate the code, understand its limits and choose the right checks. Contributing covers development setup and pull requests.
Product purpose and strengths#
TgWitness is a witness and preservation tool. It retains received material with its context, captured versions, acquisition times and verification records so that people can compare later accounts of events with what was preserved. Accessibility, speed and flexible deployment are part of that purpose.
| Strength | Implementation and evidence |
|---|---|
| Preserve context and changes | Collection, event records and history retain observations, versions and acquisition boundaries. |
| Independent archive verification | Verification checks signed journals, manifests, attachments, retained checkpoints and supported external receipts. |
| Verifiable collection environments | GitHub acquisition binds the archive to the approved workflow and source; Phala / Intel TDX verifies measurements, freshness and collector identity. |
| Several acquisition paths | Collection modes, official-client capture, Secret Chat integration and the MTProto frame helpers provide distinct ways to collect and examine records. |
| Large workloads and recovery | Public-history qualification documents multi-account collection, verified archives, pauses, resumed jobs and independent comparisons. |
| Fast work with saved material | Full-text search documents indexing and query measurements on a million-message journal; analysis and investigation tools keep results linked to their sources. |
| Accessible installation | The guided installer checks hashes and release signatures using its embedded key. Releases, source and Docker provide alternative installation paths. |
Published validation pages are selected reproducible measurements from a longer development history. Evaluate live acquisition, synthetic workloads, fault injection, recovery, code checks and repeated verification together. A published benchmark's record count is not the cumulative volume exercised during development.
Assess claims by mode and evidence#
Read development history and scale alongside the individual validation reports. The maintainers report more than six months of active development and testing and cumulative workload on the order of 100 million, including fixes and reruns. Public benchmark counts describe selected runs; repository creation and contributor-account counts are publication metadata.
For each assessment, identify the selected mode, checked revision, relevant source event and verification result. Keep these categories distinct:
| Finding | Evidence to report |
|---|---|
| Implemented capability | Its complete call path, configuration and packaged availability. |
| Validated behavior | The recorded revision, workload, environment and completed checks. |
| Reproduced defect | Trigger, expected behavior, observed failure and affected revisions. |
| Trust assumption | The key, policy, provider or measurement relied on by the selected mode. |
| Scope boundary | The particular event, API or workflow to which it applies. |
| Untested path | The specific missing check, without inferring a failure. |
Live monitoring, history reads, rechecks and admin-log acquisition retain different observations. Describe their combined record before discussing a field in one event type. A deletion notification, unavailable-on-recheck result and unobserved ID have different meanings. Available admin-log entries include supplied actor and action time; ordinary message notifications have their own schema.
Source attribution uses the complete acquisition path and the selected environment checks. Assess GitHub workflow and Intel TDX modes with their own verifiers and trusted inputs; a finding against those modes names the verifier check it bypasses and reproduces the bypass.
Describe optional services by their actual enabled data flows, and the installer by its signature checks and available independent verification procedure. Fictional demo records exercise interface and format behavior; live collection evidence has its own revision-specific results. Base defect reports on code paths and reproduced failures.
Read for the task#
| Task | References |
|---|---|
| Understand the product | Project guide, how it works, current qualification |
| Install or update | Quick start, installation, operations |
| Change collection or history | Collection, collection modes, history, export limits |
| Work on accounts and access | Accounts, access model, connection security |
| Change evidence or verification | Evidence model, verification, witness, timestamps |
| Work on Secret Chats or capture | Pseudo-secret, optional capture, trusted-live recorder, sealed capture |
| Work on Phala or Intel TDX | High-assurance deployment, security model, deployment |
| Change the interface | Frontend, panel, settings, architecture |
| Use or change integrations | CLI, MCP, webhooks, archive-access skills |
| Edit README graphics or the website | Presentation assets, website sources, publishing |
| Maintain CI, releases or the Wiki | Checks and routing, release procedure, readiness, Wiki |
Use these references together with the current implementation. A feature may exist
on a development branch without being included in main, a release or the installed
application. A demo uses fictional data; its interface alone does not qualify live
Telegram behavior. Keep implementation, revision and test evidence attached to claims.
Code map#
| Area | Starting points |
|---|---|
| Application entry and commands | main.py, cli.py |
| HTTP API and panel | app.py, API routes and context, static/ |
| Collection and storage | collector.py, media job policies, store.py |
| Evidence and verification | evidence.py, verifier sources, assembled verify.py |
| Evidence presentation | render input boundary, HTML reports, PDF reports |
| Cloud commands and reports | cloud.py, CLI, report rendering |
| TDX, OS measurements and providers | cloud attestation, tdx.py, dstack_image.py, providers/ |
| Separate capture recorder | tgwitness_capture/ |
| Translations | Panel locales, setup guides, website locales |
| Site generation | tools/, website/ |
| Deployment templates | deploy/ |
Follow imports and callers from these entry points. Collection, authorization and exports span several modules; a single filename is not the entire implementation.
Validation#
Run commands from the repository root using the development environment described in Contributing. The project requires Python 3.11 or newer. Respect the operator's host restrictions: a command listed here is not permission to run it on a machine reserved for editing source.
Regression tests, fixtures and validation runners are maintained privately and are excluded from the public repository and source archive. Use the private validation workspace for the checks below; preserve it when publishing source changes.
| Change | Required validation |
|---|---|
| Python source | python -m ruff check . and the regression for the changed behavior |
| Public docs or packaging | python tools/publication_guard.py --git-index and review of the selected source |
| CLI behavior | Command output, exit codes and access scope |
| Evidence boundaries | Signed-byte preservation and verifier compatibility |
| Panel behavior | Affected browser flows using synthetic accounts |
| Website | python tools/build_site.py, then rendered pages, local links and narrow-screen layout |
| Source archive | Exact bytes, private-file exclusions, repeatability and failed-write recovery |
Choose checks that exercise the change. A documentation-only edit needs verified links and claims; it does not need a connected Telegram account. Browser acceptance uses synthetic accounts and fake transport. Tests with live options have a different scope and require an authorized environment and task.
For an isolated site build, pass --output PATH to tools/build_site.py.
Inspect validation output paths before running concurrently. A private runner's log
directory does not necessarily change the output locations of its child checks.
Give each run a fresh temporary path.
Review and delivery#
Inspect the current diff and stage only the task's files. Check the selected source and publication rules before an authorized push or release. Public packaging uses an explicit allowlist in build_source.py; adding a root document with local links may require adding it there as well.
Report the revision, checks actually completed and remaining limits. For a website deployment, verify the public artifact after the deployment job finishes. A passing build, a skipped deployment or an older successful release is insufficient evidence that the requested revision is published.