# RFP-001 — py-peer Hardening & Modernization (3-Phase Program)

**Issued by:** Universal Connectivity
**Repository:** `github.com/sumanjeet0012/universal-connectivity` — branch `py-peer-development`
**Version:** 3.0
**Date:** 2026-08-17
**Proposal deadline:** TBD (30 days from issue)
**Contact:** repo maintainers via issues/PRs on `py-peer-development`

---

## 1. Background

`py-peer` is the Python peer of the Universal Connectivity application: a
decentralized chat on [py-libp2p](https://github.com/libp2p/py-libp2p)
(GossipSub pub/sub), with Kivy/Textual/CLI UIs, a Tornado REST + WebSocket API,
a React frontend, and Bitswap-based file sharing.

This RFP solicits proposals for a **3-phase modernization program** covering:
transport diversity (TCP-only today), file sharing on
[py-ipfs-lite](https://pypi.org/project/py-ipfs-lite/) (local mirror:
`ipfs-lite/py-ipfs-lite`, v0.1.1, Apache-2.0/MIT) with persistence and
Helia/Kubo interop, identity/key management, observability (Prometheus +
Grafana), NAT traversal, and engineering hygiene (CI, logging, graceful
shutdown, upstream dependency).

Phase dependency rationale: Phase 1 moves the foundation to upstream, adds
multi-transport support (WS/QUIC ship with upstream py-libp2p), and introduces
identity/lifecycle fixes with CI gates; Phase 2 swaps the file-sharing stack
(consuming Phase 1's upstream migration); Phase 3 adds NAT traversal, bootstrap
privacy, and observability on top of the stabilized system.

---

## 2. Selected Scope — Traceability

| # | Selected item (as issued) | Phase |
|---|---------------------------|-------|
| 1 | TCP-only → all available transports (WS, QUIC, …) | 1 |
| 2 | Direct Bitswap → py-ipfs-lite | 2 |
| 3 | Interoperable with Helia, Kubo, other clients | 2 |
| 4 | Bitswap memory store → local file store | 2 |
| 5 | Identity: drop shared default seed; random keypair; `--key-file` PKCS8 PEM; warn/refuse duplicate seeds | 1 |
| 6 | Prometheus + Grafana dashboard (health & status) | 3 |
| 7 | Files survive restart (persistent blockstore); pin semantics | 2 |
| 8 | Circuit Relay v2 + AutoNAT / hole-punching (stable upstream, else relay-stop + documented limitation) | 3 |
| 9 | Configurable bootstrap list; default OFF for public network; `--offline/private` mode | 3 |
| 10 | Private fork pin → upstream py-libp2p PyPI (≥0.7) | 1 |
| 11 | Graceful shutdown (cancel nursery; close host/pubsub/dht/lite-peer; fix cross-thread stop) | 1 |
| 12 | Structured logging; bound/remove `system_messages.txt`; core vs optional extras | 1 |
| 13 | GitHub Actions CI: ruff, mypy, pytest (unit + API contract; suites land from Phase 2) | 1 |
| 14 | Docs: README + API_REFERENCE updates; SECURITY.md threat model | 3 |
| 15 | WS-5 tests/acceptance (see per-phase breakdown) | 2/3 |

> **Maintainer note:** the security regression suite (§15.1–15.2, formerly
> Phase 1 item 1.6) is **deferred** — test-writing while the file-sharing stack
> is swapped in Phase 2 is not cost-effective. Re-introduce it as a follow-up
> phase after py-ipfs-lite integration stabilizes (see §6.K).

---

## 3. Phase 1 — Foundation, Identity & Transports

Goal: an upstream-based core with correct identity, clean lifecycle,
**multi-transport support**, and CI gates that protect everything that follows.
No test-writing in this phase — the file-sharing stack swap in Phase 2 will
break internal tests; the substantive test suites land with Phase 2 instead.

| ID | Deliverable | Acceptance criteria |
|----|-------------|---------------------|
| 1.1 | **Upstream migration** — replace `pyproject.toml` fork pin (`git+https://github.com/sumanjeet0012/py-libp2p.git@6d94f03…`) with an upstream PyPI release (`libp2p>=0.7`); absorb API drift in identify, pubsub, host, and drop removed bitswap APIs | Reproducible install from PyPI (`uv lock` clean); full suite green; no fork references remain |
| 1.2 | **Identity & keys** — remove `DEFAULT_SEED="py-peer"`; random keypair when no `--seed`; `--key-file` (PKCS8 PEM) persist/restore; refuse/warn on same-seed reuse; seed-carrying peer ID migration note | Two default nodes never share a peer ID; key file round-trip yields identical peer ID; warning emitted on duplicate seed; unit tests |
| 1.3 | **Graceful shutdown** — cancel nursery, close host/pubsub/gossipsub/DHT cleanly; replace fire-and-forget cross-thread stop in `api/service.py` with a safe signal; ensure all janus queues drained/closed | No dangling tasks/threads on shutdown; exit code 0; repeated start-stop cycles (10×) stable |
| 1.4 | **Logging & packaging** — structured logging; bound or remove `system_messages.txt` (rotate/bounded); split deps into core vs extras (`ui-kivy`, `ui-textual`, `rag`, `api`) | `pip install py-peer` minimal footprint; log format machine-parseable; no unbounded log file growth |
| 1.5 | **CI** — GitHub Actions: `ruff`, `mypy`, `pytest` (unit + API contract — substantive suites land in Phase 2 after the stack swap); matrix Python 3.12 | CI green on every PR against `py-peer-development`; ruff/mypy gates enforced from Phase 1; coverage report published from Phase 2 |
| 1.6 | **Multi-transport** — enable WS + QUIC (and any other stable transports in upstream py-libp2p) alongside TCP; transport selection/priority config; multiaddrs exported to UI/API (transports come with the 1.1 upstream migration) | `--transport tcp,quic,ws` flag; node announces all selected transports; cross-transport dial smoke-tested in local dev (automated tests deferred to Phase 2 CI) |
| 1.7 | Docs update part 1 (config/flags/queries changed in Phase 1) | README/CLI reference accurate |

## 4. Phase 2 — File Sharing & Interop (py-ipfs-lite)

Goal: replace the raw Bitswap/MerkleDag path with py-ipfs-lite so files are
persistent, provider-routed, and interoperable with Kubo/Helia.

| ID | Deliverable | Acceptance criteria |
|----|-------------|---------------------|
| 2.1 | **py-ipfs-lite integration** — replace `BitswapClient`/`MemoryBlockStore`/`MerkleDag` with `py_ipfs_lite.Peer` sharing the host/DHT where sound; expose `add_file`/`get_file` | Existing `/api/v1/files/*` contract works against the new stack; no legacy bitswap imports remain |
| 2.2 | **Persistent blockstore (§ items 4+7)** — disk-backed store for blocks (configurable path); **pin semantics** for shared files; pin-aware GC on roadmap; blocks survive restart | Restart node → previously shared CID retrievable from same node; pin list API (`/api/v1/files/pins`); GC test |
| 2.3 | **Standard CIDs** — CIDv1 base32 end-to-end; keep a hex-CID acceptance shim for one release | File message CIDs parse with standard CID tooling; Kubo `ipfs get <cid>` succeeds |
| 2.4 | **Provider routing** — DHT reprovide + optional IPNI (`cid.contact`) registration so CIDs resolve network-wide; downloader sessions/fetch without direct-only dependency | Fetch succeeds via DHT/IPNI path (direct-neighbour not required); reprovide interval configurable |
| 2.5 | **Helia & Kubo interop** — unify protocol, CID, and UnixFS layouts so browsers (Helia) and Kubo participate in add/get with py-peer | Kubo ↔ py-peer and Helia ↔ py-peer file exchange tests pass both directions (see 2.6) |
| 2.6 | **Integration tests (¶15.3–15.4)** — two-node LAN chat + file share; 100 MB file over Bitswap via py-ipfs-lite; restart-persistence test (blocks + keys); Kubo interop; Helia (js) smoke | All pass in CI; 100 MB round-trip under documented time budget; Kubo/Helia run as CI containers |
| 2.7 | API/WS event contract updates (`file_cid` as CID string, back-compat fields) | React frontend and API ref work without change beyond documented fields |

## 5. Phase 3 — NAT Traversal & Observability

Goal: NAT traversal, privacy-first defaults, and full health visibility on top
of the Phase 1 transports.

| ID | Deliverable | Acceptance criteria |
|----|-------------|---------------------|
| 3.1 | **NAT traversal** — Circuit Relay v2 + AutoNAT / hole-punching where stable upstream; otherwise relay-stop configuration with explicit documented limitation | NAT'd peers discover+chat via relay (or documented limitation + config to run a relay node); relay addresses published |
| 3.2 | **Bootstrap & privacy** — `--bootstrap <file|url|csv>`; default OFF for the global public network; `--offline/private` mode (no DHT random walk, no external bootstrap) | Zero external traffic by default; explicit opt-in documented; bootstrap reload without restart |
| 3.3 | **Prometheus metrics** — export node/pubsub/DHT/transport/file-sharing metrics (reuse py-ipfs-lite's built-in metric instrumentation where possible); `/metrics` endpoint | Metrics endpoint verified; counters/gauges for peers, mesh size, blocks, throughput, uptime |
| 3.4 | **Grafana dashboard** — dashboards-as-code (JSON provisioning) with docker-compose stack; panels for health, connectivity, DHT, pub/sub, bitswap traffic, memory | `docker compose up` serves dashboards with live data; docs screenshot; importable JSON in repo |
| 3.5 | **Performance sanity (¶15.5)** — 20-peer throughput test; bounded-memory soak test (`pytest -m memory`); message rate/flake metrics | Throughput numbers documented; no unbounded `topic_messages`/cache growth over scripted long session |
| 3.6 | **Docs consolidation (¶14)** — README + `API_REFERENCE.md` for security/config/transports/file-share; `SECURITY.md` threat model | Threat model covers the new transports, API surface, file sharing, key handling |

---