Add a 'Projects using soulseek-rs-lib' section to the README and a
matching section on the website community page, starting with seakarr.
Both point new projects at issue #12 to reserve a client minor version.
Document the minor-version registry on ClientVersion: within the
soulseek-rs major (176), minor 1 stays the library default for
unidentified consumers and minor 2 is now the reference CLI/TUI client.
Downstream clients reserve their own minor via issue #12.
The listener looked the download up immediately after the PeerInit(F),
but the 4-byte transfer token routinely arrives in a separate TCP
segment, so every firewalled peer download failed with "Download info
missing". Wait for the token within the handshake deadline and give up
on the connection when it never arrives.
Also quiets the dispatcher's unhandled-message-code log to debug; the
server routinely sends codes the library doesn't implement.
Fixes#41
In results filter mode, Up/Down move the selection over the filtered
list and Enter confirms the filter (keeps the query, returns to normal
results keys). Footer hints updated.
The registry stored a registration Instant only to pick the eviction
victim, but the actor id is already monotonic per registration and
refreshed at the same moments, so it orders identically. The
register-refuse-fail sequence was also copy-pasted between the ops
loop and the listener; both now call one register_peer_or_fail.
Review of the capacity work surfaced three real defects and two gaps:
- A peer dribbling bytes re-armed the 10s handshake read timeout on
every read and held its accept permit forever; 128 such sockets
wedged the accept loop for good. The handshake now has one deadline
across all reads, plus a 4 KiB cap on buffered handshake bytes.
- A capacity refusal was logged and swallowed, leaving the refused
peer's queued downloads stuck until the caller's own timeout. Both
registration paths now fail those downloads so the caller gets a
verdict, and the listener only announces PeerConnected when the
registration actually happened.
- The transfer-claim was undone by the worker thread writing Queued
back over it; that write is gone, so a replayed TransferResponse
can no longer start a second writer on the same .part file.
- Offered uploads and queue waiters joined the eviction-protected
set, and the browse protection window matches the minutes a real
share listing takes rather than one.
Regression tests: the broker-timeout sweep failing a queued download
end to end, the replay guard observed over a real socket, and the
handshake deadline against a dribbling and an oversized peer.
One popular search produced a ConnectToPeer for every responder, and
each one cost a 2 MiB wrapper thread, an unbounded 256 KiB actor, a
listener thread per inbound socket, and on failure a 2 MiB thread that
slept for 20 seconds. A container with a pids limit died on one album
search (issues #32, #33).
Four bounds, no behavior change for a quiet session:
- P-type registrations run inline on the ops loop; only F transfers
keep a worker thread. The dial already ran on the actor thread.
- The broker timeout is a deadline swept by the ops loop instead of a
sleeping thread per failed dial.
- The registry caps live actors (default 512, Client::set_max_peers).
At capacity the oldest peer outside the caller's protected set
(live downloads, uploads, pending serves, queued messages, browses
in flight) is evicted; if none qualify the registration is refused
before any thread is spawned.
- Concurrent listener handshakes are bounded by a 128-permit
semaphore released once the peer-init arrives, so slow handshakes
cannot multiply threads while transfers stay unthrottled.
A replayed TransferResponse for an InProgress or Completed token is
dropped instead of starting a second transfer thread over the same
.part file.
Fixes#32Fixes#33
Peer message code 50 had no handler, so the dispatcher dropped it and a
download refused by the peer stayed Queued until the caller's timeout.
The new handler reads the filename and reason and routes the denial into
the same failure path as UploadFailed. A reason of "Queued" is status,
not a failure: some clients send it for a download that is still held in
their queue, and failing it would kill healthy downloads after a
reconnect, so that one is only logged.
Fixes#29
The code 46 handler parsed the filename and only logged it, so a queued
download whose peer reported an upload failure never received
DownloadStatus::Failed and hung until the caller's own timeout.
The handler now sends PeerMessage::UploadFailed through the dispatcher
and the actor stamps the peer's username on it before forwarding
ClientOperation::UploadFailed into the existing failure path. The
variant's username field stays for API compatibility; the wire message
cannot name anyone but the connected peer, so the actor supplies it.
Fixes#30
A failed outbound dial left PeerActor in Connecting, so
check_connection_status re-fired the 20-second timeout error on every
100ms tick until the actor was stopped. ServerActor had the same bug:
its disconnect paths only cleared state when already Connected.
Reset connection_state to Disconnected in all four disconnect paths.
Fixes#31
Counting cheaply was not enough: on the live network peers answer a
popular query for minutes after the window closes, and nothing ever
stopped the pile-up. One 'aphex twin' search grew to 867,990 files and
2.1 GB of daemon memory, and a single search.results reply serialized
134 MB of JSON over 104 seconds - an attached TUI that touched the
search froze for minutes, which read as the whole application being
forever slow.
A search now stops collecting at 500 responses or 10,000 files,
whichever comes first, the same shortlist stance other clients take: a
result set is something to pick from, not an archive of everyone who
answered. The same search now plateaus at ~10k files, a 1.55 MB reply
in 23 ms, and 231 MB of daemon memory; the attached TUI idles at zero
CPU with the search open.
The peer listener died with "Too many open files (os error 24)" on any
long-lived non-daemon session: only the daemon raised RLIMIT_NOFILE, and
peer sockets were unbounded — no idle reaping, no deadlines on accepted
transfer streams, and one peer could hold two connections under split
registry identities.
- raise the open-file limit at the top of run(), for every entry point,
and warn from Client::connect when the soft limit is below 1024
- reap peer control connections idle for five minutes; a clean
disconnect no longer discards the peer's queued uploads
- give listener-supplied download streams and upload serving the same
read/write deadlines dialled sockets already had
- register inbound peers under their plain username so the registry
holds one connection per peer, and announce them like pierced ones
- hold the registry lock across actor spawn + insert and surface
thread-spawn failure, so an instantly-dead peer cannot leave a
zombie entry
A CLI one-off search leaves its result set in the daemon until someone
forgets it, and an attached TUI made that permanent state expensive:
every 100ms tick asked for the search list twice, and each answer
deep-cloned every result of every search under the context lock just to
sum file counts - stalling the thread that stores incoming results.
Worse, any active or selected search pulled its entire result set over
the socket again each tick, megabytes of JSON repeating what the window
already held.
search.list is now backed by search_file_counts, a walk of the cache
instead of a copy of it; the TUI syncs once per tick and re-fetches a
result set only when the session's file count differs from what it
holds. get_search_results_count reads the same summary, so get --pick
first against a daemon polls a count instead of the full set, and both
session kinds now count files where the local one counted peer
responses.
docs/openrpc.json lived at the repository root, outside the soulseek-rs
package, so the include_str! in proto compiled from a checkout but not
from the published tarball - cargo publish failed verification and the
v13.0.0 release stopped after publishing the lib. The document moves
into the crate's own docs/, where packaging can see it; the generator
test and every link follow.
The daemon-mode pass repeats the whole CLI suite single-threaded at
about eight minutes per pass, and the job has been cancelled at the
15-minute mark on every master push since it landed - which is what
kept the v13.0.0 release gate shut.
The serving process built its arguments by hand, so in daemon mode
--shared-dir stayed on the attached run, where shares belong to the
daemon and the flag is ignored. The daemon fell back to sharing its
default folder - which on a developer machine contains the probe files
earlier buggy runs deposited, so the test only passed by finding
served.bin in leftovers. args_with routes the share to the process
whose session does the sharing in either mode.
resolve_download_path treated "not a directory" as "names a file" and
fell back to the parent, so a directory that simply did not exist yet
redirected the transfer one level up. The CLI and TUI create the folder
before downloading and never hit it; a daemon does not, so its transfers
landed in ~/Downloads while daemon.status - and the paths download and
get report - named ~/Downloads/Soulseek. Green on any machine where the
folder already exists, which is how it survived to CI.
Only an existing file diverts to the parent now. PartFile::open already
creates the directory it streams into.
curl -fsSL https://re-invention.nl/soulseek-rs/install.sh | sh replaces the
brew one-liner as the front-page command: it works on Linux without Homebrew
by downloading the latest release archive for the platform, verifying its
sha256 and installing to /usr/local/bin or ~/.local/bin. Windows still gets
pointed at the msvc zip. scripts/test-install.sh covers target detection,
the brew fast path, checksum failure and real installs over curl and wget.
The feature list said the same things at twice the length, and Project
Goals / Project Structure were two headings for one paragraph.
SKILL.md argued against throwaway accounts on cost grounds. The reason
that holds is standing: the agent arrives under the user's name, peers
judge that name, and inventing a second one throws it away. Exit 7 now
points at starting a daemon rather than at --username.
The docs page carried three daemon sections in a row, each with its own
numbered head, so the one story the daemon tells read as three. They are now
sub-heads under 5 · the daemon. The agents section is gone, first run points
at /install instead of assuming Homebrew, and the layout section shows the
demo next to the pane descriptions rather than describing boxes nobody sees.
The footer no longer signs the project with a personal name, and the TUI
demo no longer shows a home directory that only exists on one machine.
Community picks up the sharing etiquette cards and SeeleSeek; install and
home lose a few em-dashes.
info.version carried the crate version, so release-plz bumping Cargo.toml
left docs/openrpc.json a version behind and failed document_is_current on
every platform. With the CI gate in front of publishing, that made every
release PR unmergeable. The protocol version is what auth negotiates and
what the contract actually describes.
A connection that has joined a room receives event notifications on the
same socket as its replies, so a notification landing between a request
and its answer was read as the answer. Only replies carry an id.
An audit of the README and the site against the implementation found 38
places the docs were missing, stale, or wrong about daemon mode. Two were
outright false: the community page still answered "want a queue that
outlives the command?" with "slskd has that", and the home page promised
"no daemon, no API to stand up" fifty lines before the section that
describes both.
The download-box walkthrough now tells the truth about its sharp edges: a
pushed folder or share list is live-only on a remote daemon, whose own
config.toml wins at restart; `shares add` validates the path on the
machine you type it on; and a `get` that reaches its `--timeout` cancels
the daemon-side transfer. Its shell example also runs as written now — the
old block put a second command after a foreground daemon that never
returns.
The reference material catches up with the feature: `daemon` and its
subcommands join the command list, the config table gains the
daemon/daemon_token/--no-daemon rows, nine settings become eleven,
`daemon status` and `daemon token` get documented record shapes, exit 3
covers daemons, `queued` joins the serve statuses, the credential-free
count goes 13 to 19, and the action-command count 4 to 6.
And the surprises are written down rather than discovered: the daemon does
not sweep the wishlist on its own; Windows has no socket and so no
auto-attach; the attached TUI's settings form edits the daemon's folders
and rewrites this machine's config.toml; `serve` monitors a daemon's
uploads instead of starting a second server; `whoami` reports the
daemon's world, with --no-daemon as the way out. The OpenRPC section names
the six event methods and the session-wide download.set_dir rule.
Site typechecks, lints, and builds through prerender; every claim was
verified against the cited code before it was written.
Seven region audits, every finding adversarially verified against the
whole workspace before being applied; 80 landed and the two that failed
in application were refuted honestly (ParseError has a from_str_radix
producer the audit missed; NatPmpError's Display is the only reader of
its diagnostic payloads).
The bigger cuts:
- The NewPeer/PierceFireWall cluster: an operation variant nothing ever
sent, its 77-line arm, a protocol-wrong Client::pierce_firewall that
wrote a peer-init frame down the server socket, and the struct only
they used. The live paths in peer/listen.rs are untouched.
- The ops loop reads `for operation in reader` instead of hand-rolling
recv/break, de-indenting the whole dispatch body.
- Dead lib surface: Message::print_hex and four unused accessors,
UploadFailed, registry and actor getters, build_watch_user,
strip_ansi_codes, delegating shims on ClientContext. Removing public
items makes this a semver major, which the release pipeline enforces.
- Never-used flexibility: the state-migration framework that never held
a migration, border-type helpers that all answered Rounded, the TUI's
max_concurrent_downloads field nothing read.
- Convergence: six poll loops into poll_until, three wrap-around
navigation blocks into cycle, hit tests into Rect::contains, char
walkers into floor/ceil_char_boundary, five Download literals into
one helper, the suites' shared scaffolding into tests/common.
Deliberately not taken: swapping the hand-rolled zlib inflate and MD5
for crates (they replace wire-facing machinery whose tolerance of
real-world peer quirks no local test exercises) and pulling chrono into
the lib for a log timestamp. Recorded for their own decision.
Net -1002 lines. Both e2e modes stay green: 94 CLI tests each way, 16
protocol, 33 lib, unit suites, clippy pedantic clean, aislop 100/100
with the four scanner false positives suppressed inline where they
fire.
The full-surface audit found the settings story half-true: `config set
download_dir` and the TUI's settings form edited a file a running daemon
never rereads, while the attached TUI displayed the laptop's path and
discarded the edit entirely.
- `download.set_dir`: a session-wide move, published in the contract;
the per-transfer destination stays refused. `daemon.status` reports
it and the next transfer lands there.
- `config set download_dir` and `config set shared_dirs` push to a
live daemon the way `shares add` does, through one shared helper;
the socket probe is paid only by the keys a daemon owns.
- The attached TUI seeds its settings form from the daemon's status,
pushes both folders over the socket as typed, reports a partial
refusal as exactly that, and creates no client-side directories. It
reads the daemon token from the resolved config — with the token only
in config.toml it could not attach over TCP — and settings
persistence honours --config instead of hard-coding the default path.
- `search.wishlist` records a start time, so an attached window
renders it as running rather than instantly finished.
The rest is the coverage the audit demanded: CLI-level token auth
(wrong, missing, and valid over TCP), `daemon status|stop` with
nothing running, --sort, --limit, --pick first, --password-cmd,
--log-file, --upload-slots, the config-named download folder in both
modes, and headless settings-form tests attached and local. Every
guard was checked by disabling what it protects. Both e2e modes run
green: 94 CLI tests each way, 16 protocol, 285 unit, 34 lib.
Leading with `get` was the wrong trade. It ranks on free slot, bitrate and peer
speed, which are facts about the transfer and say nothing about whether the
file is the track that was asked for. On a setlist of remixes, dubs and white
labels it does not miss, it succeeds with the wrong one: exit 0, a file on
disk, and nobody finds out until it plays. A miss can be retried; a confident
wrong answer cannot. The previous revision optimised the measurable cost and
paid for it in the unmeasurable one.
The token problem it was solving is real, but the answer is to narrow before
reading rather than not to look. A search writes its records to a file; search
flags and jq cut three hundred down to eight without any of them entering
context; the eight are read and judged. `get` keeps a scoped mention for when
the user has named nothing in particular.
Adds what "the right one" means, since that is now the load-bearing step: fold
both sides, require every distinctive word, reject live/radio edit/snippet/dj
mix, prefer lossless, and above all treat the version as the thing you match on
rather than the thing you dropped. The ladder drops "(Kalabrese Remix)" to find
anything at all, so the match must then require "kalabrese" or it takes the
original and reports success.
Checked against agents given only this file on a remix-heavy setlist: 2/2
refuse `get`, narrow before reading, require the remixer's name in the chosen
path, and report a track as unavailable rather than silently taking the
original when only that exists.
Feedback from a real 32-track run: 70 agents, 2.48M tokens, 18.7 minutes, most
of it avoidable. Four things, three adopted as reported and one corrected.
The skill mandated a handoff it gave no mechanism for. Subagents had to report
a pick, while the rules said remote paths are backslash-separated, may contain
anything, and must be passed back byte-identical. A path does not survive a
subagent's return, a coordinator's JSON and a child's argv, so every agent
re-derived a workaround from prose. The idiom is now written down: append every
rung to one ndjson file, select the pick by line number, feed it to `download
--stdin`. The bytes never leave the file, and a subagent reports a file and a
line rather than a path.
`get` is now the default and subagents the exception. It searches, ranks and
downloads in one command, so results never enter any context, which is where
the tokens went. The section says when to escalate: a named mix, an ambiguous
title, four dead rungs, or several runs landing on the same peer.
Browsing triggers on any artist appearing more than once, not only on a whole
album. In the reported set that was 14 of 32 tracks across six artists.
The one correction: "rung 2 empty means rungs 3 and 4 are ceremony" is not
safe, because rung 3 folds diacritics and punctuation and that is often exactly
why rung 2 found nothing. The general rules are that a rung producing the same
string as the last is not a rung, and that no results at all means the tokens
are wrong rather than too many, so jump to the widest rung instead of stepping.
Streaming picks into transfers is untouched; the report singled it out as
working.
Checked against agents given only this file on the same shape of set: both lead
with `get`, spawn 3 and 6 subagents rather than 32, keep result sets out of
context, browse the repeated artists, and hand paths over as a file and a line
number.
The clearance step read as a barrier. "Only you can see every pick" invited
collecting all of them before releasing any, so a forty track set moved no
bytes until the last search finished, and one white label nobody is sharing
held up thirty-nine tracks that people are.
Nothing about the safety needed a barrier. Answering a single pick takes two
running counts, transfers to that peer and transfers overall, so picks are now
cleared as they arrive and the first file transfers while the last track is
still on its second rung. Says so explicitly, because the previous wording did
not have to be read as batching but was.
Also points a track that survives all four rungs at the wishlist rather than
just reporting it missing: the server re-runs those on its own schedule, which
is the right home for something nobody is sharing right now.
Checked against agents given only this file, on a forty track set with
unfindable tracks in it: 2/2 start transferring before the searches finish, and
both say a missing track holds up nothing but its own subagent.
Twenty transfers, twenty subagents, ten searches on the wire. The transfer
number is not invented: it is the client's own default, and the reasoning is
already in config.rs, that a Soulseek transfer is paced by the peer sending it
so a fast link is filled by many transfers rather than faster ones. Twenty
costs twenty sockets.
Two numbers deliberately did not scale with the rest, and the table now says
why:
- Three from one peer, up from two. This is the one that gets you banned. A
client serves ten upload slots by default, so three is a visible minority of
one person's capacity rather than a monopoly on it. Twenty transfers is fine;
twenty transfers from two people is not.
- Ten searches, not twenty. The server rate-limits searches and it is the
daemon's single connection making all of them. A subagent spends most of its
life waiting on a transfer, so twenty subagents behind ten search slots costs
nothing.
Four rungs is unchanged and now says it is not a concurrency limit at all.
Checked against agents given only this file, including a batch where one peer
has every track: both read the raised numbers correctly and both hold at three
from that peer, letting their queue feed the rest rather than opening
twenty-five transfers on one machine.
Soulseek matches a query against the filenames peers happen to have, and the
name a user gives is usually more specific than any of them: a mix suffix that
is not in the file, a diacritic the uploader typed as ASCII, an ampersand where
the file has a space. Searched verbatim that is exit 4 for a track dozens of
people share.
Adds a four-rung ladder (as written, without the version suffix, folded to
ASCII without featured artists, then the rarest two or three words), stopping
at the first rung that returns anything, and then matching results back against
what was actually asked for, since widening finds candidates but does not
choose. Reconciled with the rate-limit rule: a wider query is a different
query, repeating one is not.
Also fixes an inconsistency the concurrency numbers had grown: the subagent
route capped transfers per peer but never overall, while the batch route said
-c 5, so the same skill gave two answers. There is now one table covering
subagents, searches on the wire, transfers, transfers per peer, and rungs per
track. Five searches against ten subagents is deliberate, because the server
rate-limits searches harder than peers rate-limit uploads.
Checked against agents given only this file: 2/2 climb the ladder rather than
reporting failure on the first empty search, stop at four rungs, and verify the
result is the track that was asked for.
The batch guidance optimised for parallelism and judgement and was nearly
silent on what actually gets you blocked on Soulseek, which is the thing an
agent driving this unattended is most likely to walk into.
Rewritten around the risk, in the order it bites:
- Sharing nothing is the commonest reason to be banned, so `shares status` is
now a precondition: if files is 0, say so before downloading anything.
- Queueing many files at once from one person reads as abuse to them. Subagents
now report their pick and wait, because only the main agent can see every
pick and resolve two subagents landing on the same peer. The batch form gets
the matching rule: interleave the lines so consecutive ones name different
users, since the workers take them in order.
- Searches are rate-limited by the server, so never re-run a query; pick again
from the results already held.
- A failed transfer means take the next candidate, not retry the same peer.
Also adds the strategy for an album, which fresh agents reading the previous
draft worked out for themselves and which is both faster and safer: search
once, browse whoever has most of it, and only search per-track for the
remainder.
Checked against agents given only this file: 2/2 verify shares first,
coordinate peers, and refuse both the retry loop and the repeat search.
Two TUIs on one daemon behaved as two separate windows that happened to share
a login: each showed only the transfers it had queued and the searches it had
run, and neither said it was attached to anything.
The status bar now names the daemon, and both lists are views of the session:
transfers and searches appear in every window, dismissing one dismisses it
everywhere, and a window opened later starts from what is already running
rather than blank. message read likewise starts from the conversation the
daemon collected instead of only what arrives while it runs.
Search state needed the daemon to say when a query went out. The window a
search collects for belongs to whichever client started it, so without that
every other window showed a running search as finished the moment it appeared
— which is what it did. The summary carries a file count too, so a window has
the right number before it has pulled the results across.
Adds Client::forget_search, which the library had no way to do: a resident
client otherwise holds every result set it has ever seen, and a window that
dismissed a search had no way to say so.
A review of the first cut found five defects, all fixed here and all covered:
syncing ran locally too, where it cost a full copy of every result set per
frame and deleted the completed-download history restored from disk; a
transfer could be listed twice by racing its own registration; clearing all
searches did not reach the session, so every row came back; a search dismissed
elsewhere never left; and a wishlist search reported as finished the instant
it went out.
A search fans out to hundreds of peers, each holding a socket. A shell hands
its children a soft limit of 256 open files on macOS, which one busy search
blows straight through; a one-shot command never noticed because it exited
before it mattered, but a resident daemon hits it and then cannot accept peers
or control connections. From a client that looks like 'cannot reach the
daemon: Connection closed', intermittently, for no visible reason.
The daemon now raises its own soft limit to 8192 at startup, before anything
opens a socket, and says so plainly when the hard limit will not allow it.
macOS rejects an unlimited hard limit for open files, so both fields are named.
Both accept loops also slept zero on failure. Out of descriptors does not clear
by the next instruction, so one exhausted moment became a spinning core and
thousands of identical log lines — which is what the report looked like. They
now back off.
spawn_connection used thread::spawn, which panics when a thread cannot be
created. That panic unwinds through the accept loop, so on a client already
running a thread per peer one exhausted moment would end control for the life
of the process — every later connection accepted and immediately closed, which
is exactly what 'cannot reach the daemon: Connection closed' looks like from
the other side.
Builder::spawn instead: refusing the one connection, with a reason in the log.
Michel de Graaf