diff --git a/README.md b/README.md index 0f84666..e014a39 100644 --- a/README.md +++ b/README.md @@ -123,21 +123,21 @@ it holds the session while everything else borrows it: ```bash soulseek-rs daemon # stays in the foreground; leave it running -soulseek-rs search 'aphex twin' # no login, no flags — uses the daemon +soulseek-rs search 'aphex twin' # no login, no flags; uses the daemon soulseek-rs # the TUI attaches to the same session ``` **Why you want this.** The server allows one session per account. Without a -daemon, two commands at once either cut each other off or push you into a -throwaway username per run — account churn on infrastructure other people -maintain and share. A daemon is one login for as long as you work, and one -share index instead of a rescan every invocation. It also lets a download +daemon, two commands at once either cut each other off, or push you into a +throwaway username per run. That second option is account churn on +infrastructure other people maintain and share. A daemon is one login for as +long as you work, and one share index instead of a rescan every invocation. It also lets a download outlive the command that queued it, and keeps collecting chat while nothing is attached. If you drive this from a script or an agent, start one. Commands find it by themselves. It listens on a Unix socket only your user can -open, so there is nothing to configure and no secret to pass around — the same -arrangement Docker and ssh-agent use. With no daemon running, everything +open, so there is nothing to configure and no secret to pass around. Docker and +ssh-agent make the same arrangement. With no daemon running, everything behaves exactly as it did before; `--no-daemon` forces that for one run. From a script, start it and *wait* for it: it takes a second or two to log in, @@ -162,14 +162,14 @@ Everything that touches the network goes through it. `config`, `skills`, `completions` and `portmap` stay local, because they are about this machine rather than a session. `shares add`/`remove` do both: they write the config file *and* update a running daemon, so a folder is served straight away rather -than at its next start. Files land on the *daemon's* filesystem — `daemon +than at its next start. Files land on the *daemon's* filesystem, and `daemon status` reports where. #### A download box you drive from your laptop This is what daemon mode is really for. Put soulseek-rs on the machine that -should be doing the downloading — a home server, a NAS, a Raspberry Pi, a -seedbox — and drive it from wherever you happen to be. +should be doing the downloading: a home server, a NAS, a Raspberry Pi, a +seedbox. Then drive it from wherever you happen to be. **On that machine**, run the daemon and let it accept connections from your network: @@ -189,9 +189,8 @@ soulseek-rs config set daemon nas.local:5030 soulseek-rs config set daemon_token ``` -That is the whole setup. Every command from now on is the normal command — -your laptop needs no Soulseek account of its own, because it is using the -server's: +That is the whole setup. Every command from now on is the normal command. Your +laptop needs no Soulseek account of its own, because it uses the server's: ```bash soulseek-rs search 'aphex twin' --json @@ -212,7 +211,7 @@ soulseek-rs daemon stop # shuts down the remote daemon ``` One caution: there is no encryption on that TCP port. On a home network that -is usually fine. Over the internet, don't expose it — reach it through SSH +is usually fine. Over the internet, don't expose it. Reach it through SSH instead, either with a tunnel (`ssh -L 5030:localhost:5030 nas`) or by running the commands over `ssh` on the box itself. @@ -220,9 +219,10 @@ the commands over `ssh` on the box itself. The daemon's interface is open and documented, so nothing about it is specific to this CLI. It speaks newline-delimited JSON-RPC 2.0 over a socket, described -by an [OpenRPC](https://open-rpc.org) document — -[`docs/openrpc.json`](docs/openrpc.json), which the daemon also serves itself -via `rpc.discover`, so a client can ask a running daemon what it can do. +by an [OpenRPC](https://open-rpc.org) document, +[`docs/openrpc.json`](docs/openrpc.json). The daemon serves that document +itself via `rpc.discover`, so a client can ask a running daemon what it can +do. OpenRPC is the JSON-RPC counterpart to OpenAPI: point a client generator at that document and you get typed bindings in whatever language you are working @@ -230,7 +230,7 @@ in. If you want an Android or iOS app that queues downloads on your home server from the sofa, or a small web page that shows what is transferring, you have everything you need and no permission to ask for. Searches, transfers, rooms, private messages and shares are all on the same interface this CLI -uses — there is no private back channel it keeps for itself, and live updates +uses. There is no private back channel it keeps for itself, and live updates are pushed to every connected client rather than polled for. [`docs/daemon-protocol.md`](docs/daemon-protocol.md) covers the parts a schema @@ -459,8 +459,8 @@ soulseek-rs serve --follow # add --upload-slots N to widen the queue Uploads only show up inside a running `serve`; every other command is short-lived with nothing to serve, so there is no `uploads list`. Managing -transfers across commands — pause, resume, a queue that outlives one -invocation — is what [daemon mode](#daemon-mode) is for. +transfers across commands (pause, resume, a queue that outlives one +invocation) is what [daemon mode](#daemon-mode) is for. Downloads run under a deadline (`--timeout`, 300s by default) and `--max-concurrent-downloads` at a time, so an unattended run always ends. diff --git a/docs/daemon-protocol.md b/docs/daemon-protocol.md index 6b68b35..4f38af6 100644 --- a/docs/daemon-protocol.md +++ b/docs/daemon-protocol.md @@ -2,8 +2,8 @@ `soulseek-rs daemon` stays logged in and answers JSON-RPC over a socket, so several clients can share one Soulseek session. The machine-readable contract -is [`openrpc.json`](openrpc.json) — an [OpenRPC](https://open-rpc.org) -document, the JSON-RPC counterpart to OpenAPI. Point a client generator at it, +is [`openrpc.json`](openrpc.json), an [OpenRPC](https://open-rpc.org) +document and the JSON-RPC counterpart to OpenAPI. Point a client generator at it, or at a running daemon via `rpc.discover`, and you have a client in your language. @@ -30,8 +30,8 @@ once. The local socket is the Docker and ssh-agent bargain: if the operating system let you open it, you are the user who started the daemon, and there is nothing -further to prove. This is what lets a local client — a script or an agent — -work with no configuration at all. +further to prove. A local client, whether a script or an agent, works with no +configuration at all. TCP always requires the token, and a token that *is* offered is always checked, on either transport, so a wrong one never passes. Failed attempts @@ -62,7 +62,7 @@ the old shape. ## Requests -Parameters are by name — a JSON object, never an array. Each method's fields +Parameters are by name: a JSON object, never an array. Each method's fields are published as individual OpenRPC content descriptors, so a generated client sends exactly the object the daemon deserializes. `rpc.discover` returns that same document from the running daemon. @@ -82,8 +82,8 @@ silence, including when it fails. ``` An error always carries an `id`, explicitly `null` when the request was too -malformed to have one — so do not read a missing `id` as "this is a -notification" and drop it. +malformed to have one. Do not read a missing `id` as "this is a notification" +and drop it. Standard codes (`-32700` parse, `-32600` invalid request, `-32601` unknown method, `-32602` bad parameters) mean what they always mean. Application @@ -102,7 +102,7 @@ as it branches on the exit code. ## Events -The daemon pushes notifications — no `id`, so never answer them: +The daemon pushes notifications. They carry no `id`, so never answer them: ```json {"jsonrpc":"2.0","method":"event.room","params":{"event":"message","room":"lobby","username":"bob","message":"hi"}} @@ -110,7 +110,7 @@ The daemon pushes notifications — no `id`, so never answer them: `event.room`, `event.message`, `event.upload`, `event.download_status`, `event.browse`, `event.session_loss`. Payload schemas are in `openrpc.json`, -where they appear as entries marked `x-notification` — OpenRPC has no +where they appear as entries marked `x-notification`. OpenRPC has no first-class notion of a server-pushed message, so that flag is how they are distinguished from methods you may call. @@ -129,7 +129,7 @@ later, with one exception: private messages are also kept, and nobody was attached. Every connection gets its own queue, bounded at 1024 events. **A client that -stops reading its socket is disconnected** once that queue fills — the daemon +stops reading its socket is disconnected** once that queue fills. The daemon will not let one stalled consumer cost the others their events. Read continuously; do not treat the socket as request/response only. @@ -139,7 +139,7 @@ matched on `username` + `filename`. ## Behaviour worth knowing **Files land on the daemon's filesystem.** The directory is the daemon's -configuration and a client cannot name one — that would be an arbitrary write +configuration and a client cannot name one, which would be an arbitrary write on the daemon's host. `daemon.status` reports the directory in use. There is no method to fetch the bytes over the control socket. @@ -158,8 +158,8 @@ rejoined on startup. **Searching is two steps.** `search.start` puts the query on the wire and returns immediately; peers answer over their own connections for as long as you care to wait. Decide the window yourself, then read `search.results`. This -is deliberate — a client's search window must not be time the daemon spends -not answering everybody else. +is deliberate: a client's search window must not be time the daemon spends not +answering everybody else. ## A session by hand diff --git a/soulseek-rs/src/SKILL.md b/soulseek-rs/src/SKILL.md index e6edbde..bb252c1 100644 --- a/soulseek-rs/src/SKILL.md +++ b/soulseek-rs/src/SKILL.md @@ -30,15 +30,15 @@ soulseek-rs daemon status --json >/dev/null 2>&1 || { ``` Wait for it, do not just launch it. A daemon takes a second or two to log in, -and `soulseek-rs daemon status … || soulseek-rs daemon &` does **not** do this -— `&` backgrounds the whole list, so it returns instantly and the next command +and `soulseek-rs daemon status … || soulseek-rs daemon &` does **not** do this. +`&` backgrounds the whole list, so it returns instantly and the next command races the login it is supposed to be using. A command that arrives too early finds no daemon and quietly opens a session of its own, which is the outcome this whole section exists to avoid. Then use every other command exactly as documented below. They find the daemon -by themselves — it listens on a Unix socket only your user can open, so there -is nothing to configure, no address to pass, and no token to handle. +by themselves. It listens on a Unix socket only your user can open, so there is +nothing to configure, no address to pass, and no token to handle. **Why this matters, and why it is not optional for you.** Soulseek is a real network of other people's computers, and the server allows **one session per @@ -46,7 +46,7 @@ account**. Without a daemon, every invocation is its own login, and running two at once leaves you two bad choices: share a username and they silently cut each other off (exit 7), or give each run a throwaway name and *register a new account on a public server for every search you make*. The second is what a -naive script does, and it is the antisocial one — it is account churn on +naive script does, and it is the antisocial one: account churn on infrastructure other people maintain and share. A daemon is one login for as long as you work, however many commands you run @@ -156,8 +156,9 @@ participant. **`daemon`** — stay logged in as a service other commands share. Prints progress on stderr and nothing on stdout; it does not return until stopped. While it runs, every other command on this machine uses its session -automatically — no flags, no credentials, no second login. `--bind ADDR` also -accepts connections from other machines, which need the token. +automatically, with no flags, no credentials, and no second login. +`--bind ADDR` also accepts connections from other machines, which need the +token. **`daemon status`** — one object: `user`, `server`, `version`, `protocol`, `listening`, `listen_port`, `shared_folders`, `shared_files`, `download_dir`, `session_loss`, `clients`, `uptime_secs`. Exit 3 when nothing is running. @@ -165,8 +166,8 @@ accepts connections from other machines, which need the token. **`daemon token`** — the secret a client on another machine needs. Local clients never need it. -The daemon can also be on a *different* machine — a home server or NAS doing -the downloading. Point at it once with `config set daemon host:5030` and +The daemon can also be on a *different* machine, such as a home server or NAS +doing the downloading. Point at it once with `config set daemon host:5030` and `config set daemon_token `, and every command afterwards drives that machine; files land there, not here. The interface is an open JSON-RPC schema (`rpc.discover` serves it), so a purpose-built client is a reasonable thing to @@ -218,7 +219,7 @@ is the normal way to be selective. ## Running several at once -With a daemon running this needs no thought — they all share its one session: +With a daemon running this needs no thought. They all share its one session: ```bash for q in 'gary beck fold' 'marcal steady'; do @@ -230,7 +231,7 @@ wait Without one, the same loop is a bug: the runs share a username, so each login displaces the last and they kill each other's session (exit 7). Giving each run its own `--username` avoids the collision by registering a fresh account on the -server for every search, which is worse — do not do it. Start the daemon. +server for every search, which is worse. Do not do it. Start the daemon. The listener port needs no care either way: a run that finds the configured port taken binds a free one and tells the server about that one instead. @@ -250,8 +251,8 @@ port taken binds a free one and tells the server about that one instead. - **Exit 4 is not a failure.** A search that finds nothing means the query was too narrow or misspelled. Widen it; use `whoami` to rule out the connection. - **Exit 7 is not an answer.** The session ended mid-command, so nothing it saw - counts. It means something else logged in as you — almost always a second - run of your own. Start a daemon and route through it rather than hunting for + counts. Something else logged in as you, almost always a second run of your + own. Start a daemon and route through it rather than hunting for an unused name. - **Never put a password in argv** — `ps` shows it to every user on the box. Use `--password-stdin`, the OS keychain, or `SOULSEEK_PASSWORD_CMD`. diff --git a/soulseek-rs/src/daemon/mod.rs b/soulseek-rs/src/daemon/mod.rs index a8eaab1..676c393 100644 --- a/soulseek-rs/src/daemon/mod.rs +++ b/soulseek-rs/src/daemon/mod.rs @@ -199,7 +199,7 @@ pub fn run( downloads: Arc::clone(&downloads), download_dir: ctx.download_dir.clone(), server: ctx.settings.server_address.to_string(), - token, + token: token.clone(), started: Instant::now(), stop: Arc::clone(&stop), open: Arc::new(std::sync::atomic::AtomicUsize::new(0)), @@ -225,7 +225,7 @@ pub fn run( "logged in as {}, sharing {files} files in {folders} folders", session.client.username() )); - ctx.out.status("ready"); + ready(ctx, bind, &token); // The listeners run on their own threads; this one only has to notice a // `daemon.stop` and write state back on the way past. @@ -247,6 +247,57 @@ pub fn run( Ok(()) } +/// Say what is now possible, once, on stderr. +/// +/// The token goes here rather than only into a file because the moment you +/// need it is the moment you bind a port, and hunting for `daemon token` +/// afterwards is a poor first experience. stdout stays empty either way, so +/// `daemon` still composes in a pipeline. +fn ready(ctx: &Ctx, bind: Option<&str>, token: &str) { + ctx.out.status(""); + ctx.out.status("ready — leave this running."); + ctx.out.status(""); + ctx.out + .status(" On this machine, nothing else to do. Other commands find"); + ctx.out + .status(" it on their own and share this one login:"); + ctx.out.status(""); + ctx.out.status(" soulseek-rs search 'aphex twin'"); + ctx.out + .status(" soulseek-rs (the TUI attaches)"); + ctx.out.status(" soulseek-rs daemon status"); + ctx.out.status(" soulseek-rs daemon stop"); + ctx.out.status(""); + + let Some(address) = bind else { + ctx.out.status( + " To drive it from another machine, restart with --bind ADDR;", + ); + ctx.out + .status(" `soulseek-rs daemon token` prints the secret it needs."); + ctx.out.status(""); + return; + }; + + ctx.out.status(&format!( + " From another machine, point a client at {address} with" + )); + ctx.out.status(" this token:"); + ctx.out.status(""); + ctx.out.status(&format!(" {token}")); + ctx.out.status(""); + ctx.out + .status(" soulseek-rs config set daemon :PORT"); + ctx.out + .status(" soulseek-rs config set daemon_token "); + ctx.out.status(""); + ctx.out.status( + " That port has no encryption. Keep it off the open internet and \ + reach it over SSH instead.", + ); + ctx.out.status(""); +} + /// Remove the socket file so the next start does not have to reason about /// whether it is stale. fn cleanup() { diff --git a/soulseek-rs/src/daemon/server/transport.rs b/soulseek-rs/src/daemon/server/transport.rs index 84169f1..e54a02f 100644 --- a/soulseek-rs/src/daemon/server/transport.rs +++ b/soulseek-rs/src/daemon/server/transport.rs @@ -155,14 +155,20 @@ fn spawn_connection(stream: Stream, daemon: Arc) { if open > MAX_CONNECTIONS { daemon.open.fetch_sub(1, Ordering::Relaxed); soulseek_rs::warn!( - "refusing a control connection: {MAX_CONNECTIONS} already open" + "refusing a control connection: {MAX_CONNECTIONS} already open. \ + This is a leak unless you really have that many clients — please \ + report it with `soulseek-rs daemon status`." ); stream.shutdown(); return; } std::thread::spawn(move || { if let Err(e) = handle(stream, daemon.as_ref()) { - soulseek_rs::debug!("control connection ended: {e}"); + // Loud, not debug: a client sees only "the connection closed", so + // this line is the only account of why it did. + soulseek_rs::warn!( + "control connection dropped before it could be served: {e}" + ); } daemon.open.fetch_sub(1, Ordering::Relaxed); }); diff --git a/soulseek-rs/tests/daemon_protocol_e2e.rs b/soulseek-rs/tests/daemon_protocol_e2e.rs index 2ac33ba..32982f3 100644 --- a/soulseek-rs/tests/daemon_protocol_e2e.rs +++ b/soulseek-rs/tests/daemon_protocol_e2e.rs @@ -565,3 +565,105 @@ fn a_config_driven_remote_client_reports_the_daemons_world() { let _ = std::fs::remove_dir_all(&config); let _ = std::fs::remove_dir_all(&state); } + +/// Several clients at once, all seeing the same session. +/// +/// This is the promise daemon mode makes and the reason the fan-out exists: +/// the server allows one login, so the alternative to sharing is displacing. +/// Each client gets its own connection, its own event queue, and the same +/// answers. +#[test] +fn many_clients_share_one_session_and_agree_about_it() { + let daemon = daemon_or_skip!(); + + // Held open together, not one after another: the point is concurrency. + let mut clients: Vec = (0..8) + .map(|_| { + let mut connection = daemon.connect_tcp().expect("connect"); + let reply = connection + .authenticate(Some(&daemon.token())) + .expect("an answer"); + assert!(reply.contains("\"result\""), "client rejected: {reply}"); + connection + }) + .collect(); + + // Every one of them is told the same thing about the same session. + let mut seen = Vec::new(); + for (i, client) in clients.iter_mut().enumerate() { + let reply = client + .call(&format!( + r#"{{"jsonrpc":"2.0","id":{},"method":"daemon.status"}}"#, + 100 + i + )) + .unwrap_or_else(|| panic!("client {i} lost its connection")); + let status: serde_json::Value = + serde_json::from_str(&reply).expect("valid JSON"); + assert_eq!( + status["id"], + serde_json::json!(100 + i), + "replies must not be crossed between clients: {reply}" + ); + seen.push(( + status["result"]["username"].clone(), + status["result"]["server"].clone(), + )); + } + assert!( + seen.windows(2).all(|pair| pair[0] == pair[1]), + "every client should describe the same session: {seen:?}" + ); + + // And the daemon counts them all, rather than having quietly dropped some. + let reply = clients[0] + .call(r#"{"jsonrpc":"2.0","id":200,"method":"daemon.status"}"#) + .expect("still connected"); + let status: serde_json::Value = + serde_json::from_str(&reply).expect("valid JSON"); + assert_eq!( + status["result"]["clients"], + serde_json::json!(8), + "all eight should still be attached: {reply}" + ); +} + +/// Connecting and dropping, over and over, must not degrade the daemon. +/// +/// Every command makes at least one throwaway connection while it works out +/// whether a daemon is there, so a per-connection resource that is not +/// released shows up as a daemon that works for a while and then refuses +/// everyone. +#[test] +fn rapid_connect_and_drop_does_not_wear_the_daemon_out() { + let daemon = daemon_or_skip!(); + + // Comfortably more than the connection cap, so a leak of one slot per + // connection would have exhausted it well before the end. + for _ in 0..200 { + drop(daemon.connect_tcp().expect("connect")); + } + // Half-open ones too: authenticated, then abandoned mid-conversation. + for _ in 0..50 { + let mut connection = daemon.connect_tcp().expect("connect"); + connection.authenticate(Some(&daemon.token())); + drop(connection); + } + + // Give the daemon a moment to notice the hangups, then prove it is well. + std::thread::sleep(Duration::from_secs(2)); + let mut connection = daemon.connect_tcp().expect("connect"); + let reply = connection + .authenticate(Some(&daemon.token())) + .expect("the daemon should still be answering"); + assert!(reply.contains("\"result\""), "{reply}"); + let status = connection + .call(r#"{"jsonrpc":"2.0","id":1,"method":"daemon.status"}"#) + .expect("an answer"); + let parsed: serde_json::Value = + serde_json::from_str(&status).expect("valid JSON"); + assert_eq!( + parsed["result"]["clients"], + serde_json::json!(1), + "abandoned connections must not still be counted: {status}" + ); +} diff --git a/website/src/pages/docs.tsx b/website/src/pages/docs.tsx index bada849..e2adfe7 100644 --- a/website/src/pages/docs.tsx +++ b/website/src/pages/docs.tsx @@ -228,7 +228,7 @@ const Daemon = () => (
Soulseek allows one session per account. Run a daemon and everything else borrows - it — no second login, nothing to configure. + it. No second login, nothing to configure. ( { t: 'cmd', text: 'soulseek-rs daemon &' }, { t: 'cm', text: '# once; it holds the session' }, { t: 'cmd', text: 'soulseek-rs search "netlabel 2004" --json' }, - { t: 'cm', text: '# no login, no flags — it finds the daemon' }, + { t: 'cm', text: '# no login, no flags; it finds the daemon' }, { t: 'cmd', text: 'soulseek-rs' }, { t: 'cm', text: '# the TUI attaches to the same session' }, { t: 'cmd', text: 'soulseek-rs daemon status' }, @@ -251,10 +251,10 @@ const Daemon = () => ( running, everything behaves exactly as it did before.

- Why bother: the server permits a single session per account, so two one-off - commands at once either cut each other off or push you into a throwaway - username per run — account churn on a network other people maintain. A - daemon is one login for as long as you work, one share index instead of a + Why bother: the server permits a single session per account. Two one-off + commands at once either cut each other off, or push you into a throwaway + username per run, which is account churn on a network other people maintain. + A daemon is one login for as long as you work, one share index instead of a rescan every invocation, and downloads that outlive the command that queued them. If you drive this from a script or an agent, start one.

@@ -293,16 +293,16 @@ const DownloadBox = () => ( />

- Your laptop needs no Soulseek account of its own — it is using the box’s. - Queue what you want, then close the lid: the downloads keep going, because they + Your laptop needs no Soulseek account of its own; it uses the box’s. Queue + what you want, then close the lid. The downloads keep going, because they belong to the daemon rather than to the command that asked for them. Come back tomorrow, run daemon status, and pick up where you left off. The files are on the box, which is where you wanted them.

- There is no encryption on that port. On a home network that is usually fine; - over the internet, reach it through SSH instead —{' '} - ssh -L 5030:localhost:5030 nas, or just run the commands on the box. + There is no encryption on that port. On a home network that is usually fine. + Over the internet, reach it through SSH instead:{' '} + ssh -L 5030:localhost:5030 nas, or run the commands on the box.

@@ -325,9 +325,9 @@ const OpenSchema = () => (

OpenRPC is the JSON-RPC counterpart to OpenAPI: point a generator at it and you - get typed bindings in whatever language you are working in. An Android or iOS app - that queues downloads on your home server from the sofa, a small web page showing - what is transferring — you have everything you need and nobody to ask. + get typed bindings in whatever language you are working in. Want an Android or + iOS app that queues downloads on your home server from the sofa, or a web page + showing what is transferring? You have what you need and nobody to ask.

Searches, transfers, rooms, private messages and shares are all on the same diff --git a/website/src/pages/home.tsx b/website/src/pages/home.tsx index 9d1a21d..5fdeb68 100644 --- a/website/src/pages/home.tsx +++ b/website/src/pages/home.tsx @@ -66,8 +66,8 @@ const FEATURES = [ cmd: 'soulseek-rs daemon', body: ( <> - Hold one session and let every other command borrow it — the network allows - one login per account, so this is the neighbourly way to run several at once. It + Hold one session and let every other command borrow it. The network allows one + login per account, so this is the neighbourly way to run several at once. It speaks JSON-RPC, so other tools can drive it too. ),