docs: publish the daemon protocol and describe daemon mode in the README

The OpenRPC document is the machine-readable contract; daemon-protocol.md
covers what a schema cannot say — framing, the auth handshake, and that
events are pushed, unreplayable, and will disconnect a client that stops
reading.
This commit is contained in:
Michel de Graaf
2026-07-27 10:46:05 +02:00
parent a0354f66a8
commit 834fbcd35b
3 changed files with 452 additions and 7 deletions
+49 -4
View File
@@ -58,7 +58,7 @@ them freely.
## Planned Features ## Planned Features
- [ ] Headless daemon mode with remote control - [x] Headless daemon mode with remote control
## Project Structure ## Project Structure
@@ -116,6 +116,52 @@ soulseek-rs
Everything else is a one-shot command that runs headless. Everything else is a one-shot command that runs headless.
### Daemon mode
A one-shot command logs in, does its job, and exits, which means nothing
outlives it and two commands at once fight over the server's one-session-per-
account rule. Run a daemon instead and it holds the session while everything
else borrows it:
```bash
soulseek-rs daemon # stays in the foreground; leave it running
soulseek-rs search 'aphex twin' # no login, no flags — uses the daemon
soulseek-rs # the TUI attaches to the same session
```
Commands find it by themselves. The daemon listens on a Unix socket only your
user can open, so there is nothing to configure and no secret to pass around —
the same arrangement Docker and ssh-agent use. With no daemon running,
everything behaves exactly as it did before; `--no-daemon` forces that for one
run.
What this buys: downloads that outlive the command that queued them, several
clients (or agents) on one account at once, and a chat window that keeps
collecting while nothing is attached.
```bash
soulseek-rs daemon status # who it is, what it shares, who is attached
soulseek-rs daemon stop
```
For a client on another machine, bind a TCP port as well and give it the
token:
```bash
soulseek-rs daemon --bind 0.0.0.0:5030
soulseek-rs daemon token # the secret that connection needs
```
There is no transport encryption, so put SSH or a TLS proxy in front of
anything non-loopback. Note that files land on the *daemon's* filesystem.
Third-party clients are first-class: the daemon speaks newline-delimited
JSON-RPC 2.0 described by an [OpenRPC](https://open-rpc.org) document
([`docs/openrpc.json`](docs/openrpc.json), also served by `rpc.discover`), so
a generator produces a client in any language.
[`docs/daemon-protocol.md`](docs/daemon-protocol.md) covers the framing, the
handshake, and how pushed events behave.
### Scripting ### Scripting
The one-shot commands follow three rules, so they compose with other tools: The one-shot commands follow three rules, so they compose with other tools:
@@ -339,9 +385,8 @@ soulseek-rs serve --follow # add --upload-slots N to widen the queue
Uploads only show up inside a running `serve`; every other command is Uploads only show up inside a running `serve`; every other command is
short-lived with nothing to serve, so there is no `uploads list`. Managing short-lived with nothing to serve, so there is no `uploads list`. Managing
transfers across commands (pause, resume, a queue that outlives one transfers across commands — pause, resume, a queue that outlives one
invocation) needs the resident daemon `serve` would grow into, which is not invocation — is what [daemon mode](#daemon-mode) is for.
there yet.
Downloads run under a deadline (`--timeout`, 300s by default) and Downloads run under a deadline (`--timeout`, 300s by default) and
`--max-concurrent-downloads` at a time, so an unattended run always ends. `--max-concurrent-downloads` at a time, so an unattended run always ends.
+152
View File
@@ -0,0 +1,152 @@
# The daemon control protocol
`soulseek-rs daemon` stays logged in and answers JSON-RPC over a socket, so
several clients can share one Soulseek session. The machine-readable contract
is [`openrpc.json`](openrpc.json) — an [OpenRPC](https://open-rpc.org)
document, the JSON-RPC counterpart to OpenAPI. Point a client generator at it,
or at a running daemon via `rpc.discover`, and you have a client in your
language.
This page covers what a schema cannot: framing, the handshake, and how pushed
events behave.
## Framing
Newline-delimited JSON-RPC 2.0. One JSON value per line, in both directions,
UTF-8, no length prefix. A line that does not parse gets a `-32700` reply and
the connection stays open.
## Transports and authentication
| Transport | Address | Authentication |
|---|---|---|
| Unix socket | `<state-dir>/daemon.sock`, mode 0600 | The socket's permissions. `token` may be omitted. |
| TCP | opt-in via `daemon --bind ADDR` | The token in `<state-dir>/daemon.token`, mode 0600. |
The local socket is the Docker and ssh-agent bargain: if the operating system
let you open it, you are the user who started the daemon, and there is nothing
further to prove. This is what lets a local client — a script or an agent —
work with no configuration at all.
TCP always requires the token, and a token that *is* offered is always
checked, on either transport, so a wrong one never passes. Failed attempts
sleep one second. The token is 256 bits from the OS, generated on first start
and reused across restarts so existing clients keep working;
`soulseek-rs daemon token` prints it.
There is no transport encryption. Binding non-loopback TCP means putting SSH
or a TLS-terminating proxy in front of it.
`<state-dir>` is `$SOULSEEK_STATE_DIR`, else the platform data directory
(`~/.local/share/soulseek-rs/state` on Linux and macOS).
### The handshake
`auth` must be the first request on a connection. Anything else is answered
`-32600` and the connection is dropped. Until it succeeds the connection is
told nothing and receives no events.
```json
--> {"jsonrpc":"2.0","id":1,"method":"auth","params":{"protocol":1}}
<-- {"jsonrpc":"2.0","id":1,"result":{"protocol":1,"daemon_version":"13.0.0","username":"you"}}
```
Compare `protocol` with your own and fail loudly on a mismatch rather than
guessing; it is bumped whenever a change would break a client written against
the old shape.
## Requests
Parameters are by name — a JSON object, never an array. Every method and its
parameter and result schemas are in `openrpc.json`; `rpc.discover` returns
that same document from the running daemon.
Requests may be pipelined and are answered out of order, so match replies by
`id`. A request without an `id` is a notification and is answered with
silence, including when it fails.
## Errors
```json
{"jsonrpc":"2.0","id":4,"error":{"code":-32000,"message":"no results","data":{"exit":4}}}
```
Standard codes (`-32700` parse, `-32600` invalid request, `-32601` unknown
method, `-32602` bad parameters) mean what they always mean. Application
failures use `-32000` and carry `data.exit`: the CLI exit status the same
operation would have produced locally, so a wrapper can branch on it exactly
as it branches on the exit code.
| `exit` | Meaning |
|---|---|
| 2 | Usage or configuration |
| 3 | Could not reach the server, or the login was rejected |
| 4 | Succeeded but produced nothing |
| 5 | Timed out |
| 6 | A transfer started but did not complete |
| 7 | The server session ended mid-operation |
## Events
The daemon pushes notifications — no `id`, so never answer them:
```json
{"jsonrpc":"2.0","method":"event.room","params":{"event":"message","room":"lobby","username":"bob","message":"hi"}}
```
`event.room`, `event.message`, `event.upload`, `event.download_status`,
`event.browse`, `event.session_loss`. Payload schemas are in `openrpc.json`,
where they appear as entries marked `x-notification` — OpenRPC has no
first-class notion of a server-pushed message, so that flag is how they are
distinguished from methods you may call.
**Events are the only way to observe these things, and they are not
replayable.** The underlying library hands over its event buffer and empties
it, so the daemon drains once and copies to every attached connection. A
client that is not connected when something happens does not learn about it
later, with one exception: private messages are also kept, and
`message.history` returns the conversation including what arrived while
nobody was attached.
Every connection gets its own queue, bounded at 1024 events. **A client that
stops reading its socket is disconnected** once that queue fills — the daemon
will not let one stalled consumer cost the others their events. Read
continuously; do not treat the socket as request/response only.
A `download.start` you issued reports progress through `event.download_status`
matched on `username` + `filename`.
## Behaviour worth knowing
**Files land on the daemon's filesystem.** `download_directory` is the
daemon's configuration; a remote client cannot see that filesystem and does
not choose. `daemon.status` reports the directory in use. There is no method
to fetch the bytes over the control socket.
**Several clients, no locking.** Last write wins. Two clients cancelling the
same transfer is two hands on one keyboard, not an error.
**The daemon owns durable state.** Transfers, joined rooms, and the
conversation survive a restart: unfinished transfers are re-queued and rooms
rejoined on startup.
**Searching is two steps.** `search.start` puts the query on the wire and
returns immediately; peers answer over their own connections for as long as
you care to wait. Decide the window yourself, then read `search.results`. This
is deliberate — a client's search window must not be time the daemon spends
not answering everybody else.
## A session by hand
```console
$ nc -U ~/.local/share/soulseek-rs/state/daemon.sock
{"jsonrpc":"2.0","id":1,"method":"auth","params":{"protocol":1}}
{"jsonrpc":"2.0","id":1,"result":{"protocol":1,"daemon_version":"13.0.0","username":"you"}}
{"jsonrpc":"2.0","id":2,"method":"search.start","params":{"query":"aphex twin"}}
{"jsonrpc":"2.0","id":2,"result":{"ok":true}}
{"jsonrpc":"2.0","id":3,"method":"search.results","params":{"query":"aphex twin"}}
{"jsonrpc":"2.0","id":3,"result":{"results":[…]}}
```
That the whole protocol is reachable from `nc` and a text editor is the reason
it is shaped this way.
+251 -3
View File
@@ -30,9 +30,11 @@ const EXIT_SESSION_LOST: i32 = 7;
/// Every environment variable the CLI reads, cleared for each run so a /// Every environment variable the CLI reads, cleared for each run so a
/// developer's shell (or a stray `.env`) cannot change what a test observes. /// developer's shell (or a stray `.env`) cannot change what a test observes.
const CLI_ENV_VARS: [&str; 13] = [ const CLI_ENV_VARS: [&str; 15] = [
"SOULSEEK_CONFIG_DIR", "SOULSEEK_CONFIG_DIR",
"SOULSEEK_STATE_DIR", "SOULSEEK_STATE_DIR",
"SOULSEEK_DAEMON",
"SOULSEEK_DAEMON_TOKEN",
"SOULSEEK_USERNAME", "SOULSEEK_USERNAME",
"SOULSEEK_PASSWORD", "SOULSEEK_PASSWORD",
"SOULSEEK_PASSWORD_CMD", "SOULSEEK_PASSWORD_CMD",
@@ -48,16 +50,66 @@ const CLI_ENV_VARS: [&str; 13] = [
/// Build a command for the binary, isolated from the developer's environment /// Build a command for the binary, isolated from the developer's environment
/// and from the repository's own working directory. /// and from the repository's own working directory.
///
/// The state directory is not merely cleared but pointed somewhere empty: a
/// client with no state directory of its own would look in the real one, find
/// whatever daemon the developer happens to be running, and route the whole
/// suite through their live Soulseek session.
fn command(args: &[&str]) -> Command { fn command(args: &[&str]) -> Command {
let mut command = Command::new(BIN); let mut command = Command::new(BIN);
for name in CLI_ENV_VARS { for name in CLI_ENV_VARS {
command.env_remove(name); command.env_remove(name);
} }
command.env("SOULSEEK_STATE_DIR", isolated_state_dir());
command.current_dir(std::env::temp_dir()); command.current_dir(std::env::temp_dir());
command.args(args); command.args(args);
command command
} }
/// An empty state directory shared by every run in this test process, so
/// auto-routing finds nothing.
fn isolated_state_dir() -> PathBuf {
static DIR: std::sync::OnceLock<PathBuf> = std::sync::OnceLock::new();
DIR.get_or_init(|| {
let path = std::env::temp_dir()
.join(format!("soulseek-cli-e2e-state-{}", std::process::id()));
std::fs::create_dir_all(&path).expect("state directory");
path
})
.clone()
}
/// Which kind of session the suite drives.
///
/// The point of the switch is that the *same* tests run both ways: identical
/// records, identical exit codes, whether the session is this process's own or
/// a daemon's. CI runs the suite twice, once per mode, and that equivalence is
/// the guarantee daemon mode rests on.
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
enum Mode {
Local,
Daemon,
}
fn mode() -> Mode {
match std::env::var("SOULSEEK_E2E_MODE").as_deref() {
Ok("daemon") => Mode::Daemon,
_ => Mode::Local,
}
}
/// Skip the rest of a test that has no meaning against a daemon — one about
/// logging in, or about what happens when a listener port is taken, both of
/// which are the daemon's business rather than a client's.
macro_rules! local_only {
($why:expr) => {
if mode() == Mode::Daemon {
println!("skipped in daemon mode: {}", $why);
return;
}
};
}
fn run(args: &[&str]) -> Output { fn run(args: &[&str]) -> Output {
command(args).output().expect("the binary should run") command(args).output().expect("the binary should run")
} }
@@ -488,6 +540,121 @@ struct TestServer {
child: Option<Child>, child: Option<Child>,
db: Option<PathBuf>, db: Option<PathBuf>,
_gate: std::sync::MutexGuard<'static, ()>, _gate: std::sync::MutexGuard<'static, ()>,
/// Daemons started for this server, one per (user, session flags). Dropped
/// with the server, which kills them.
daemons: std::sync::Mutex<std::collections::HashMap<String, DaemonFixture>>,
}
/// A `soulseek-rs daemon` running against the test server, with a state
/// directory of its own so its socket and token cannot collide with another
/// fixture's — or with the developer's.
struct DaemonFixture {
child: Child,
socket: PathBuf,
state: PathBuf,
}
impl DaemonFixture {
fn start(
server: &TestServer,
user: &str,
session_flags: &[String],
) -> Self {
static COUNTER: std::sync::atomic::AtomicU32 =
std::sync::atomic::AtomicU32::new(0);
let n = COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let state = std::env::temp_dir().join(format!(
"soulseek-e2e-daemon-{}-{n}",
std::process::id()
));
std::fs::create_dir_all(&state).expect("daemon state directory");
let mut args = vec![
"--no-config".to_string(),
"--server".to_string(),
server.address(),
"--username".to_string(),
user.to_string(),
"--password".to_string(),
"pw".to_string(),
];
// A listener of its own unless the test asked for something specific:
// transfers arrive on a connection the uploader opens back to us.
if !session_flags.iter().any(|flag| {
flag == "--listener-port" || flag == "--no-listener"
}) {
args.push("--listener-port".to_string());
args.push(free_port().expect("listener port").to_string());
}
args.extend(session_flags.iter().cloned());
args.push("daemon".to_string());
let mut command = Command::new(BIN);
for name in CLI_ENV_VARS {
command.env_remove(name);
}
command.env("SOULSEEK_STATE_DIR", &state);
command.current_dir(std::env::temp_dir());
command.args(&args);
command.stdout(Stdio::null()).stderr(Stdio::null());
let child = command.spawn().expect("the daemon should start");
let socket = state.join("daemon.sock");
let fixture = Self {
child,
socket,
state,
};
fixture.wait_until_ready();
fixture
}
/// Wait until the socket actually accepts, not merely exists: the daemon
/// binds before it logs in, so a client that connects too early would be
/// answering questions about a session that is not there yet.
fn wait_until_ready(&self) {
let deadline = Instant::now() + Duration::from_secs(45);
while Instant::now() < deadline {
#[cfg(unix)]
if std::os::unix::net::UnixStream::connect(&self.socket).is_ok() {
// Binding happens first, so also wait for the login to land.
if self.answers() {
return;
}
}
std::thread::sleep(Duration::from_millis(100));
}
panic!("the daemon never became ready at {}", self.socket.display());
}
/// Whether the daemon answers `daemon status` with a session.
fn answers(&self) -> bool {
let mut command = Command::new(BIN);
for name in CLI_ENV_VARS {
command.env_remove(name);
}
command.env("SOULSEEK_STATE_DIR", &self.state);
command.current_dir(std::env::temp_dir());
command.args([
"--no-config",
"--json",
"--daemon",
&self.socket.display().to_string(),
"daemon",
"status",
]);
command
.output()
.is_ok_and(|output| output.status.success())
}
}
impl Drop for DaemonFixture {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
let _ = std::fs::remove_dir_all(&self.state);
}
} }
impl TestServer { impl TestServer {
@@ -508,6 +675,9 @@ impl TestServer {
child: None, child: None,
db: None, db: None,
_gate: gate, _gate: gate,
daemons: std::sync::Mutex::new(
std::collections::HashMap::new(),
),
}); });
} }
@@ -540,6 +710,7 @@ impl TestServer {
child: Some(child), child: Some(child),
db: Some(db), db: Some(db),
_gate: gate, _gate: gate,
daemons: std::sync::Mutex::new(std::collections::HashMap::new()),
}) })
} }
@@ -573,6 +744,34 @@ impl TestServer {
] ]
} }
/// Client arguments pointing at a daemon logged in as `user` with
/// `session_flags`, starting one if this combination has not been seen.
///
/// Keyed by the flags as well as the user: a test that shares a different
/// folder needs a daemon that shares it, and reusing one configured for
/// something else would quietly test the wrong thing.
fn daemon_args(&self, user: &str, session_flags: &[String]) -> Vec<String> {
let socket = self.daemon_for(user, session_flags);
vec![
"--no-config".to_string(),
"--quiet".to_string(),
"--daemon".to_string(),
socket.display().to_string(),
]
}
fn daemon_for(&self, user: &str, session_flags: &[String]) -> PathBuf {
let key = format!("{}|{user}|{}", self.address(), session_flags.join(" "));
let mut daemons = self.daemons.lock().expect("daemon registry");
if let Some(existing) = daemons.get(&key) {
return existing.socket.clone();
}
let handle = DaemonFixture::start(self, user, session_flags);
let socket = handle.socket.clone();
daemons.insert(key, handle);
socket
}
/// An in-process client, used to stand up the other end of a test. /// An in-process client, used to stand up the other end of a test.
fn client(&self, user: &str, shares: Vec<String>) -> Client { fn client(&self, user: &str, shares: Vec<String>) -> Client {
let port = free_port().expect("peer port"); let port = free_port().expect("peer port");
@@ -700,13 +899,62 @@ fn probe_bytes() -> Vec<u8> {
} }
/// Run the binary against `server` as `user`, appending `args`. /// Run the binary against `server` as `user`, appending `args`.
///
/// In daemon mode the same call is served by a daemon instead. The flags that
/// shape a *session* rather than a command — shares, the listener, where
/// downloads land — are handed to the daemon rather than the client, because
/// that is where they now belong. The test itself does not change, which is
/// the whole point: if the records and exit code come out the same either way,
/// daemon mode is equivalent.
fn cli(server: &TestServer, user: &str, args: &[&str]) -> Output { fn cli(server: &TestServer, user: &str, args: &[&str]) -> Output {
let mut all = server.args(user); let mut all = match mode() {
all.extend(args.iter().map(|a| (*a).to_string())); Mode::Local => server.args(user),
Mode::Daemon => {
let (session_flags, _) = split_session_flags(args);
server.daemon_args(user, &session_flags)
}
};
let command_args: Vec<String> = match mode() {
Mode::Local => args.iter().map(|a| (*a).to_string()).collect(),
Mode::Daemon => split_session_flags(args).1,
};
all.extend(command_args);
let refs: Vec<&str> = all.iter().map(String::as_str).collect(); let refs: Vec<&str> = all.iter().map(String::as_str).collect();
run(&refs) run(&refs)
} }
/// Split arguments into the ones that configure a session and the ones that
/// are the command. Only meaningful in daemon mode, where the two halves go to
/// different processes.
fn split_session_flags(args: &[&str]) -> (Vec<String>, Vec<String>) {
/// Flags that take a value and describe the session, not the command.
const SESSION_PAIRS: [&str; 4] = [
"--shared-dir",
"--listener-port",
"--download-dir",
"--max-concurrent-downloads",
];
/// Session flags that stand alone.
const SESSION_SWITCHES: [&str; 2] = ["--no-listener", "--listener"];
let mut session = Vec::new();
let mut command = Vec::new();
let mut rest = args.iter();
while let Some(arg) = rest.next() {
if SESSION_PAIRS.contains(arg) {
session.push((*arg).to_string());
if let Some(value) = rest.next() {
session.push((*value).to_string());
}
} else if SESSION_SWITCHES.contains(arg) {
session.push((*arg).to_string());
} else {
command.push((*arg).to_string());
}
}
(session, command)
}
/// Wait out the SetWaitPort registrations so peer lookups resolve. /// Wait out the SetWaitPort registrations so peer lookups resolve.
fn settle() { fn settle() {
std::thread::sleep(Duration::from_secs(1)); std::thread::sleep(Duration::from_secs(1));