Skip to main content

Probability & Bayesian Updating at LOKLOK

This page collects the probability functions that actually run in the LOKLOK engine and then develops the Bayesian view of the station: what is already Bayesian today (the Thompson-sampling RL mood arms), and where a Bayesian updating layer could be added to make the station's beliefs about its listeners explicit, honest about uncertainty, and updated continuously from telemetry.

The intent is deliberately "design-first": the RL arms are live code (smart-scheduler.js), while several of the ideas in §4–§5 are proposals for a future engine revision. Every formula that is production code is marked LIVE; every proposal is marked PROPOSED.

It is a companion to Equations & Math Reference (the full closed forms), Rotation Science (the radio context), and Smart Scheduler (the implementation).


1. The probability functions already in the engine​

LOKLOK runs four families of probability machinery. Three are distribution draws used for stochastic-but-constrained scheduling; one is the online Bayesian learning loop.

1.1 Dirichlet genre sampling (LIVE)​

The category of the next music slot is sampled from a Dirichlet distribution with concentrations proportional to the configured GENRE_SPLIT weights. The Dirichlet is the conjugate prior of the multinomial — it is the "distribution over distributions" that keeps the long-run mix on target while every individual hour follows a different path (Blei, Ng, & Jordan, 2003).

wc,αc=6 wc,Xc∼Gamma(αc),pc=Xc∑jXjw_c,\quad \alpha_c = 6\, w_c,\qquad X_c \sim \text{Gamma}(\alpha_c),\qquad p_c = \frac{X_c}{\sum_j X_j}

The Gamma draw uses the Marsaglia–Tsang algorithm for integer/real shapes and the Box–Muller transform for the normal part (Marsaglia & Tsang, 2000; Box & Muller, 1958). See Equations §2.

1.2 Beta–Bernoulli Thompson sampling (LIVE)​

The three mood arms (CHILL / BALANCED / PARTY) are a Beta–Bernoulli bandit. Each arm carries a Beta posterior over "probability that this arm is rewarded":

αk, βk,θk∼Beta(αk,βk)\alpha_{k},\ \beta_{k},\qquad \theta_k \sim \text{Beta}(\alpha_k, \beta_k)

A Bernoulli reward r∈{0,1}r \in \{0,1\} is observed after each on-air track, and the active arm's posterior is updated by the conjugate rule:

Beta(α+r, β+1−r)\text{Beta}(\alpha + r,\ \beta + 1 - r)

In the concrete engine the counts are updated by signal type — a listener count that grew is a success (α+1\alpha{+}1), a drop is a failure (β+1\beta{+}1), a hype vote is a strong success (α+2\alpha{+}2), a chill vote a strong failure (β+2\beta{+}2), and a flat count between 5 s polls is neutral (no update, to keep a steady audience from hammering β\beta). The posterior is then sampled for each arm (sampleBeta = the ratio of two Gamma draws), the argmax arm wins, and the winning arm's config patch is blended into SCHEDULER_CONFIG at rate 0.15 so manual/agent overrides still dominate instantly.

This is textbook Bayesian updating: the prior is Beta(1,1)\text{Beta}(1,1) (a uniform prior), each observation updates the posterior, and the posterior is the policy via Thompson sampling (Thompson, 1933; Russo et al., 2018). Priors are persisted to E:\radionew\models\rl_arms.json so the belief state survives restarts — the engine never forgets what it has learned. See Equations §9.

1.3 Anti-fatigue proportional control (LIVE)​

Not Bayesian, but a probability-family neighbor: the scheduler tracks observed category frequency p^c\hat p_c over the last 12 plays and applies a proportional correction to the target weight:

wc′=max⁡(0.02, wc+(wc−p^c)⋅0.5)w_c' = \max(0.02,\ w_c + (w_c - \hat p_c) \cdot 0.5)

This is a classical negative-feedback controller (Åström & Murray, 2021), the same update family a Bayesian model would use as its observation model — §4 shows how the two can be unified. See Equations §4.

1.4 The NN/VAE scoring blend (LIVE)​

Track selection scores candidates with a 0.7/0.3 blend of a trained LSTM success predictor and a station-vibe VAE (see Equations §7). These are discriminative models — they output point estimates, not posteriors. A Bayesian treatment of the same objects (Gaussian-process or MC-dropout uncertainty over the score) is one of the §5 proposals.


2. The Bayesian view of a radio station​

A Bayesian formulation makes the station's decision process explicit as a belief-update loop:

The three terms every Bayesian system needs map cleanly onto LOKLOK:

TermMeaningLOKLOK equivalent
Priorbelief before dataBeta(1,1) uniform; config GENRE_SPLIT defaults
Likelihoodhow data is generated given the stateBernoulli "reward": listeners grew / hype vote
Posteriorbelief after datarl_arms.json α,β counts; the sampled θ per arm

What is still missing to call the whole station Bayesian (rather than just the arm selector) is a single joint model that treats listener count, category, daypart, and track features as one graph. §4–§5 develop that.


3. Why Bayesian updating is the right frame for a live station​

Three properties of the live radio problem make the Bayesian frame a natural fit rather than a fashion:

  1. Data is sequential and cheap. Every track produces a listener-count sample and (occasionally) a vote. The natural estimator for "does this policy retain listeners" is a recursive filter, not a batch re-fit — exactly the conjugate-update structure of §1.2.
  2. Uncertainty is information. A bandit needs to know how sure it is, not just the point estimate. A policy played 3 times deserves more exploration than one played 3,000 times; the Beta posterior's variance encodes this automatically, and Thompson sampling turns that variance into action.
  3. Non-stationarity is the default. Radio audiences shift by daypart, season, and trend. A static optimum does not exist; a posterior that is continually updated (and whose old evidence can be discounted) tracks a moving target. The engine's neutral-observation suppression (skip updates when the count is flat) is itself a variance-control choice that prevents noise from masquerading as signal.

The reader should note the difference between Bayesian updating (recursive posterior refinement, cheap, online — what LOKLOK's arms already do) and Bayesian modeling (a generative joint model with priors over everything — the §5 proposals). The former is live; the latter is a roadmap.


4. Proposal: a unified listener-count Bayesian layer (PROPOSED)​

The RL arms already consume a binary "grew/dropped" signal. A richer, still cheap upgrade is to model the listener count itself as a Poisson-Gamma conjugate model per (arm, daypart) cell.

4.1 The Poisson–Gamma model​

Listeners arrive over a polling window; the natural generative model is λk,d\lambda_{k,d}, the expected listener count for arm kk in daypart dd:

λk,d∼Gamma(a0,b0),nk,d∼Poisson(λk,d)\lambda_{k,d} \sim \text{Gamma}(a_0, b_0), \qquad n_{k,d} \sim \text{Poisson}(\lambda_{k,d})

Because Gamma is conjugate to Poisson, observing nn listeners in one poll updates a cell's posterior in closed form:

Gamma(a0+n, b0+1)\text{Gamma}\big(a_0 + n,\ b_0 + 1\big)

The posterior mean E[λ]=a/b\mathbb{E}[\lambda] = a/b is the smoothed listener-count estimate — a recursively computed, daypart-aware average that never needs to store raw history. Comparing posteriors across arms within the same daypart then yields a per-daypart arm ranking with calibrated uncertainty, which is strictly more informative than the current single global arm switch.

4.2 Hierarchical daypart pooling​

A hierarchical version borrows strength across dayparts (Gelman et al., 2013): each cell draws its rate from a global distribution,

λk,d∣μk, κ∼Gamma(μkκ, κ)\lambda_{k,d} \mid \mu_k,\ \kappa \sim \text{Gamma}(\mu_k \kappa,\ \kappa)

so a daypart with little data inherits the arm's overall rate, and the global μk\mu_k is itself updated by every cell. In practice this means the 03:00 audience's tiny sample does not over-fit its own daypart cell — it is shrunk toward the station-wide arm behavior, which is exactly what a professional programmer does implicitly ("early mornings behave like overnights").

4.3 Decay: the non-stationary conjugate​

Raw conjugacy assumes a stationary rate. Radio is not stationary, so the proposal adds an exponential discount of past evidence — the standard trick for conjugate models under drift. After each update the counts are shrunk toward the prior:

a←ρ a+(1−ρ) a0,b←ρ b+(1−ρ) b0,ρ∈(0.9,0.99)a \leftarrow \rho\, a + (1-\rho)\, a_0, \qquad b \leftarrow \rho\, b + (1-\rho)\, b_0, \qquad \rho \in (0.9, 0.99)

A decay of ρ=0.95\rho = 0.95 gives a half-life of roughly 14 observation windows — recent audience behavior dominates, ancient history fades, and the posterior never collapses into a point mass (which would kill exploration). The existing observe() neutral-suppression stays: flat counts should still not add evidence.


5. Proposal: Bayesian track scoring (PROPOSED)​

The current track score is a point estimate from an LSTM + VAE blend (Equations §7). A Bayesian scoring layer would replace the point with a posterior predictive over "success" for each candidate track:

y^t=Ep(θ∣D)[ fθ(featurest) ]\hat y_t = \mathbb{E}_{p(\theta \mid \mathcal{D})}\big[\, f_\theta(\text{features}_t) \,\big]

with a confidence band. Two practical routes, in increasing order of effort:

  1. MC-dropout on the existing LSTM — run the network MM times with dropout active and treat the variance across passes as epistemic uncertainty. Near zero new code, a principled uncertainty per track.
  2. Gaussian-process surrogate over a small feature vector (energy, category, time-of-day, recency) — a GP gives exact posterior predictive variance and a natural acquisition function (e.g. "play the track with the highest lower-confidence-bound-adjusted score"). GPs are the standard tool for Bayesian optimization of expensive objectives; the "objective" here is listener retention per candidate.

The scheduling benefit: when two tracks score near-identically, the system can prefer the one with lower uncertainty (exploit the reliable pick) or higher (build evidence) according to a tunable exploration schedule — a principled replacement for the fixed 0.7/0.3 blend and the flat anti-fatigue gain.


6. What is live today vs. what is proposed​

ComponentStatusWhere
Dirichlet genre drawLIVEsmart-scheduler.js _pickMusicCategory
Beta–Bernoulli Thompson armsLIVEReinforcementLearningEngine, rl_arms.json
Anti-fatigue proportional controlLIVEsmart-scheduler.js category-weight correction
NN/VAE point-score blendLIVETrackSuccessPredictor + VAE
Poisson–Gamma listener layerPROPOSED§4.1
Hierarchical daypart poolingPROPOSED§4.2
Exponential evidence decayPROPOSED§4.3
Bayesian (MC-dropout / GP) track scoringPROPOSED§5

The live core already performs genuine Bayesian updating in production, 24/7, with a persisted belief state. The proposals generalize the same machinery from "which mood arm" to "how many listeners, in which daypart, with how much confidence" — the same closed-form conjugacy, applied more broadly.


7. FAQ​

Q: Is the station's learning already Bayesian? A: The mood-arm selector is: Beta–Bernoulli conjugacy with a uniform prior, posterior sampling, and persisted beliefs (rl_arms.json). The rest of the scheduler (genre draw, anti-fatigue, NN/VAE scoring) is stochastic + learned, but not Bayesian.

Q: Why Beta(1,1) as the prior? A: It is the uniform prior on [0,1] — no arm is preferred before data. With listener counts of tens, α and β grow quickly and the prior influence disappears within hours, so the choice is behaviorally neutral.

Q: Why neutral observations update nothing? A: Because a flat count between 5 s polls is evidence about the count, not about the arm. Updating β on every flat poll would punish the current arm for a steady audience. The existing observe() guards exactly this (see Equations §9).

Q: Would the Poisson–Gamma layer change scheduling? A: Not the draw mechanism — it changes the reward signal the arms consume (from a binary grew/dropped to a smoothed per-daypart rate) and would let the scheduler tune per-daypart weights from data instead of the fixed multipliers. Both are backward-compatible changes to observe().

Q: What about a fully Bayesian prior over GENRE_SPLIT? A: That is the natural end-state: a Dirichlet prior over the category weights per daypart, updated by realized category frequencies — turning §1.1's static Dirichlet draw into the posterior predictive of a learned Dirichlet-multinomial model. It is the cleanest of the proposals and a direct upgrade of live code. See Equations §2 for the current static form.