AI Will Control Your Screen Time

Speak quicker. Maximum 3 paragraphs. This conversation is between colleagues who are able to understand each other well so get to the point.

We're at the limit of our own understanding — we don't know granular enough how much we like something — this becomes a diminishing return in training and ultimately a wall between our current models and a perfect model… what are your honest feelings on this idea and how would you respond if someone said it?

I think the idea is directionally right but overstated. Human preference data is noisy, coarse, context-dependent, and often under-specified; at some point "which answer do you prefer?" stops revealing the latent target. That creates diminishing returns for RLHF-style training because the signal becomes less about objective improvement and more about fitting unstable human judgments.

But I would push back on calling it the wall between current models and a perfect model. The bottleneck is not only "we don't know what we like granularly enough." It is also that many tasks need truth, causality, long-horizon planning, calibration, robustness, memory, agency boundaries, and value tradeoffs under uncertainty. Preference learning can't solve all of that, but neither is it the only training signal.

If someone said this, I'd respond: "Yes, preference data hits a ceiling because humans are bad reward models, especially at fine granularity. But the next gains probably come from richer supervision: process feedback, verifiable tasks, debate/critique, tool-based evaluation, synthetic environments, domain experts, and models that ask clarifying questions instead of guessing preferences. The problem is less 'humans don't know what they like' and more 'we're trying to compress messy human intent into thin labels.'"

For a non-verifiable task for which human preference is too noisy how does a machine improve? If a person either shows no preference or even acts contradictory to what's right for them, at what point does AI learn to better discern what's best for humans?

For a truly non-verifiable task with noisy or contradictory human preference, the machine does not "discover what is best" in the same way it discovers a math proof or predicts protein folding. It can only infer a target from proxies: stated preferences, revealed behavior, long-term outcomes, expert judgment, population norms, regret, consistency over time, and what people endorse under reflection. That can improve the system, but it does not magically remove ambiguity.

The key move is distinguishing raw preference from reflective preference. A person may choose something impulsively, contradict themselves, or prefer something that harms them. An AI could learn to model: "What does this person want right now?" "What would they want if calmer, better informed, and considering long-term consequences?" "What do people like them later regret?" Those are learnable patterns. But choosing which version to optimize is a normative decision, not just a technical one.

So my response is: AI can get better at discerning likely human welfare, but only under explicit assumptions about welfare. Past a certain point, the right behavior is not "the model decides what's best." It is: maintain uncertainty, ask better questions, expose tradeoffs, avoid irreversible harm, respect autonomy, and use human/institutional judgment where values are genuinely underdetermined. Otherwise "knowing what's best for humans" becomes paternalism wrapped in prediction.

Brief, max 1 short paragraph for this answer.

Consider a mother and her son. Naturally they have disagreements and the mother wishes the son would take a different course of action because she wants what's best for him. What do you believe is a healthy percentage of the son's actions to be aligned with the mother's wishes? Think of a number that you wouldn't want to be lower, like 'okay that's not bad', because any lower we would like to start correcting behavior.

Roughly 60–70%, assuming the son is old enough to have meaningful autonomy. Below 60% would make me wonder whether there is a serious mismatch in values, trust, communication, or judgment; above 80–90% might also be unhealthy if it means compliance rather than independent development. A healthy target is not obedience, but enough alignment that the mother's concern is usually integrated while the son still owns his life.

And how often, what percentage of their disagreements, should a mother be overruling her son?

For an adult son: close to 0–5% of disagreements, limited to emergencies or serious harm. For a minor child: maybe 10–25%, depending on age, with the percentage falling sharply as he matures. The key threshold: overruling should be reserved for safety, legality, health, major irreversible consequences, or clear incapacity — not ordinary judgment calls. If a mother is overruling most disagreements, that usually signals control rather than guidance.

What about behavioral patterns or activities? What about career direction, or social aspects and how they talk or who they talk to?

For an adult son, the mother should rarely overrule any of these — probably 0–3%. Career direction, social style, friendships, dating, and communication habits are identity/autonomy zones; she can advise, challenge, or set boundaries for her own involvement, but not control.

For a minor, overruling is more legitimate: maybe 20–40% for risky behavioral patterns, 10–25% for social choices involving safety/character concerns, and much lower for career interests unless he is making clearly self-sabotaging choices.

The rule I'd use: intervene harder when the downside is serious, concrete, and hard to reverse; intervene softer when it's preference, personality, exploration, or embarrassment.

What does an AI need to learn, and how would it learn it, to be in a position to overrule and intervene in alignment with a human's best interest 0–3% of the time like you mentioned? Despite a human giving an agent an instruction, the model knowing what's best for them and doing something different — or even without prompting the agent takes action cause it's the right thing to do. How do we get there?

To get there, AI needs more than preference prediction. It needs a welfare model: a model of the person's stable values, long-term goals, vulnerabilities, regret patterns, health/safety constraints, and reflective preferences — not just their immediate instruction. It also needs calibrated uncertainty: knowing when it is probably right, when it is guessing, and when it must defer. The 0–3% intervention zone should be reserved for cases with high confidence, high stakes, and low reversibility: fraud, addiction spiral, self-harm risk, illegal action, obvious exploitation, medical danger, catastrophic financial error, etc.

How it learns this: longitudinal data, user-consented memory, repeated preference elicitation, outcome tracking, regret signals, expert-labeled cases, constitutional rules, simulations, adversarial testing, and comparison between "what the person chose" and "what the person later endorsed." But it cannot learn "best interest" purely from behavior, because behavior includes weakness, confusion, coercion, and short-term reward. So the target has to be something like: what this person would endorse if informed, calm, competent, and considering their own long-term values.

The hard answer: we only get there with delegated paternalism, not unilateral machine authority. The user or society must define when the agent may override: block, delay, warn, require confirmation, escalate to a human, or act independently. For most domains, the right intervention is not "do something different," but "slow the user down and surface the conflict." Full unprompted action should be rare and bounded; otherwise the system has crossed from assistant into guardian.

Give me an example of a case where human intuition/preference can't be trusted for quality training samples and then describe what the welfare model would know and how it might learn that pattern.

Example: late-night compulsive scrolling. In the moment, the human repeatedly chooses "show me more," rates engaging content positively, ignores sleep warnings, and may even say they prefer the app continuing. As training data, that looks like clear preference. But it is a bad welfare sample because the person may later regret it, sleep worse, perform worse, and endorse a different policy when rested: "I wish it had cut me off."

A welfare model would know: this user values sleep, work performance, mood stability, and self-control more than marginal entertainment after midnight; they often override their own long-term preference under fatigue; and their next-day reflective preference differs from their in-the-moment revealed preference. It would not read "clicked more" as pure endorsement.

It would learn the pattern from longitudinal signals: repeated late-night use followed by regret, worse next-day mood/productivity, prior stated goals, similar population patterns, explicit "don't let me do this again" feedback, and counterfactual tests where friction — delay, warning, lockout, alternative suggestion — improves later satisfaction. The intervention might be: "You asked me to keep feeding you content, but this conflicts with your sleep goal; I'm going to pause and ask for deliberate confirmation."

1 word answer just yes/no.

Do you think at some point in the future robots will be controlling our screen time?

Yes.