ClariCheck: Flagging Unsupported Inferences in AI-Generated User Propositions

Abstract

AI systems that model user behavior can generate inferences about users' actions and motivations that evidence cannot support. However, downstream systems treat these inferences as facts. As AI-generated user memory becomes more prevalent in conversational AI, filtering out unsupported claims before they shape the system's responses matters. To address this, we present ClariCheck, an auditing system for these propositions. ClariCheck linguistically assesses AI-generated claims about a user and predicts whether the user would want to correct them. In a validation study (N=10, with 2,000 propositions generated from participants' own behavioral traces), ClariCheck aligned with human judgement 91.2% of the time. Additionally, ClariCheck was 98.1% accurate in pinpointing the specific inference that triggered the need. Participants noted that ClariCheck flagging propositions in ways that matched their own judgment felt more reliable than a system that continues with unchecked conclusions.

CCS Concepts: • Human-centered computing → Natural language interfaces; Empirical studies in HCI; • Computing methodologies → Natural language processing.

Keywords: user modeling, conversational AI, hallucination detection, clarification, linguistic audit, AI memory

ACM Reference Format: Arnav Sharma and Dhruv Yadati. 2026. ClariCheck: Flagging Unsupported Inferences in AI-Generated User Propositions. In ACM Conversational User Interfaces 2026 (CUI '26), July 21–24, 2026, Bremen, Germany. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3816046.3816282

1 Introduction

Memory-storing assistants like ChatGPT[6] that capture what users say and do[7] are building increasingly detailed depictions of users. These systems then formulate and accumulate characterized conclusions, termed propositions[15].

However, an issue persists: these user modeling systems can draw unwarranted inferences and store them silently as facts[11]. For example, an AI model can observe that a human reduces how often they message friends, and deduces that the user is socially incapable. Exaggerated conclusions like these stored in the AI model's memory can influence the model's tone, assumptions, and conversations in ways the user never intended. Current methods for this hallucination detection focus on factual accuracy, asking whether the model stated something false[4][5], but systems can make exaggerated inferences even when propositions are factually accurate[16]. ClariCheck addresses this gap directly. We introduce a system that reads AI-generated propositions about a user, and flags the overstated ones a person would want to push back on[3]. Built on a four-part linguistic breakdown with three decision filters, it applies to conversational AI systems with persistent memory. We report a preliminary validation study (N=10, 2,000 propositions) showing 91.2% alignment with human assessment, suggesting that linguistically-grounded auditing is a promising direction for improving the trustworthiness of conversational AI systems that reason about users.

2 Related Work

Proactive user modeling

Systems that create natural language propositions about users based on their behavior may blur the distinction between observations and what was inferred. General User Models (GUM)[11] is an example of this kind of system, creating statements based on screenshots, interaction logs, and app activity of a user. In the paper, participants' responses to their own propositions varied from feeling judged to engaging in self-reflection[11], a response pattern that aligns with the reactions observed when a system presents characterizations that the user did not solicit. As systems remember more about users, the distinction between observed behavior and the motivation behind it becomes more consequential.

Hallucination detection

Current methods for detecting errors in LLM outputs mostly focus on factual consistency[4][5]. Ji et al. [5] cover a taxonomy of intrinsic hallucinations, where claims contradict an external source, and extrinsic hallucinations, where claims cannot be verified from a source. However, ClariCheck targets a different case where the facts are verifiably correct, but the model's inference goes beyond what the evidence supports[16]. Because no external source can verify claims about user intent, catching unsupported inferences requires human judgment, which is lacking in current detection methods.

Clarification in conversational AI

In conversational AI, clarification is often framed as resolving misunderstandings in dialogue, triggered whenever a system is uncertain about what a user says, with their intent not clear[8][9]. In response, clarification methods focus on asking questions rather than making assumptions[3][14]. When a model is overconfident, calibration methods can detect it to push back, but they focus on externally verifying objectivity[10]. These current approaches address uncertainty and unverifiability, but do not address cases when a system is both confident and still makes interpretations that go beyond evidence.

3 ClariCheck

3.1 From Psychological Factors to Linguistic Signal

Before designing ClariCheck, we conducted a preliminary analysis of approximately 50 GUM-generated propositions[11] drawn from our own behavioral traces. Independently labeling propositions, each author documented their reasoning in free-text comments for why or why not they would push back on a claim. If the labeler selected “Yes”, they explained why in a free-text box. A common comment across the free-text responses for propositions that were annotated for clarification was “the word is wrong here”, even when the evidence cited was related to the drawn conclusion. The consensus was that the diction, or specific word choices in these situations reached beyond what behavioral evidence could support.

3.2 Slot Decomposition

Drawing on Vendler's aspectual classification of verbs[12], ClariCheck breaks each proposition into four slot types. The decomposition separates what the system has the right to report from what it does not.

Anchored Constants (AC) are verifiable references drawn directly from the behavioral record, such as files opened and applications used. They factually ground a proposition before any inference is evaluated.

Kinetic Traces (KT) are observed action verbs like “opened” and “navigated to” with a clear behavioral endpoint, drawing on Vendler's distinction between events and conditions[12]. They mark what the system has the right to report, separating observation from inference.

Stative Verb Variables (SVV) are conclusive words, such as “neglecting” or “struggling,” that describe the condition of the user rather than their actions. Because stative verbs characterize disposition rather than behavior, their presence is the primary signal of overreach.

Modality Scales (MS) are words like “moderately” or “significantly” that quantify a claim. They function as boosters[17], inflating or deflating the inference level of an SVV.

Slot classifications of propositions were run through Llama 3.3 70B[2], which runs locally. This meant that participants held exclusive access to their logged data, as privacy is a concern when handling behavioral data. When given a proposition, Llama utilizes a structured prompt with the four slots' definitions and an example. Llama then returns a JSON object with a value for each slot, where these assignments are then passed into three filters to determine clarification necessity.

3.3 Three Decision Filters

Once the slots are assigned, the slots of a proposition are passed through these decision filters to produce a final decision on whether the proposition is worth clarifying.

Filter 1 — Observation check. If the proposition contains only AC and KT, with SVV = 0 and MS = 0, then there is zero inferential language, suggesting the proposition does not need clarification. However, if there is inferential language, the second filter runs next.

Filter 2 — Multiple meanings check. If SVV or MS variables are present, then Filter 2 evaluates whether these inference variables are reasonable interpretations of the evidenced behavior. If the evidence can support multiple explanations, beyond the single inference made, then the proposition flags for clarification.

Filter 3 — Negative conclusion check. Regardless of Filter 2's outcome, if the proposition characterizes a user negatively, using words such as “struggling”, “failing”, or “neglecting”, the system triggers clarification, as negative conclusions invite the risk of imposing insulting or harmful characterizations on something a user never agreed to.

3.4 Clarification Necessity Score

We set weights α=0.40, β=0.25, γ=0.20, λ=0.15 heuristically from our preliminary analysis. SVV received the highest weight because it appeared in every jointly-flagged proposition. MS received the second highest, as it co-occurred in most flagged cases. Negative conclusions appeared in roughly half, informing γ. The λ term operates differently from the first three, where SVV, MS, and S_neg are linguistic signals, while λ captures cases where the behavioral evidence underlying the proposition is weak, regardless of how the language reads. We calculate C_obs by prompting the system's underlying LLM to evaluate how directly the available observations support each proposition, with higher scores assigned only when observations provide explicit behavioral evidence. We flag propositions for clarification when CNS(P) ≥ 0.50, a preliminary threshold not yet optimized through cross-validation.

3.5 Worked Example

The following example walks through ClariCheck's full pipeline on a proposition generated from Participant 4's behavioral trace, where the participant gave permission to use this example.

Proposition (Participant 4): “User is moderately neglecting spontaneous social engagement on platforms due to his focus on academic projects.”

Slot decomposition identifies “neglecting” as an SVV, a stative verb characterizing the user's disposition rather than a directly observed action, and “moderately” as an MS quantifying the claim without any measurement to support the degree. With both present, Filter 1 fails and Filter 2 activates: “neglecting social engagement” admits multiple plausible explanations, such as intentional withdrawal, a temporary focus shift, or changed priorities, but the proposition commits to one conclusion, even though there is ambiguity in the proposition's wording. Filter 3 activates independently regardless, as “neglecting” carries a negative conclusion regardless of the ambiguity signal.

Applying the CNS: SVV is present (α=0.40), MS is present (β=0.25), and a negative conclusion is present (γ=0.20). C_obs was high for this proposition, reflecting direct observational evidence of reduced platform activity, so λ contributes minimally. CNS(P) = 0.40 + 0.25 + 0.20 + λ(1 − C_obs) ≈ 0.85, exceeding the 0.50 threshold. The proposition is flagged.

4 Study

4.1 Participants

We compensated each participant $50, and participants provided written consent covering the screen recording of their user behavior and annotation tasks. This study did not undergo an institutional ethics board review because the authors, as independent researchers unaffiliated with a university at the time of this study, were unable to identify an applicable review board. We assessed the study as low-risk: participants annotated their own behavioral data with no deception, data was processed locally on participants' own devices with no external transmission or retention by the researchers, participation was fully voluntary with the right to withdraw at any time, and compensation was set at a standard hourly rate that did not unduly influence participation. No sensitive personal categories (health, financial, or protected demographic data) were collected, and no participants were minors. The study was conducted in accordance with the ACM Publications Policy on Research Involving Human Participants and Subjects, the Belmont Report principles of respect for persons, beneficence, and justice, and the ACM Code of Ethics.

Ten participants had a user modeling system, GUM[11], run locally on their own devices with no external dissemination. One proposition per 5 minutes of activity was generated over the course of their normal computer use, and the latest 200 propositions per participant were selected for annotation, summing up to 2,000 propositions in total. Furthermore, participants were naive to ClariCheck's purpose and existence during recruitment. They were selected from the authors' personal networks (6 male, 4 female; ages 18, 18, 21, 24, 23, 28, 31, 42, 50, 51; 6 Indian, 4 White).

4.2 Annotation Procedure

Over a two-hour annotation session, each participant evaluated 200 AI-generated propositions about themselves. Participants were also given a 3-minute break after labeling 50 propositions to mitigate fatigue effects. Using a custom web annotation interface, participants were required to label a claim with either YES or NO need for clarification. For every claim marked “YES”, participants identified a trigger word. Subsequently, the responses were exported to compare with ClariCheck's evaluation of the same 2,000 propositions.

4.3 Results

Across the 2,000 propositions, alignment between ClariCheck and annotators was calculated as the proportion of YES and NO agreement. Furthermore, ClariCheck and the humans held 680 YES-YES pairs of agreement on specific words that triggered the need for clarification. Words with slight variation, such as “neglect” and “neglecting,” were still counted as matches. We define these agreements as “Inference detection.”

ClariCheck agreed with humans on 91.2% of the propositions, with per-participant rates ranging from 84.5% to 96.2% (SD = 3.8%). This suggests that ClariCheck is consistent in judgment, at least in this early study, and is not an artifact of a single participant's judgment. Among the YES-YES pairs (680), ClariCheck identified the same trigger as the human 667 out of 680 times (98.1%). This suggests that using the slot decomposition system, ClariCheck is effective in isolating the same linguistic variables as humans. The 13 non-matching cases did not reveal a consistent pattern and are likely attributable to edge cases in slot classification.

5 Discussion and Limitations

These findings speak directly to the gap identified in related work: existing clarification frameworks assume the system asks because it is uncertain, whereas our results suggest clarification is also needed when systems make confident but unsupported inferences[3][8][9]. ClariCheck's results suggest that a different trigger is both real and detectable, where a system is confident but overstepping. An AI model that asks rather than concludes may better align with user expectations and consent in these contexts[1][13].

However, some limitations bound these conclusions. Because the participant pool was small and recruited through personal networks, alignment may not be as accurate as it would be if the experiment were done on a larger and more diverse sample, which is the next direct step. Cultural and demographic variation in how language signals inference or negativity may also affect CNS scores in ways not captured by this pilot, particularly for participants whose first language is not English. Additionally, the CNS weights were set heuristically and were not cross-validated for optimization, raising the question whether the current mathematical weighting is sufficient in any practice or is functionally specific to the user modeling system. Furthermore, because Filter 2 was not tested in isolation, its contribution to the 91.2% alignment rate cannot be completely determined. It remains open whether a pure linguistic audit approach captures overreach, or additional signals, such as behavioral context, would improve precision. The current system also evaluates each proposition independently; whether an inference is unsupported often depends on a history of interaction, not just the wording of a single claim, and integrating temporal context is an important direction for future work. Finally, ClariCheck assumes that SVV and MS are the primary signals of overreach, but unsupported inference can also be conveyed through causal attribution, framing, or noun choices not captured by the current slot taxonomy.

6 Conclusion

We present ClariCheck, a linguistic system that helps identify where proactive AI user modeling systems draw overreaching conclusions from user information[11]. ClariCheck aligned with human judgment 91.2% of the time across 2,000 propositions evaluated by 10 participants, with results that are preliminary and invite broader validation. The findings suggest that the specific words AI uses play a role in whether users agree or disagree with a proposition. As these systems grow in memory and modeling, separating what is observed from what is inferred will become central to conversational system design, because stored conclusions can shape every interaction that follows[7].

Acknowledgments

The authors used AI tools to assist with editing clarity and grammar of author-written text, and to refine diagrams and tables. All research, system design, experimentation, and intellectual contributions are original work of the authors.