Auto-detect rich vs. entity markdown on send

Classify typed markdown on send: structure the message-entity set can't
represent (headings, lists, tables) is sent as a rich message
(RichTextMessageAttribute carrying an InstantPage); everything else takes
the existing entity path. Adds the parse-then-inspect classifier in
BrowserMarkdown, gates it in ChatControllerNode.sendCurrentMessage
(replacing the debugRichText flag), drops formulas/dividers as triggers,
and cleans up heading blocks for the chat path. Documented in CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

CLAUDE.md

@@ -128,6 +128,27 @@ The `ChatMessageDateAndStatusNode` mirrors TextBubble's placement, adapted to th
 - **`linkSelectionRects` and the bubble tap path check all six interactive keys** (URL + the five entity keys), not just URL, so press-highlight and the link-loading shimmer cover entities too.
 - **Rich-data text selection must reach a line's trailing edge.** This is general to rich-data selection, not just entities: `InstantPageTextItem.attributesAtPoint(_:orNearest:)`'s `orNearest: true` (selection-drag) path returns `line.range.upperBound` (via `CTLineGetStringRange`) when the point is at/past `lineFrame.maxX`. `TextSelectionNode` uses that index as the **exclusive** upper bound, so clamping to the last character's index — as the `orNearest: false` hit-testing path correctly does — would leave the last character/item of every line unselectable. Mirrors `Display.TextNode`. Do not collapse the two `orNearest` paths back together.
 
+## Markdown send: entity vs. rich detection
+
+On message send, the app auto-decides: if the typed markdown maps onto the regular message-entity set (bold/italic/code/strikethrough/spoiler/links/blockquote/fenced-code) it sends a **normal message** via the existing entity path; if it contains structure the entity set can't represent it sends a **rich message** (`RichTextMessageAttribute` carrying an `InstantPage`, rendered by `ChatMessageRichDataBubbleContentNode`). Always-on (no flag). **Effective rich triggers are headings, lists, and tables only.**
+
+### Where things live
+
+| File | Responsibility |
+|---|---|
+| `submodules/BrowserUI/Sources/BrowserMarkdown.swift` | The classifier `richMarkdownAttributeIfNeeded(context:text:)` (pre-filter `markdownMightNeedRichLayout` → parse via existing `inputRichTextAttributeFromText` → block inspection `instantPageNeedsRichLayout`/`blockIsEntityExpressible`/`richTextIsEntityExpressible`), plus the markdown→InstantPage conversion (`markdownWebpage`, `markdownBlocks(from:)`, `markdownBlocksWithGeneratedAnchors`). |
+| `submodules/TelegramUI/Sources/ChatControllerNode.swift` (`sendCurrentMessage`, ~line 4860) | The gate: `if !isSpecialChatContents, let attribute = richMarkdownAttributeIfNeeded(context:, text: effectiveInputText.string)` routes to the rich branch; the unchanged `else` is the entity path. |
+
+### Non-obvious invariants
+
+- **Boundary rule:** send rich iff the parse yields an `InstantPageBlock` with no entity equivalent. Entity-expressible whitelist (→ normal): `.paragraph`, `.preformatted`, `.blockQuote` (empty caption), `.anchor`, `.unsupported`, **and `.divider`/`.formula`** — the latter two are *deliberately excluded as triggers* (`---` and `$…$` are too common in casual text, e.g. `$5-$10`/`$FOO=$BAR`). Inline `.formula` is likewise treated as expressible. So effective triggers = headings, lists, tables.
+- **Approach A (parse-then-inspect):** the classifier reuses the real parser, so "what triggers rich" can't drift from "what the rich renderer shows." `markdownMightNeedRichLayout` is a cheap necessary-condition over-approximation — it may over-trigger a parse but must **never** false-negative. It detects `#`, list markers, dash-lines (`-{1,}`, which also catches setext-H2 underlines → heading blocks), `\n=` (setext H1), `|`, `![`. It deliberately does **not** detect `$`/`\(` (formulas don't trigger rich).
+- **Chat vs. document path = `file == nil` / `context.documentURL == nil`.** `inputRichTextAttributeFromText` passes `file: nil`; the document-attachment path passes a real file. Two chat-only behaviors key off this: (a) generated heading anchors are **skipped** (`markdownBlocksWithGeneratedAnchors` runs only for documents — anchors exist for intra-document `#slug` links and otherwise prepend a spurious invisible `.anchor` block per heading); (b) a level-1 `#` heading maps to `.heading(text:, level: 1)`, not `.title` (the document/article-title treatment). H2–H6 → `.heading(level: 2…6)` for both paths. This converter only ever emits `.title` (H1-doc) or `.heading` — never `.header`/`.subheader`.
+- **The classifier is fed the RAW `effectiveInputText.string`**, not the post-`convertMarkdownToAttributes` `inputText`, so inline `**bold**` survives into the rich render. The entity branch still uses the converted `inputText`.
+- **Bypassed for `.customChatContents`** (business links / quick replies) via `isSpecialChatContents`. **Editing is out of scope** — the gate is only on the compose/send path; edits route earlier to `editMessage`.
+- **Transmission:** `RichTextMessageAttribute` → `Api.InputRichMessage` via `messages.sendMessage(richMessage:)` (flag bit 23, `StandaloneSendMessage.swift`); recipients reconstruct it from the incoming `richMessage` field (`StoreMessage_Telegram.swift`). The rich branch sends `text: ""` + the attribute, nils `mediaReference` (no separate webpage preview), and bypasses 4096-char chunking. iOS < 15 / oversize markdown → `inputRichTextAttributeFromText` returns nil → entity path (which chunks). 
+- **`debugRichText` experimental flag is now orphaned** — it previously gated this path and is no longer read anywhere, though `DebugController`/`ExperimentalUISettings` still define/persist it. Optional cleanup.
+
 ## Postbox → TelegramEngine refactor (in progress)
 
 A gradual migration is underway to eliminate direct `import Postbox` from consumer submodules in favor of `TelegramEngine`.

submodules/BrowserUI/Sources/BrowserMarkdown.swift

@@ -996,6 +996,126 @@ public func inputRichTextAttributeFromText(context: AccountContext, text: String
     return RichTextMessageAttribute(instantPage: instantPage._parse())
 }
 
+// MARK: - Markdown classification (entity-expressible vs. rich layout)
+
+private let markdownBlockHeuristicRegex = try? NSRegularExpression(
+    pattern: "(^|\\n)[ \\t]*(#{1,6}[ \\t]|[-*+][ \\t]|\\d{1,9}[.)][ \\t]|-{1,}[ \\t]*(\\n|$))",
+    options: []
+)
+
+// Cheap necessary-condition pre-filter. A safe over-approximation: if it
+// returns false, the text cannot contain a rich-only construct, so the
+// expensive markdown parse is skipped and the entity path is used.
+private func markdownMightNeedRichLayout(_ text: String) -> Bool {
+    // Tables ('|') and images ('![') can appear anywhere on a line. This over-approximates
+    // (prose containing '|' also triggers a parse); the parse + block inspection resolves it.
+    if text.contains("|") || text.contains("![") {
+        return true
+    }
+    // Setext H1 heading: a line of '=' underlining the previous line.
+    // (Setext H2 dash-underlines are caught by the dash-line branch in the regex.)
+    if text.contains("\n=") {
+        return true
+    }
+    // Line-anchored block markers: headings, list items, setext-H2 dash underlines.
+    if let regex = markdownBlockHeuristicRegex {
+        let range = NSRange(text.startIndex..., in: text)
+        if regex.firstMatch(in: text, options: [], range: range) != nil {
+            return true
+        }
+    }
+    return false
+}
+
+// True when this inline RichText maps onto Telegram message entities.
+// Returns false for inline content that forces the rich path (inline image,
+// sub/superscript, highlight). Inline formulas are intentionally treated as
+// expressible: '$'-delimited spans are too common in casual text ('$5-$10').
+private func richTextIsEntityExpressible(_ text: RichText) -> Bool {
+    switch text {
+    case .empty, .plain:
+        return true
+    case .bold(let inner), .italic(let inner), .underline(let inner), .strikethrough(let inner), .fixed(let inner):
+        return richTextIsEntityExpressible(inner)
+    case .superscript, .marked, .`subscript`:
+        return false
+    case .url(let inner, _, _):
+        return richTextIsEntityExpressible(inner)
+    case .email(let inner, _):
+        return richTextIsEntityExpressible(inner)
+    case .concat(let items):
+        return items.allSatisfy(richTextIsEntityExpressible)
+    case .phone(let inner, _):
+        return richTextIsEntityExpressible(inner)
+    case .image:
+        return false
+    case .anchor(let inner, _):
+        return richTextIsEntityExpressible(inner)
+    case .formula:
+        return true
+    case .textCustomEmoji:
+        return true
+    case .textAutoEmail(let inner), .textAutoPhone(let inner), .textAutoUrl(let inner), .textBankCard(let inner), .textBotCommand(let inner), .textCashtag(let inner), .textHashtag(let inner), .textMention(let inner), .textSpoiler(let inner):
+        return richTextIsEntityExpressible(inner)
+    case .textMentionName(let inner, _):
+        return richTextIsEntityExpressible(inner)
+    }
+}
+
+private func isEmptyRichText(_ text: RichText) -> Bool {
+    switch text {
+    case .empty:
+        return true
+    case .plain(let value):
+        return value.isEmpty
+    default:
+        return false
+    }
+}
+
+// Block types that do NOT trigger a rich-layout message. Besides the genuinely
+// entity-expressible blocks (paragraph/preformatted/blockQuote/anchor), dividers
+// and formulas are intentionally excluded as triggers too: '---' and '$'-delimited
+// spans are too common in casual text to justify a rich message. Effective rich
+// triggers are therefore headings, lists, and tables.
+private func blockIsEntityExpressible(_ block: InstantPageBlock) -> Bool {
+    switch block {
+    case .paragraph(let text):
+        return richTextIsEntityExpressible(text)
+    case .preformatted(let text, _):
+        return richTextIsEntityExpressible(text)
+    case .blockQuote(let text, let caption):
+        return isEmptyRichText(caption) && richTextIsEntityExpressible(text)
+    case .anchor, .unsupported:
+        return true
+    case .divider, .formula:
+        return true
+    default:
+        return false
+    }
+}
+
+private func instantPageNeedsRichLayout(_ blocks: [InstantPageBlock]) -> Bool {
+    return blocks.contains { !blockIsEntityExpressible($0) }
+}
+
+// Returns a RichTextMessageAttribute IFF the markdown in `text` produces an
+// InstantPage block with no entity equivalent. Returns nil (-> send via the
+// regular entity path) for plain text, pre-iOS-15, oversize markdown, or
+// markdown that maps cleanly onto entities.
+public func richMarkdownAttributeIfNeeded(context: AccountContext, text: String) -> RichTextMessageAttribute? {
+    guard markdownMightNeedRichLayout(text) else {
+        return nil
+    }
+    guard let attribute = inputRichTextAttributeFromText(context: context, text: text) else {
+        return nil
+    }
+    guard instantPageNeedsRichLayout(attribute.instantPage.blocks) else {
+        return nil
+    }
+    return attribute
+}
+
 @available(iOS 15.0, *)
 private func markdownWebpage(context: AccountContext, file: (file: FileMediaReference, url: URL)?, data: Data) -> TelegramMediaWebpage? {
     let limits = markdownSafetyLimits
@@ -1029,7 +1149,15 @@ private func markdownWebpage(context: AccountContext, file: (file: FileMediaRefe
     guard let pageResult = markdownPageResult(from: attributedString, context: conversionContext) else {
         return nil
     }
-    let blocks = markdownBlocksWithGeneratedAnchors(pageResult.blocks)
+    // Heading anchors exist for intra-document navigation ([link](#slug)); they are
+    // noise in a chat message, where they prepend an invisible block per heading.
+    // Only generate them for the document path (file != nil).
+    let blocks: [InstantPageBlock]
+    if file != nil {
+        blocks = markdownBlocksWithGeneratedAnchors(pageResult.blocks)
+    } else {
+        blocks = pageResult.blocks
+    }
     guard !blocks.isEmpty, budget.validateFinalBlocks(blocks) else {
         return nil
     }
@@ -1190,7 +1318,12 @@ private func markdownBlocks(from node: MarkdownIntentNode, context: MarkdownConv
         }
         switch level {
         case Int.min ... 1:
-            return [.title(text)]
+            if context.documentURL == nil {
+                // Chat message: a single '#' is a normal heading, not a document title.
+                return [.heading(text: text, level: 1)]
+            } else {
+                return [.title(text)]
+            }
         default:
             return [.heading(text: text, level: Int32(max(2, min(level, 6))))]
         }

submodules/TelegramUI/Sources/ChatControllerNode.swift

@@ -4857,9 +4857,12 @@ class ChatControllerNode: ASDisplayNode, ASScrollViewDelegate {
                     }
                 }
                 
-                if self.context.sharedContext.immediateExperimentalUISettings.debugRichText, let attribute = inputRichTextAttributeFromText(context: self.context, text: inputText.string) {
-                    var attributes: [MessageAttribute] = []
-                    attributes.append(attribute)
+                var isSpecialChatContents = false
+                if case .customChatContents = self.chatPresentationInterfaceState.subject {
+                    isSpecialChatContents = true
+                }
+                if !isSpecialChatContents, let attribute = richMarkdownAttributeIfNeeded(context: self.context, text: effectiveInputText.string) {
+                    let attributes: [MessageAttribute] = [attribute]
                     messages.append(.message(text: "", attributes: attributes, inlineStickers: [:], mediaReference: nil, threadId: self.chatLocation.threadId, replyToMessageId: self.chatPresentationInterfaceState.interfaceState.replyMessageSubject?.subjectModel, replyToStoryId: nil, localGroupingKey: nil, correlationId: nil, bubbleUpEmojiOrStickersets: []))
                     mediaReference = nil
                 } else {