RichText.textCustomEmoji: inline animated custom emoji in rich-data bubbles

Add the RichText.textCustomEmoji(fileId:alt:) case end-to-end: model +
Postbox coding, FlatBuffers schema/codec, and Api.RichText
parsing/serialization (lossless), plus display in the InstantPage V2
renderer. The emoji renders as an InlineStickerItemLayer that participates
in the streaming reveal (pops in as the reveal cursor crosses it) and is
gated by the bubble's visibility rect, propagated recursively through the
nested V2 view tree. Also frees the CTRunDelegate extent buffers for the
image/formula/custom-emoji attachment arms and documents the feature in
CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
isaac 2026-05-23 22:12:47 +04:00
parent ded413ae74
commit b48b96ecd9
14 changed files with 408 additions and 148 deletions

View file

@ -74,6 +74,31 @@ Spec: [`docs/superpowers/specs/2026-05-19-richdata-streaming-animation-design.md
- **`Thinking…` header positioning matches TextBubble.** `streamingTextFrame.origin = (bubbleInsets.left - textInsets.left, topInset - textInsets.top)`. `streamingHeaderOffset` = visible bottom + 1pt spacing = where pageView's `frame.origin.y` and statusFrame y-shift attach. Bubble minimum width includes `visible_thinking_width + bubbleInsets.left + bubbleInsets.right + 2`.
- **Display-link tick re-layouts on extent change.** Tick reads `revealedContentSize` at the new cursor; if the height differs from the previous cursor, calls `requestFullUpdate`. So the bubble grows in flight when the cursor crosses a line/item boundary, not just between chunks. Tick passes `animated: true` to `applyReveal` to fire the snippet pop-in.
## Inline custom emoji (RichText.textCustomEmoji)
`RichText.textCustomEmoji(fileId:alt:)` renders an inline **animated** custom emoji inside rich-data bubbles. Covers API parsing, Postbox + FlatBuffers serialization, and display in the InstantPage V2 renderer; the emoji participates in the streaming reveal above.
### Where things live
| File | Responsibility |
|---|---|
| `submodules/TelegramCore/Sources/SyncCore/SyncCore_RichText.swift` | Enum case `textCustomEmoji(fileId: Int64, alt: String)` + Postbox coding (discriminator 17, keys `ce.f`/`ce.a`), `==`, `plainText` (returns `alt`), and FlatBuffers codec. |
| `submodules/TelegramCore/FlatSerialization/Models/RichText.fbs` | FlatBuffers schema — `RichText_CustomEmoji` union member + table. **Source of truth**; the Bazel `flatc` genrule regenerates `*_generated.swift` at build time (the checked-in `Sources/*_generated.swift` is stale). |
| `submodules/TelegramCore/Sources/ApiUtils/RichText.swift` | `Api.RichText.textCustomEmoji` ⇄ Swift, lossless both ways. |
| `submodules/InstantPageUI/Sources/InstantPageTextItem.swift` (`attributedStringForRichText`) | Emits a single placeholder char carrying `ChatTextInputAttributes.customEmoji` (a `ChatTextInputTextCustomEmojiAttribute`) + a `CTRunDelegate` sized `font.pointSize · 24/17`. |
| `submodules/InstantPageUI/Sources/InstantPageV2Layout.swift` (line-breaker) | Collects per-line `InstantPageTextLine.emojiItems`; overwrites each placeholder char's `characterRect` with a full cell (`width = itemSize`) so it feeds the reveal cost map. |
| `submodules/InstantPageUI/Sources/InstantPageRenderer.swift` (`InstantPageV2View`) | Owns the `InlineStickerItemLayer`s: `updateInlineEmoji` (create/reuse/remove/position), `updateEmojiReveal` (reveal-driven pop-in), `updateEmojiVisibility` + `propagateVisibilityRect`. Layers attach to each text view's `emojiContainerView`. |
### Non-obvious invariants
- **flatc casing/`required` gotchas.** Edit `RichText.fbs`, not the generated Swift. Scalars (`long`) cannot be `(required)` — only strings/tables can. A union member `RichText_CustomEmoji` generates the Swift enum case `.richtextCustomemoji` (everything after the suffix's first letter is lowercased); the table type stays `TelegramCore_RichText_CustomEmoji` and field accessors keep `.fbs` casing (`value.fileId`). See the `flatbuffers-codegen` memory.
- **`ChatTextInputTextCustomEmojiAttribute` is reused end-to-end** (display layer ⇄ layout model). The attribute is written to the placeholder in `attributedStringForRichText` and read back by the V2 line-breaker under the SAME key (`ChatTextInputAttributes.customEmoji`); `InlineStickerItemLayer.init` consumes it directly and resolves the file lazily from `fileId`.
- **Emoji participates in the streaming reveal.** Its placeholder char's `characterRect` is overwritten to a full cell (width = `itemSize`, baseline-relative bottom at `y=0`), so the width-based cost map charges it like other content. `updateEmojiReveal` pops the layer in (alpha 0→1 + scale) when `charIndexInItem < currentRevealCharacterCount`; unrevealed → opacity 0.
- **Layers sit ABOVE the reveal mask.** They attach to `InstantPageV2TextView.emojiContainerView` (a sibling above `renderContainer`), NOT inside it — so the reveal mask wipes glyphs while emoji pop in independently. Adding a CTRunDelegate-glyph to the mask would clip-wipe them instead.
- **Layers are owned by `InstantPageV2View`, not the text view.** Keyed by `InlineStickerItemLayer.Key(id: fileId, index: occurrence)`. The pageView is rebuilt per `stableVersion` bump (see streaming section), so the dict starts fresh each chunk — no orphan/leak across rebuilds; within one view, stale keys are pruned.
- **`visibilityRect` gates looping; `nil` means "not visible".** The bubble's `visibility` override pushes a full-width sub-rect to the root `pageView.visibilityRect`, re-pushed in the apply closure after `pageView.frame` is set (because `streamingHeaderOffset` shifts across chunks without a `visibility` change). `propagateVisibilityRect` converts the rect into each nested V2View's coordinate space (`self.convert(_:to:)`) for details bodies / table cells+title, fanning out via each child's `didSet`.
- **CTRunDelegate extent buffers must be freed.** Every inline-attachment arm (`.image`/`.formula`/`.textCustomEmoji`) in `attributedStringForRichText` allocates an `extentBuffer`; the `dealloc` callback must `deallocate()` it (it re-runs per layout pass).
## Postbox → TelegramEngine refactor (in progress)
A gradual migration is underway to eliminate direct `import Postbox` from consumer submodules in favor of `TelegramEngine`.

View file

@ -1,138 +0,0 @@
# RichText.textCustomEmoji — parsing, serialization & display
**Date:** 2026-05-22
**Status:** Design approved, ready for implementation plan
## Goal
Add full support for inline custom emoji (`textCustomEmoji`) in `RichText`:
1. **API parsing** — decode `Api.RichText.textCustomEmoji(documentId, alt)` into a real Swift `RichText` case instead of dropping it to `.plain("")`.
2. **Serialization** — round-trip the case through the API, Postbox coding, and FlatBuffers.
3. **Display** — render the emoji as a **full animated** inline sticker inside the InstantPage V2 text renderer used by `ChatMessageRichDataBubbleContentNode`, including participation in the AI streaming reveal.
## Context
`RichText` is the InstantPage rich-text tree. AI "rich data" chat bubbles render a synthesized InstantPage V2 document via `ChatMessageRichDataBubbleContentNode``InstantPageV2View`.
Today `Api.RichText.textCustomEmoji` (which carries `documentId: Int64, alt: String`) is mapped to `.plain("")` in `submodules/TelegramCore/Sources/ApiUtils/RichText.swift:56-58` — the emoji is silently lost. The Swift `RichText` enum, Postbox coding, and FlatBuffers schema have no custom-emoji case.
The V2 text renderer (`InstantPageRenderer.swift`, `TextRenderView`) currently draws only glyphs + decorations; inline images render as grey placeholders. Inline emoji rendering is therefore greenfield, but the codebase has a mature reference: `InteractiveTextNodeWithEntities` drives `InlineStickerItemLayer`s for inline custom emoji in chat text.
### Key facts established during exploration
- The Swift `RichText` enum, Postbox `init(decoder:)`/`encode(_:)`, `==`, `plainText`, and the FlatBuffers extension all live in `submodules/TelegramCore/Sources/SyncCore/SyncCore_RichText.swift`. The enum has 17 cases today (`empty``formula`), Postbox discriminators 016.
- FlatBuffers `*_generated.swift` files are **regenerated at build time** by a Bazel `flatc` genrule (`submodules/TelegramCore/FlatSerialization/BUILD`, target `GenerateModels`) from the `.fbs` sources. The checked-in `RichText_generated.swift` is stale (it is already missing `formula`). **The `.fbs` is the source of truth.**
- `attributedStringForRichText` (`submodules/InstantPageUI/Sources/InstantPageTextItem.swift:669`) is **shared** by the V1 and V2 layout paths — single edit point for attribute insertion.
- `InstantPageTextLine` (`InstantPageTextItem.swift:79`) carries per-line `imageItems`/`formulaItems`; the V2 line-breaker (`InstantPageV2Layout.swift:~2553-2746`) collects them and computes per-character ink rects for the reveal mask.
- `InlineStickerItemLayer` lives in `submodules/TelegramUI/Components/EmojiTextAttachmentView`; it takes a `ChatTextInputTextCustomEmojiAttribute`, an `AnimationCache`, and a `MultiAnimationRenderer`, resolves the file lazily from `fileId` when `file == nil`, and is driven by `frame` + `isVisibleForAnimations` + `dynamicColor`. Reference usage: `InteractiveTextNodeWithEntities.updateInteractiveContents` (`submodules/TelegramUI/Components/InteractiveTextComponent/Sources/InteractiveTextNodeWithEntities.swift:279`).
- InstantPageUI already depends on `TextFormat` and `AccountContext` but **not** on `EmojiTextAttachmentView`/`AnimationCache`/`MultiAnimationRenderer`. Adding those is **cycle-free** (verified: none of them depend on InstantPageUI).
- No visibility plumbing exists in the bubble or `InstantPageV2View` today. `ChatMessageTextBubbleContentNode:129` is the reference for forwarding `visibility``visibilityRect`.
### Decisions
- **Display fidelity:** full animated `InlineStickerItemLayer` (lazy file resolution, animation, visibility-driven looping). Managed at the `InstantPageV2View` level.
- **Streaming reveal:** emoji **participates in the reveal** — contributes its width to the cost map and pops in (alpha + scale) when the reveal cursor crosses its glyph position.
- **Attribute representation (Approach A):** reuse `ChatTextInputAttributes.customEmoji` + `ChatTextInputTextCustomEmojiAttribute` end-to-end. The layout model carries the chat-input attribute directly (TextFormat is already a dependency), and `InlineStickerItemLayer.init` consumes it without translation.
- **Enum payload:** `case textCustomEmoji(fileId: Int64, alt: String)`. Raw `Int64` (not `MediaId`) — namespace is always `Namespaces.Media.CloudFile`, the API gives `documentId: Int64`, and the display layer speaks `fileId: Int64`.
## Section 1 — Data model, parsing & serialization
### `SyncCore_RichText.swift`
- Add `case textCustomEmoji(fileId: Int64, alt: String)` to the `RichText` enum.
- Add `case textCustomEmoji = 17` to the private `RichTextTypes` discriminator enum.
- `init(decoder:)`: decode `Int64` (key `"ce.f"`) + `String` (key `"ce.a"`).
- `encode(_:)`: encode discriminator 17 + the two fields.
- `==`: add the case comparison.
- `plainText`: return `alt` (so text extraction / accessibility / copy yields the fallback glyph).
- FlatBuffers extension: add the `.richtextCustomEmoji` case to `init(flatBuffersObject:)` and the `.textCustomEmoji` case to `encodeToFlatBuffers`.
### `submodules/TelegramCore/Sources/ApiUtils/RichText.swift`
- `init(apiText:)`: replace the stub at lines 5658 with
`self = .textCustomEmoji(fileId: data.documentId, alt: data.alt)`.
- `apiRichText()`: add
`case let .textCustomEmoji(fileId, alt): return .textCustomEmoji(.Cons_textCustomEmoji(documentId: fileId, alt: alt))`.
(Confirmed constructor: `Api.RichText.Cons_textCustomEmoji(documentId: Int64, alt: String)`.)
### `submodules/TelegramCore/FlatSerialization/Models/RichText.fbs`
- Append `RichText_CustomEmoji` to the `RichText_Value` union (after `RichText_Formula` — appending keeps existing discriminators stable).
- Add `table RichText_CustomEmoji { fileId:long (id: 0, required); alt:string (id: 1, required); }`.
- The Bazel build regenerates `RichText_generated.swift`; no manual edit of the generated file is required for the Bazel build.
## Section 2 — Layout & reveal participation
### Attribute insertion — `attributedStringForRichText` (`InstantPageTextItem.swift:669`)
New `.textCustomEmoji(fileId, alt)` case emits **one placeholder character** carrying:
- `ChatTextInputAttributes.customEmoji` = `ChatTextInputTextCustomEmojiAttribute(interactivelySelectedFromPackId: nil, fileId: fileId, file: nil, custom: nil)` — file resolves lazily in the layer; this attribute doubles as the line-breaker's marker.
- the current style-stack font + foreground color (correct baseline/line metrics).
- a `CTRunDelegate` sized `itemSize = font.pointSize · 24.0/17.0`, ascent/descent from the font, width = `itemSize` (identical to the `InteractiveTextNodeWithEntities` reference).
### Line model — `InstantPageTextItem.swift`
- New struct: `struct InstantPageTextEmojiItem { let frame: CGRect; let range: NSRange; let emoji: ChatTextInputTextCustomEmojiAttribute }`.
- `InstantPageTextLine` gains `let emojiItems: [InstantPageTextEmojiItem]`, added to its initializer with default `[]` so V1 and all other call sites compile unchanged.
### V2 collection — `InstantPageV2Layout.swift` line-breaker (~25532746)
Mirror the existing `pendingImages` flow:
- In the run-attribute walk (~2566) add an `else if` reading `ChatTextInputAttributes.customEmoji` → append to a `pendingEmoji` list with `xOffset` (`CTLineGetOffsetForStringIndex`), `range`, `itemSize`, and the attribute.
- Add an emoji ascent loop alongside the image/formula loops so a tall emoji grows `lineAscent`.
- Build `InstantPageTextEmojiItem` frames from `baselineY`, vertically centered on the line, and pass `emojiItems: lineEmojiItems` into the `InstantPageTextLine(...)` constructor.
### Reveal participation
After `lineCharacterRects` is built (~27002742), overwrite each emoji placeholder char's slot with its full cell rect (`x = CTLineGetOffsetForStringIndex(...)`, width/height = `itemSize`, baseline-relative to match the glyph-rect convention). The glyph-ink path would otherwise leave the slot ~empty (the run-delegate run has no real glyph). With a real rect:
- the emoji automatically contributes `itemSize` points of width to `InstantPageV2RevealCostMap` (cost unit = points of width), and
- the reveal cursor/mask gets a position to cross.
Expected: **no change** to `InstantPageV2RevealCost.swift` beyond verifying the cost sum picks the new rect up.
## Section 3 — Display lifecycle & bubble wiring
### Layer management in `InstantPageV2View` (`InstantPageRenderer.swift`)
Mirrors `InteractiveTextNodeWithEntities.updateInteractiveContents`:
- New state: `inlineStickerItemLayers: [InlineStickerItemLayer.Key: EmojiLayerData]`, where `EmojiLayerData` holds the layer, a weak ref to its owning `InstantPageV2TextView`, the emoji's local frame, and its global char index (for reveal gating).
- New `updateInlineEmoji()`, called at the end of `update(layout:theme:animation:)`. No-op when `renderContext == nil` (consistent with the grey-placeholder fallback). It walks `currentLayout` text items → lines → `emojiItems`, allocates `InlineStickerItemLayer.Key(id: fileId, index:)` by occurrence order (`nextIndexById`), and creates/reuses/removes layers:
```swift
InlineStickerItemLayer(
context: rc.context, userLocation: .other, attemptSynchronousLoad: false,
emoji: item.emoji, file: item.emoji.file,
cache: rc.context.animationCache, renderer: rc.context.animationRenderer,
placeholderColor: <theme secondary/link color>,
pointSize: CGSize(width: floor(itemSize · 1.3), height: floor(itemSize · 1.3)),
dynamicColor: <text color>)
```
- Layers attach to a new `emojiContainerView` on `InstantPageV2TextView`, layered **above** `renderContainer` so the reveal mask wipes glyphs while emoji pop in independently.
### Visibility + reveal pop-in
- `InstantPageV2View.visibilityRect: CGRect?` (new) — `didSet` updates every layer's `isVisibleForAnimations = enableLooping && rect.intersects(visibilityRect) && isRevealed`, matching the reference. `enableLooping = rc.context.sharedContext.energyUsageSettings.loopEmoji`.
- `applyReveal(revealedCount:costMap:animated:)` (`InstantPageV2RevealCost.swift`) — after it computes each text view's revealed char count, it sets each emoji's revealed state. Crossing into the revealed range with `animated` → alpha 0→1 + small scale pop (matches the snippet pop-in aesthetic); unrevealed → alpha 0 and not animating.
### Bubble wiring — `ChatMessageRichDataBubbleContentNode.swift`
- Override `visibility` (none today), following `ChatMessageTextBubbleContentNode:129`: compute the full-width `subRect` and forward to `pageView.visibilityRect`.
- No render-context change: the bubble already builds `InstantPageV2RenderContext` with an `AccountContext`; the V2View reads `animationCache`/`animationRenderer` from it.
### BUILD
`submodules/InstantPageUI/BUILD` gains deps: `EmojiTextAttachmentView`, `AnimationCache`, `MultiAnimationRenderer` (cycle-free, verified).
## Out of scope / noted
- The legacy V1 InstantPage line-breaker (`InstantPageTextItem.swift` layout) is **not** taught to collect emoji items, so a `textCustomEmoji` inside a classic Instant View would render as a blank placeholder. Acceptable — this task targets the rich-data bubble. The shared `attributedStringForRichText` change is harmless there (the placeholder simply isn't picked up).
## Verification
- Build the full `Telegram/Telegram` target via `Make.py` (no per-module build; no unit tests in this project).
- In the simulator: confirm a `textCustomEmoji` node decodes from the wire (not blank), animates inline at the correct baseline/size, pops in as the AI stream reveals across it, and loops only while on-screen.
- Confirm Postbox + FlatBuffers round-trip (a re-decoded message preserves `fileId`/`alt`).

View file

@ -1956,6 +1956,8 @@ private func markdownDroppingPrefixLength(_ length: Int, from text: RichText) ->
case let .anchor(inner, name):
let dropped = markdownDroppingPrefixLength(length, from: inner)
return dropped == .empty ? .empty : .anchor(text: dropped, name: name)
case .textCustomEmoji:
return text
}
}
@ -1985,6 +1987,8 @@ private func markdownHasDisplayableContent(_ richText: RichText) -> Bool {
return true
case let .formula(latex):
return !latex.isEmpty
case .textCustomEmoji:
return true
}
}
@ -2014,6 +2018,8 @@ private func markdownIsWhitespaceOnly(_ richText: RichText) -> Bool {
return false
case let .formula(latex):
return latex.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
case .textCustomEmoji:
return false
}
}

View file

@ -344,6 +344,8 @@ private func trimStart(_ input: RichText) -> RichText {
break
case .formula:
break
case .textCustomEmoji:
break
}
return text
}
@ -388,6 +390,8 @@ private func trimEnd(_ input: RichText) -> RichText {
break
case .formula:
break
case .textCustomEmoji:
break
}
return text
}
@ -433,6 +437,8 @@ private func trim(_ input: RichText) -> RichText {
break
case .formula:
break
case .textCustomEmoji:
break
}
return text
}
@ -477,6 +483,8 @@ private func addNewLine(_ input: RichText) -> RichText {
break
case let .formula(latex):
text = .concat([.formula(latex: latex), .plain("\n")])
case .textCustomEmoji:
break
}
return text
}

View file

@ -120,6 +120,8 @@ extension RichText {
case .formula:
//TODO:localize
return "Fx"
case let .textCustomEmoji(_, alt):
return alt
}
}
}

View file

@ -19,6 +19,9 @@ swift_library(
"//submodules/TelegramPresentationData:TelegramPresentationData",
"//submodules/TelegramUIPreferences:TelegramUIPreferences",
"//submodules/TextFormat:TextFormat",
"//submodules/TelegramUI/Components/EmojiTextAttachmentView:EmojiTextAttachmentView",
"//submodules/TelegramUI/Components/AnimationCache:AnimationCache",
"//submodules/TelegramUI/Components/MultiAnimationRenderer:MultiAnimationRenderer",
"//submodules/GalleryUI:GalleryUI",
"//submodules/MusicAlbumArtResources:MusicAlbumArtResources",
"//submodules/LiveLocationPositionNode:LiveLocationPositionNode",

View file

@ -8,6 +8,10 @@ import TelegramUIPreferences
import AccountContext
import GalleryUI
import ComponentFlow
import TextFormat
import EmojiTextAttachmentView
import AnimationCache
import MultiAnimationRenderer
// MARK: - Stable item identity (for view reuse on re-layouts)
@ -74,6 +78,19 @@ public final class InstantPageV2RenderContext {
}
}
// MARK: - Emoji layer data
private final class InstantPageEmojiLayerData {
let itemLayer: InlineStickerItemLayer
weak var textView: InstantPageV2TextView?
var charIndexInItem: Int = 0
var revealed: Bool = false
init(itemLayer: InlineStickerItemLayer) {
self.itemLayer = itemLayer
}
}
// MARK: - Public renderer
public final class InstantPageV2View: UIView {
@ -88,6 +105,20 @@ public final class InstantPageV2View: UIView {
public let renderContext: InstantPageV2RenderContext?
private var inlineStickerItemLayers: [InlineStickerItemLayer.Key: InstantPageEmojiLayerData] = [:]
private var emojiEnableLooping: Bool = true
/// Scroll-visibility rect in this view's coordinate space; gates emoji animation looping.
/// `nil` means "not visible" emoji don't animate. The root rect is set by the bubble and
/// propagated down the nested tree (details/table) by `propagateVisibilityRect`.
public var visibilityRect: CGRect? {
didSet {
if oldValue != self.visibilityRect {
self.updateEmojiVisibility()
}
}
}
// Weak references to every media wrapper in the tree, keyed by `InstantPageMedia.index`.
// Used by `transitionArgsFor` and `applyHiddenMedia` so the gallery transition + hidden-source
// state can find a wrapper without walking the view hierarchy. Nested V2Views (details body,
@ -191,6 +222,172 @@ public final class InstantPageV2View: UIView {
self.itemViewStableIds = newStableIds
self.currentLayout = layout
self.currentTheme = theme
self.updateInlineEmoji()
}
func updateInlineEmoji() {
guard let rc = self.renderContext else { return }
let context = rc.context
let cache = context.animationCache
let renderer = context.animationRenderer
self.emojiEnableLooping = context.sharedContext.energyUsageSettings.loopEmoji
var nextIndexById: [Int64: Int] = [:]
var validIds: [InlineStickerItemLayer.Key] = []
for view in self.itemViews {
guard let textView = view as? InstantPageV2TextView else { continue }
let textItem = textView.item.textItem
let boundsWidth = textItem.frame.size.width
for line in textItem.lines {
if line.emojiItems.isEmpty { continue }
let lineFrame = v2FrameForLine(line, boundingWidth: boundsWidth, alignment: textItem.alignment)
for emojiItem in line.emojiItems {
let index = nextIndexById[emojiItem.emoji.fileId] ?? 0
nextIndexById[emojiItem.emoji.fileId] = index + 1
let id = InlineStickerItemLayer.Key(id: emojiItem.emoji.fileId, index: index)
validIds.append(id)
let itemSize = emojiItem.frame.width
let localX = lineFrame.minX + (emojiItem.frame.minX - line.frame.minX)
let itemFrame = CGRect(
x: localX + v2TextViewClippingInset,
y: emojiItem.frame.minY + v2TextViewClippingInset,
width: itemSize,
height: itemSize
)
var textColor: UIColor?
if emojiItem.range.location < textItem.attributedString.length {
textColor = textItem.attributedString.attribute(.foregroundColor, at: emojiItem.range.location, effectiveRange: nil) as? UIColor
}
let data: InstantPageEmojiLayerData
if let existing = self.inlineStickerItemLayers[id] {
data = existing
if data.itemLayer.superlayer !== textView.emojiContainerView.layer {
textView.emojiContainerView.layer.addSublayer(data.itemLayer)
}
} else {
let pointSize = floor(itemSize * 1.3)
let layer = InlineStickerItemLayer(
context: context,
userLocation: .other,
attemptSynchronousLoad: false,
emoji: emojiItem.emoji,
file: emojiItem.emoji.file,
cache: cache,
renderer: renderer,
placeholderColor: UIColor(white: 0.5, alpha: 0.3),
pointSize: CGSize(width: pointSize, height: pointSize),
dynamicColor: textColor
)
layer.opacity = 0.0
data = InstantPageEmojiLayerData(itemLayer: layer)
self.inlineStickerItemLayers[id] = data
textView.emojiContainerView.layer.addSublayer(layer)
}
data.itemLayer.dynamicColor = textColor
data.itemLayer.frame = itemFrame
data.textView = textView
data.charIndexInItem = emojiItem.range.location
}
}
}
var removeKeys: [InlineStickerItemLayer.Key] = []
for (key, data) in self.inlineStickerItemLayers where !validIds.contains(key) {
removeKeys.append(key)
data.itemLayer.removeFromSuperlayer()
}
for key in removeKeys {
self.inlineStickerItemLayers.removeValue(forKey: key)
}
self.updateEmojiReveal(animated: false)
}
func updateEmojiReveal(animated: Bool) {
for (_, data) in self.inlineStickerItemLayers {
let revealed: Bool
if let textView = data.textView, let count = textView.currentRevealCharacterCount {
revealed = data.charIndexInItem < count
} else {
revealed = true
}
if data.revealed == revealed {
continue
}
data.revealed = revealed
if revealed {
if animated {
data.itemLayer.opacity = 1.0
let opacityAnim = CABasicAnimation(keyPath: "opacity")
opacityAnim.fromValue = 0.0
opacityAnim.toValue = 1.0
opacityAnim.duration = 0.2
data.itemLayer.add(opacityAnim, forKey: "emojiRevealOpacity")
let scaleAnim = CABasicAnimation(keyPath: "transform.scale")
scaleAnim.fromValue = 0.1
scaleAnim.toValue = 1.0
scaleAnim.duration = 0.2
scaleAnim.timingFunction = CAMediaTimingFunction(name: .easeOut)
data.itemLayer.add(scaleAnim, forKey: "emojiRevealScale")
} else {
data.itemLayer.opacity = 1.0
}
} else {
data.itemLayer.opacity = 0.0
}
}
self.updateEmojiVisibility()
}
func updateEmojiVisibility() {
for (_, data) in self.inlineStickerItemLayers {
let onScreen: Bool
if let visibilityRect = self.visibilityRect, let textView = data.textView {
let rectInSelf = textView.convert(data.itemLayer.frame, to: self)
onScreen = rectInSelf.intersects(visibilityRect)
} else {
// No visibility rect == not tracked / off-screen don't animate. The root view's
// rect is propagated down the nested tree (details bodies, table cells/title) by
// `propagateVisibilityRect`, so a nil here genuinely means "not visible".
onScreen = false
}
data.itemLayer.isVisibleForAnimations = self.emojiEnableLooping && data.revealed && onScreen
}
self.propagateVisibilityRect()
}
// Pushes this view's `visibilityRect` down into every nested V2 view (details body, table
// title + cells), converted into each child's coordinate space. Each child's `visibilityRect`
// didSet re-runs `updateEmojiVisibility`, which propagates one level further so a single
// root assignment fans out across the whole tree.
private func propagateVisibilityRect() {
for view in self.itemViews {
for nested in InstantPageV2View.nestedV2Views(of: view) {
let childRect = self.visibilityRect.map { self.convert($0, to: nested) }
if nested.visibilityRect != childRect {
nested.visibilityRect = childRect
}
}
}
}
private static func nestedV2Views(of view: InstantPageItemView) -> [InstantPageV2View] {
if let detailsView = view as? InstantPageV2DetailsView {
return detailsView.bodyView.map { [$0] } ?? []
} else if let tableView = view as? InstantPageV2TableView {
var result: [InstantPageV2View] = []
if let titleSubView = tableView.titleSubView {
result.append(titleSubView)
}
result.append(contentsOf: tableView.cellSubViews)
return result
}
return []
}
/// Returns the input view typed-updated against `item`, or `nil` if the existing view's
@ -451,6 +648,7 @@ final class InstantPageV2TextView: UIView, InstantPageItemView {
private let renderContainer: UIView
private let renderView: TextRenderView
let emojiContainerView: UIView = UIView()
// Reveal mask state populated in Task 5.
private var maxCharacterDrawCount: Int?
@ -474,6 +672,10 @@ final class InstantPageV2TextView: UIView, InstantPageItemView {
self.renderView.frame = self.bounds
self.renderContainer.addSubview(self.renderView)
self.emojiContainerView.frame = self.bounds
self.emojiContainerView.isUserInteractionEnabled = false
self.addSubview(self.emojiContainerView)
}
@available(*, unavailable)
@ -486,6 +688,7 @@ final class InstantPageV2TextView: UIView, InstantPageItemView {
self.item = item
self.renderView.item = item
self.renderView.setNeedsDisplay()
self.emojiContainerView.frame = self.bounds
}
func updateRevealCharacterCount(value: Int?, animated: Bool) {
@ -496,6 +699,10 @@ final class InstantPageV2TextView: UIView, InstantPageItemView {
self.updateRevealMask(animateNewSegments: animated)
}
var currentRevealCharacterCount: Int? {
return self.maxCharacterDrawCount
}
private struct RevealLineInfo {
let lineFrame: CGRect
let lineHeight: CGFloat

View file

@ -51,6 +51,12 @@ struct InstantPageTextFormulaRun {
let attachment: InstantPageMathAttachment
}
struct InstantPageTextEmojiItem {
let frame: CGRect
let range: NSRange
let emoji: ChatTextInputTextCustomEmojiAttribute
}
public struct InstantPageTextAnchorItem {
public let name: String
public let anchorText: NSAttributedString?
@ -78,11 +84,12 @@ public final class InstantPageTextLine {
let markedItems: [InstantPageTextMarkedItem]
let imageItems: [InstantPageTextImageItem]
let formulaItems: [InstantPageTextFormulaRun]
let emojiItems: [InstantPageTextEmojiItem]
public let anchorItems: [InstantPageTextAnchorItem]
let isRTL: Bool
public let characterRects: [CGRect]? // line-local, one rect per character in `range`; nil = not computed
init(line: CTLine, range: NSRange, frame: CGRect, strikethroughItems: [InstantPageTextStrikethroughItem], underlineItems: [InstantPageTextUnderlineItem], markedItems: [InstantPageTextMarkedItem], imageItems: [InstantPageTextImageItem], formulaItems: [InstantPageTextFormulaRun], anchorItems: [InstantPageTextAnchorItem], isRTL: Bool, characterRects: [CGRect]? = nil) {
init(line: CTLine, range: NSRange, frame: CGRect, strikethroughItems: [InstantPageTextStrikethroughItem], underlineItems: [InstantPageTextUnderlineItem], markedItems: [InstantPageTextMarkedItem], imageItems: [InstantPageTextImageItem], formulaItems: [InstantPageTextFormulaRun], emojiItems: [InstantPageTextEmojiItem] = [], anchorItems: [InstantPageTextAnchorItem], isRTL: Bool, characterRects: [CGRect]? = nil) {
self.line = line
self.range = range
self.frame = frame
@ -91,6 +98,7 @@ public final class InstantPageTextLine {
self.markedItems = markedItems
self.imageItems = imageItems
self.formulaItems = formulaItems
self.emojiItems = emojiItems
self.anchorItems = anchorItems
self.isRTL = isRTL
self.characterRects = characterRects
@ -754,7 +762,8 @@ func attributedStringForRichText(_ text: RichText, styleStack: InstantPageTextSt
}
let extentBuffer = UnsafeMutablePointer<RunStruct>.allocate(capacity: 1)
extentBuffer.initialize(to: RunStruct(ascent: 0.0, descent: 0.0, width: dimensions.cgSize.width))
var callbacks = CTRunDelegateCallbacks(version: kCTRunDelegateVersion1, dealloc: { (pointer) in
var callbacks = CTRunDelegateCallbacks(version: kCTRunDelegateVersion1, dealloc: { pointer in
pointer.assumingMemoryBound(to: RunStruct.self).deallocate()
}, getAscent: { (pointer) -> CGFloat in
let d = pointer.assumingMemoryBound(to: RunStruct.self)
return d.pointee.ascent
@ -789,7 +798,8 @@ func attributedStringForRichText(_ text: RichText, styleStack: InstantPageTextSt
}
let extentBuffer = UnsafeMutablePointer<RunStruct>.allocate(capacity: 1)
extentBuffer.initialize(to: RunStruct(ascent: attachment.rendered.ascent, descent: attachment.rendered.descent, width: attachment.rendered.size.width))
var callbacks = CTRunDelegateCallbacks(version: kCTRunDelegateVersion1, dealloc: { _ in
var callbacks = CTRunDelegateCallbacks(version: kCTRunDelegateVersion1, dealloc: { pointer in
pointer.assumingMemoryBound(to: RunStruct.self).deallocate()
}, getAscent: { pointer -> CGFloat in
let data = pointer.assumingMemoryBound(to: RunStruct.self)
return data.pointee.ascent
@ -819,6 +829,37 @@ func attributedStringForRichText(_ text: RichText, styleStack: InstantPageTextSt
let result = attributedStringForRichText(text, styleStack: styleStack, url: url)
styleStack.pop()
return result
case let .textCustomEmoji(fileId, _):
struct RunStruct {
let ascent: CGFloat
let descent: CGFloat
let width: CGFloat
}
let attributes = styleStack.textAttributes()
let font = (attributes[NSAttributedString.Key.font] as? UIFont) ?? UIFont.systemFont(ofSize: 17.0)
let itemSize = font.pointSize * 24.0 / 17.0
let extentBuffer = UnsafeMutablePointer<RunStruct>.allocate(capacity: 1)
extentBuffer.initialize(to: RunStruct(ascent: font.ascender, descent: font.descender, width: itemSize))
var callbacks = CTRunDelegateCallbacks(version: kCTRunDelegateVersion1, dealloc: { pointer in
pointer.assumingMemoryBound(to: RunStruct.self).deallocate()
}, getAscent: { pointer -> CGFloat in
let d = pointer.assumingMemoryBound(to: RunStruct.self)
return d.pointee.ascent
}, getDescent: { pointer -> CGFloat in
let d = pointer.assumingMemoryBound(to: RunStruct.self)
return d.pointee.descent
}, getWidth: { pointer -> CGFloat in
let d = pointer.assumingMemoryBound(to: RunStruct.self)
return d.pointee.width
})
let delegate = CTRunDelegateCreate(&callbacks, extentBuffer)
let emojiAttribute = ChatTextInputTextCustomEmojiAttribute(interactivelySelectedFromPackId: nil, fileId: fileId, file: nil)
let mutableAttributedString = attributedStringForRichText(.plain(" "), styleStack: styleStack, url: url).mutableCopy() as! NSMutableAttributedString
mutableAttributedString.addAttributes([
kCTRunDelegateAttributeName as NSAttributedString.Key: delegate as Any,
ChatTextInputAttributes.customEmoji: emojiAttribute
], range: NSMakeRange(0, mutableAttributedString.length))
return mutableAttributedString
}
}

View file

@ -2443,6 +2443,13 @@ private struct PendingV2FormulaAttachment {
let baselineOffset: CGFloat
}
private struct PendingV2EmojiAttachment {
let xOffset: CGFloat
let range: NSRange
let emoji: ChatTextInputTextCustomEmojiAttribute
let size: CGFloat
}
func layoutTextItem(
_ string: NSAttributedString,
boundingWidth: CGFloat,
@ -2554,6 +2561,8 @@ func layoutTextItem(
var lineFormulaItems: [InstantPageTextFormulaRun] = []
var pendingImages: [PendingV2ImageAttachment] = []
var pendingFormulas: [PendingV2FormulaAttachment] = []
var lineEmojiItems: [InstantPageTextEmojiItem] = []
var pendingEmoji: [PendingV2EmojiAttachment] = []
var isRTL = false
if let glyphRuns = CTLineGetGlyphRuns(line) as? [CTRun], !glyphRuns.isEmpty {
if let run = glyphRuns.first, CTRunGetStatus(run).contains(CTRunStatus.rightToLeft) {
@ -2572,6 +2581,11 @@ func layoutTextItem(
let xOffset = CTLineGetOffsetForStringIndex(line, range.location, nil)
let baselineOffset = (attributes[NSAttributedString.Key.baselineOffset] as? CGFloat) ?? 0.0
pendingFormulas.append(PendingV2FormulaAttachment(xOffset: xOffset, range: range, attachment: attachment, baselineOffset: baselineOffset))
} else if let emoji = attributes[ChatTextInputAttributes.customEmoji] as? ChatTextInputTextCustomEmojiAttribute {
let xOffset = CTLineGetOffsetForStringIndex(line, range.location, nil)
let font = (attributes[NSAttributedString.Key.font] as? UIFont) ?? UIFont.systemFont(ofSize: 17.0)
let itemSize = font.pointSize * 24.0 / 17.0
pendingEmoji.append(PendingV2EmojiAttachment(xOffset: xOffset, range: range, emoji: emoji, size: itemSize))
}
}
}
@ -2593,6 +2607,11 @@ func layoutTextItem(
lineDescent = formula.attachment.rendered.descent
}
}
for emoji in pendingEmoji {
if emoji.size > lineAscent {
lineAscent = emoji.size
}
}
let baselineY = workingLineOrigin.y + lineAscent
for image in pendingImages {
@ -2615,6 +2634,15 @@ func layoutTextItem(
)
lineFormulaItems.append(InstantPageTextFormulaRun(frame: formulaFrame, range: formula.range, attachment: attachment))
}
for emoji in pendingEmoji {
let emojiFrame = CGRect(
x: workingLineOrigin.x + emoji.xOffset,
y: baselineY - emoji.size,
width: emoji.size,
height: emoji.size
)
lineEmojiItems.append(InstantPageTextEmojiItem(frame: emojiFrame, range: emoji.range, emoji: emoji.emoji))
}
extraDescent = max(0.0, lineDescent - baselineToNextTopSlack)
@ -2739,11 +2767,22 @@ func layoutTextItem(
}
}
}
for emoji in pendingEmoji {
let localIndex = emoji.range.location - lineRange.location
if localIndex >= 0 && localIndex < rects.count {
let x = CTLineGetOffsetForStringIndex(line, emoji.range.location, nil)
// characterRects are baseline-relative (positive-up). The emoji cell sits
// bottom-on-baseline (see frame loop: y = baselineY - emoji.size), so its
// baseline-relative bottom is 0 and maxY = emoji.size the width feeds the
// reveal cost map; maxY feeds the reveal-mask y conversion in the renderer.
rects[localIndex] = CGRect(x: x, y: 0.0, width: emoji.size, height: emoji.size)
}
}
lineCharacterRects = rects
} else {
lineCharacterRects = nil
}
let textLine = InstantPageTextLine(line: line, range: lineRange, frame: CGRect(x: workingLineOrigin.x, y: workingLineOrigin.y, width: lineWidth, height: height), strikethroughItems: strikethroughItems, underlineItems: underlineItems, markedItems: markedItems, imageItems: lineImageItems, formulaItems: lineFormulaItems, anchorItems: anchorItems, isRTL: isRTL, characterRects: lineCharacterRects)
let textLine = InstantPageTextLine(line: line, range: lineRange, frame: CGRect(x: workingLineOrigin.x, y: workingLineOrigin.y, width: lineWidth, height: height), strikethroughItems: strikethroughItems, underlineItems: underlineItems, markedItems: markedItems, imageItems: lineImageItems, formulaItems: lineFormulaItems, emojiItems: lineEmojiItems, anchorItems: anchorItems, isRTL: isRTL, characterRects: lineCharacterRects)
lines.append(textLine)
imageItems.append(contentsOf: lineImageItems)

View file

@ -350,6 +350,7 @@ public extension InstantPageV2View {
for view in self.itemViews {
clearRevealOn(view: view, animated: animated)
}
self.updateEmojiReveal(animated: animated)
return
}
@ -368,6 +369,7 @@ public extension InstantPageV2View {
clearRevealOn(view: self.itemViews[i], animated: animated)
}
}
self.updateEmojiReveal(animated: animated)
}
}

View file

@ -20,7 +20,8 @@ union RichText_Value {
RichText_Phone,
RichText_Image,
RichText_Anchor,
RichText_Formula
RichText_Formula,
RichText_CustomEmoji
}
table RichText {
@ -98,3 +99,8 @@ table RichText_Anchor {
table RichText_Formula {
latex:string (id: 0, required);
}
table RichText_CustomEmoji {
fileId:long (id: 0);
alt:string (id: 1, required);
}

View file

@ -53,9 +53,8 @@ extension RichText {
case let .textAnchor(textAnchorData):
let (text, name) = (textAnchorData.text, textAnchorData.name)
self = .anchor(text: RichText(apiText: text), name: name)
case .textCustomEmoji:
//TODO:localize
self = .plain("")
case let .textCustomEmoji(data):
self = .textCustomEmoji(fileId: data.documentId, alt: data.alt)
case let .textMath(textMath):
self = .formula(latex: textMath.source)
case let .textAutoEmail(email):
@ -141,6 +140,8 @@ extension RichText {
return .textAnchor(Api.RichText.Cons_textAnchor(text: text.apiRichText(), name: name))
case let .formula(latex):
return .textMath(Api.RichText.Cons_textMath(source: latex))
case let .textCustomEmoji(fileId, alt):
return .textCustomEmoji(Api.RichText.Cons_textCustomEmoji(documentId: fileId, alt: alt))
}
}
}

View file

@ -20,6 +20,7 @@ private enum RichTextTypes: Int32 {
case image = 14
case anchor = 15
case formula = 16
case textCustomEmoji = 17
}
public indirect enum RichText: PostboxCoding, Equatable {
@ -40,7 +41,8 @@ public indirect enum RichText: PostboxCoding, Equatable {
case image(id: MediaId, dimensions: PixelDimensions)
case anchor(text: RichText, name: String)
case formula(latex: String)
case textCustomEmoji(fileId: Int64, alt: String)
public init(decoder: PostboxDecoder) {
switch decoder.decodeInt32ForKey("r", orElse: 0) {
case RichTextTypes.empty.rawValue:
@ -83,6 +85,8 @@ public indirect enum RichText: PostboxCoding, Equatable {
self = .anchor(text: decoder.decodeObjectForKey("t", decoder: { RichText(decoder: $0) }) as! RichText, name: decoder.decodeStringForKey("n", orElse: ""))
case RichTextTypes.formula.rawValue:
self = .formula(latex: decoder.decodeStringForKey("l", orElse: ""))
case RichTextTypes.textCustomEmoji.rawValue:
self = .textCustomEmoji(fileId: decoder.decodeInt64ForKey("ce.f", orElse: 0), alt: decoder.decodeStringForKey("ce.a", orElse: ""))
default:
self = .empty
}
@ -154,6 +158,10 @@ public indirect enum RichText: PostboxCoding, Equatable {
case let .formula(latex):
encoder.encodeInt32(RichTextTypes.formula.rawValue, forKey: "r")
encoder.encodeString(latex, forKey: "l")
case let .textCustomEmoji(fileId, alt):
encoder.encodeInt32(RichTextTypes.textCustomEmoji.rawValue, forKey: "r")
encoder.encodeInt64(fileId, forKey: "ce.f")
encoder.encodeString(alt, forKey: "ce.a")
}
}
@ -261,6 +269,12 @@ public indirect enum RichText: PostboxCoding, Equatable {
} else {
return false
}
case let .textCustomEmoji(lhsFileId, lhsAlt):
if case let .textCustomEmoji(rhsFileId, rhsAlt) = rhs, lhsFileId == rhsFileId, lhsAlt == rhsAlt {
return true
} else {
return false
}
}
}
}
@ -306,6 +320,8 @@ public extension RichText {
return text.plainText
case let .formula(latex):
return latex
case let .textCustomEmoji(_, alt):
return alt
}
}
}
@ -398,6 +414,11 @@ extension RichText {
throw FlatBuffersError.missingRequiredField()
}
self = .formula(latex: value.latex)
case .richtextCustomemoji:
guard let value = flatBuffersObject.value(type: TelegramCore_RichText_CustomEmoji.self) else {
throw FlatBuffersError.missingRequiredField()
}
self = .textCustomEmoji(fileId: value.fileId, alt: value.alt)
case .none_:
self = .empty
}
@ -520,8 +541,15 @@ extension RichText {
let start = TelegramCore_RichText_Formula.startRichText_Formula(&builder)
TelegramCore_RichText_Formula.add(latex: latexOffset, &builder)
offset = TelegramCore_RichText_Formula.endRichText_Formula(&builder, start: start)
case let .textCustomEmoji(fileId, alt):
valueType = .richtextCustomemoji
let altOffset = builder.create(string: alt)
let start = TelegramCore_RichText_CustomEmoji.startRichText_CustomEmoji(&builder)
TelegramCore_RichText_CustomEmoji.add(fileId: fileId, &builder)
TelegramCore_RichText_CustomEmoji.add(alt: altOffset, &builder)
offset = TelegramCore_RichText_CustomEmoji.endRichText_CustomEmoji(&builder, start: start)
}
return TelegramCore_RichText.createRichText(&builder, valueType: valueType, valueOffset: offset)
}
}

View file

@ -59,6 +59,35 @@ public class ChatMessageRichDataBubbleContentNode: ChatMessageBubbleContentNode
// to request a full bubble re-layout (so the bubble grows with the reveal).
private var lastAppliedRevealedCount: Int = 0
override public var visibility: ListViewItemNodeVisibility {
didSet {
if oldValue != self.visibility {
self.updatePageViewVisibilityRect()
}
}
}
// Pushes the current `visibility` sub-rect into `pageView.visibilityRect`, translated into the
// page view's coordinate space (the page view sits at `streamingHeaderOffset` inside the bubble).
// Re-invoked from the apply closure after `pageView.frame` is set, because that offset shifts
// across streamed chunks without a `visibility` change, which would otherwise leave the
// animation-gating rect stale.
private func updatePageViewVisibilityRect() {
guard let pageView = self.pageView else {
return
}
switch self.visibility {
case .none:
pageView.visibilityRect = nil
case let .visible(_, subRect):
var rect = subRect
rect.origin.x = 0.0
rect.size.width = 10000.0
rect.origin.y -= pageView.frame.minY
pageView.visibilityRect = rect
}
}
required public init() {
self.containerNode = ContainerNode()
self.containerNode.clipsToBounds = true
@ -646,6 +675,7 @@ public class ChatMessageRichDataBubbleContentNode: ChatMessageBubbleContentNode
origin: CGPoint(x: -1.0, y: streamingHeaderOffset),
size: pageLayout.contentSize
)
self.updatePageViewVisibilityRect()
} else {
self.currentPageLayout = nil
self.pageView?.update(