1.1 What Are Tokens in NLP?

Tokens are the atomic units of text processing in natural language processing (NLP), representing the smallest meaningful segments of text that a model can interpret. In modern transformer-based architectures, tokens typically correspond to subword units, words, or even characters, depending on the tokenization strategy employed. The process of tokenization involves breaking down raw text into these discrete elements, which are then mapped to numerical representations (token IDs) through a vocabulary lookup.

Mathematical Representation of Tokenization

Given an input string S of length N, tokenization can be formalized as a function f that splits S into a sequence of k tokens:

where each token t_i belongs to a predefined vocabulary V of size |V|. The vocabulary is constructed during the tokenizer's training phase, typically using algorithms like Byte-Pair Encoding (BPE), WordPiece, or Unigram.

Token Embeddings and Numerical Representation

Each token t_i is mapped to a dense vector representation e_i ∈ ℝ^d through an embedding layer, where d is the model's hidden dimension. This mapping is formally expressed as:

where E ∈ ℝ^|V|×d is the embedding matrix learned during training. For subword tokenizers, this approach enables handling of out-of-vocabulary words through composition of subword embeddings.

Advanced Tokenization Techniques

Modern NLP systems employ sophisticated tokenization strategies that balance vocabulary size with semantic granularity:

• Byte-Pair Encoding (BPE): Iteratively merges frequent character pairs to create a variable-length subword vocabulary.
• WordPiece: Similar to BPE but uses likelihood maximization rather than frequency for merge operations.
• SentencePiece: Implements unsupervised tokenization that works directly on raw text without requiring pre-tokenization.
• Unigram Language Model: Probabilistically segments text based on a trained language model of subword likelihoods.

The choice of tokenization significantly impacts model performance, particularly for morphologically rich languages or specialized domains. For instance, clinical text processing often benefits from domain-specific tokenizers that preserve meaningful medical terminology.

Practical Considerations in Tokenization

Several key factors influence tokenization effectiveness in real-world applications:

• Vocabulary Coverage: The percentage of encountered tokens that exist in the vocabulary, affecting the frequency of unknown token occurrences.
• Sequence Length: The number of tokens generated per input, which directly impacts computational requirements.
• Semantic Granularity: The degree to which tokens preserve meaningful linguistic units.
• Language Agnosticism: The ability to handle multiple languages with a single tokenizer.

Transformer models typically impose maximum sequence length constraints (e.g., 512 tokens for BERT), making efficient tokenization crucial for processing long documents. Advanced techniques like sliding window approaches or hierarchical models address this limitation while maintaining contextual integrity.

Tokenization Techniques and Their Impact

Subword Tokenization and BPE

Byte Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent pairs of bytes or characters to form a vocabulary of subword units. Given a corpus, BPE starts with a base vocabulary of individual characters and computes the frequency of each adjacent pair. The most frequent pair is merged into a new token, and the process repeats until a target vocabulary size is reached. The probability of merging a pair (x, y) is given by:

where V is the current vocabulary. BPE's strength lies in its ability to handle rare words by decomposing them into known subword units, reducing out-of-vocabulary (OOV) errors. For example, the word "unhappiness" might be tokenized as ["un", "happiness"] if these subwords exist in the vocabulary.

WordPiece and Unigram Language Modeling

WordPiece, used in models like BERT, employs a similar merge strategy but selects pairs based on likelihood rather than raw frequency. The merge criterion maximizes the language model probability of the training data:

Unigram Language Modeling, used in SentencePiece, takes a different approach by starting with a large vocabulary and iteratively pruning the least probable tokens under a unigram language model. The probability of a token sequence X = (x_1, ..., x_n) is:

where p(x_i) is the unigram probability of token x_i. This method allows for probabilistic sampling during tokenization, enabling multiple valid segmentations for the same input.

Impact on Model Performance

Tokenization directly affects model efficiency and generalization. Larger vocabularies reduce sequence lengths but increase embedding matrix size, creating a trade-off between memory usage and computational cost. Subword methods improve handling of rare words but may split semantically meaningful units. For instance, BPE might segment "transformer" into ["trans", "former"], losing the connection to the original concept.

Comparative studies show that Unigram LM often outperforms BPE on morphologically rich languages, while WordPiece strikes a balance between robustness and simplicity. The choice of tokenizer also influences positional encoding efficiency in transformers, as longer sequences require more memory for attention computations.

Practical Considerations

In practice, tokenization must align with the model's pretraining data. Mismatched tokenizers between pretraining and fine-tuning degrade performance, as seen when using a BERT tokenizer trained on Wikipedia text for biomedical NLP tasks. Hybrid approaches, such as using BPE for pretraining and rule-based tokenizers for domain-specific fine-tuning, can mitigate this issue.

Recent advancements like BPE-dropout introduce stochasticity during training by randomly preventing some merges, improving robustness to tokenization variations. Similarly, dynamic tokenization adapts the vocabulary during training based on evolving language patterns, though this requires careful handling of embedding matrices.

Role of Tokens in Controlled Text Generation

Tokens serve as the fundamental building blocks in transformer-based language models, acting as discrete units that encode semantic, syntactic, and positional information. In controlled generation tasks, tokens take on additional significance as carriers of guidance signals that steer the model's output toward desired attributes. The probability distribution over the vocabulary at each generation step can be decomposed into:

2.1 Prompt Engineering with Tokens

Token-Level Control in Language Models

Modern transformer-based language models process text as sequences of tokens, where each token corresponds to a subword or word fragment. The tokenization process directly influences model behavior, as the model's attention mechanisms operate over these discrete units. By manipulating tokens at a granular level, practitioners can achieve precise control over generation outputs.

def constrain_generation(prompt, required_tokens):
    logit_processor = LogitsProcessor(
        lambda input_ids, scores: adjust_for_tokens(scores, required_tokens)
    return generator(prompt, logits_processor=logit_processor)
    

2.2 Constrained Decoding Strategies

Constrained decoding enforces strict adherence to predefined rules during text generation, ensuring outputs comply with syntactic, semantic, or domain-specific constraints. Unlike unconstrained beam search or sampling, these strategies dynamically modify token probabilities or search paths to satisfy hard or soft constraints.

Hard Constraint Methods

Hard constraints eliminate invalid tokens entirely from the candidate set at each decoding step. The most common approaches include:

• Finite-State Machines (FSMs): Define valid token transitions as a graph, pruning branches that violate grammar rules or keyword requirements. For example, generating valid SQL queries requires adhering to clause ordering (SELECT → FROM → WHERE).
• Lexical Constraints: Force inclusion of predefined keywords by masking all tokens except the required ones at specific positions. The probability distribution becomes:
  $$ P(w_t | w_{<t}) = \begin{cases} 0 & \text{if } w_t \notin C \\ \frac{\exp(s(w_t))}{\sum_{w' \in C} \exp(s(w'))} & \text{otherwise} \end{cases} $$
  where C is the constraint set and s(w_t) is the model's logit for token w_t.

Soft Constraint Techniques

Soft constraints bias sampling toward desired attributes without absolute enforcement:

• Logit Modification: Additive or multiplicative adjustments to token logits. For instance, boosting the probability of rare words to improve lexical diversity:
  $$ \tilde{s}(w_t) = s(w_t) + \lambda \cdot \log(1/p_{\text{unigram}}(w_t)) $$
  where λ controls the strength of the bias.
• Discriminative Guidance: Use auxiliary classifiers to steer generation. A toxicity classifier can downweight harmful continuations by modifying logits via:
  $$ s'(w_t) = s(w_t) - \eta \cdot \nabla_{x_t} \log p_{\text{classifier}}(y | x_{1:t}) $$

Dynamic Beam Search Variants

Standard beam search fails with constraints due to early pruning of valid but low-probability paths. Improved variants include:

• Constraint-Aware Beam Search (CABS): Maintain separate beams for each constraint state, allowing temporary low-probability paths that may satisfy future constraints.
• Stochastic Beam Search: Sample beams proportionally to P(beam) × 1[beam satisfies constraints], enabling exploration of diverse valid outputs.

Case Study: Chemistry-Aware Generation

Generating valid SMILES strings (chemical notation) requires strict syntactic validity. A hybrid approach combines:

1. FSM-based pruning of invalid characters
2. Valency checks via real-time molecular graph updates
3. Rejection sampling for rare but valid constructions

This achieves >99% validity compared to 12% with unconstrained decoding, as demonstrated in Journal of Chemical Information and Modeling (2022).

Dynamic Token Masking and Biasing

Dynamic token masking and biasing refine autoregressive generation by selectively constraining or amplifying token probabilities during inference. Unlike static approaches, these methods adapt in real-time based on contextual cues, syntactic rules, or external knowledge bases.

Mathematical Formulation

Given a vocabulary V and logits l_t at step t, dynamic masking applies a binary mask M_t ∈ {0,1}^|V|:

where log(0) is set to −∞. For biasing, a learned or heuristic-based weight vector w_t ∈ ℝ^|V| modulates the distribution:

The scaling factor α controls intervention strength. In transformer architectures, these operations occur between the final layer and softmax.

Implementation Strategies

Three dominant paradigms exist for dynamic control:

• Rule-based masking: Enforces hard constraints via finite-state machines (e.g., ensuring XML tag closure)
• Neural biasing: Uses auxiliary networks to predict w_t from hidden states
• Hybrid approaches: Combins symbolic rules with learned bias models

For example, in code generation, a rule-based mask might enforce syntactic validity by disabling all non-contextual tokens:

def apply_syntax_mask(logits, valid_tokens):
    mask = torch.zeros_like(logits)
    mask[valid_tokens] = 1
    return logits + torch.log(mask)

Applications and Tradeoffs

Dynamic masking proves essential in:

• Controlled dialogue systems (avoiding harmful outputs)
• Structured text generation (JSON/XML compliance)
• Domain-specific generation (enforcing chemical notation rules)

However, aggressive masking risks mode collapse, while excessive biasing may degrade fluency. Empirical studies show optimal α values typically fall in [0.3, 1.0] for most tasks.

3.1 Content Moderation and Safe Generation

Modern language models generate text autoregressively, sampling tokens from a probability distribution conditioned on prior tokens. While this enables creative outputs, it also introduces risks of generating harmful, biased, or unsafe content. Content moderation techniques aim to guide generation toward safer outputs without sacrificing coherence or fluency.

Token-Level Safety Filtering

One approach involves modifying the token sampling process to exclude unsafe continuations. Given a sequence of tokens \(x_{1:t}\), the model computes logits \(l_{t+1}\) for the next token. A safety filter \(S(x_{1:t}, w)\) assigns a risk score to each candidate token \(w\):

The modified sampling distribution becomes:

This effectively sets the probability of unsafe tokens to zero during generation. The challenge lies in defining \(S\) accurately—overly aggressive filtering may harm output diversity, while weak filtering allows harmful content.

Discriminator-Guided Decoding

An alternative approach uses a separate safety classifier \(D\) to evaluate full sequences. At each step, beam search candidates are scored by:

where \(\lambda\) controls the strength of safety guidance. The classifier \(D\) is typically trained on labeled datasets of harmful vs. benign text, using architectures like BERT or RoBERTa. This method allows more nuanced safety judgments than token-level filtering.

Constrained Decoding with Automata

For strict compliance with predefined safety rules, finite-state automata can enforce hard constraints during generation. The decoding process is guided by a product automaton combining:

• The language model's token transition probabilities
• A safety automaton that only accepts valid token sequences

Mathematically, this corresponds to:

where \(q_t\) is the automaton state at step \(t\), and \(A_{\text{safe}}\) encodes the safety constraints. This approach guarantees compliance but requires careful automaton design.

Real-World Implementation Challenges

Practical systems often combine these techniques. For example:

• Token filters block obviously harmful words or phrases
• Discriminator guidance handles more subtle cases
• Automata enforce structural constraints (e.g., preventing personal information generation)

The trade-off between safety and creativity remains an active research area, with recent work exploring reinforcement learning from human feedback (RLHF) to better align models with human values during generation.

3.2 Domain-Specific Text Generation

Domain-specific text generation leverages guided tokens to enforce constraints that align with specialized vocabularies, stylistic conventions, or factual correctness in fields like medicine, law, or engineering. Unlike general-purpose models, domain-specific generation requires fine-grained control over output structure and terminology.

Token-Level Domain Constraints

Given a vocabulary V partitioned into domain-specific (V_d) and generic (V_g) tokens, the generation probability at step t is modified through masking:

Interactive and Adaptive Dialog Systems

Modern dialog systems leverage guided token generation to achieve dynamic, context-aware interactions. The core mechanism involves constrained decoding where the model's token probabilities are modified in real-time based on dialog state, user intent, and external knowledge. This goes beyond simple prompt engineering by enabling fine-grained control over response properties like style, factual accuracy, and task completion.

Token-Level Control Mechanisms

The generation process can be formalized as modifying the standard language model probability distribution P(w_t|w_<t, c) through learned or heuristic constraints:

class DialogController:
    def __init__(self, model, constraints):
        self.model = model  # Base LM
        self.constraints = constraints  # List of Constraint modules
        
    def generate(self, prompt, state):
        logits = self.model(prompt)
        for constraint in self.constraints:
            if constraint.active(state):
                logits = constraint.apply(logits, state)
        return self.model.sample(logits)

4.1 Overfitting to Token Constraints

When models are fine-tuned or explicitly constrained to generate outputs matching specific token sequences (e.g., predefined prefixes, templates, or guardrails), they risk overfitting to these constraints. This manifests as:

• Loss of generalization: The model becomes overly reliant on the constrained tokens, failing to adapt to unseen prompts or variations outside the training distribution.
• Degraded fluency: Outputs may rigidly adhere to token patterns even when contextually inappropriate, sacrificing natural language coherence.
• Pathological repetition: The model may exploit token constraints as shortcuts, leading to degenerate outputs (e.g., infinite loops of constrained tokens).

Mechanisms of Overfitting

Mathematically, overfitting arises when the model's conditional probability distribution p(y|x) becomes sharply peaked around the constrained tokens. For a model with parameters θ and a constraint set C of allowed tokens, the log-likelihood gradient during training pushes the distribution toward:

This imbalance causes the model to underestimate the probability mass of valid but unconstrained continuations. In transformer architectures, the attention mechanism exacerbates this by reinforcing token-specific key-value patterns across layers.

Empirical Observations

Studies on GPT-3 and T5 show that models trained with strict token constraints exhibit:

• Up to 68% higher perplexity on unconstrained evaluation tasks compared to baseline.
• Reduced entropy in the output distribution, with top-1 token probability often exceeding 0.9 for constrained positions.
• Failure modes where the model ignores semantic context to satisfy syntactic constraints (e.g., forcing a verb token despite nonsensical meaning).

Mitigation Strategies

1. Constraint Relaxation

Replace hard token constraints with learned soft constraints via auxiliary loss terms. For a constraint set C, modify the cross-entropy loss L with a balancing term λ:

2. Adversarial Training

Introduce a discriminator network D that penalizes the generator G for over-reliance on constrained tokens. The adversarial objective becomes:

3. Dynamic Token Masking

During training, randomly mask subsets of constrained tokens with probability p_mask to force generalization. The masking rate can follow a curriculum:

where t is the training step and T the total steps.

4.2 Balancing Control and Creativity

Guided generation techniques often face a fundamental trade-off: excessive control leads to rigid, deterministic outputs, while too little control results in incoherent or undesired generations. The challenge lies in dynamically adjusting the influence of control tokens or constraints without stifling the model's generative capacity.

Quantifying the Control-Creativity Trade-off

The degree of control can be formalized through conditional probability distributions. Let G represent the base generative model, and C denote the control mechanism (e.g., constrained decoding, token-level guidance). The modified distribution becomes:

4.3 Computational Overhead and Efficiency

Guided token generation introduces computational overhead primarily through two mechanisms: dynamic masking and sequential constraint validation. The former requires real-time modification of the model's output logits, while the latter enforces grammar or syntax rules during autoregressive decoding. Both operations scale with sequence length n and vocabulary size V, leading to a theoretical time complexity of:

where C represents the cost of constraint checking (e.g., finite-state automaton transitions for grammar validation). For transformer-based models, this compounds with the standard O(n²d) self-attention cost, where d is the hidden dimension.

Memory Bandwidth Bottlenecks

Token-level guidance often requires storing intermediate decoding states (e.g., partial parse trees or masked logits) in GPU memory. The memory footprint grows linearly with batch size B and constrained decoding depth k:

where S is the state size of the guiding automaton. For large V (e.g., 50k+ tokens in modern LLMs), this creates significant pressure on memory bandwidth, particularly when:

• Running beam search with width w, multiplying memory usage by w
• Processing long sequences where n > 2048 tokens

Optimization Strategies

1. Prefiltering and Early Pruning

Reduce V dynamically by prefiltering invalid tokens before softmax computation. For regex-guided generation, this can be implemented via bitmask operations:

def apply_token_mask(logits, valid_token_mask):
    # Set invalid tokens to -inf before softmax
    return logits.masked_fill(~valid_token_mask, float('-inf'))

2. Parallel Constraint Verification

Offload constraint checks to dedicated CUDA kernels that operate concurrently with attention computations. For JSON schema validation, this involves:

• Maintaining a stack of open JSON scopes in shared memory
• Using warp-level primitives for concurrent state updates

Hardware-Specific Considerations

On TPU architectures, the overhead manifests differently due to:

• XLA compiler's static graph requirements complicating dynamic masking
• Higher penalty for irregular memory access patterns during constraint checks

Empirical measurements on A100 GPUs show a 15-40% throughput degradation when enforcing complex constraints (e.g., type-aware Python code generation) compared to unconstrained decoding. The tradeoff becomes nonlinear when n exceeds the GPU's L2 cache capacity (40MB on A100), causing frequent DRAM accesses.

5.1 Key Research Papers on Token-Guided Generation

• Expert-Guided Extinction of Toxic Tokens for Debiased Generation — The LLMs are guided by the debiasing expert to distinguish between toxic and unbiased tokens and suppress the undesired attributes. The performance of EXPOSED is evaluated from three perspectives: (1) open-ended text generation with toxic prompts to measure the toxicity of generation, (2) reading comprehension to evaluate the stereotypical bias,
• PDF GeDi: Generative Discriminator Guided Sequence Generation - ACL Anthology — tion probabilities for possible next tokens at each gen-eration timestep using only element-wise operations. These classification probabilities can then be used to guide generation from a language model (e.g., GPT-2) to achieve attribute control across domains. If a class conditional language model was trained on movie re-
• Token-Picker: Accelerating Attention in Text Generation with Minimized ... — In the generation phase, tokens are sequentially generated un-til the maximum sequence length or an end-of-sequence token (< >) is encountered. At a given time , the model takes an input token + to produce the following token + +1. The input is con-structed as a vector from a single token, leading to the execution
• DeepPress: guided press release topic-aware text generation using ... — Guided text generation is one of the key issues when it comes to creating human-like artificial intelligence writing machines. Humans can use their writing skills depending on the topic of the text and the pieces of information they want to include. The context and style also play an important role in mediating the engagement level of the press release. However, current research does focus on ...
• PDF TokenMixup: Efficient Attention-guided Token-level Data ... - NIPS — controls the minimum amount of saliency gain required for a token to be replaced. By setting = 0, tokens are mixed in a way that maximizes total saliency. If is maximal, no tokens are mixed, as shown in middle Table 1. Finally, (main paper section 3.4) is the number of tokens to be pooled from each previous layers when VTM is adopted.
• PDF Study of Various Methods for Tokenization - Springer — tokens. TokenizerME class † Using the Tokenizer Model class, it loads the en-token.bin model. † Instantiate the TokenizerME class. † Tokenization of sentences can be done using tokenize method of this class. 2.2 Byte Pair Encoding (BPE) [2] In 2016, Byte Pair Encoding has been used to prepare sub-word dictionary. In 2019,
• From Tokens to Tales: Semantic Similarity in Story Generation - Springer — By shifting token distribution or selecting specific tokens for generated sequence, the method generates text that satisfies required controls. This approach is used in Keyword2Text method [ 17 ], which shifts language model's token distribution to the guide word using cosine similarity in the semantic vector space of word2vec or GloVe model.
• A Critical Look At Tokenwise Reward-Guided Text Generation — tokenwise reward-guided text generation (RGTG) techniques that a void any fine-tuning of the LLM. More precisely, the LLM remains frozen (i.e., not finetuned) and the reward model is used at
• Frame Representation Hypothesis: Multi-Token LLM Interpretability and ... — Concept probing evolution during model generation for the 8 languages supported by Llama 3.1 70B using Top-k Concept-Guided Decoding with k = 3. Hindi and Thai are more susceptible to the ...
• A recent survey on controllable text generation: A causal perspective — Recently, several reviews have emerged in the field of CTG. These reviews primarily concentrate on aspects other than causality. For instance, [1, 4] focus on specific methodologies employed in CTG, and [5, 6] explore avenues for enhancing the quality of the texts generated.Different from these reviews, our paper will analyze existing CTG methods from a new perspective: causality.

5.2 Open-Source Libraries and Tools

• Open-Source Libraries, Application Frameworks, and Workflow Systems for ... — The chapter is organized as follows: corpus datasets are discussed in Section 2.In Section 3, we list datasets that are essential for developing statistical and machine learning models for performing various NLP tasks.Treebanks are listed in Section 4 and software libraries and frameworks for machine learning are presented in Section 5.Task-specific NLP tools are discussed in Section 7.
• Recent Advances in Intelligent Source Code Generation: A Survey on ... — Source Code Generation (SCG) is a prevalent research field in the automation software engineering sector that maps specific descriptions to various sorts of executable code. ... The booming large-scale dataset generated by open-source code repositories and Q&A resources, the innovation of machine learning algorithms, and the development of ...
• Doxygen homepage — It automates the generation of documentation from source code comments, parsing information about classes, functions, and variables to produce output in formats like HTML and PDF. By simplifying and standardizing the documentation process, Doxygen enhances collaboration and maintenance across diverse programming languages and project scales.
• MeshKit | SIGMA - Argonne National Laboratory — MeshKit is an open-source library of mesh generation functionality. Its design philosophy is two-fold: it provides a collection of meshing algorithms for use in real meshing problems, along with other tools commonly needed to support mesh generation (coordination of BREP-based meshing process, mesh smoothing, etc.); and it serves as a platform in which to perform mesh generation algorithm ...
• More than a framework: Sketching out technical enablers for natural ... — SantaCoder [12] achieved comparable and stronger performance than previous open-source multilingual code generation models through better preprocessing methods, despite being a substantially smaller model. These works show the active state of the entire field, and it is worth investigating how these algorithms are introduced and why they can be ...
• PDF Automatic Code Generation for a Seamless Low-cost Development Platform — The open source tool Scilab / Xcos is used as the central CAE environment, which is suitable for simulating hybrid systems from a wide variety of domains (Nikoukhah, 2006). With the help of the model library, various continuous-time and discrete-time model components can be reused and parame-terized. Through the LoRra code generator, efcient
• Code Generation for Unknown Libraries via Reading API Documentations — Our framework for code generation via exploiting API documentations. Figures - available via license: Creative Commons Attribution 4.0 International Content may be subject to copyright.
• VeriGen: A Large Language Model for Verilog Code Generation — A promising new approach comes via the proliferation of technically capable code-writing large language models (LLMs) [].LLMs are deep neural networks, typically based on transformer [] architectures, that aim to model the underlying distribution of a natural or structured language corpus.Given a sequence of words (or "tokens") LLMs predict a distribution over the next word/token.
• GitHub - CAMeL-Lab/camel_tools: A suite of Arabic natural language ... — See Available Packages for a list of all available datasets.. By default, data is stored in C:\Users\your_user_name\AppData\Roaming\camel_tools.Alternatively, if you would like to install the data in a different location, you need to set the CAMELTOOLS_DATA environment variable to the desired path. Below are the instructions to do so (on Windows 10):
• PDF Turning Coders into Makers: The Promise of Embedded Design Generation — maps user input to a set of library components according to well-defined composition rules that can be verified statically. PBD-based tools solve the synthesis problem by opportunistically composing el-ements from a library to generate systems of constraints which can be solved by external solvers. For instance, METRO II [Davare et al.

5.3 Recommended Books and Tutorials

• Integrated Electronic Payment Technologies for Smart Cities 3031382218 ... — Integrated Electronic Payment Technologies for Smart Cities 3031382218, 9783031382215 This book addresses the use of existing and emerging electronic payment technologies within a smart city in the context
• PDF Mastering Generative AI and Prompt Engineering - Data Science Horizons — 6.1. Content generation and creative writing 6.2. Data analysis and visualization 6.3. Chatbots and conversational AI 6.4. Anomaly detection and pattern recognition Conclusion Appendices A. Recommended books, articles, and blogs B: Online communities and forums for discussions and collaboration 1
• Inference-Time Alignment in Diffusion Models with Reward-Guided ... — Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review Masatoshi Uehara 1*, Yulai Zhao 2, Chenyu Wang 3, Xiner Li 4, Aviv Regev 1, Sergey Levine 5 ∗, Tommaso Biancalani 1Genentech, 2Princeton University, 3 MIT, 4 Texas A&M University, 5 UC Berkeley Abstract This tutorial provides an in-depth guide on inference-time guidance and alignment methods
• GEDI: GENERATIVE DISCRIMINATOR GUIDED SEQUENCE GENERATION - OpenReview — Figure 1: A toy example of how GeDi-guided generation uses Bayes rule to efficiently compute classification probabilities for possible next tokens at each generation timestep using only element-wise operations. These classification probabilities can then be used to guide generation from a language model (e.g., GPT-2) to achieve
• GuidedDiscreteDiffusionforElectronicHealthRecord Generation - arXiv.org — generation,includingmedGANChoietal.(2017b),medBGAN(Baowalyetal.,2018),EHRWGAN ... we focus on developing a guided discrete diffusion model tailored specifically for ... we introduced multinomial diffusion with a single token,x ∈RK. In the context of categorical EHRs, we aim to generate a sequence of N tokens with K= 2, denoted by x = ...
• A Critical Look At Tokenwise Reward-Guided Text Generation — tokenwise reward-guided text generation (RGTG) techniques that a void any fine-tuning of the LLM. More precisely, the LLM remains frozen (i.e., not finetuned) and the reward model is used at
• (PDF) Reward-Guided Controlled Generation for Inference ... - ResearchGate — This tutorial provides an in-depth guide on inference-time guidance and alignment methods for optimizing downstream reward functions in diffusion models.
• Inference-Time Alignment in Diffusion Models with Reward-Guided ... — The simplest such approach is best-of-N sampling in Figure 2 a, which involves generating multiple designs (N samples) from a pre-trained diffusion model and selecting the best one based on reward functions (e.g., Nakano et al. ). However, this method can be highly inefficient when the reward functions are difficult to optimize.
• LeadRec: Towards Personalized Sequential Recommendation via Guided ... — 2.1 Sequence Recommendation. Sequential recommendation aims to capture the evolution of user interests in sequential information. With the advancement of deep learning, RNNs have been applied to sequential recommendations for better modeling user preferences, such as GRU4Rec [].CNNs have also demonstrated promising effectiveness in capturing sequential patterns, exemplified by Caser [].