Peter pszemraj

Applying Mamba-3 Ideas to an OLMo-Hybrid-Style Gated DeltaNet

Recommendation

The best target architecture is not a Mamba-3 hybrid. It is an OLMo-Hybrid-style interleaved Gated DeltaNet model with Mamba-3's hardware-aware recurrence upgrades ported into the GDN mixer.

The highest-value variant is:

Hybrid MIMO-HGDN: [GDN, GDN, GDN, Attention] × N, with negative-eigenvalue GDN preserved, rank-R MIMO updates added to the GDN state update, short convolutions retained for the first ablation, and optional data-dependent q/k rotations as a lower-priority expressivity experiment.

Decorative Monospace

A formatting failure mode in modern LLM outputs where code blocks are used as a rhetorical device for signaling rigor rather than as semantic markup for content where whitespace, syntax, or literal reproduction matters.

The model isn't trying to communicate "this is code." It's trying to communicate "this is structured thinking" by borrowing the visual texture of a technical document. The texture is decorative; the underlying content is prose.

	"""
	parcae_reference.py
	═══════════════════

	A single-file, heavily annotated reference implementation of the Parcae
	architecture (Prairie, Novack, Berg-Kirkpatrick, Fu — arXiv 2604.12946).

	PURPOSE
	───────
	This is a pedagogical reference, not a training rig. Every architectural

	#!/usr/bin/env bash
	set -e

	# a script for running https://hf.co/unsloth/gemma-4-31B-it-GGUF
	# on 5090 hardware, targeting long ctx

	MODEL_PATH="$HOME/model-weights/llm/gguf/gemma4/gemma-4-31B-it-UD-Q4_K_XL.gguf"
	MMPROJ_PATH="$HOME/model-weights/llm/gguf/gemma4/mmproj-BF16.gguf"
	CONTEXT_LENGTH=65536
	PORT=8674

	#!/usr/bin/env python3
	"""
	CLI tool to check Python files for missing docstrings, type hints, and lazy docstrings.

	A 'lazy' docstring is one that only contains a summary without documenting
	arguments or return values when the function has 2+ parameters or returns something.

	Usage:
	python doc_check.py src/ --check-lazy-docstrings

	#!/usr/bin/env python3
	"""
	Image distortion tool for creating "deep fried" meme-filters.

	Dependencies:
	pip install universal-pathlib Pillow numpy scipy matplotlib

	Usage:
	python deepfryer.py --help

	#!/usr/bin/env bash
	set -euo pipefail

	# Modern PDF compressor with LLM-optimized defaults
	# Requires: ghostscript (gs)

	VERSION="1.0.0"
	SCRIPT_NAME=$(basename "$0")

	# Defaults (LLM preset)

	import torch

	import time
	from transformers import AutoTokenizer, AutoModelForCausalLM


	# 1. Basic Timer Context Manager
	class Timer:
	def __init__(self, name="Task"):
	self.name = name

	"""
	slightly optimized image tiling for vlms based on "Jina-VLM: Small Multilingual Vision Language Model"

	Based on the pseudocode in Appendix A.1: https://arxiv.org/abs/2512.04032
	"""
	import math
	from typing import List, Tuple

	import torch
	import torch.nn.functional as F

	#@markdown add auto-Colab formatting with `IPython.display`
	from IPython.display import HTML, display
	# colab formatting
	def set_css():
	display(
	HTML(
	"""
	<style>
	pre {
	white-space: pre-wrap;

Peter pszemraj

Applying Mamba-3 Ideas to an OLMo-Hybrid-Style Gated DeltaNet

Recommendation

Decorative Monospace

What it looks like