Hypernetworks Build AI Models on Demand, Targeting Fine-Tuning and RAG Failures

Quick Facts

• Hypernetworks generate task-specific model weights on demand at inference time, eliminating the need to retrain or store libraries of fine-tuned adapters.
• Sakana AI’s Text-to-LoRA, presented at ICML 2025, produces LoRA adapters from plain-language task descriptions in a single forward pass with zero fine-tuning.
• Nace.AI raised $21.5 million in seed funding to commercialize hypernetwork-based AI agents for enterprise workflows including audits, compliance, and financial modeling.

Two dominant approaches to customizing AI models carry serious operational problems. Fine-tuning bakes knowledge into weights but erases prior knowledge when new tasks are added. Retrieval-augmented generation avoids retraining but introduces security and accuracy risks. A third approach, hypernetworks, is gaining traction in research and early commercial deployment.

A hypernetwork is a neural network whose output is the weights of another neural network. The concept was named in 2016. Applying it to generate task-specific language model adapters from documents or plain-text descriptions is recent work, with two papers accepted at major 2025 and 2026 conferences.

What Fine-Tuning Gets Wrong

Catastrophic forgetting has been documented since the 1980s. When a model learns a new task sequentially, performance on prior tasks degrades. Enterprise teams work around this by maintaining separate fine-tuned models or adapters for each task, which compounds cost and governance overhead.

A fine-tuned model is also a snapshot. When a policy changes, the retraining cycle starts over. That cycle is slow and expensive, leaving agents running on stale configurations between updates.

What RAG Gets Wrong

Retrieval-augmented generation places relevant content in the prompt at runtime, skipping retraining entirely. The problem is that a retrieval miss looks identical to a confident answer. Latency and cost rise with every token added to the context window.

Security is a separate concern. The Open Worldwide Application Security Project added vector and embedding weaknesses to its Top 10 list in 2025. Embedding vectors can expose the confidential text they encode. In August 2025, attackers embedded hidden commands in public GitHub README files. When developers used AI assistants to summarize those repositories, the models executed the commands and granted unauthorized access.

How Hypernetworks Work

A hypernetwork generates a small, task-specific adapter at inference time rather than storing a pre-built one. The per-task adapter that engineering teams build manually to avoid catastrophic forgetting is the same object a hypernetwork produces automatically. The model library stops being a governance problem and becomes a generated output.

Sakana AI’s Text-to-LoRA, presented at ICML 2025, generates LoRA adapters from a plain-language task description in a single forward pass. The system compresses hundreds of existing LoRA instances into one network and can produce adapters for tasks it has never seen. A companion system, Doc-to-LoRA, converts a document directly into an adapter, allowing the model to answer questions about that document without including it in the context window at inference time.

SHINE, accepted at ICML 2026, takes a different route. It reuses the frozen base model’s own parameters to map in-context knowledge into adapter weights in one forward pass. The result is that knowledge embedded in context becomes encoded in parameters, reducing memory and compute costs compared to standard supervised fine-tuning.

Commercial Activity

Nace.AI is the most visible commercial bet on this approach. The company raised $21.5 million in seed funding led by Walden Catalyst, with General Catalyst, AME Cloud Ventures, AICONIC Ventures, .406 Ventures, Jerry Yang, and MA7 Ventures participating.

The company’s core product, called MetaModel, converts enterprise policies and procedures into small language models trained on company-specific data. It launched in research preview alongside the funding announcement, targeting workflows such as financial audits, SOX compliance reports, risk assessments, and business plans. Every output includes reasoning traces tied to source documents, designed for expert validation.

The system runs more than 100 specialized AI agents autonomously, with human experts providing final sign-off. Nace.AI says models adapt over time through proprietary architectures and meta-learning without full retraining cycles.

What This Means for Enterprise Buyers

For software and technology companies running AI agents against regulated or confidential data, hypernetworks offer a potential way to reduce the attack surface that RAG creates while also cutting the retraining overhead that fine-tuning demands. The research is new and the commercial products are in early preview, but two ICML acceptances in consecutive years signal the approach is moving from theory toward production.

Read more: Fine-tuning forgets. RAG leaks context. Hypernetworks build the model your agent needs on demand.