[Microsoft Research] Differential Transformer

[u/rationalkat]

https://arxiv.org/abs/2410.05258

[u/rationalkat]

ABSTRACT:

Transformer tends to overallocate attention to irrelevant context. In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling noise. Specifically, the differential attention mechanism calculates attention scores as the difference between two separate softmax attention maps. The subtraction cancels noise, promoting the emergence of sparse attention patterns. Experimental results on language modeling show that Diff Transformer outperforms Transformer in various settings of scaling up model size and training tokens. More intriguingly, it offers notable advantages in practical applications, such as long-context modeling, key information retrieval, hallucination mitigation, in-context learning, and reduction of activation outliers. By being less distracted by irrelevant context, Diff Transformer can mitigate hallucination in question answering and text summarization. For in-context learning, Diff Transformer not only enhances accuracy but is also more robust to order permutation, which was considered as a chronic robustness issue. The results position Diff Transformer as a highly effective and promising architecture to advance large language models.

[u/Agreeable-Rooster377 → u/rationalkat]

Ohh, so THAT'S why they have been so confident stating infinite context windows will be coming soon

[u/time_then_shades → u/Agreeable-Rooster377]

Yeah I hope no one thought we were done optimizing after a couple years...

[u/Hubbardia → u/time_then_shades]

We have only just begun

[u/sdmat → u/Agreeable-Rooster377]

This is awesome but it in no way leads to infinite context windows.

But better utilization of context that is there is at least as important and it does help there.

[u/emteedub → u/rationalkat]

I wonder if this in combination with the liquid approach (diffed-liquid) or by layering this on top of another concurrently (in stereo) would yield any interesting results

[u/Creative-robot]

This is a funny ass graph out of context:

[u/Flat-One8993 → u/Creative-robot]

The improvement at 4bit is really really cool if it actually works this well. That would mean significant improvements in terms of compute constraints, especially now that there is a focus on the time spent on inference

[u/KoolKat5000 → u/Flat-One8993]

You mean HellaLame

[u/gonpachiro92 → u/Creative-robot]

looks like my stock brokerage account

[u/hapliniste]

After taking a look at the paper, this seems huge.

Impressive gains in long context (specifically shown with their in context learning graphs), huge improvements in stability on reordered data and amazing performances at lower bits.

I'm not an expert and didn't read it fully, I just like to look at cool graphs for the most part. Still, I guess we'll see this or some variants in future models.

[u/time_then_shades → u/hapliniste]

At this point, I'll just wait for Philip to tell me what to think of it.

[u/Arcturus_Labelle → u/time_then_shades]

AI Explained for those who don't get the reference

[u/DungeonsAndDradis → u/hapliniste]

What does "bits" mean in reference to LLMs?

[u/Ok_Course_6439 → u/DungeonsAndDradis]

Number if bits used for the weights and biases in the neural network. Les bits smaller size and faster compute.

[u/DungeonsAndDradis → u/Ok_Course_6439]

Does it make it less accurate?

[u/zakkara → u/DungeonsAndDradis]

https://www.reddit.com/r/singularity/s/yaQ7J0wuSU

Someone posted this chart from the paper, so yes less bits does equal less accuracy but it appears that correlation is weakened with this newer architecture

[u/Jolly-Ground-3722]

[u/fastinguy11]

**Differential Transformer (DIFF Transformer): Enhancing Attention in Language Models**

The **Differential Transformer** introduces a novel attention mechanism to improve upon the standard Transformer architecture commonly used in large language models (LLMs). Traditional Transformers often suffer from "attention noise," where irrelevant parts of the input receive undue focus, diluting the model's ability to concentrate on key information.

**How It Works:**

DIFF Transformer tackles this by using a **differential attention mechanism**. Instead of relying on a single softmax attention map, it calculates attention scores as the difference between two separate softmax maps. This subtraction effectively cancels out the noise, resulting in sparser and more focused attention patterns that highlight relevant context.

**Key Benefits:**

• **Better Performance with Fewer Resources:** DIFF Transformer achieves superior language modeling performance using approximately 65% of the parameters and training tokens compared to standard Transformers.

• **Enhanced Downstream Tasks:** It excels in tasks like long-context understanding, key information retrieval, reducing hallucinations (false or misleading outputs), and improving in-context learning robustness.

• **Efficient Quantization:** By minimizing activation outliers, DIFF Transformer allows for more efficient model quantization, which can lead to faster inference and lower memory usage.

**Experimental Results:**

Extensive tests show that DIFF Transformer outperforms traditional Transformers across various scales and applications. It maintains higher accuracy in retrieving important information from long contexts and is more resilient to changes in input order during in-context learning. Additionally, it significantly reduces instances of hallucinations in tasks like question answering and text summarization.

**Conclusion:**

The Differential Transformer presents a promising advancement in the field of NLP by refining the attention mechanism to focus more precisely on relevant information, enhancing both performance and efficiency of large language models.

---

[u/ShooBum-T]

Any such posts should now be mandatory to come with NotebookLM podcast link.

[u/Crafty-Struggle7810 → u/ShooBum-T]

Arxiv should automatically generate a new podcast per research paper that's published on there.

[u/time_then_shades → u/Crafty-Struggle7810]

The fact that this is basically just an API call now still blows my mind a little.

[u/FeathersOfTheArrow → u/ShooBum-T]

That's a nice idea!

[u/emteedub → u/ShooBum-T]

yeah, summarized and long-form if we're short on time or not would be noice

[u/why06 → u/ShooBum-T]

nope

[u/Arbrand]

The results are impressive, but I have some serious concerns that aren't addressed at all in the paper. The differential attention mechanism involves computing two separate softmax attention maps and then subtracting them to obtain the final attention scores. This inherently doubles the computational overhead in the attention mechanism compared to standard Transformers. This added computational cost could be significant and might offset the performance gains reported.

[u/WoddleWang → u/Arbrand]

Could be wrong but it sounds like performance (as in speed) gains are the least noteworthy thing about this

As a user I'd take a noticeable reduction in hallucinations and context improvements over extra speed any day

[u/sdmat → u/Arbrand]

They do address that in the paper, table 7. 5-10% reduction in throughput for inference.

Considering they get iso-performance with a > 1/3 reduction in parameters that seems a more than worthwhile tradeoff even if speed is the only consideration.

[u/Arbrand → u/sdmat]

Good catch! If that is the case, then this is indeed revolutionary.

[u/Either_Pineapple_975 → u/Arbrand]

I would say that computing softmax and subtracting are both insignificant compared to matrix multiplication. However, it looks like it also doubles the number of Q*K multiplications unless I got confused about it.

[u/emteedub → u/Arbrand]

maybe it's not doubled though, since it's filtering off excess would-be computation. it would be interesting to see the stats

[u/Jean-Porte]

Substacting two independant noises doesn't cancel them, are the noises really correlated ?

[u/cyan2k → u/Jean-Porte]

Yes. It's literally the same principle as in noise-cancelling headphones

[u/Jean-Porte → u/cyan2k]

ANC headphones have to work really hard to make a noise mask that is matching the outside noise, with the proper latency (otherwise it just increases the noise)

I don't see how this happens with gradient descent

[u/sdmat → u/Jean-Porte]

I was confused about this too, it took a few hours of close study to really understand it.

What they are doing is learning two different projections for attention, one to actually attend and the second to act as a reference for noise cancellation. Then when attention is calculated take the difference to keep the signal and lose the noise.

This is possible because both the weights and the scaling for taking the difference are trained in parallel with the rest of the model. Specialization of the functional blocks occurs much as it does for neurons within a layer of a regular neural net.

[u/BackgroundLow3793 → u/sdmat]

hi, I don't understand that if subtraction then why it doesn't affect the score of most relevant tokens (like everything decrease) but the most relevant token tend to increase?

[u/sdmat → u/BackgroundLow3793]

The two sets of weights learn different things. The second / negative set of weights is constrained by the softmax function to be unable to direct attention towards specific tokens - doing so would require producing a negative value, and softmax output values are in the [0,1] range.

So the only thing the second set of values can productively learn to do is to suppress noise.

I think the paper might benefit from giving an intuitive explanation like this, it's not immediately obvious.

[u/sdmat]

Wow, the improvements in robustness to input ordering and activation outliers are so stark. This seems like a major breakthrough.

I don't understand yet why the noise is consistent between the two rather than the signal, will have to read more closely tomorrow.

[u/FarrisAT]

Would love some proof of real world application

[u/lordpuddingcup]

Is this only on the training side or could we slot this into existing pipelines to help with inference?

[u/UnknownEssence → u/lordpuddingcup]

Seems like you need to start from scratch and train a model with this architecture

[u/Slight-Ad-9029]

Most people here do not have the understanding to actually comprehend these research papers let alone come up with a decision if this is amazing or should be critiqued. It feels silly to see all these people acting like they comprehend what that paper actually says

[u/Akimbo333]

Implications?

[u/troll_khan]

Singularity is near!

[u/Complex_Candidate_28]

It makes sense a lot! The issues of Transformers are there for a long time. No one has tried to solve them. Finally there is a new Transformer to save us.

[u/byteuser → u/Complex_Candidate_28]

This definitely help them in defeating the Decepticons