{"id":1400,"date":"2024-11-11T10:00:00","date_gmt":"2024-11-11T10:00:00","guid":{"rendered":"https:\/\/netguide.io\/news\/2024\/11\/11\/mit-beats-agi-pub-benchmark-record-by-20-with-new-llm\/"},"modified":"2026-08-17T23:21:10","modified_gmt":"2026-08-17T23:21:10","slug":"mit-beats-agi-pub-benchmark-record-by-20-with-new-llm","status":"publish","type":"post","link":"https:\/\/netguide.io\/news\/en\/2024\/11\/11\/mit-beats-agi-pub-benchmark-record-by-20-with-new-llm\/","title":{"rendered":"MIT beats AGI-PUB benchmark record by 20% with new LLM"},"content":{"rendered":"<div id=\"netgu-3669199448\" class=\"netgu-before-content netgu-entity-placement\"><script async src=\"\/\/pagead2.googlesyndication.com\/pagead\/js\/adsbygoogle.js?client=ca-pub-6258556257245998\" crossorigin=\"anonymous\"><\/script><ins class=\"adsbygoogle\" style=\"display:block;\" data-ad-client=\"ca-pub-6258556257245998\" \ndata-ad-slot=\"3494115342\" \ndata-ad-format=\"auto\"><\/ins>\n<script> \n(adsbygoogle = window.adsbygoogle || []).push({}); \n<\/script>\n<\/div><p class=\"wp-block-paragraph\">A team from MIT has made significant progress in abstract task solving using an 8 billion parameter Language Model (LLM) and an innovative technique called Test-Time Training (TTT). With a performance of 61.9% on the ARC-AGI-PUB benchmark, they significantly outperformed the previous record of 42%. These results are remarkable as the ARC-AGI-PUB is a challenging collection of tasks requiring visual pattern recognition and complex reasoning.<\/p>\n<p class=\"wp-block-paragraph\"><strong>What is Test-Time Training (TTT)?<\/strong> TTT is a technique that enables the model to make small adjustments during test time in order to react to new tasks. TTT uses the context of the input data to temporarily optimize the model for the current task. This approach differs from traditional methods in which the model is trained for a task in advance. Especially in complex and novel tasks, TTT shows strong performance improvements.<\/p><div id=\"netgu-4131941556\" class=\"netgu-content netgu-entity-placement\"><aside class=\"deals-top deals-top--compact\">\n\n\t\t\t<h3 class=\"deals-top__title\">Top-Deals<\/h3>\n\t\n\t\t\t<ul class=\"deals-top__list\">\n\t\t\t\t\t\t\t<li class=\"deals-top__item\">\n\t\t\t\t\t<a class=\"deals-top__link\" href=\"https:\/\/netguide.io\/deals\/de\/deals\/ml2u-226-watch-face-fuer-wearos-im-google-play-store\/\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img class=\"deals-top__image\" src=\"https:\/\/netguide.io\/news\/wp-content\/uploads\/sites\/15\/2026\/10\/deal-3-150x150.png\" alt=\"\" width=\"52\" height=\"52\" loading=\"lazy\" \/>\n\t\t\t\t\t\t\n\t\t\t\t\t\t<span class=\"deals-top__body\">\n\t\t\t\t\t\t\t<span class=\"deals-top__name\">ML2U 226 Watch Face f\u00fcr WearOS im Google Play Store<\/span>\n\n\t\t\t\t\t\t\t<span class=\"deals-top__meta\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-top__price\">Gratis<\/span>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<s>1.39 \u20ac<\/s>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-discount\">-100%<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t<\/span>\n\n\t\t\t\t\t\t<span class=\"deals-top__temperature\">\n\t\t\t\t\t\t\t55\u00b0\n\t\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<li class=\"deals-top__item\">\n\t\t\t\t\t<a class=\"deals-top__link\" href=\"https:\/\/netguide.io\/deals\/de\/deals\/star-citizen-twitch-drops-deko-fuer-den-hangar\/\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img class=\"deals-top__image\" src=\"https:\/\/netguide.io\/news\/wp-content\/uploads\/sites\/15\/2026\/10\/source-150x150.jpg\" alt=\"\" width=\"52\" height=\"52\" loading=\"lazy\" \/>\n\t\t\t\t\t\t\n\t\t\t\t\t\t<span class=\"deals-top__body\">\n\t\t\t\t\t\t\t<span class=\"deals-top__name\">Star Citizen Twitch Drops \u2013 Deko f\u00fcr den Hangar<\/span>\n\n\t\t\t\t\t\t\t<span class=\"deals-top__meta\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-top__price\">Gratis<\/span>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t<\/span>\n\n\t\t\t\t\t\t<span class=\"deals-top__temperature\">\n\t\t\t\t\t\t\t55\u00b0\n\t\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<li class=\"deals-top__item\">\n\t\t\t\t\t<a class=\"deals-top__link\" href=\"https:\/\/netguide.io\/deals\/de\/deals\/moon-knight-digital-watch-face-fuer-wear-os-4\/\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img class=\"deals-top__image\" src=\"https:\/\/netguide.io\/news\/wp-content\/uploads\/sites\/15\/2026\/09\/deal-27-150x150.png\" alt=\"\" width=\"52\" height=\"52\" loading=\"lazy\" \/>\n\t\t\t\t\t\t\n\t\t\t\t\t\t<span class=\"deals-top__body\">\n\t\t\t\t\t\t\t<span class=\"deals-top__name\">MOON KNIGHT Digital Watch Face f\u00fcr Wear OS 4+<\/span>\n\n\t\t\t\t\t\t\t<span class=\"deals-top__meta\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-top__price\">Gratis<\/span>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<s>0.89 \u20ac<\/s>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-discount\">-100%<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t<\/span>\n\n\t\t\t\t\t\t<span class=\"deals-top__temperature\">\n\t\t\t\t\t\t\t55\u00b0\n\t\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<li class=\"deals-top__item\">\n\t\t\t\t\t<a class=\"deals-top__link\" href=\"https:\/\/netguide.io\/deals\/de\/deals\/battlefield-6-code-fuer-spielerkartenhintergrund-und-titel\/\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img class=\"deals-top__image\" src=\"https:\/\/netguide.io\/news\/wp-content\/uploads\/sites\/15\/2026\/10\/battlefield-6-how-to-redeem-codes-asset-code-9-2026_de-150x150.webp\" alt=\"\" width=\"52\" height=\"52\" loading=\"lazy\" \/>\n\t\t\t\t\t\t\n\t\t\t\t\t\t<span class=\"deals-top__body\">\n\t\t\t\t\t\t\t<span class=\"deals-top__name\">Battlefield 6 \u2013 Code f\u00fcr Spielerkartenhintergrund und Titel<\/span>\n\n\t\t\t\t\t\t\t<span class=\"deals-top__meta\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-top__price\">Gratis<\/span>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t<\/span>\n\n\t\t\t\t\t\t<span class=\"deals-top__temperature\">\n\t\t\t\t\t\t\t55\u00b0\n\t\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t\t\t<li class=\"deals-top__item\">\n\t\t\t\t\t<a class=\"deals-top__link\" href=\"https:\/\/netguide.io\/deals\/de\/deals\/train-sim-world-7-starter-pack-kostenlos-fuer-ps4-ps5-xbox-pc\/\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img class=\"deals-top__image\" src=\"https:\/\/netguide.io\/news\/wp-content\/uploads\/sites\/15\/2026\/09\/capsule_616x353-11-150x150.jpg\" alt=\"\" width=\"52\" height=\"52\" loading=\"lazy\" \/>\n\t\t\t\t\t\t\n\t\t\t\t\t\t<span class=\"deals-top__body\">\n\t\t\t\t\t\t\t<span class=\"deals-top__name\">Train Sim World 7: Starter Pack kostenlos f\u00fcr PS4, PS5, Xbox, PC<\/span>\n\n\t\t\t\t\t\t\t<span class=\"deals-top__meta\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"deals-top__price\">Gratis<\/span>\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t<\/span>\n\n\t\t\t\t\t\t<span class=\"deals-top__temperature\">\n\t\t\t\t\t\t\t55\u00b0\n\t\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/li>\n\t\t\t\t\t<\/ul>\n\n\t\t\t\t\t<p class=\"deals-top__more\">\n\t\t\t\t<a href=\"https:\/\/netguide.io\/deals\/\">Alle Deals ansehen \u2192<\/a>\n\t\t\t<\/p>\n\t\t\t<\/aside>\n<\/div>\n<p class=\"wp-block-paragraph\"><strong>How did the breakthrough at MIT work?<\/strong> The team found that a combination of initial fine-tuning and the application of special transformations to test data was crucial to success. These adjustments improved the model&#8217;s ability to recognize patterns. In addition, an ensemble with modern programming approaches was created to increase the performance value to 61.9% &#8211; a performance equivalent to that of an average human.<\/p>\n<p class=\"wp-block-paragraph\"><strong>The significance of this development<\/strong> The results show that language models can solve abstract tasks even without explicit symbolic methods.<\/p>\n<p class=\"wp-block-paragraph\">Read more in the <a href=\"https:\/\/ekinakyurek.github.io\/papers\/ttt.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">official paper<\/a>.<\/p>\n<div id=\"netgu-357906517\" class=\"netgu-after-content netgu-entity-placement\"><script async src=\"\/\/pagead2.googlesyndication.com\/pagead\/js\/adsbygoogle.js?client=ca-pub-6258556257245998\" crossorigin=\"anonymous\"><\/script><ins class=\"adsbygoogle\" style=\"display:block;\" data-ad-client=\"ca-pub-6258556257245998\" \ndata-ad-slot=\"4559785002\" \ndata-ad-format=\"auto\"><\/ins>\n<script> \n(adsbygoogle = window.adsbygoogle || []).push({}); \n<\/script>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>An MIT team hit 61.9% on the ARC-AGI-PUB benchmark with an 8B-parameter LLM and Test-Time Training, smashing the previous record of 42% by almost 20 points.<\/p>\n","protected":false},"author":1,"featured_media":1624,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_affilinode_tracking":"","_affilinode_autolink_off":false,"_affilinode_rating":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[1698],"tags":[],"class_list":["post-1400","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ki-en"],"brizy_media":[],"_links":{"self":[{"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/posts\/1400","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/comments?post=1400"}],"version-history":[{"count":1,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/posts\/1400\/revisions"}],"predecessor-version":[{"id":1626,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/posts\/1400\/revisions\/1626"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/media\/1624"}],"wp:attachment":[{"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/media?parent=1400"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/categories?post=1400"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/netguide.io\/news\/wp-json\/wp\/v2\/tags?post=1400"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}