{"id":1458787,"date":"2024-02-22T19:45:00","date_gmt":"2024-02-23T00:45:00","guid":{"rendered":"https:\/\/bugaluu.com\/news\/?p=1458787"},"modified":"2024-02-22T19:45:00","modified_gmt":"2024-02-23T00:45:00","slug":"groq-ais-lpu-the-breakthrough-answer-to-chatgpts-gpu-woes","status":"publish","type":"post","link":"https:\/\/bugaluu.com\/news\/groq-ais-lpu-the-breakthrough-answer-to-chatgpts-gpu-woes\/1458787\/","title":{"rendered":"Groq AI’s LPU: The Breakthrough Answer To ChatGPT’s GPU Woes?"},"content":{"rendered":"

Groq AI’s LPU: The Breakthrough Answer To ChatGPT’s GPU Woes?<\/span><\/p>\n

\n

Authored by Savannah Fortis via CoinTelegraph.com,<\/em><\/a><\/p>\n

Groq’s LPU chip emerges as a potential solution to the challenges faced by AI developers relying on GPUs, sparking comparisons with ChatGPT.<\/strong><\/em><\/p>\n

<\/a><\/p>\n

The latest artificial intelligence (AI) tool to capture the public\u2019s attention is the Groq LPU Inference Engine, which became an overnight sensation on social media after its public benchmark tests\u00a0went viral, outperforming the top models<\/a>\u00a0by other Big Tech companies.\u00a0<\/p>\n

Groq, not to be confused with Elon Musk\u2019s AI model called Grok, is, in fact, not a model itself but a chip system through which a model can run.<\/strong><\/p>\n

The team behind Groq developed its own \u201csoftware-defined\u201d AI chip which they called a language processing unit (LPU), developed for inference purposes. The LPU allows Groq to generate roughly 500 tokens per second.<\/p>\n

Comparatively, the publicly available AI model ChatGPT-3.5, which runs off of scarce and costly graphics processing units (GPUs), can generate around 40 tokens per second. Comparisons between Groq and other AI systems have been flooding the X platform.<\/p>\n

Groq is a Radically Different kind of AI architecture<\/p>\n

Among the new crop of AI chip startups, Groq stands out with a radically different approach centered around its compiler technology for optimizing a minimalist yet high-performance architecture. Groq’s secret sauce is this\u2026 pic.twitter.com\/Z70sihHNbx<\/a><\/p>\n

\u2014 Carlos E. Perez (@IntuitMachine) February 20, 2024<\/a><\/p>\n

Cointelegraph heard from Mark Heaps, the Chief Evangelist at Groq, to better understand the tool and how it can potentially transform how AI systems operate.\u00a0<\/p>\n

Heaps said that the founder of Groq, Jonathan Ross, initially wanted to create a system technology that would prevent AI from being \u201cdivided between the haves and have nots.\u201d<\/p>\n

At the time tensor processing units (TPUs) were only available to Google for their own systems, however, LPUs were born because:<\/p>\n

\u201c[Ross] and the team wanted anyone in the world to be able to access this level of compute for AI to find innovative new solutions for the world.\u201d<\/strong><\/em><\/p>\n

The Groq executive explained that the LPU is a \u201csoftware-first designed hardware solution,\u201d by which the nature of the design simplifies the way data travels \u2014 not only over the chip but from chip to chip and throughout a network.\u00a0<\/p>\n

\u201cNot needing schedulers, CUDA libraries, Kernels, and more improves not only performance but the Developer experience,\u201d he said.<\/p>\n

\u201cImagine commuting to work and every red light turned green right as you hit it because it knew when you’d be there. Or the fact is, you wouldn’t need traffic lights at all. That’s what it’s like when data travels through our LPU.\u201d<\/p>\n

A current issue plaguing developers in the industry is the scarcity and cost of powerful GPUs \u2014 such as\u00a0Nvidia\u2019s A100 and H100 chips<\/a>\u00a0\u2014 needed to run AI models.<\/p>\n

However, Heaps said they don\u2019t have the same issues as their chip is made using 14nm silicon. \u201cThis size of die has been used for 10 years in chip design,\u201d he said, \u201cand is very affordable, and readily available. Our next chip will be 4nm and also made in the United States.\u201d<\/p>\n

He said GPU systems still have a place when talking about running smaller-scale hardware deployments. However, the choice of GPU vs. LPU comes down to multiple factors including the workload and model.<\/p>\n

\u201cIf we’re talking about a large-scale system, serving thousands of users with high utilization of a large language model, our numbers show that [LPUs] are more efficient on power.\u201d<\/p>\n

LPU usage remains to be implemented by many of the major developers in the space.<\/strong> Heaps said several factors result in this, one of which being the relatively new \u201cexplosion of LLMs\u201d over the last year.<\/p>\n

\u201cFolks still wanted a one-size-fits-all solution like a GPU which they can use for both their training and inference. Now the emerging market has forced people to find differentiation and a general solution won’t help them accomplish that.\u201d<\/p>\n

Aside from the product itself, Heaps also touched on the elephant in the room \u2014 the name \u201cGroq.\u201d<\/strong><\/p>\n

Groq was created in 2016 with the name trademarked shortly after. However, Elon Musk\u2019s chatbot, Grok,\u00a0only appeared on the scene in November 2023<\/a>,\u00a0becoming widely recognized in the AI space in a short time.<\/p>\n

Heaps said there have been \u201cElon fans\u201d who have assumed they tried to \u201ctake the name\u201d or that it was a sort of marketing strategy. However, once the company\u2019s history became known he said, \u201cthen folks [got] a little quieter.\u201d<\/p>\n

\u201cIt was challenging a few months ago when their LLM was getting a lot of press, but right now I think people are taking notice of Groq, with a Q.\u201d<\/strong><\/em>\n<\/div>\n

Tyler Durden<\/a><\/span>
\nThu, 02\/22\/2024 – 14:45<\/span><\/p>\n

\u200bRead More<\/a>\u00a0<\/p>\n

\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"

Groq AI’s LPU: The Breakthrough Answer To ChatGPT’s GPU Woes? Authored by Savannah Fortis via CoinTelegraph.com, Groq’s LPU chip emerges as a potential solution to…<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-1458787","post","type-post","status-publish","format-standard","hentry","category-news","wpcat-1-id"],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pbimBl-67uP","jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/posts\/1458787","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/comments?post=1458787"}],"version-history":[{"count":0,"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/posts\/1458787\/revisions"}],"wp:attachment":[{"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/media?parent=1458787"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/categories?post=1458787"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bugaluu.com\/news\/wp-json\/wp\/v2\/tags?post=1458787"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}