Large language models like Generative Pre-trained Transformer 3 (or GPT-3) have boosted the branch of artificial intelligence called natural language processing (NLP). But while the applications for NLP algorithms are myriad, they operate on text data, which makes them complex and expensive and often limits their usefulness. For example, GPT-3 contains 175 billion parameters, costs roughly $12 million to train, and requires roughly 20 GPUs for inference alone.
But one LLM could change all that. Recently, Google (GOOGL) subsidiary DeepMind published a novel type of LLM architecture called Retrieval-Enhanced Transformer, or RETRO.
RETRO models are unique because they reference a separate database for information instead of relying on learned knowledge contained in their model parameters. Therefore, RETRO models can shrink to a fraction of the size of other LLMs. Though RETRO is roughly 5% the size of GPT-3 with 7.5 billion parameters, RETRO performs on par with and GPT-3.
Simon Barnett, genomics analyst at ARK Investment Management, recently wrote that “RETRO-style models will be useful in healthcare applications.”
“Unlike many machine-learning tasks with immutable truth labels, healthcare, and biology knowledge changes frequently,” Barnett wrote. “Traditional LLMs would require constant retraining to remain relevant in biology. Because RETRO references a separate database, the models can change without recalibration and could be useful in analyzing patient medical records and informing genetic counselors.”
One ARK fund that targets firms that develop AI and machine learning — and has holdings in DeepMind’s parent — is the ARK Autonomous Technology & Robotics ETF (CBOE: ARKQ). ARKQ is an actively managed fund that invests in autonomous technology and robotics companies relevant to the theme of disruptive innovation.
Companies within ARKQ are focused on and are expected to substantially benefit from the development of new products or services, technological improvements, and advancements in scientific research related to energy, automation & manufacturing, materials, artificial intelligence, and transportation. Its top holdings are Tesla Inc. (TSLA), at 10.47%; UiPath Inc (PATH), at 7.99%; and Trimble Inc. (TRMB), at 7.22%.
ARKQ has an expense ratio of 0.75%.
For more news, information, and strategy, visit the Disruptive Technology Channel.
Read more on ETFtrends.com.
The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.
www.nasdaq.com