Today, 12:57 PM
While we're on the topic, Stanford has effectively jail-broken GPT-3 (maybe even GPT-4?) their self-training AI model called Alpaca...
Alpaca is based on Facebook's Large Language Model called Llama. The biggest version of Llama is less than half the size of GPT-3, and there are smaller versions of Llama that retain most of the capabilities of GPT-3. Facebook used a smarter, less "brute-force" approach to training Llama, so it gives you more goodness for a given model-size. Stanford's Alpaca then "fine-tunes" the Llama model using a mechanism they're calling "self-training" -- basically, they queried GPT-3 with 52,000 example queries, stored its replies, and then trained Alpaca to use Llama to respond in a similar way. I'm oversimplifying because the training is not as deterministic as it sounds. The end-result is that you have a query-response AI based on a large-language model that, in its smaller sizes, can run on a single GPU card. So, all you have to do is train your own local Llama instance (costs maybe a few hundred dollars) and voila, you have something with GPT-3-like capabilities on your local computer. Obviously, Bing chat and OpenAI's models are still going to be way more powerful, but you have the basic framework to perform natural language question-answer queries which is extremely powerful.
Kudos to Stanford!
Find latest posts
View Articles
View Blog Entries
Connect With Us