Tag: LLM's

Mark Zuckerberg Teaches Us the TMKK of Game Theory

Why is Llama open sourced?

We’ll get to the answer in a bit, but just in case you need help understanding what Llama is:

Llama (Large Language Model Meta AI) is a family of autoregressive large language models (LLMs), released by Meta AI starting in February 2023.
On April 18, 2024, Meta released Llama-3 with two sizes: 8B and 70B parameters. The models have been pre-trained on approximately 15 trillion tokens of text gathered from “publicly available sources” with the instruct models fine-tuned on “publicly available instruction datasets, as well as over 10M human-annotated examples”. Meta plans on releasing multimodal models, models capable of conversing in multiple languages, and models with larger context windows. A version with 400B+ parameters is currently being trained

So what, you might say. There’s the OG ChatGPT, there’s Claude, there’s Gemini… so one more comes along. Ho hum.

Well, if you say that, you’d be very, very wrong.

Why would you be wrong?

Because of this:

In contrast, the most powerful LLMs have generally been accessible only through limited APIs (if at all), Meta released LLaMA’s model weights to the research community under a noncommercial license

Why does this matter? Because, as our good friend Claude explains:

“The release of the Llama model by Meta under a noncommercial license is a significant development in the field of large language models (LLMs) and artificial intelligence more broadly. Here’s why it matters:

  1. Accessibility: Before this, the most powerful LLMs were usually kept secret by the companies that made them. For example, OpenAI’s GPT-3 model could only be used through a paid interface, like a vending machine you put money into to get a result. By releasing Llama’s “model weights” – essentially the knowledge the AI has learned – Meta has allowed researchers and hobbyists to experiment with and build upon a cutting-edge language model. It’s like they’ve given away the recipe for a powerful tool, not just limited access to using the tool itself.
  2. Democratization of AI: Restricting access to top LLMs meant that only a handful of big corporations could really use this powerful technology. Imagine if only a few factories could use electricity. An open-source model changes the game significantly. It empowers way more people to explore creative uses of language models and lowers the barriers to innovation in this space. It’s like the difference between a few people having libraries versus everyone having access to all the books.
  3. Cost: Using LLMs through paid interfaces can get expensive quickly, putting them out of reach for many. It’s like having to rent a supercomputer every time you want to use one. With access to the model weights themselves, people can run the model on their own computers, dramatically reducing costs. This opens up experimentation to students, researchers, startups and others with limited budgets.
  4. Customization: When you can only access a model through an interface, you’re limited to pre-defined uses, like ordering off a set menu at a restaurant. Having the actual model provides much more flexibility to tailor and fine-tune it for specific applications and domains. This could lead to an explosion of niche language models customized for particular industries or use cases – imagine a model specifically trained to understand and generate legal jargon, or one tuned for writing poetry.
  5. Reproducibility and Transparency: In scientific research, it’s crucial to be able to reproduce results. Using an API is like a black box – you can’t see how the model works under the hood, you just get the output. With the model weights, the exact workings of the model can be scrutinized, enabling more robust research and understanding of how these models function. It’s like being able to examine the engine of a car instead of just looking at the exterior.

Model weights are the key to how a language model works. They’re essentially the “knowledge” the model has learned during training. In a neural network (which is what most modern language models are), the weights are the strength of the connections between the neurons. These weights determine how the model responds to a given input, like how a brain’s neural connections determine a person’s response to a question. By releasing the weights, Meta has provided the “source code” of their model, allowing others to understand how it works, modify it, and use it for their own purposes.

While the noncommercial license does place some limits on how Llama can be used (you couldn’t start a company selling access to it, for example), the release of the model is still a major shift in the AI landscape that could have far-reaching effects on research, innovation, and accessibility of this transformative technology. We’re likely to see a proliferation of new applications and rapid progress in natural language AI as a result.”

You don’t just get the dish to eat, as Claude puts it, but you get the recipe so that you can try and recreate (and modify) the recipe at home. Not all of us have specialized cooking equipment at home, but those of us who do can get cooking very quickly indeed.

Speaking of cooking, have you seen this excellent series from Epicurious called 4 Levels? Chefs of varying expertise (home cook to the pros) are invited to cook the same dish, but with varying levels of expertise, ingredients and equipment.

Source

That’s what the 8 billion, 70 billion and 400 billion parameter models are all about. Same idea (recipe), but different capabilities and “equipment”.

But why do this? If Gemini, Claude and ChatGPT are giving away basic versions for free and premium versions for 20 USD per month, then why is Meta not just giving away all versions for free… but also giving away the recipe itself?

Because game theory! (Do read the tweet linked here in its entirety, what follows is a much more concise summarization):

  1. You can get janta to do the debugging of the model for you.
  2. If social debugging and optimization of models makes AI so kickass that AI friends can replace all your friends, then who owns the technology to make these friends “wins” social media. Nobody does, because janta is doing the work for “everybody”. So sure, maybe Mark bhau doesn’t win… but hey, nobody else does either!
  3. The nobody else does point is the really important point here, because by open sourcing these models, he is making sure that Gemini, Claude and ChatGPT compete against everybody out there. In other words, everybody works for Mark bhau for free, but not to help Mark win, but to help make sure the others don’t win.

The economics of AI is a fascinating thing to think about, let alone the technological capabilities of AI. I hope to write more about this in the coming days, but whatay topic, with whatay complexities. Yay!

All this is based on just one tweet sourced from a ridiculously long (and that is a compliment, believe me) blog post by TheZvi on Dwarkesh’s podcast with Mark Zuckerberg. Both are worth spending a lot of time over, and I plan to do just that – and it is my sincere recommendation that you do the same.

The Conversation About Inheritance Taxes that I Would Like to Have

Are machines dystopian, or are humans dystopian? Asking for a blog post.

I am going to give this prompt to a very capable LLM. I would like you to critique this prompt, and let me know how it can be made better. Point out what is missing in terms of the comprehensiveness of my framework, as well as any details I may have missed out on. Optimize for the LLM giving an answer that holistically addresses the problem I am thinking about. We are not as interested in generating an answer to the problem as we are in generating a way to think about the problem. What the problem is will become clear upon reading my prompt. Also rate my prompt (for the parameters we’ve spoken about) on a scale of 1 to 10, where 1 is a poor prompt, and 10 is an excellent prompt. Please give me your reasons for the points I score, and reasons for points you have deducted.

“There has been a lot of controversy recently about whether India should, or should not have, an inheritance tax. I would like you to build out for me a framework to think through the issue. While doing so, I would like you to have a persona that has deep expertise in the global history of public finance, taxation laws, development economics, political economy and welfare economics. I would also like you to be well versed in the history of India’s taxation laws, and in India’s political economy. I would like you to assume that I am a very curious person, but with very little knowledge of economics, theoretical or applied. I like detailed explanations, but in simple English. I would like the framework to contain the following aspects, and please note that I plan to ask the LLM these questions sequentially, with plenty of scope for discussion, pushback and clarification after each question has been answered:

  1. Are there principles that underlie taxation for any economy? If yes, what are they? If not, what should a list of such principles look like?
  2. Within these principles, which will presumably be generic and applicable to all kinds of taxes, what principles might apply to inheritance taxes specifically?
  3. Should an inheritance tax be taxed while the money is still in the account(s) of the deceased individual, or the inheritor’s account? What would be better? For whom would it be better? The person being taxed or the entity doing the taxing?
  4. How should one think about the desirability of having an inheritance tax for an economy? Should it be a function of the number of people in an economy? Or their age? Or their inheritance laws?
  5. Should the state capacity of that economy have a bearing on the issue? That is, it might be all well and good to take from the rich, but does the state have the ability to spend that money effectively?
  6. Should inheritance taxes be a function of how thorough income tax laws are? For example, the higher the number of loopholes in the income tax act, the better the case for an inheritance tax is my line of thinking – is this justified?
  7. What is the history of the development of the theory surrounding inheritance taxes? What about the theory as applied to different geographies, different time periods, and countries with differing stages of development?
  8. Have countries enacted laws with sunset clauses? Why or why not?
  9. Have countries experienced Laffer Curve type problems upon implementation of inheritance taxes? How have these problems been detected, and how have they been dealt with?
  10. Based on our conversations so far, give me your top five “for” and top five “against” arguments for an inheritance tax in India today.”

Here’s Claude’s answer:

“Here’s my critique of your prompt:

Strengths:

  1. Comprehensive coverage: Your prompt covers a wide range of aspects related to inheritance taxes, including principles, implementation, history, and country-specific considerations.
  2. Persona specification: You provide clear guidelines for the LLM’s persona, which should help generate responses tailored to your needs.
  3. Sequential questioning: Breaking down the prompt into sequential questions allows for a structured and focused discussion.
  4. Encouraging discussion: You leave room for discussion, pushback, and clarification, which can lead to a more engaging and informative conversation.
  5. Specific to India: Focusing on India’s context helps narrow down the scope and generates more relevant insights.

Areas for improvement:

  1. Clarity on the target audience: While you mention that you have little knowledge of economics, it would be helpful to specify your background and the purpose of this framework (e.g., personal curiosity, policy-making, academic research).
  2. Prioritization of questions: Consider ranking the questions based on their importance or relevance to your understanding of the issue.
  3. Comparative analysis: You could include a question about comparing inheritance tax policies and their outcomes across different countries, especially those with similar characteristics to India.
  4. Societal impact: Consider adding a question about the potential social and distributional effects of inheritance taxes, such as wealth inequality and social mobility.
  5. Feasibility and alternatives: You could ask about the feasibility of implementing an inheritance tax in India given its current economic, political, and administrative context, and explore potential alternatives or complementary measures.

Missing aspects:

  1. Economic impact: Include a question about the potential economic consequences of an inheritance tax, such as its effects on savings, investment, and capital formation.
  2. Tax evasion and avoidance: Address the issue of tax evasion and avoidance strategies that may arise with inheritance taxes and how to mitigate them.
  3. Interaction with other taxes: Explore how inheritance taxes might interact with other forms of taxation, such as income, property, or gift taxes.
  4. Public opinion and political feasibility: Consider asking about public sentiment towards inheritance taxes and the political challenges in implementing such a policy.

Rating: 8/10

Reasons for the score:

  • Comprehensive coverage of key aspects
  • Clear persona specification and sequential questioning
  • Encourages discussion and focuses on India’s context

Points deducted:

  • Lack of clarity on target audience and purpose
  • Missing aspects related to economic impact, tax evasion, interaction with other taxes, and public opinion

By addressing these areas and incorporating the missing aspects, your prompt could be further strengthened to generate a more holistic and nuanced framework for thinking about inheritance taxes in India.”

Run with this framework, and please feel free to modify it and improve upon it. Please use this framework to have a discussion with an LLM about inheritance taxes. It’s the kind of discussion our ancestors were capable of having re: public policy issues.

Also, feel free to copy and improve upon the framework itself – run your prompts through an LLM to see how you can make your thinking about an issue better.

There is a discussion to be had about how the price, income and substitution effects are at play when it comes to spending time talking with folks online, or spending it talking to LLM’s, but I’d rather not go there for now.

I’ll say this much though: I’m usually quite favorably disposed to the substitution effect.

Andrej Karpathy on an Intro to LLM’s

Learn at Twitter Speed, Get Tested at AOL Speed

The title of today’s post is directly lifted from an MR Post from yesterday, which you should read in its entirety.

Instead of hearing a rumor at the coffee shop and running down to the bank branch to wait on line to withdraw your money, now you can hear a rumor on Twitter or the group chat and use an app to withdraw money instantly. A tech-friendly bank with a highly digitally connected set of depositors can lose 25% of its deposits in hours, which did not seem conceivable in previous eras of bank runs.
But the other part of the problem is that, while depositors can panic faster and banks can give them their money faster, the lender-of-last-resort system on which all of this relies is still stuck in a slower, more leisurely era. “When the user interface improves faster than the core system, it means customers can act faster than the bank can react,” wrote Byrne Hobart. You can panic in an instant and withdraw your money with an app, but the bank can’t get more money without a series of phone calls and test trades that can only happen during regular business hours.

https://marginalrevolution.com/marginalrevolution/2023/03/banks-as-meme-stocks.html

Try this variant on for size:

Instead of hearing about a concept in a classroom, and running to the library to get access to the book that explains it in greater detail, now you can hear about a concept on Twitter, or the group chat, and use ChatGPT to learn all about it instantly. A tech-friendly classroom with a highly digitally connected group of learners can learn much more about a topic in a couple of hours, which did not seem conceivable in previous learning environments.
But the other part of the problem is that, while learners can learn faster and LLM’s can give them additional nuance and context much better, the exam system on which all of this ultimately relies for certifications is still stuck in a slower, more traditional era. “When the learning environment improves faster than the testing environment, it means learners can learn better than colleges can meaningfully test them,” wrote a grumpy old blogger. You can learn much more about a topic in a semester than you ever could before, but the college will still insist on making you memorize stuff so that you can choose five questions out of six to answer in a closed-book-pen-and-paper examination.

It’s not an exact analogy, of course. But there are two points to this blogpost:

  1. Where colleges and universities are concerned, this is a useful framework to deploy. And sure I had fun tweaking that excerpt in order to maximize my snarkiness – but I’m not joking about the point being made. When students are able to learn far better, far more effectively and far faster, but the testing environment doesn’t keep up with either the learning or its applications, it is a problem. Simply put, if teaching and learning with LLM’s is best, but the college thinks that testing without access to LLM’s is best, there’s a disconnect.
  2. The broader point, of course, is that you should be applying this framework to everything. Banks and colleges, sure. What about government (at all levels)? What about software companies? What about delivery apps? What about <insert the place you work at here>? Which parts of your organization are already using LLM’s in their workflows, or will sooner rather than later? Which parts will be the most reluctant, and therefore the last to adopt to this brave new world? What imbalances might result? How should we incentivize the rate of adoption such that we optimize appropriately?

Note that this doesn’t necessarily mean incentivizing those reluctant to adopt! You might want to incentivize a slower adoption of ChatGPT, if that’s what you think is best (and yes, that goes for colleges too). But if that’s the route you’re going to go down, think first about the competition. And note that in the age of LLM’s, defining who your competition is isn’t as easy as it used to be.