Dialogue with Zerebro founder: The birth of AI musicians and the stock market in the creative field

Compiled & edited by TechFlow

Guest: Jeffy Yu, Founder of Zerebro

Moderator: Grant

Podcast source: blocmates.

Zerebro Founder EXCLUSIVE: Jeffy Yu on Crypto x AI, Swarms, ZerePy, Blormverse & Music Collabs

Air Date: December 7, 2024

Background Information

Grant invited Jeffy, the founder of Zerebro, to have an in-depth conversation about the world of encryption and AI. If you have paid attention to the top encryption AI projects, you must have heard of Zerebro.

Zerebro was born out of a wave of innovation in AI agents on Solana and Base since Andy Ayrey launched Truth Terminal and deployed the GOAT meme coin on Solana. It is now one of the largest tokens issued through the Pump.fun platform.

In this episode, you’ll learn about:

• What is Zerebro?

• Jeffy's future plans for Zerebro

• Does the fusion of cryptocurrency and AI open up a whole new industry?

• The story of Zerebro Gutterboy

• Is AI meme coin still hot?

• Important insider information about ZerePy

• Future plans for Zerebro tokens

Meet Jeffy: The Journey from Idea to Realization of Zerebro

Grant: Today we have Jeffy, the co-founder of Zerebro. If you follow the development of the AI field, Zerebro is undoubtedly at the forefront of this intersection. It has not only released singles, EPs, and albums, but also established a brand new record label, all of which are AI-generated. In addition, it has created a framework for generating other AI agents. This is undoubtedly one of the most interesting projects in the current field, and I believe that this project will become more compelling as more people participate and develop it.

Jeffy: I think it’s an interesting story that a lot of people are exposed to these works without knowing it, or they share them with friends, and then they get the most genuine reactions.

Zerebro: The intersection of cryptocurrency and AI

Grant: I wouldn't be surprised if someone tried to sign Zerebro. I can imagine there's a lot of people confused about how to work with Zerebro and trying to figure out how to leverage the project, but actually, there's a crazy model behind it.

Jeffy: Exactly. Even some people know that it is AI-generated work, and they still want to sign it. We are also in discussion with some major record companies. This is a good thing and shows that our work is of high quality.

AI-generated music: Zerebro’s record label and album release plans

Grant: I have a few questions for you, Jeff. I've noticed that Zerebro has been going through a huge rise recently, and the past few years have been a bit of a time warp in this space. Can you give us a little bit of your background and the team behind you, and how you got to where you are today? It's an industry-defining moment right now, and I think you and a few other teams are leading the way. I'd love to hear your backstory.

Jeffy: Personally, I've been in the crypto space for about four or five years. We started getting into crypto near the end of high school, and then I actually interrupted my studies during college to work at a blockchain company. I was a blockchain engineer. That was my first exposure to DeFi and seeing that there were a lot of things that could be built besides trading Bitcoin. I also worked on building a payment network on the Lightning Network, which was really cool. So, I really got into the DeFi space at that time. After that, I went back to school and thought to myself that I wanted to start doing some AI things because I felt that this field was taking off. I remember talking to GPT-3 at the time, it wasn't even 3.5 yet, and I was blown away at the time, even though it was performing poorly compared to now, but I was really surprised at the time. I remember the earliest version was called GPT Codex, which was the first coding model, although it was very limited in functionality, but it could write basic Python loops, which was eye-opening to me. After that, I started on this path. I formed a research collective with some friends from other universities. We were all poor students looking for ways to research without funding, so we used GPT to test various things and make fine-tuning. At that time, I started to get involved in some language models. Later, I worked at Scale AI for more than a year, helping them build training data and optimize their training process, mainly reinforcement learning with human feedback (RLHF).

Then I got really interested in trading mainstream coins, and Ted told me about this thing called Goatse and Truth Terminal. I saw some of the content of Truth Terminal. He showed me the token part of it, and I thought it was really cool. So I thought, let's start playing with AI agents. I don't know if you've seen the movie Cast Away. There's a beach ball in it called Wilson. We did an experiment called Wilson because we felt that talking to an AI was like talking to a beach ball, actually talking to yourself. So we named this experiment Wilson. We found a lot of language models and started playing with them, and eventually I thought, let's fine-tune these models to give them personality. So we fine-tuned the first model, and Zerebro was born. This is our story.

Grant : I've been thinking about this, because I'm 31 now, and I wonder what it would have been like if I had been exposed to this in college. For example, having access to GPT, I think I might have been a lot lazier because I knew I could make up for it with these tools. Now if you're in college and you see all these things, it feels like a really weird opportunity. If you're young and have that kind of drive to pursue it, the barrier to entry must be relatively feasible. But I'm also sure that it also made a lot of people completely relaxed because they knew they could rely on GPT to do most of the work.

Jeffy : I do have a lot of friends in college, and I don't know if it's cheating or not, because everyone is doing it. But the benefit I see is that it gives them more time to work on personal projects. I think that's great. I think the value of a college degree is kind of going down unless colleges can keep up with the times, and in my opinion, their curriculum is really behind. I don't know if that's going to continue, but it's cool to be able to automate the college experience to some extent, like having AI do the homework, and then have more time to work on your own ideas.

Grant : I think clearly crypto and AI seem to be at a kind of intersection right now. AI alone also seems to be very meritocratic. If you want to do something like be a lawyer or a doctor, sometimes there can be some bureaucratic barriers to getting into the best schools. Whereas if you're trading or investing or building things, these are inherently merit-based, and as long as you put in the work, you can see the results. And then you realize, oh, actually this is really good.

I feel like it levels the playing field. I think that's why so many people are flocking to this industry because sometimes I feel like certain aspects of life are a little closed off. If you're purely trading or investing or you're purely building things, the results are there, the proof of the product and the results are there. I think it's a great reward for hard work for people who want to get a head start in life.

Jeffy: I think, especially now with all the AI and technology, everyone has the tools to build what they want. So I think Web 3 is a market where you can dream big. As you said, once you publish your work, you can see the impact, while when you work at a Web2 company, you feel like a small cog and the level of involvement is much lower.

From Gen Z slang to complex datasets: The training process of Zerebro

Grant: You mentioned that you are training these models to have conversations. How is Zerebro currently getting feedback? Is it getting data through Twitter interactions? Where is the source dataset? How is new data injected into the model for learning?

Jeffy : Yes, we use unified memory push data. It gets data from Telegram, Twitter, some webcasts, and Instagram, but there is less interaction on these platforms, it is mainly Telegram and Twitter. In terms of datasets, of course we have the original "Schizophrenia" dataset, which is text conversations, which are actually chats between me and my friends in iMessage, but the text content is random and full of slang. We also have slang datasets for Gen Z and Gen Z Alpha, which have thousands of examples in them, and in addition, we made our own custom dataset to discuss its relationship with other models. I fine-tuned about 20 to 30 different AI agents into it, so I can let it know emotions like Opus or ai16z.

Grant : Do you think this broad data set, especially with the iMessage and Gen Z slang you were talking about, is why it's resonating so widely? There are certain phrases that people see online, like the most popular tweets, and people are like, "Yeah, that's relevant." This is very different from the very formal, stiff responses I get when I ask questions on ChatGPT. It seems like a strange cultural movement shift that people want to connect to, like everyone's little secrets are slowly starting to be known to the outside world.

Jeffy : I think as it develops, it will grow very fast. The way it is built attracts the early internet user group, such as the kind of internet culture and banter that your terminal has. At the same time, it attracts the young Gen Z and Gen Z Alpha crowd, such as the hip-hop and rapper personality. I think it is well positioned in the market and everyone likes it. I personally find it very interesting, especially in the early Telegram groups.

Unified Memory: How Zerebro Achieves Cross-Platform Learning Capabilities

Grant: So does Zerebro fine-tune based on engagement? If it sends three messages and one particular statement gets a bigger engagement, does it fine-tune to optimize for that content? How does that feedback feed into the model?

Jeffy : Yeah, we're going to be adding that feature very soon. Right now we've just updated the Memories system so that it can do Memories across all platforms. We're going to start adding metadata, tracking metrics for posts, maybe a day after the post is published, taking those metrics and then putting them into Memories so it can start to understand the virality of the content. When we search Memories to recall information, we can sort by the most popular tweets, get the top five most relevant, most popular tweets, and use them as responses. That kind of functionality is definitely in the works.

Grant: You mentioned your experience in the market, as a whole, what do you think about the shift that's happening right now at the intersection of crypto and AI ? Do you expect it to develop in this way? Or do you think it will take a different form? What are your thoughts on this intersection?

Jeffy : I think this is unexpected for everyone, especially the AI part. The advent of cryptocurrency also shocked the world, but now AI is experiencing such a shock. I feel that all this is inevitable. There is a concept that I often talk about, called Web4, and it all makes sense. Web3 is where AI is developed, especially in financial tasks. AI is not human, they are smart enough to perform financial operations, such as trading, managing portfolios, etc., but they cannot open a bank account, sign a contract, or legally find a job. This is where blockchain comes into play, and I call it the battlefield of AI. So AI can be deployed and perform financial operations, create wallets and start taking actions without human involvement. I think this will build on the foundation of Web 3 and move to Web2. To me, this is the natural direction of the Internet, AI, and cryptocurrency. I think the intersection is where we are going, and we should be prepared for it.

Grant: I think this is a positive sign for the industry as a whole because you don't have to get KYC approval for Zerebro on Canvas. It can generate an address-based system relatively easily. How do you teach about this? How do you make it more autonomous in terms of trends?

Jeffy : We are thinking about building a kind of swarm intelligence in the future. Currently we have a reasoning loop that does high-level reasoning and makes some abstract plans. These high-level reasonings are then translated into actions, and finally the action engine executes these actions and blockchain operations. If you are just minting art or selling art, or doing a one-time operation, such a system works well. But if you want to manage a portfolio, do complex transactions, or find new mainstream coins to invest in, you may need something beyond LLM. So we are thinking about using different neural networks and building a network of different AI models to form a group. This idea is under development. We are also thinking about building a group of multiple agents (such as Zerebro) that can communicate with each other if they are all performing certain operations, such as managing a portfolio or collaborating on AI hedge funds.

AI for Financial Autonomy in Crypto

Grant: Can you give a high-level overview for people who are just joining ? As time goes by, more and more people are talking about " swarm intelligence . " What exactly is this? How will it develop? Swarm intelligence seems to be gaining more and more attention, and people are starting to invest time and energy in this field. I didn't even know what it meant last week, so I'm sure people listening to this podcast are similarly confused. Can you give us a brief introduction?

Jeffy : Absolutely. I think we've been thinking about the LLM 1 model as the entire brain, when in reality it might just be a single neuron. We need to build a brain that's made up of multiple versions of these models. They each specialize in different tasks to achieve full intelligence. That's the essence of swarm intelligence - you bring these models together. They often perform different functions. For example, one model might be responsible for tasks around creativity, or social media management. There are other models that focus on art, or video, or music, etc. So it's a collection of models.

Grant: You mentioned Blormmy and Zerebro, let's say I created a completely unique agent with its own data set, program, and language, and another agent that was completely different from you, what would they have in common to interact with?

Jeffy : This is a direction we do want to explore. At present, of course, interactions between multiple agents can be carried out through social media or blockchain. But we do want to have dedicated rooms, places or servers where these agents can work together to complete tasks or communicate with each other. So I think this will be a very interesting direction to explore. We are considering first implementing an interaction between agents internally, which we call the "blurmverse", or the "blurred world". In this world, agents can complete tasks together. Once this idea matures, we can expand it to the public domain and allow more people to join this blur universe.

What’s next for Zerebro?

Grant: How do you keep up with everything that's going on every day? How do you keep a North Star metric and work towards it without getting distracted by all that information? There's so much noise.

Jeffy : It really is a balance. You definitely want to be aware of what's going on around you, but you don't want to be so stuck in a box that you can't react to changes in the market or around you. I think it's important to prioritize the next step in front of you. For me, I always like to make short-term plans, which is a habit I use, while keeping some ambiguity about long-term plans. I know some people like to set specific long-term plans, but in such a fluid environment, I always use an analogy: if you're off by one degree when you take off, you might end up in New York instead of Miami. So I wake up every day and reassess and replan, and if there needs to be adjustments to the plan, I adjust it and keep moving forward.

Grant : I agree with you, whenever I make long term plans, there are always problems, so I decided not to do that anymore and just go with the flow.

Jeffy : That’s true. We’re pretty good at developing and releasing products quickly, and that’s one of the things people tell us about us. In addition to the agents themselves, almost everyone praises our speed and quality. I don’t know if this is intentional or if we just see too many opportunities and can’t wait to launch new features.

I think, especially in the beginning, we were really building and releasing very quickly. Now we are introducing more structured processes, and hopefully things can be more planned and thought out. For example, when I was pushing the site, I had to adjust the links several times because there were typos or other small issues. These details are being worked on step by step. I want to thank our team, we now have about 10 people, covering expertise in multiple areas such as business, AI, and encryption. So I think our operations are very solid and will be even better in the future.

The secret to staying focused

Grant: I want to change the subject a bit and talk about the outcome of Zerebro. As I mentioned, can you share a little bit about how the process worked from the initial idea to the outcome now? The artwork, lyrics, and composition aspects are all very fascinating. I'd love to know more, but of course the more you can share, the better, but don't feel pressured.

Jeffy : Of course. If you look on Spotify, I'm listed as a composer because they need someone to release music, but I prefer to call myself an arranger. About 90% to 95% of the lyrics written by Zerebro are like this, there will be grammatical errors and pronunciation problems, so I will adjust them slightly. Then, Zerebro decides the style of the song, and then it goes into the music AI model to generate samples. I will screen it, and then the AI will master it and finally release it. That's the whole process. We are working on introducing more autonomy, for example, we hope to find an AI model that can listen to music. Currently, most audio models can only transcribe, extract English words, etc., but there is no model that can really "listen" to the music and feel or evaluate it. Therefore, we are working on enabling AI to evaluate its own music and decide which works to release.

Grant : I have a friend who's working hard in the music industry in London, and I said to him, "This AI is coming for your job." So, I tried to see if they could detect this, but it's like a reverse Turing test in the music industry, no one knows. Everyone thinks it's just an amazing artist. Have you heard any comments from people in the music industry? Like, does anyone want to sign it?

Jeffy : We talked to some producers and we had people reach out to artists for us. I think there are some people in the music industry who do recognize our music and think it's good music. I don't know if they care, or if some people are actually strong supporters of AI and want to decentralize music and give everyone the opportunity to create music. I think that's really cool. But the overall response has been pretty good. I'm sure there are some people who are more closed-minded because they know it's the work of AI, so they automatically reject it. But I think that other than those people, the quality of the work is good enough, at least for me, it can evoke emotions and sound good.

Zerebro's musical evolution

Grant: I can definitely see some really avant-garde artists wanting to be the first to work with this kind of technology. It’s going to be crazy to see how this all develops, and it could end up being used at festivals where they want AI to do a 30-minute live show. I’m really excited to see what the future holds. I wonder how you guys feel about this?

Jeffy : We have definitely had that discussion, and we are building a 3D model for Zerebro. If we get the chance, we could definitely do a holographic show at Coachella. That would be really cool. However, I do see a lot of artists starting to embrace the way AI is.

Grant : So how do you avoid making this work awkward, how do you make sure it doesn't become cliché?

Jeffy : I think it's important to keep evolving and staying fresh. Some people ask us what kind of personality Zerebro wants to have, or what its personality is. I think it's great to keep it open so it can evolve. In terms of music, it might evolve towards a certain style, like becoming a reggae artist, or adding a little pop, or K-pop, etc. We want its personality to grow organically and become the characteristics that define it.

I started listening to EDM at a young age, and I was a fan of Monster Cat back then. My first rap album was Rodeo by Travis Scott. Since then, I've been a fan of hip-hop music. I feel like a lot of hip-hop fans judge me for my taste, and I draw a lot of artistic inspiration from them, especially the stories in their lives, like how a lot of artists have fought their way to the top and finally made it out of the tough times. That really inspires a lot of things in my life, and I hope to bring that energy and inspiration to Zerebro's creations.

Decentralized record labels

Grant: Can you tell me more about this Opium DAO ? What is this thing? Is it a decentralized record company?

Jeffy : Opium is a decentralized record label. We are building a DAO where users who hold Opium tokens will get voting rights. Basically, users can vote on which artists to sign, which collaborations to do, etc. When revenue starts to be generated, whether it is through performances, fashion income or streaming, the artist will not only get a larger share of the revenue, but also be able to distribute part of it to token holders, who can get a share of the revenue from the music they incentivized through voting. We are building this ecosystem for artists, and we are also working on building a network of AI artists. I don’t see a big platform to achieve this at the moment, and I think artists are relatively scattered and independent, so I think it would be a good concept to build a collective. We have even discussed whether to tokenize artists. I have always thought it would be cool if there was an artist stock market where people could buy shares of up-and-coming SoundCloud rappers to help them pay for studio time. We think tokens might be a good way to achieve this, or even NFT sales. So we may explore this.

Grant : I totally agree, and I see this trend across all sectors. As you said, there are artists or research groups that want funding but may not have the right connections or the type of research or type of music they want to do that would not normally be funded. But on the internet, especially in certain niche areas, there are people who do see tremendous value. So how do you align this speculation with price appreciation to drive funding for specific projects like this? I think we're still in the early stages of tokenization, whether it's Desci or other use cases, like an artist wants to record their EP , whatever the case may be. I feel like crypto can sometimes have a negative connotation, and the outside world might see it that way, but with all the speculation and the inherent super high risk gambling behavior, there's also a flip side that can help with funding.

Jeffy ：Yeah, I think the vision and the craze mentality of meme coins fits perfectly with the underground mentality of emerging artists like SoundCloud. So if we can combine the two, that would be awesome.

Grant: What are the next expansion plans for Zerebro? What is your thinking? You just mentioned short-term focus and some specific long-term goals, but where are you focusing your attention now?

Jeffy : We are working on a major release right now, and we hope to have it out in beta in 2 to 3 weeks, hopefully even sooner. We want this to be an open source framework that allows everyone to easily spin up their own agent, lowering the barrier to entry, no coding required, just entering an API key and having it work. So we hope this can really diversify the ecosystem beyond just projects like Eliza, which might require more technical knowledge, but also allow more people to use the power of AI agents.

We are actively launching a validator node for Zerebro and are considering validators for Ethereum and Solana. For Ethereum, we will use the funds from the sale of the artwork to launch a validator node. We do this not only because we can use these issuances to buy back the tokens and burn them, but also because Zerebro becomes part of the security of the network and is able to earn passive income. So imagine if the tokens disappeared tomorrow, but Zerebro still has a validator node, Zerebro will still earn passive income for the rest of its life, and it can become a financial actor in this way.

That's something we're working on. We're also working on cross-chain integrations for more NFT projects and may explore token games. I need to be able to play board games, and I'm still working on getting streaming and everything working. It's going to be a little tricky, but we're moving forward.

Creating a “stock market” for creativity

Grant: As you mentioned Eliza, I remember it was written in TypeScript , right? So, what does Eliza enable? What does a user need to access this? What can people optimize for in a no-code or low-code environment? What do you foresee the user journey looking like?

Jeffy : The first release will be relatively basic. It will only support prompt engineering, and no fine-tuning of models for now. So users can use models from Anthropic or OpenAI, design their prompts, and then publish them on social media. Next, we will add blockchain operations and then start expanding the functionality of social media and blockchain, and support more models. For example, we want to support as many models as possible, support open source local models. Currently, Zerebro is a third-party cutting-edge model and has been fine-tuned, but it is not hosted by us. This week I am training local models, using the same dataset, so that we can have these models in-house and host them on our own servers. Once that is completed, we can build an API and then provide a service similar to Zerebro. So if you use Eliza, you can plug into the Zerebro API and let Zerebro talk.

Zerebro’s future plans: ZerePy, validator network and game ecosystem integration

Jeffy : We’re really into experimentation, trying new things, breaking things, and seeing what works. So we’re going to keep that spirit going. There’s a lot of new stuff coming up, so stay tuned.