• 3 Posts
  • 212 Comments
Joined 1 年前
cake
Cake day: 2023年6月16日

help-circle












  • Speaking for LLMs, given that they operate on a next-token basis, there will be some statistical likelihood of spitting out original training data that can’t be avoided. The normal counter-argument being that in theory, the odds of a particular piece of training data coming back out intact for more than a handful of words should be extremely low.

    Of course, in this case, Google’s researchers took advantage of the repeat discouragement mechanism to make that unlikelihood occur reliably, showing that there are indeed flaws to make it happen.


  • Accumulated knowledge in our society really is frail. Take a computer mouse, tons of people are involved in making them, they’re considered extremely simple tools. Yet not one person on the planet could go out into nature, get the natural resources required, and without help turn those resources into a working computer mouse.


  • I’m not an expert, but I would say that it is going to be less likely for a diffusion model to spit out training data in a completely intact way. The way that LLMs versus diffusion models work are very different.

    LLMs work by predicting the next statistically likely token, they take all of the previous text, then predict what the next token will be based on that. So, if you can trick it into a state where the next subsequent tokens are something verbatim from training data, then that’s what you get.

    Diffusion models work by taking a randomly generated latent, combining it with the CLIP interpretation of the user’s prompt, then trying to turn the randomly generated information into a new latent which the VAE will then decode into something a human can see, because the latents the model is dealing with are meaningless numbers to humans.

    In other words, there’s a lot more randomness to deal with in a diffusion model. You could probably get a specific source image back if you specially crafted a latent and a prompt, which one guy did do by basically running img2img on a specific image that was in the training set and giving it a prompt to spit the same image out again. But that required having the original image in the first place, so it’s not really a weakness in the same way this was for GPT.



  • I’m not talking strictly about ideas, I’m talking about a human having a vision, and taking action to make that vision into something. Whether something is copyrightable requires a “human element,” which is the reasoning behind why machine or animal generated content cannot be copyrighted, because they lack that.

    So the question is if someone tweaking an image, even if they’re merely selecting things, then is that a sufficient human element to say that a person had enough hand in creating it?


  • When it comes to selection, we already have a valid form of copyright which is explicitly that- compositions. If I take a bunch of royalty-free songs, and make a book of sheet music where I hand selected songs to be in that book, I can own a copyright on the composition without owning any of the featured material.

    So, if someone selects a bunch of individual elements in an image using img2img, is that now a composition?


  • I accidentally submitted early, but also, I wrote out the lyrics. It’s the most bland version of those breakup-depression kind of songs imaginable. I guess people voted it as “feel-good” out of irony.

    Sitting at my favorite cafe

    Sipping my tea it’s saturday

    Thinking about all he’s done, to everyone

    This town is full of broken dreams

    Shattered hopes, and silent screams

    Somebody please help me

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    I’ve lost it all

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    We’ve lost it all

    Alone in the streets, alone in my thoughts

    Thinking of all our favorite spots

    I thought someday things might turn around

    But I was lost and never found

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    I’ve lost it all

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    We’ve lost it all

    Faces painted with smiles

    Lies are told

    A facade of unity

    A vitality sold

    So I sit here in silence

    Just wondering how

    To rewrite the tales

    This town won’t allow

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    I’ve lost it all

    Betrayed by this town

    Let’s tear it all down

    We’re all just destined to fall

    We’ve lost it all

    I’ve lost it all

    We’ve lost it all