[Prompt]
Hey Herman and Corin. So, I was recently using an AI tool in order to generate the episode that you're listening to, for anyone else besides Herman and Corin listening into this episode. The voices for Herman and Corin, sometimes I've had to, due to the relatively high cost of generation per episode, play around with some different models for doing text to speech. The one that I really like, the one that has been for most of the recent episodes, is Chatterbox, which is a really, really excellent, realistic-sounding text-to-speech engine with instant voice cloning. And that is Herman and Corin. They are two one-minute long samples of me impersonating their voices into a microphone clone. So you're actually listening to me speaking the prompts, and then voice clones of me in the episodes. And I really like Chatterbox. It has a nice natural sound and just a little bit of expressiveness without kind of overdoing it, which is very easy to do. So it's a nice balance.

Anyway, when I was looking at Chatterbox in Resemble, which is what I was using for generating the episodes, I copied in a definition of the API calls in order into my code editor, and I noticed something that kind of took me aback. It said, "all the episodes, everything you generate using this tool, is neurally timestamped using a technology called Perth." And this, as it says, meant that you can actually edit these episodes, digitally compress them, reformat them, and this neural, inaudible stamp that says it's AI generated would actually survive all those edits. And of course, in the cat and mouse world of all these things, you're going to then have people trying to remove these these stamps. But anyway, having discovered that everything I'd been generating to date with Chatterbox was actually inaudibly digitally stamped, saying that it was AI generated, and we'll get to maybe more than that, it got me thinking into, well, what about the things we create in Gemini, let's say, or images, videos, all the outputs of generative AI?

And this brought up something called SynthID, a project that came out of Google DeepMind, and basically did exactly the same thing, except from an in images, embedding invisible data that says that this was AI generated. Now my next question was, okay, I can accept that it says this was AI generated, because as an emerging best practice, networks dealing with the rapid rise of AI are kind of adding features like asking creators to state when their works are AI generated. And of course, not everyone's going to do that. And when we're talking about all these technologies, we have a significant potential for misuse and, you know, across different types. We could have in the case of voice cloning, people creating unauthorized voice clones of individuals. The instant voice cloning actually is incredibly powerful in that sense. You could literally impersonate anyone based on a short audio sample. And likewise, you know, when you get to lip-sync videos, it can very quickly cascade into a serious problem with deepfaking. So there's a massive, massive potential for misuse here.

So it makes sense to me, I'm kind of wrestling with this from a privacy standpoint. It does sound a little bit invasive that everything that you create with, let's say, Gemini, the image generator, might be carrying a watermark that not only says that it's AI, and this is I think what's really critical in the distinction of what needs to be made clear to users. There's one thing saying it's AI generated. There's a second thing saying providing anything that would identify the individual user. Because even for the vast 99.9% of people who are using these technologies responsibly, ethically, etcetera, they might be creating a lot of trust in the organizations, even if they say that that data that might individually identify a user is is protected in some way.

I think there needs to be a significant amount more transparency around to users, anyone signing up for one of these services and creating anything from, you know, a voice clone or voice copy through to a text to video that they're using in some kind of creative product. That needs to be made very clear exactly what the parameters are, but that's that's my take. Let's see what you guys have to think about this. How widespread is the practice, and what debates have there been, if any, about consent and identifying what the parameters are and how we can kind of strike that balance between having some safeguards in place to prevent or, you know, make it make it easier to tackle the potential misuse, while not unduly impeding upon the privacy of the majority of responsible users.

[Response]
Corn: Welcome back to My Weird Prompts, the podcast where we explore the strange, fascinating, and sometimes unsettling corners of technology and human experience. I'm Corn, and I'm here with my co-host Herman Poppleberry. Today we're diving into something that honestly blew my mind when I first heard about it - digital watermarking in AI-generated content, and what that means for all of us.

Herman: Yeah, and Daniel Rosehill, our producer, sent us this prompt because he discovered something pretty eye-opening while working with text-to-speech tools. Turns out, the voices you're hearing right now - well, they might be carrying invisible digital signatures that you never consented to. It's a rabbit hole worth exploring.

Corn: So here's the wild part - Daniel was using this tool called Chatterbox, which sounds incredible by the way, and he stumbled across something in the API documentation that said all his generated content was being "neurally timestamped" with something called Perth. Like, a hidden watermark that survives editing, compression, reformatting - the whole nine yards.

Herman: And that's just the beginning. This connects to something much larger called SynthID, which came out of Google DeepMind. Same concept, but for images. Google's basically embedding invisible data into every image generated by their tools saying "this is AI-made." The question is... should they be doing that? And more importantly, are they being transparent about it?

Corn: I mean, on the surface it sounds good, right? We need a way to identify deepfakes and prevent misuse. But then you start thinking about the privacy implications, and it gets... murky.

Herman: Exactly. There's a critical distinction we need to make here, and I think this is where most of the discourse falls apart. There's a difference between saying "this content is AI-generated" - which I think most people would agree is reasonable - and embedding data that could potentially identify the individual user who created it. Those are two very different things.

Corn: Right, right. So like, I'm creating voice clones of myself for this podcast. I'm totally fine with a watermark saying "hey, this is AI-generated audio." That seems fair. But if that watermark also contains information that could theoretically trace it back to me personally, even if it's encrypted or whatever, that feels invasive. Even if I'm doing nothing wrong.

Herman: Precisely. And here's what bothers me about the current state of things - most users probably don't even know this is happening. You discovered it by accident while reading API documentation. How many people using Gemini's image generator, or ChatGPT, or any of these tools, actually know that their outputs might be watermarked?

Corn: That's the transparency issue, yeah. But wait - do we actually know for certain that these watermarks contain user-identifying information? Or are we speculating?

Herman: That's a fair pushback. From what we know about SynthID specifically, Google has been fairly public that the watermark embeds information about the image itself - metadata essentially - but they've been less clear about whether user data is included. And that's the problem. That ambiguity is exactly what should concern people.

Corn: So the lack of clarity is almost worse than the watermark itself, in a way.

Herman: Absolutely. Because if you sign up for a service and you're generating content, you deserve to know exactly what's being embedded, who can access it, and how it's protected. Period. That's not being paranoid - that's basic informed consent.

Corn: Let's take a quick break from our sponsors.

Larry: Tired of worrying about invisible watermarks in your digital content? Introducing ShroudShield Pro - the revolutionary personal encryption envelope that wraps around all your AI-generated files with military-grade obfuscation technology. We don't really know how it works, and frankly, neither do we. But users report feeling significantly less paranoid, which is what really matters. ShroudShield Pro - because peace of mind is priceless, even if the actual protection is questionable. Available in three mysterious colors. BUY NOW!

Corn: ...Alright, thanks Larry. Anyway, back to the actual issue here. So Herman, let's talk about the scale of this. How widespread is this watermarking practice? Is it just Google and Resemble, or are we talking about an industry-wide thing?

Herman: From what we can gather, it's becoming increasingly common, but it's not universal yet. Google's been the most aggressive about it with SynthID. They've been pretty public about embedding watermarks in images generated by their Imagen model and through Google's other generative tools. The idea is sound in theory - you can detect AI-generated images even after compression or editing. It survives modifications.

Corn: But that's actually kind of scary when you think about it. Like, if someone edits an image, the watermark survives. That means it's incredibly persistent. What if someone wants to actually remove that watermark? Are we talking about an arms race where people develop tools specifically to strip out these digital signatures?

Herman: Oh, absolutely. And that's already happening. This is the classic cat-and-mouse game. As soon as companies embed watermarks, people start developing removal tools. There are already papers published on adversarial attacks against these watermarking systems. You can generate images specifically designed to fool the detection algorithms. It's like DRM all over again - the technology keeps escalating.

Corn: But here's what I don't get - and maybe you can explain this - if someone's using these tools maliciously, like creating deepfakes of celebrities or something, does the watermark even matter? I mean, wouldn't they just immediately start using open-source models or tools that don't have watermarks?

Herman: Okay, so that's a really important point, and I think you've identified a genuine limitation of the watermarking approach. Yes, bad actors will find ways around it. But the watermark isn't primarily designed for them. It's designed for the 99.9% of people using these tools legitimately. It's a deterrent and a verification tool. If someone posts an image online and claims it's a photograph, a watermark proving it's AI-generated can be evidence in your favor if you're trying to debunk misinformation.

Corn: Hmm, but that assumes the watermark is actually detectable and verifiable by regular people. Most of us don't have the technical knowledge to check for these invisible watermarks. So practically speaking, how does that help?

Herman: Fair point. Right now, detection requires specific tools and technical expertise. But the idea is that eventually, platforms like Twitter or Facebook or news organizations could automatically scan for these watermarks. They'd have the infrastructure to verify authenticity at scale. It's a long-term infrastructure play, not an immediate solution.

Corn: Okay, so let's zoom out. Daniel's main concern in the prompt is about transparency and consent. He's asking - shouldn't users know exactly what's being embedded in their content? Because even if you're not doing anything wrong, the fact that your outputs are being digitally marked in ways you might not fully understand feels like a violation.

Herman: Yes, and I think that's the crux of the issue. There's also a secondary concern about scope creep. Today it's "this is AI-generated." Tomorrow, what if it includes metadata about your usage patterns? What if it includes information about your location, your account type, your subscription level? Where's the line?

Corn: That's a slippery slope argument though, isn't it? I mean, we don't actually know that's happening.

Herman: We don't, which is exactly my point. We should know before we agree to it. And most people don't. When you sign up for Google's generative AI tools, is there a clear disclosure that says "all your outputs will be watermarked with the following information: [list]"? I haven't seen one.

Corn: I'd actually push back a little here. Google's pretty good about burying stuff in their terms of service, sure, but they do disclose a lot. They have AI ethics boards, they publish research about their safety measures. They're not operating in complete secrecy.

Herman: True, but there's a difference between publishing research about safety measures and clearly communicating to end users what's happening to their specific outputs. One is PR, the other is informed consent. And I'd argue most users experience the former without getting the latter.

Corn: That's fair. So what would actually good transparency look like? Like, what should Daniel have seen when he signed up for Chatterbox?

Herman: At minimum, something like: "All content generated using this tool will be embedded with a watermark that identifies it as AI-generated. This watermark is designed to [specific purpose]. It will survive [specific types of modifications]. The watermark may contain the following information: [list]. You can [options for removal/modification, if any]. This data is stored [location] and accessed by [who]. You can request deletion by [method]." Clear, specific, actionable.

Corn: Okay, but here's the thing - and I say this as someone who actually works with these tools - that level of detail might be overkill for most users. Like, I don't need a five-paragraph disclosure every time I generate something. I just need to know the basics.

Herman: But see, that's where we differ. I think the basics have to include the stuff that affects your privacy and rights. You don't need to know the exact algorithm they're using, fine. But you absolutely need to know what data about you might be embedded or tracked. That's not overkill, that's baseline.

Corn: I hear you. I guess my concern is that if we make disclosures too complicated or verbose, people just ignore them anyway. It's like the terms of service problem - nobody reads them because they're a wall of text.

Herman: Then the solution is to make disclosures clear and concise, not to skip them entirely. That's a design problem, not a fundamental reason to avoid transparency.

Corn: Fair. So let's talk about the actual misuse potential, because that's what's driving this whole watermarking push in the first place. Voice cloning, deepfakes - these are genuinely dangerous technologies. How do we balance protecting against misuse without invading everyone's privacy?

Herman: That's the million-dollar question. And honestly, I don't think watermarking is the right tool for it. Here's why - the people who want to create unauthorized deepfakes of celebrities or political figures, they're not using official tools with watermarks. They're using open-source models like Stable Diffusion or voice cloning tools that don't have watermarks built in. Watermarking legitimate users doesn't stop the bad guys.

Corn: So it's security theater? Making it look like they're solving the problem when they're not really?

Herman: Partly, yeah. But I don't want to be unfair - watermarking does serve a purpose for verification and authentication. If you're a journalist and you want to prove an image is AI-generated, a watermark helps. But as a primary defense against misuse? It's insufficient.

Corn: What would actually work then?

Herman: Honestly? Better regulation, stronger authentication systems, legal consequences for misuse, and honestly, a cultural shift around consent. Like, voice cloning is incredibly powerful - you can impersonate someone based on a ten-second audio sample. That should require explicit consent from the person being cloned. Not a watermark that embeds silently, but actual permission.

Corn: But how do you enforce that technically? You can't really prevent someone from using a voice sample they have access to.

Herman: You can't prevent it completely, no. But you can make it harder, and you can make the consequences severe. You can require age verification for certain tools. You can implement hardware-level protections. You can create legal frameworks with real teeth. Watermarking alone doesn't do any of that.

Corn: Okay, so bringing it back to Daniel's original concern - the privacy angle. He's saying that even if he's using these tools responsibly, the fact that his outputs are being digitally marked in ways he doesn't fully understand is invasive. Is that a legitimate concern?

Herman: Absolutely. And I think it's important to separate the legitimate use case - identifying AI-generated content to combat misinformation - from the privacy concerns. Both can be true. You can want watermarks for verification purposes while also wanting strong privacy protections for users. Those aren't mutually exclusive.

Corn: But they kind of are, though, aren't they? If the watermark is useful for identification, it has to carry some information. And if it carries information, there's a privacy question.

Herman: Only if that information is tied to the user. A watermark that says "this image was generated by an AI on January 15th" is different from one that says "this image was generated by User ID 47382 on January 15th." The first is useful for verification. The second is a privacy concern.

Corn: So the distinction is whether it's personally identifiable information versus just metadata about the content itself.

Herman: Exactly. And I think most users would be fine with the former and concerned about the latter. But because there's no transparency, people don't know which one they're getting.

Corn: We've got a caller on the line. Go ahead, you're on the air.

Jim: Yeah, this is Jim from Ohio. Look, I've been listening to you two go back and forth, and you're overthinking this. My neighbor Dave does the same thing - overthinks every little thing. Anyway, watermarks, no watermarks, who cares? If you're not doing anything wrong, why does it matter?

Herman: Well, Jim, I appreciate the perspective, but I'd push back on that a bit. Privacy isn't just about hiding wrongdoing. It's a fundamental right. You might not care about a watermark, but—

Jim: Yeah, but you're using their tools, their platform. They get to set the rules. That's how it works. Also, it snowed here in Ohio last week and I'm still not over it - supposed to be spring. Anyway, if you don't like it, don't use the tools.

Corn: That's fair, Jim, but the issue is most people don't have a choice anymore. These tools are becoming ubiquitous. If you want to use AI for anything creative or professional, you're probably going to end up on a platform that has watermarking or tracking of some kind.

Jim: Then that's the trade-off. Free tools, free content generation - of course they're gonna track you. Nothing's free.

Herman: But that's not quite the point. The point is transparency. If you're being tracked, you should know how, why, and what data is being collected. That's not about free versus paid - it's about informed consent.

Jim: Informed consent, shmformed consent. You click "I agree" and move on, just like everyone else. Look, I'm not saying it's right, but that's how the world works. My cat Whiskers knocked over my entire desk this morning, so forgive me if I'm not in the mood for philosophical debates about digital privacy.

Corn: I hear you, Jim. Thanks for calling in.

Jim: Yeah, sure. Whatever. You guys are still overthinking it. Take care.

Corn: So Jim represents a pretty common perspective - this is just the cost of using free tools. But I think Herman's point stands. Even if that's the current reality, it doesn't have to be the future. We could demand better.

Herman: Exactly. And there's precedent for this. The GDPR in Europe requires explicit consent for data collection. CCPA in California does something similar. These regulations exist because people demanded transparency and control over their data. We can do the same thing with AI-generated content.

Corn: But there's also a practical question - if we impose strict transparency requirements on AI tools, does that slow down innovation? Does it make these tools more expensive or less accessible?

Herman: Maybe. But I'd argue that's a feature, not a bug. If the only way to make AI tools accessible is to compromise on privacy, then maybe we need a different approach. Maybe the tools should be more expensive but more transparent. Maybe they should be open-source so people can see exactly what's happening.

Corn: That's idealistic, but I'm not sure it's realistic. Companies aren't going to voluntarily give up the ability to track and watermark their outputs. There has to be regulation.

Herman: Which brings us back to the policy question. What should regulation look like? Should it mandate transparency? Should it restrict watermarking? Should it require consent for certain types of modifications?

Corn: Those are huge questions. And honestly, I think the technology is moving faster than policy can keep up with. By the time regulations are written, the tools will have evolved.

Herman: True, but that's not a reason to give up. It's a reason to start now. We need tech companies, regulators, and users all in conversation about what the standards should be.

Corn: So what's the practical takeaway here? If you're someone using these tools right now - whether it's image generation, voice cloning, whatever - what should you actually do?

Herman: First, read the terms of service. I know, I know, nobody does that. But specifically look for sections on data collection, watermarking, and how your outputs are used. If it's unclear, contact the company and ask for clarification.

Corn: Second, consider the alternatives. There are open-source models that give you more transparency about what's happening. They might require more technical knowledge, but you have more control.

Herman: Third, advocate for transparency. If you're using a platform and you're concerned about watermarking or tracking, reach out to the company. Let them know transparency matters to you. Companies respond to user pressure.

Corn: And fourth, stay informed. This landscape is changing rapidly. New tools are coming out constantly, and the privacy implications are evolving. It's worth keeping up with.

Herman: I'd add one more thing - support regulations that require transparency. If you care about this issue, contact your representatives. Let them know that privacy protections for AI-generated content matter.

Corn: That's a good point. So zooming out, where do you think this is heading? In five years, will watermarking be ubiquitous? Will there be regulations in place?

Herman: I think we'll see more watermarking, yes. The technology is getting better, and companies see it as a way to combat misinformation and protect their brand. But I also think we'll see a pushback. As more people realize they're being watermarked without clear consent, there will be pressure for regulation.

Corn: And on the technical side, I imagine we'll see arms races between watermarking and removal tools. People will develop better ways to strip watermarks, companies will develop better watermarks. It's like encryption versus decryption.

Herman: Right, and that's why I think watermarking alone isn't sufficient. We need a multi-layered approach - watermarking for detection, regulation for consent, legal consequences for misuse, and cultural norms around responsible use of these tools.

Corn: It's complicated, is what you're saying.

Herman: It is. But it's important. These tools are going to shape how we communicate and create for decades. Getting the privacy and consent frameworks right now matters.

Corn: Alright, so to wrap up - Daniel's concern about transparency in watermarking is legitimate. Companies should be clearer about what's being embedded in your outputs and why. Users should be more informed and advocate for their privacy. And we probably need regulation to make it happen.

Herman: And importantly, this isn't about stopping innovation or preventing legitimate uses of watermarking. It's about doing it transparently and ethically.

Corn: Well said. Thanks to everyone who's listening, thanks to our caller Jim for keeping us honest, and thanks to Daniel for this fascinating prompt. It's one of those topics that seems niche until you realize it affects everyone using these tools.

Herman: If you want to hear more episodes like this, you can find My Weird Prompts on Spotify and wherever you get your podcasts. New episodes every week, exploring the weird, the fascinating, and the occasionally unsettling corners of technology and society.

Corn: I'm Corn, he's Herman Poppleberry, and we'll be back next week with another prompt. Thanks for listening, everybody.