feat: add ambient logging (not continuous, technically)

I turned off every smart feature on my LG TV years ago, watched the network anyway, and blocked it when it kept talking. This month Gamers Nexus showed what LG TVs do in standby, LG answered that its TVs do not 'continuously' record conversations, and Jensen Huang told Dreamforce that AI safety is an engineering problem and the market already keeps companies in check. This episode does a code review on the word 'continuously,' reviews the market as a safety system (detection, latency, correction) through the Vizio settlement and the 737 MAX, and argues that good regulation is branch protection for an industry.

September 25, 202624 min

Show Notes

I turned off every smart feature on my LG TV years ago, then watched the network anyway. It was still talking. So I blocked it. This month, Gamers Nexus spent hundreds of hours showing what LG TVs do in standby, and LG answered that its TVs do not "continuously" record conversations. Continuously. So I did a code review on that sentence.

The same month, Jensen Huang told Dreamforce that AI safety is an engineering problem, that the market already keeps companies in check, and that we don't need new laws. I agree with the first half. But in engineering, the author never approves their own pull request. I give Huang a fair hearing, walk through what the investigation actually showed (including the caveat that some of the scariest audio demos used a compromised TV), and explain why an offline audio buffer that uploads on reconnect is a design decision and not an accident.

Then I review the market like any other safety system: can it detect the problem, can it signal fast enough, can it produce a fix. From automatic content recognition to the Vizio settlement and the 737 MAX, it fails all three, and for data collection the market isn't failing at all. It's working, just not for you. I close with three guardrails engineering already requires internally: an independent reviewer, logs someone outside can read, and an off switch that's actually off.

If you build software, care about AI regulation, or own a TV that knows too much about you, this one is for you.

Read full transcript

I have an old LG smart TV, and I want to be clear, it's a good TV. The picture's great. The smart part, I turned off years ago. Every setting I could find. Voice features, off. Viewing information, off. The little "help us improve your experience" toggles, off, off, off, off. I went through those menus like I was defusing something.

And then, because I am who I am, I didn't just trust the menu. I watched what was actually on the network. And the TV was still talking. Opted out of everything, and it was still sending packets. Not a lot. Not constantly. Just, you know, checking in. Like a coworker who says they're "not really on Slack this week" and then reacts to every single message with a thumbs up.

I did the only thing that has ever actually worked in my house, which is I stopped asking nicely and blocked it at the network. My TV lives in a firewall timeout now. It can show me things. It cannot tell anyone what it saw.

I thought that was me being a little extra. And then this month happened.

Your TV might be listening to you. LG would like you to know that's a misconception. According to their statement, their smart TVs "do not continuously record or transmit" users' conversations.

Continuously.

Let's do a code review on that sentence. If you told me your service doesn't continuously write to the database, I would not feel reassured. I would ask the obvious follow-up. Okay, when does it write?

Because Gamers Nexus bought LG TVs off a store shelf, spent hundreds of hours on them, and showed one keeping logs of ambient audio long after you'd assume the assistant had stopped listening. Not a vibe. Not a rumor.

And LG's response wasn't a patch. It wasn't a changelog. It was an adverb. That's not a fix. That's a PR merged with the comment "works as intended."

Here's what I want to know. If a company gets caught shipping something that listens when it shouldn't, and the market's response is a carefully worded press release, is the market actually the safety system? Because this month, on a stage in San Francisco, Jensen Huang said yes.

Welcome back to Chaotic Commits. I'm Joanne Skiles. I'm an engineer, I teach cloud-native systems, and my research is on transparent systems, which is just a fancy way of saying I spend a lot of time asking, "Okay, but how would anybody outside the company actually know what this thing does?" Which, it turns out, is the whole question this month.

Two stories landed within about ten days of each other. In one, the CEO of the most valuable chip company on the planet stood up at Dreamforce and said AI safety is an engineering problem, not a legal one, and that we don't need any new laws because the market already keeps companies in check. In the other, a hardware review channel spent roughly seventy thousand dollars and five hundred hours finding out what a TV does when you think it's off, and the company's answer hinged on one word.

I don't think these are two different stories. I think the second one is a test case for the first one. Huang made a claim about how safety works. LG just ran the experiment. And I want to be fair to both of them, because I actually agree with part of what Huang said. I just think he stopped the sentence early. So let's get into it.

Here's what he said, as close to verbatim as I can get it. On stage at Dreamforce, talking with Marc Benioff in front of something like twelve thousand people, Jensen Huang said, "Safety is paramount in a lot of ways. It's job one." And then, "However, safety is an engineering problem." And then, "The market forces are already there. We don't need any new laws. We don't need new regulations."

That put him on the other side of the stage, philosophically, from Dario Amodei and Sam Altman, who both argued for a more cautious approach at the same conference.

Now, like always, I will steelman this, because it deserves that. Safety is an engineering problem. I'm not going to pretend it isn't. Nobody ever made a bridge safe by passing a law about bridges. A bridge is safe because somebody did the load calculations, picked the right steel, and put in a margin for the day that twice as many trucks show up as anybody planned for. The law didn't do the math. An engineer did.

And there's a real version of the regulation argument that I have zero patience for either. The version where people who have never deployed anything write rules about systems they don't understand, and what we get is a checkbox. Think about cookie banners. Every website you've visited in the last several years asks you to click "accept" on something you did not read, so that a company can say you consented. That's a lot of friction, paid by every single person, and I'm not convinced it made anyone meaningfully safer. We talked about that exact shape of tax in the last episode. Bad regulation is real, it's expensive, and it lands on the honest people.

So when Huang says safety is an engineering problem, my first reaction is: yes, obviously, you're correct. My second reaction is: have you met engineering?

We ship bugs to prod constantly. Constantly. I have taken down prod. You've probably taken down prod. Some of you are taking down prod right now while listening to this. I'm sorry. Also, check your connection pool.

And the thing is, we already know that engineers left alone don't produce safe systems, because we built an entire profession around not trusting ourselves. That's what code review is. That's what CI is. Branch protection, required approvals, separation of duties, audit logs, two-person rules on production access. None of that exists because engineers are bad people. It exists because engineers are people. And people under deadlines grade their own homework generously.

There is one rule in almost every serious engineering org I've ever worked in, and it's so basic we don't even talk about it anymore. You do not approve your own PR. That's it. That's the whole rule. It doesn't matter how senior you are. It doesn't matter how sure you are. It doesn't matter if it's a one-line change. The author does not get to be the reviewer, because the author is the person with the most context and the most incentive to believe it's fine.

So when someone says safety is an engineering problem, and therefore the companies building it should be the ones who decide it's safe, what I hear is an engineer asking for admin rights to merge their own PR to main with no reviewer, and a promise that the customers will file a bug if anything goes wrong.

And I want to say the next part carefully, because I think it's fair and not a cheap shot. Nvidia sells the hardware nearly everyone is training on. More AI, faster, is very good for Nvidia. That doesn't make Huang wrong. But in any code review I've ever run, when the person arguing against a review step is also the person whose code would be reviewed, we note that. We don't throw out their argument. We just don't let them be the tiebreaker.

Safety is an engineering problem. That is true. It's just not the end of the sentence. The end of the sentence is: and in engineering, the author doesn't approve the merge.

Okay, let's look at the test case.

Earlier this month, Gamers Nexus, working with Level1Techs and a handful of independent security researchers, put out a video that's more than two hours long called "216,000,000 Spy TVs." They bought retail LG OLED TVs, including a 2025 model, the G5, and spent something like five hundred hours and seventy thousand dollars figuring out what those TVs do when no one's actively watching them.

And here's roughly what they reported. The TVs scan the home network and log the other devices on it. Phones, smartwatches, computers, nearby Wi-Fi network names. So your TV isn't just a TV. It's kind of an inventory system for your house that you didn't ask for.

The TVs run something called ACR, automatic content recognition. That's a feature that fingerprints the audio and video on screen to figure out what you're watching, and not just in LG's own apps, but on things coming in over HDMI too. So your game console, your cable box, whatever's plugged in.

And the one that got everyone's attention: in their testing, the TV captured microphone audio in standby. When they cut it off from the internet, audio kept being stored locally, and when the connection came back, it uploaded.

Now, I have to be really fair here, because it matters. Some of the scariest audio demonstrations in that investigation involved a TV that had been compromised, running researcher-controlled code. So if I want to frame it honestly, they showed a capability and a security risk. It's not accurate to say every LG TV in the world is secretly recording your family dinner at all times. I am not saying that, and you should not take that away from this either.

But LG's response, and I will read it carefully, because carefulness is the point, is that LG said its smart TVs "do not continuously record or transmit" users' conversations. They said voice data is only processed when you press the voice button on the remote, or when a wake word like "Hi LG" is detected, and only if you've turned on the far-field voice feature. They said wake-word listening happens locally on the TV, and if no wake word is detected, the audio is discarded. Not converted to text, not stored, not transmitted. And they framed the whole thing as clearing up misconceptions.

And some of that may well be completely true. Wake-word detection is a real, normal design. Your phone does it. Your smart speaker does it. The little chip listens for one specific sound pattern and throws everything else away. That's a legitimate architecture.

But here's the thing. The Verge pointed out that the word "continuously" is doing a lot of heavy lifting in that statement, and it is. So let's do the code review.

If I got a design doc that said "the service does not continuously write user data to disk," I'd leave four comments before I finished my coffee.

Comment number one: what's the duty cycle? "Not continuously" is satisfied by writing once a day. It's satisfied by writing ninety-nine percent of the time. It rules out exactly one thing, which is one hundred percent.

Comment number two: what's the retention? You told me when you don't record. You didn't tell me how long you keep what you do.

Comment number three: what's "users' conversations"? Is my kid a user? Is the person on the couch who never touched a remote a user? Is a conversation still a conversation if it's been turned into a fingerprint instead of an audio file?

And comment number four, and this is the one that actually matters to me as an engineer: explain the offline buffer.

Because if what Gamers Nexus showed holds up, audio stored while offline and uploaded when the network came back, that's not an accident. That's a store-and-forward queue. That's retry-on-reconnect logic. I've built that pattern. It's a really good pattern. It's what you build when data is important enough that you don't want to lose it just because the Wi-Fi dropped. Nobody accidentally writes a durable local buffer with a flush on reconnect. Somebody designed that. Somebody code reviewed it. Somebody wrote a test for it.

It's like a restaurant putting out a statement that says, "We do not continuously serve raw chicken." Great. I'm so glad. I have a lot of follow-up questions, and, um, some of those are an entire health inspection.

That's the part that gets me. The statement isn't a lie, necessarily. It's a spec written to be technically true. And we all know that kind of spec, because we've all been in the meeting where somebody asks, "Can we say it's encrypted?" And the honest answer is, "In transit, sometimes, yes." And the marketing copy comes back as "encrypted."

So let's take Huang's claim seriously, as an engineering claim. "The market forces are already there." Fine. Let's treat the market like a safety system we're being asked to put into production, and review it the way I'd review any safety system.

A safety system has to do three things. It has to detect the problem, it has to signal fast enough to matter, and it has to produce a correction. Smoke gets detected, alarm goes off, you get out of the house. If any one of those three fails, it isn't a safety system. It's really just decoration.

So, detection. The market can only punish what buyers can see. And look at what it took to see this: a small team of specialists, hundreds of hours, tens of thousands of dollars in equipment and TVs. That's not a signal a normal consumer can generate. Your aunt is not running packet captures in her living room. I did, but I'm also a professor with a PhD who got weirdly into it, and even I only saw that it was talking. I couldn't tell you what it said. A safety system whose detection layer requires a YouTube channel with a lab budget is not a detection layer. It's a lottery.

So let's now talk about latency. Say the signal does get out. How fast does the market respond? We actually have data on this, because we've done this before. Vizio, another TV maker, settled with the FTC and the state of New Jersey back in 2017 for $2.2 million, because its TVs had been collecting what people watched, second by second, on about eleven million sets, starting around 2014, without meaningful consent. Years of collection, then a settlement, which was really a slap on the wrist. That's the loop.

And this is the part where I do have to give a point back to Huang. What corrected Vizio wasn't the market. It was existing law. The FTC already had the authority. So in fairness, "we have existing laws that cover this" is not a crazy claim. But you do need to notice what it took. Someone had to find it first, then years went by, then the correction was a number that, for a company selling millions of TVs, is closer to a rounding error than a lesson. The existing law was the reviewer. It just showed up years after the merge and left one comment.

So now let's get into correction. This is where the market as a safety system really falls apart for me. When a safety system fires, the output should be a fix. What did the market produce this time? A statement with an adverb in it. The TVs on people's walls this morning are, as far as I know, the same TVs they were before the video.

And then we get to the part that nobody really wants to say out loud, but we do need to say it. The market isn't failing here. The market is working, just not for you.

Smart TVs are cheap, in part, because the data is worth something. Viewing data, ad targeting, content recognition: that's a revenue line. The feature isn't a bug the market will eventually punish. The feature is something the market is actively paying for. So asking the market to be the safety system for this particular behavior is like asking a vending machine to report that it's been eating your quarters. The machine is not confused about what it's doing. The machine is doing its job. You're just not who it works for.

And honestly, the stakes don't stay at "my TV knows I've rewatched the same baking show twenty times." Market pressure is also what people point to in one of the most studied engineering failures of the last decade. When Boeing built the 737 MAX, the competitive pressure from Airbus was enormous. A lot of the certification work had been delegated to Boeing's own people. The company was, in a real sense, reviewing its own PR, and 346 people died in two crashes. Market forces were absolutely there. They were the gas pedal. They were not the brake.

And that's the problem with "the market will handle it." The market is a fantastic accelerator. It's a terrible reviewer. It doesn't read the diff. It reads the earnings call.

So what do I actually want, then? Because "regulate everything" isn't the answer either. I just spent five minutes complaining about cookie banners. I want us to take Huang at his word. Let's treat safety like an engineering problem, and then let's bring in the parts of engineering that we already know work, the ones we would never ship without internally.

One: the author doesn't approve the merge. Independent review. Not a committee of people who've never seen a stack trace, but someone outside the company who can actually check. We already do this in other places: financial audits, building inspections, electrical certification on the charger you plugged in last night. Nobody thinks that stuff is anti-innovation. It's just the reviewer.

Two: logs that someone outside can read. That's my research area, so I'll be quick, and you can yell at me later. The whole question with the TV is: how would anyone know what it does without spending seventy thousand dollars? What if the answer was just, "It tells you"? A plain-language, machine-readable disclosure of what the device sends, where, and when. Like a nutrition label, but for packets. "This device transmits device inventory every fifteen minutes, content fingerprints while on, audio only after wake word, retained for this long." If your privacy statement needs the word "continuously," your system needs a spec. Publish the spec.

Three: a kill switch that's actually a kill switch. When I turned off every smart feature on my TV, that should have meant the TV stopped talking. It didn't. The opt-out was a UI element, not a system state. In engineering terms, the toggle was writing to a preference nobody read. That's a bug. If a user turns something off, off should be verifiable. It should be enforced in the architecture, not just displayed in a menu.

And that's where law, good law, actually fits. Not as the thing that does the engineering. As the thing that makes the engineering non-optional. Think of it as branch protection for an industry. Branch protection doesn't write your code. It doesn't slow down good code very much. It just makes sure that nobody, no matter how senior, no matter how confident, no matter how good their quarter was, gets to push straight to main.

Here's how I'd put it to Jensen Huang if I ever got two minutes with him, which, let's be honest, I will not. You're right. Safety is an engineering problem. So engineer it like we engineer everything else that matters: with a reviewer who isn't the author, with logs somebody can read, with an off switch that's actually off. That's not anti-innovation. It's just a pull request with a second approver.

So, my TV is still in network jail. It's been there for years. It's going to be there for the foreseeable future. And I'll be honest, there's a part of me that feels a little vindicated this month. I watched the traffic. I blocked it. I was right to be weird about it.

But the bigger feeling is that it shouldn't have taken that. My safety system for my own living room is a one-line firewall rule that I wrote because I happen to know how. That works for exactly one household: mine. It doesn't scale to my neighbor, or my mom, or the 216 million TVs in that video title. Being an engineer who's suspicious of her own television isn't really a safety system. It's just my weird hobby.

The market didn't catch this. A YouTube channel caught it, and the market's response was pathetic. So when someone tells you the market is the safety system, ask them the same question I'd ask about any safety system. What does it detect? How fast? What's the fix? If the answer is a press release, that's not a safety system. That's a status page that says "All systems operational" while the room is still on fire.

And if the only way your customers can find out what your product does is to spend $70,000 finding out, you didn't build a product with a privacy problem. You built a privacy problem with a very nice screen.

That's this episode of Chaotic Commits. I'm Joanne Skiles, the only person I know who put her television in network jail and occasionally checks on its parole request for fun. New episodes wherever you get podcasts. No highlight reel. Just the commit history.