The author appears to have gotten the slide exactly backwards. She said the slide showed a query of “children’s clothing” that Google rewrites to be a “Nikolia kidswear” query so that it can sell more ads. But in reality, the slide is describing a fuzzy keyword matching system that takes a query of “Nikolia kidswear” and allows it to match ads with “children's clothing” keywords.
I’m surprised WIRED allowed such an obviously incorrect article to be published in the first place, particularly when it was by a known partisan (the article discloses that the author is a former Duck Duck Go executive with an obvious bias).
More precisely, the slide shows, for an advertiser who is bidding on the keywords “+kids +clothing” and has this sort of broad match enabled, three columns of examples of searches that would also match:
1. (because of [kids → children]) ads with keywords “+kids +clothing” would also match searches like “clothing for young child” and “newborn children's clothing”
2. (because of [kids clothing → kidswear]) ads with keywords “+kids +clothing” would also match searches like ”nikolai kidswear” and “kidswear outlet”
3. (because of [clothing → apparel / outlet]) ads with keywords “+kids +clothing” would also match searches like “creative apparel for kids” and “kids outfits”
That is what the slide's title (“Advertisers benefit via closing recall gaps”) refers to: the gaps in recall (matching) are being closed, by being broader.
The WIRED article misunderstood the slide, and was entirely based on the premise that if you searched for “children’s clothing” you'd get results for “NIKOLAI-brand kidswear” which is not true (and would indeed have been “startling”, not to mention obvious, if it were true). In fact, the organic (non-ads) part of the search results in Google are always completely independent of anything in ads, something that the Search team in Google have maintained for several decades as a fundamental principle.
> In fact, the organic (non-ads) part of the search results in Google are always completely independent of anything in ads, something that the Search team in Google have maintained for several decades as a fundamental principle.
Are you sure? There's an email from the ads team proposing multiple measures to increase the number of search queries so they can reach their target revenue. One of the mentioned measures include "ranking tweaks."
Well, it's a line I've heard a few times over the years, and it's also being stated externally (https://twitter.com/searchliaison/status/1709726778170786297 says “The organic (IE: non-sponsored) results you see in Search are not affected by our ads systems.”) so I'm fairly sure it's still true.
That email thread you linked is between the Ads and Chrome (not Search) teams, is about the number of search queries (not the results of search queries), and “ranking tweaks” there refers to the ranking that Chrome uses to show the suggestions in the omnibox (address bar). (To get a sense of these “ranking tweaks”, try this experiment in a (new?) Chrome profile with default settings: type "flowers" in the Chrome address bar and don't hit Enter, and look at the suggestions: what mix of search suggestions, entities, and bookmarks/history do you see? Try again with other commercial queries like “insurance” and “mortgage”, and also some less commercial queries like, I don't know, “Minnesota” or “economics”.)
(And FWIW, I think that whole email thread actually shows Google in a “good” light relative to the popular impression here on HN as a company whose every action is some Machiavellian scheme to increase ads revenue: it shows that Chrome actually launched something to production before its negative impact on revenue became a concern, that Ads leads had to work hard to persuade them to either roll back or find some other way to undo the decrease in search query volume, that starting to include search query volume as a launch criterion would be a “cultural shift” for Chrome, etc: that Ads having an influence on Chrome is a rare occurrence.)
I disagree that it shows Google in a "good" light. Here is a excerpt from the emails between Ads to the Google Chrome team. From Jerry Dirschler (Ads) to Anil Sabharwal (Google Chrome)
>"Thanks Anil (Google Chrome Lead) for pushing your team and being open to this whole line of thinking... We are short REDACTED% queries and are ahread on ads launches so are short REDACTED% vs. plan...The Search team is working together with us (ADS) to accelerate a lunch out of a new mobile layout by the end of May that will be very revenue positive (exact numbers still moving) but that still won't be enough. Our best shot at making the quarter is if we get an injection of at least REDACTED%, ideally REDACTED%, queries ASAP from Chrome... I also don't want the message to be "we're doing this thing because the Ads team needs revenue." That's a very negative message. But my question to all of you is - based on above - what do we think is the best decision for Google overall?
>In that spirit, do we think it's worth reconsidering a rollback? Or are there very scrappy tactical tweaks we can launch with holdback that we know will increase queries? (For example, can we increase vertical space between the search box/icons/feed on new tab to make search more prominent? are there other ranking tweaks we can push out very quickly? Are there other entry points we haven't focused on that we could push on soon?) Just to be clear, the reason I haven't pushed harder on a rollback so far is because I don't want the message to be..."
That's the same as the link I was responding to, and that's why I wrote “relative to” — the Ads team pushing on Chrome for revenue shows Google in a poor light relative to an imagined world where Chrome never cares about Google revenue, but a good light relative to an imagined world where everything that Chrome does is for revenue or some short-term profit to Google, rather than what's good for users.
Why are people not talking about this document more?
“I also don’t want the message to be we’re doing this because the Ads team needs more revenue…but what is the best for Google overall?”
Clearly, the ads team has influence over search to the point of saying more-or-less screw culture and team morale, let’s do what’s best for Google overall which is hitting our quarterly targets.
reading through the email chain, it seems ads did indeed get its way, and the product was indeed made worse to drive revenue numbers – chrome team was unable to say "no" when pressured by ad team
And I looked at the article at the time, and then showed my righteous indignation on HN. An example of why it’s important to dog deeper before commenting.
Unfortunately par for the course these days. At least they did remove the article. Citing "editorial standards" is weak sauce though, the point of having editorial standards is so they can be applied by the editor before publication...
I have always found this to be a weird take from a forum largely populated by software people. Every piece of software I have ever used has been riddled with bugs and security defects. This despite the fact that we have dedicated sub-fields for both (security engineering / test engineering). So when a journalist notices a bug in their article and has to "retract a release", how is that not what we do every single day?
I think you might just be disappointed in humanity, rather than any particular agency or person. Humans come together to get the right answer eventually; not multiple times a day, every day, without fail. Sometimes the software doesn't do what you want. Sometimes that article has a factual error. Eventually we figure it out. It's not the end of the world.
In this case, it's probably justified to have gripes with the editors.
They got an opinion piece with an extraordinary claim about something the writer saw in a slide presented at a trial. She wrote the entire article about it creating theories and then how it needs to be stopped. Not to mention, she had previously worked with a competitive search engine. The central piece of fact checking was seeing that slide, that would have been the second question after "Are you sure you saw something like that?". The entire article hinges on that one slide she saw and what she understood from it.
This is not a case of missing a boundary condition. This is missing the central premise. At any level, it would be inexcusable.
PS: In a sense, as the artifacts are becoming public, i think more such confusions would surface in the coming weeks where merely misunderstanding what is on the slide would lead to a lot of rumors. Probably best we have an initial case and some intuition that this can happen.
I don't think you should derive a universal QA standard from specific release practices borne out of a specific risk/reward situation that depends on things like your particular business model or lack thereof.
There's a wide spectrum of how fucked your software can be when you release it without it being a huge deal, like if I push a broken release for my hypothetical build system that jumbles the error messages a little that's probably fine, if I push a broken build of the google dot com landing page that's raising a few more sirens and I really shouldn't make a habit of it, if I deploy critically broken firmware to your pacemaker or your crewed rocket ship I should probably be exiled from the field. I imagine there's a similar spectrum for journalists and we can't really figure out where on the spectrum to the article from the OP should be with just analogies to a different field. Complaining about a trend of publications slipping towards the yolo'ier end of the spectrum doesn't seem on its face hypocritical.
It's also an understandable mistake. The slide the GP mentioned doesn't provide a clear explanation and can be interpreted the way the original article did. It's no wonder multiple people didn't catch the mistake.
An article should not be published based on one person's interpretation of one slide. Basic journalistic standards would require having other sources that corroborated what the writer thought the slide meant.
Isn't one of the key elements of ethical journalism that you need to ask the subject of your article for comments? I have a hard time believing they asked Google to comment on this piece and didn't get "that is categorically false and if you publish it you will be hearing from out attorney" as an answer. So I don't really see how this one got through unless they didn't ask for comments.
This would be like the SWE equivalent of pushing to production without code review. It's not an excusable mistake like when a bug gets through.
I don't know about "need", but it's certainly common to reach out to them... But that doesn't mean that they respond quickly. Google does not have a rep for reliably getting back to people quickly or even at all, and when they do get back to people, they very often just say they can't comment, because there's no way for the people handling the press contact to reach a person who would know what was going on in a predictable manner. (Yeah, that sounds stupid. It is. And yet, nonetheless, you can regularly find that no one at Google can figure out who owns something or knows why it's the way it is.)
A difference being software engineering is a laughingstock no one expects much of, while worldclass publishing has a past excellence against which it's still frequently judged.
People expect their software to work perfectly nearly all of the time. And for the most part, it does. So not really sure who you think these people are who think it's a laughingstock.
We also have the notion that fixing bugs at design time is easier than in dev, which is in turn easier than in prod.
Publications can simultaneously be praised for “rolling back a bad build” while being criticised for letting that build roll out to prod in the first place.
Journalists suck. They pretend to be bearers of the truth, and yet go forth with horribly biased and inaccurate claims. Nothing like software which does not wield anywhere close to the same cultural power to control narratives as the news media.
The same way we expect "QA" to pick up those bugs before release?
Think about the opposite. Imagine we pickup a bug in a live application and update it citing "our QA standards". This magical word does not absolve us of the fact that we let a bug through to production. And it's implied that we'll do better next time to pick up this kind of bug.
The post you are responding to is making the point that even though we have QA, dedicated jobs or even departments for these things … We, as a profession, still fail in releasing bug-free software. And that’s 100% correct.
Is that really something you are challenging? Which pieces of popular software you can think of are bug-free? And are you thinking of more than a handful?
They said "So when a journalist notices a bug in their article and has to "retract a release", how is that not what we do every single day?"
And they themselves were responding to someone criticizing media for using "editorial standards" as an excuse when they "retract a release".
And I explained how we are the same and supposedly strive to not release bad content/software, we just don't get to magically absolve fault with an excuse that "it doesn't meet our QA standards" as an analogy to "editorial standards".
Anywho, the whole thing breaks down when we have to beat it with a stick. It's a discussion, assume a charitable interpretation.
Retracting an article due to "editorial standards" is, to extend your analogy, like attributing the need for an emergency hotfix to "QA practices".
Editorial standards should stop bad stories from being published. Saying "we retracted this story because it doesn't meet our editorial standards" begs the question "why are you publishing things that don't meet your editorial standards in the first place?"
It doesn't take responsibility for or explain the mistakes in the article. It doesn't state that the article had factual errors. It's a frustrating cop-out. I sincerely hope this is a temporary measure while Wired gets a more comprehensive retraction put together.
Wouldn't it be better to add a correction at the top of the article instead of deleting it?
Deleting it I'd think would just fuel (current or budding) conspiracy theorists, as they can point to something that existed but got removed, rather than something that got corrected.
But in the end, it's probably a loosing battle anyway...
I don't know if you've ever had the misfortune of interacting much with conspiracy theorists, but literally whatever happens it always proves their beliefs are correct.
I recall an anecdote by a mathematician about his approach to peer-reviewing proofs of theorems. He said he glosses over most of it, but very thoroughly checks anything the author claims to be "obvious" because it's the things that seem obvious that screw people up.
Had a discussion about Assange the other day (contentious I know but bear with me). I brought up how he had a show on RT, the Russian state media channel, mentioning that it’s pretty bad optics regardless of what one thinks of the dude. They immediately went “no they didn’t that’s got to be a CIA myth.” Hand to god that’s what they said.
I showed him the IMDb and Wikipedia pages for it and a clip. Without missing a beat - remember they literally had never heard this before and have already decided it’s misinformation planted by the CIA - he said “Well it’s an independent show he put on that happened to be shown on RT.”
Seriously! Like I said, I get it’s a very contentious issue and I’m sure people have varied feelings on this, but to walk into a conversation literally knowing nothing about the topic at hand and then hand waving it away so quickly…I mean there’s no point in even continuing the conversation at that point.
Assange knew every western government was likely to do literally nothing to defend him against the US. In his shoes you'd also probably be drip-feeding information and money to the Russians. Info which should be out in the open anyway.
Whether they removed the article or not, I think it’s ridiculous that WIRED didn’t explain why the article was retracted.
They should have mentioned 1) what the article got wrong and 2) that the article was written by a former Duck Duck Go executive who should know better.
Ya, I'd prefer if they just corrected the article, made a note at the top of the article about what was corrected, and have a disclaimer that it was written by Google's competitor.
Adding a correction at the top would just fuel (current or budding) conspiracy theorists, as they can point to something that was, in their eyes, forcibly corrected against WIRED's will.
Whereas the sources spoon-feeding them this attitude are, of course, unimpeachable. (They don't redact, or admit fault.)
I generally find that anyone expecting error-free perfection in any subjective field from an entire industry can be safely dismissed as a complete idiot. Intelligent people look at the processes that produce outcomes, not the outcomes.
(In this case, the process that produced the outcome was a former DDG employee mailing in an oped of incredibly questionable quality that criticized a DDG competitor, that was eventually pulled. Wired should look into reading op-eds before publishing them.)
Because journalism is not the way to solve this, and we all think that it will.
The way to solve it is for "investigators" to get access to said source code and give us an honest assessment. Get someone neutral, get NDAs in place, have them look at the source in an air-gapped clean room, anonymize DB access, etc etc.
Instead we go around in circles. Journalism at the end of the day then becomes "we got some info that points to X, but we have no proof, and we won't look for it. We'll just wait till someone, somewhere, at some point, who knows when, finds some proof and puts it out in the public, only then will we say it's conclusive. Until then, you all go nuts, especially the conspiracy theorists. Don't worry, we'll ride this gravy train and report on those nut jobs too!".
It's a mess, and I'm not at all surprised that conspiracy-theorists have a field day with it.
Hey, let's get a bounty up so that a whistle-blower from Google comes forward. Maybe Mozilla foundation can take some of that sweet sweet endowment they use for "Social Justice" and pony up to fund this investigative effort. After all they claim to be fighting for a free internet. I'd say this is a much better use of money than paying random DEI consultancies huge speaking fees.
That would be showing too much respect for or deference to conspiracy theorists, who aren’t really worth the trouble. Copies probably exist in the Internet Archive, archive.is and in PDFs people could have saved when the article was up so it’s not like it truly disappeared. WIRED just took down a bad article that shouldn’t have been published.
Transparency is the key to trust and journalistic integrity. Even an article that was mostly wrong/written from a failed premise should remain accessible with a timestamped correction statement explaining the problem.
Making your mistakes suddenly disappear as if they never happened is tempting, but it's going to leave users feeling gaslit, distrustful, or even just misinformed.
> the point of having editorial standards is so they can be applied by the editor before publication...
“Ain't nobody got time for that”.
The role of journalists is to provide facts, but the business model of media in the internet age is about “creating content” and “engagement” and they don't work well together …
Even providing facts is a subjective, opinionated process. Which facts are provided, which are left out, which are contextualized, whether or not active or passive voice is used[1] and which are not can completely invert the meaning of a story.
[1] 'The gun went off and the shot hit Mr. Smith's head' versus 'Officer Sloan shot Mr. Smith in the head.' [2]
[2] Both of these are 'just the facts', but one of them blames the gun for going off, the other blames the person who made the gun go off. Passive voice versus active voice, very different presentation of the same facts, both are correct, and both are biased.
Was this piece marked as opinion or an editorial? Allowing 3rd party writers to publish isn't new, but back when we pretended we had standards, they were placed in an obvious part of the publication that it wasn't staff writers working within the normal flow of editorial process. i must be old, because a lot of my comments are now "used to be", "back when", "remember back when" type stories
Honestly, it just seems like something where somebody looked at it too quickly, got excited, and then didn’t look a second time. They are probably an experienced writer so everyone approving it went “sounds great.“ Honest mistake, but a bad one, and very unprofessional.
>I’m surprised WIRED allowed such an obviously incorrect article to be published in the first place
Is wired supposed to be a very accurate news source? I'm not surprised to hear bullshit from the media. This statement implies wired is supposed to be better than normal?
The author appears to have gotten the slide exactly backwards. She said the slide showed a query of “children’s clothing” that Google rewrites to be a “Nikolia kidswear” query so that it can sell more ads. But in reality, the slide is describing a fuzzy keyword matching system that takes a query of “Nikolia kidswear” and allows it to match ads with “children's clothing” keywords.
I’m surprised WIRED allowed such an obviously incorrect article to be published in the first place, particularly when it was by a known partisan (the article discloses that the author is a former Duck Duck Go executive with an obvious bias).