Setting this case aside because it's sloppy reporting based on what another classifier found, this is a hard problem. Youtube gets too many videos and comments for humans to manually review them. Flagging helps, and Google likely uses ML more complex than just text and classifier (who is commenting, is the post trending, patterns in comments, etc.) When things do reach manual review, remember that humans make mistakes, possibly more often than ML, this could be human error.
You actually want the algorithm to be a black box so that people don't know that you can trick it by only commenting between 5pm and 8pm on an IP in the Netherlands from Chrome, and avoid using words like "cactus."
The important part here is a transparent appeal process, not that ML is only pretty good at these problems.
People forget that they get these free services for free, and a big portion of that is because ML tools are able to cheaply do a really hard job of moderating. You hear about things like this because they're so unusual, which is a testament to the amount of time they work correctly without being noticed.
Granted, there have been some issues with people getting banned from Google services for what appear to be wrong reasons and then not being able to get any kind of a human response. Those aren't great, but if you ask people whether they want to pay $10/month for YouTube so human moderators can stop this from happening or get it for free and have the .0001% chance that they get banned purely by accident, they'll take the latter. And that's putting aside the fact that there will still be errors with humans doing it, and there will still need to be the same appeal process to resolve those.
Not entirely true. First the platform is free, but also is the content that people provide and without it Youtube wouldn't make sense - so it's not a free service, it's a partnership. Second, for many people YouTube is the primary source of income, and I'm sure there would be plenty of content authors who'd be glad to pay a service with human moderators rather than bots - GIVEN THAT they get to keep access to the same traffic that YouTube provides. That's the key thing, you can't just go somewhere else when all the users are on a few popular platforms. If it's not for that, many high-profile authors who get their videos demonetized on stupidities would have left YouTube long time ago - but they can't, big players keep them in a checkmate position due to the form of monopoly on viewers that they have.
And that means you can't just say "If you don't like it, leave, it's a free service", because it's not that simple, as leaving in reality means "give up on your business", and most of people just can't afford that. Youtube has them cornered and thus needs to be careful, since it's not a game, it affects people's lives. For many accidental blocking of channel can mean they'll be living in their cars until the Gods of Google show mercy and fix the error.... and if they do it, since sometimes you don't even get a meaningful reply on what happened, just a template message.
The cost to a competitor is in getting a user to switch. Think of it in terms of paying users: how much would you have to pay someone to switch from Youtube to your site? Multiply the average by the number of users, and that's the advantage Youtube has over competitors.
Ooh, this is something I've often wondered (and has become more interesting since Alphabet recently started revealing revenue numbers for YouTube) - can YouTube be profitable as a standalone entity?
The reason I ask is because, despite all its flaws, YouTube is one of the treasures of the internet. And I wonder if we'd lose it by breaking up Alphabet.
I am not pro-Alphabet and 100% believe Alphabet needs more regulation, and quite frankly does need some trust busting, but I'd be really sad to lose YouTube. Yeah, there's vimeo but it's clearly not a direct competitor. I also fondly remember stage6 (https://en.wikipedia.org/wiki/Stage6).
I would bet it could. YouTube definitely benefits from Google infrastructure. Especially since a lot of their cost is video transcoding and batch pipelines which can easily be slotted into unused CPU around the world. YouTube almost certainly also gets very cheap storage not only because Google has optimized storage cost for them but also because of the similar flexibility that they have on where videos are stored. Furthermore they use Google's CDN which is likely the best in the world hand has many relationships with ISPs to get caches close to eyeballs.
However as I understand it the product itself isn't tied to the Google infrastructure. YouTube's main value is the user base, both of viewers and of publishers. It would be a technical marvel to move it off of Google's infrastructure however I don't think that there is any feature that they wouldn't be able to provide anymore. It would certainly be more expensive, but at YouTube's size you could probably work out similar deals with other providers (or with GCP) so I'm not sure that the prices would rise that much. (IDK, maybe 15% increase in cost between provider cut and raw cost increase?)
YouTube doesn't really fit the definition of a natural monopoly, particularly because there's nothing fundamentally stopping other services from popping up. YouTube's status as an effective monopoly doesn't come from some sort of scarcity of the means of production.
> The cost to a competitor is in getting a user to switch.
Only if we're assuming that users and uploaders will only ever use exactly one service at any given time. No reason why that needs to be the case; nothing stopping people from uploading to YouTube and Vimeo and Twitch and DailyMotion and PornHub and LBRY and PeerTube and whatever other platforms, and nothing stopping people from viewing from those platforms, either.
Thing is that the free market approach just doesn't work in markets where there's only a few players and the cost of entering the market is extremely high. Simply there's no enough competition to make things actually competitive. That's why google provides no support and shuts down peoples accounts without warnings... they can, they just don't give a shit. They know they will not loose any clients over that, and in the end that attitude detriments the quality of services for both viewers and content producers (and especially them).
> and the cost of entering the market is extremely high.
The cost to enter the video sharing market is extremely low, especially in this day and age where you can rent computing and storage capacity on the cheap around the world.
Then where is the YouTube competitor? The closest seems to be odysee (https://odysee.com/) where you're lucky to break a few hundred views.
Suppose YouTube disappeared tomorrow. Where would everyone go? Probably odysee. So why don't they go there now? That's why Youtube is a natural monopoly.
Yeah the fact that said service is the very first result on a major search engine, who's conveniently the owner of said service, has nothing to do with it. Organic congregation and all.
> ...that is because ML tools are able to cheaply do a really hard job of moderating
I mean , YouTube is huge, but we had moderation-at-scale before ML was a popular phrase to bandy around.
'ML' and 'Algorithms' are just convenient non-human bogeyman that any corporation can pin all their woes on without batting an eye right now -- a convenient explanation tool for when things go wrong.
I am going to take a bold stance with this next statement:
An algorithm cannot be at fault -- it is always the fault of those that implement such algorithms.
YouTube and their community is huge -- YouTube has revenue fifteen times greater than Yahoo. 'Too big to X' should not be a tolerable excuse from any industry -- especially so since YouTube/Google has been pushing the idea for years that being a YTer is an actual profession.
Imagine for a moment coming into work and being told that you're fired because an internal corporate policy checker decided that something you did in your past work history was worth of termination; and then 24 hours later being told to continue coming into work because a mistake had been made, but with no further explanation.
This kind of 'workplace' would be intolerable in any real 'work environment'. Just more evidence that YTers are not really treated like any other partner or employee anywhere else in the world -- and that they should probably curtail that language if they want expectations to match the reality that their 'employees' experience.
>People forget that they get these free services for free,
YouTube is not a free service. This fallacy needs to go.
Okay, fine, so an algorithm can't be at fault. Ultimately it doesn't matter whether it's an algorithm. The point is that at this scale, moderation mistakes will be made. Doesn't matter if it's humans or ML or what have you. People rush to condemn the service and the people who make it as a result of these very, very seldomly occurring mistakes, but there is no system that exists that would allow perfect moderation at this scale.
Your work comparison just isn't valid because there is no employer with a number of employees within several orders of magnitude of the number of videos on YouTube.
> You hear about things like this because they're so unusual
I feel like it's the exact opposite: we're hearing about them because these are the ones that slipped into public awareness. The inadequacy of automated moderation without any real semblance of human oversight has been rampant and frequent for about as long as the concept has existed.
Just because the user doesn't need to pay up-front doesn't mean the service operates at a loss (YouTube pulled in $15 billion in 2019, according to a quick search). Furthermore, YouTube Premium is a paid offering. IMO Alphabet just uses YouTube as a playground for its ML experiments because it can moreso than because it needs to for viability.
Youtube brought in 15 billion dollars in revenue last year. It is probably most important to the customers of youtube (the ad buyers) that their ads aren't shown on controversial videos.
Also, I'd be really interested to know how much the Youtube team payed to build this classifier and how that compares to paying human moderators.
> Those aren't great, but if you ask people whether they want to pay $10/month for YouTube so human moderators can stop this from happening or get it for free and have the .0001% chance that they get banned purely by accident, they'll take the latter.
At this point, I view YouTube like I viewed bars that allowed smoking--it was sufficiently popular that nobody could compete against it even though it was harmful. So, we needed legislation to move us to a different local minimum.
And, I find this very painful because I firmly believe that smoking bans have killed live music. But I also understand that nobody was ever going to do the right thing while doing the bad one was so profitable.
Is an astoundingly extreme claim. First we have to accept the premise that live music is “dead”. Second it’s due to some orthogonal factor like a smoking ban as opposed to recorded music becoming way more accessible through technology.
Intermediate size venues got wiped out because smokers are extremely profitable--smoking and drinking go together.
You can have a small venue (<100) and it will trundle along. You can have a big venue (>1000) because it probably reached self-supporting.
However, we lost a LOT of 100-1000 size venues from about 1990 to about 2005. And those are the ones that working musicians can make a living off of. But those need the stupidly profitable contingent that goes along with smoking and drinking. The single craft beer drinker isn't going to generate enough money to support such a place.
You could see this even before the smoking bans in areas which had sophisticated dancing (swing, ballroom, etc.) groups. If your venue somehow attracted the young dancing contingent, the venue was going to vaporize within 6 months. Everybody loved them--they tended to be polite, didn't harass the waitstaff, looked good, prevented highly skewed M/F ratios, etc. -- except that they spent next to no money compared to the general public so the owners HATED them. This was in stark contrast to the elder dancing contingent who smoked and drank like fish and could keep clubs alive long beyond their expiration date.
There must be other factors at play, because pre-pandemic I attended several live music events every month, a lot of them small <250 people venues. A friend of mine has a goal of seeing 1000 artists in a year, and he got within spitting distance of it in 2019.
Of course, all venues make sure to have outside areas for smoking, because that's the sensible thing to do.
Okay, I'm going to ask "Where?" because my musician friends probably want to move there once Covid is done.
Although, if you say Nashville or New York, you're not helping--those are mega anomalies and all the musicians are already there which makes the situation untenable.
Denmark, but it goes for most of Europe in general, at least the places I've been. Any city over a certain size will have a bunch of small venues, and the laws against smoking indoors are EU-wide.
However, I am concerned that a lot of these smaller venues and promoters will struggle to survive the pandemic lockdowns, unless our governments pull themselves together and support cultural venues, instead of focusing so hard on sports. Culture isn't just something you play with a ball :-)
As for the downvotes, probably general disagreement or spillover from other discussions where someone has taken offense to what I wrote.
Intermediate size venues got wiped out because smokers are extremely profitable--smoking and drinking go together.
Same in pubs. The smoking-and-drinking crowd got edged out by parents who for some reason want to bring their kids to the pub. They will have brought their own drinks and snacks for the kids too. The mother will have a small glass of wine and the father will drink half a pint of "craft beer", stretched out over a couple of hours while their kids run around screaming and annoying the few remaining paying drinkers.
I loved pubs in the old days but even before COVID they were dying out because they just don't want their loyal paying customers any more.
I’ve yet to see a case of the hoi poloi get a manual review - you either are a well enough known YouTuber where they have humans dedicated to you, you know someone who works at YouTube and can intercede on your behalf, or you have to whip up a Twitter mob big enough that it hurts YouTube's image or stirs up people with contacts at YouTube.
Even then, sometimes that doesn’t work - look at the Terraria dev.
YouTube effectively has no manual review for 99.9% of people which is why its automated systems accuracy is so important.
Be careful not to confuse manual review with an appeal. When I say "manual review," I mean the classifier isn't sure, so a human double checked, or it was sent to a human to validate the classifier. You're right--appeals are hard to get--but we have no idea how much Youtube manually reviews content.
It's a genuinely hard problem, but they do have appeal systems that creators can use that go to human reviewer queues for things like video removals and demonetization even though the first pass is done via automation, but don't think humans aren't immune from mistakes either.
As of 2018 youtube had 10,000 manual reviewers (and I imagine it's significantly more than that today), but at their scale, that realistically won't go that far.
Actually watching every YouTube video uploaded would take ~100,000 people which is possible, but quite expensive. Actually watching every video that’s flagged for banning might take ~100 people and seem like a reasonable effort.
As such arguments about using automated systems are meaningless, they can pre-filter without having final decision making authority.
> 100,000 people might not be enough to just deal With flags...
You’re off by orders of magnitude.
500 * 60 * 24 * 365 = 262800000 hours per year / 1.3x playback speed / 2,000 hours of work per person per year = ~101,077 people to watch every uploaded video.
Less if people are watching at higher playback speeds or working significantly more than a 40h week. I would personally be surprised if even 1 in 1,000 videos where flagged by ML at which point you’re talking ~100 people. Though that would be a truly terrible job.
> 500 * 60 * 24 * 365 = 262800000 hours per year / 1.3x playback speed / 2,000 hours of work per person per year = ~101,077 people to watch every uploaded video.
When you have people watch the video at that fast speed and instantly make a decision, you might as well keep the algorithm - the failure rate won't be too different.
Additionally, these people need holidays, managers, infrastructure and an actual flag will probably not instantly be clear - you'll need to point out the specific offence and make a case. You might even need to look things up. Then you'll have to handle appeals and discussions, because changing the AI blackbox to an appeal-less human black box wouldn't improve the situation much[0]. Next, you need people who can understand the specific language of that video - most will be english, but what happens when a Nigerian video is flagged? So you need people from that country or at least familiar with it. Plus infrastructure.
Overall, you're probably looking at something approaching 400 or 500 people, with this very low flag rate. Assuming conservative 30k$/year [1], this is 15 million USD alone. Doable, yes, but its not the no-brainer you make it out to be.
Reading the article, "human error" really isn't a reasonable description of what happened:
> The Facebook furor began when the social network deleted the famous photo from Norwegian author Tom Egeland’s Facebook page, where it was part of a series of memorable wartime imagery.
This could be a human error, sure.
> When Egeland subsequently posted his shocked reaction to the removal of the “napalm girl” photo, he found his account suspended.
But not this.
> Norway’s largest newspaper, Aftenposten, published Egeland’s story on the censorship, only to find that its own Facebook posts were also quickly deleted.
Definitely not this.
> Espen Egil Hansen, Aftenposten’s editor, then took to the front page of his paper to slam Facebook in an open letter to CEO Mark Zuckerberg
> Prime Minister Solberg joined the debate on Friday, only to find that her comments and posts about the suppressed photo were also deleted by Facebook.
This is a (ridiculously severe) problem in several of Facebook's policies, not a classification error.
> Reading the article, "human error" really isn't a reasonable description of what happened:
I'm pretty sure to have read that the initial blocking was done by a contractor - I can find a better source, if you want.
> This is a (ridiculously severe) problem in several of Facebook's policies, not a classification error.
Yes, and that is exactly the point I tried to make:
>> changing the AI blackbox to an appeal-less human black box wouldn't improve the situation much
Said another way, having underpaid contractors with no time for consideration (and possibly strange policies) is exactly as bad. This is a pretty clear case: That photo should not have been blocked and the idea behind having a human do the evaluation is that he recognizes the historical relevance and, if not outright allowing it, at least consults the relevant authorities whether this is exempt. This could've been done by an AI all the way without a different outcome.
People often watch YouTube videos at 2x speeds, it’s part of their app to select 1.25, 1.5, 1.75, 2x because you can generally listen to most speakers at that speed just fine. Let alone 11 hours of ocean waves etc.
Only averaging 1.3x assumed significant overhead and going back to re listen to segments, have an escalation queue for flagged videos, and randomly assigning multiple people to the same videos to ensure people are paying attention not just playing solitaire while the video is on. Plus other random crap, in other words ~30% overhead.
[disclaimer] I work at Google, all opinions are my own. I don't deal with Youtube but have had exposure to moderation operations. It is not an easy task.
You are incorrect about your assumptions in many respects. Let me list out a few:
1) There is a big difference between watching a 15 min video at 2X for fun vs watching 8 hours of videos a day while having to follow laid down policy with complexity. Videos are not taken down only for one reason and there is a lot of complexity involved with edge cases. Humans are not robots and this is not a task we are inherently good at. Give it a shot yourself for a day and see how easy it is.
2) Your overhead does not consider any additional complexity in terms of languages, specialty, regional expertise etc. This is not a simple problem either. Sure maybe you can hire 1000 reviewers in lets say Indonesia but you cannot find a native Kswahili speaker there. Its not cheap to setup an office for 2 people in Kenya. Scale this to the world which has 193 countries and 6,500 languages.
3) People dont work without any management overhead. You need frontline reviewers. Then you need a layer of experts above them. The you need managers to take care of the operations. The managers to manage those managers. Then recruiters to hire those people. Then HR to deal with their issues.
4) Turnover is a big deal. Very few people in the world can watch beheadings 40 hours a week. An even smaller proportion can handle child safety material. What happens when those people need to take time off? You need additional people. If you want a humane operation then you might have to get them to work only 2 hours a day.
It continues to amaze me how some of the smartest and technically savvy people on HN either do not recognize or refuse to admit how complicated a problem planet scale moderation is.
> It continues to amaze me how some of the smartest and technically savvy people on HN either do not recognize or refuse to admit how complicated a problem planet scale moderation is.
It's probably a variation of the trivial-core-problem [0]. When looked at roughly, it seems very easy; only when you start implementing it, you'll see all the edge cases appear. I made pretty similar points above to the ones you made - but on a quick napkin calculation, Retric's math checked out, to be fair. And when you're not in the business, missing the inherent complexity is easy.
Building systems is clearly a major effort which was specifically excluded from that estimate. However, while you’re right it’s complicated, I would point out my overall estimate was close to Google’s own.
“Since we started using machine learning to flag violent and extremist content in June, the technology has reviewed and flagged content that would have taken 180,000 people working 40 hours a week to assess.”
That said, I am surprised Google isn’t using machine translation for obscure languages. Props to them that’s going above and beyond in my personal opinion.
I'm not saying it's not complicated, but complaints about expense and overheads for a company that makes something like $40 billion a year in profit ring a little hollow. That's roughly the size of the UK's defence budget.
They could just manually review the videos the algo flagged as potentially ban worthy. It might even given YT an incentive to hone their algorithm to be more accurate since it would reduce their bottom-line.
Right. The algorithm should just identify videos that need to be reviewed, not ban them. It should also be a sliding scale. If a video has 5 total views, it shouldn't need a ban or a review because no one is watching it. Places like this chess channel with a million subscribers and hundreds of thousands of views per video should all easily get a manually review. Automating bans for channels with that level of a following is just lack of respect for their content producers.
So why bother screening any of it? So what if some nefarious content makes it through. Don’t like it, don’t watch it. Not sure why everything has to be sanitized.
Their actual customers, aka advertisers, want to avoid being associated with some things. It’s the same reason you don’t see Pepsi etc advertised on porn websites.
I think it's hard to be perfect but easy to be much better:
1) warn content providers before blocking whenever possible. It's rare something is so bad it requires immediate takedown. Allow people to reply to the warnings.
2) next step is to show a warning to content viewers and allow followers and searchers to find, but not autosuggest.
3) have a transparent appeals process at each step.
I'm not saying that Google's ML isn't doing some very fancy things but:
- Ultimately, any supervised technique will depend on some system that can provide ground truth, which in these applications must be from humans. You can't avoid some humans in the loop and still learn a fuzzy human concept like "is hate speech". But if your aren't gathering that ground truth info in a very deliberate way, you can end up with a range of gaps or artifacts.
- There's a broad swath of technical approaches which would consider multiple kinds of information, but ensemble them together such that a very strong signal from one can have a determinative impact. It would be easy to have a situation where if the video itself talks about white/black and attack/threat/defend many times, the score from some language model into a top level ensemble model is very high and the other components may not matter.
- But a lot of those other components may often be in agreement because of natural structure. A video says black/white a lot, several other videos on the channel do the same, and a bunch of interested users also look at other channels that have very similar feature vectors, suddenly a content level, channel level and community level signal are all in agreement, and your model can believe it found a subcommunity of hate speech.
Incorporating many kinds of info into an ML model isn't that hard ... But doing so in a way which is tightly integrated (eg I'm likely to use a specific meaning of a term in my video which matches a use in a video of a related channel linked in my video description, and that use of that word is also related to what's visible in the video at around that time) is pretty hard. Doing so in a way which distinguishes predictions from confidences is harder.
All of which is just to say that it's hard to build a really good black box for a complex problem, especially one based around subjective human concepts, and maybe both we and the black boxes would do better by attaching more uncertainty to their predictions.
If it's worth the channel owner spending many hours making videos, and viewers spending many many more watching them, I think it's not crazy to ask that if the model predicts that it's likely disallowed content, that a manual reviewer can take a quick look.
Is it more complex than a text classifier though? If it truly took into account who is commenting, and context, etc., it would have noticed very few chess videos were being flagged for what seemed like racist language.
It seems more than likely somebody put in a very crude black box to process text without context. (Or should I say, opaque box, so the horde doesn't flag me?)
Youtube actually could manually review literally every video. [1] In 2019, users only uploaded on average 500 hours of video per minute. This is ~30,000 hours per hour. If you had people manually watching every video at 1x speed for their entire duration, you would need 30,000 reviewers for each 8 hour shift. Even if we assume that a fully-burdened reviewer costs Youtube an astronomical $80,000/yr, that only amounts to $7.2B/yr on manual review if we assume that they have people literally watching every second of every video. In the same year, 2019, Youtube made a revenue of at least $15.15B in ad revenue alone, so the cost of such a form of manual review only amounts to ~50% of Youtube's yearly revenue which is absolutely within the realm of possibility. Now, I am not saying they could do so cost-competitively or profitably, but it is not directly ludicrous on its face to suggest that they could so if they wanted to.
Obviously, just watching every video in real-time a single time is not an adequate review process. Things can be missed, misunderstood, or just take more time to process, so it is likely that more review would be necessary for non-obvious cases. But, on the other hand, it is not necessary to review literally every second of every video. If they only reviewed reported videos and reports were required to specify the timestamp, they would probably instantly reduce this total burden by a factor of 10x-100x (assuming they filter out obvious spam reports). This would allow them to allocate more independent reviewers to the actual non-obvious cases resulting in reasonable accuracy with a human appeals process while reducing their overall cost burden significantly lower than the previously stated ~50%. They could then tier their review process based on the amount of views to allowing them to limit how many resources they spend on unpopular and irrelevant videos to focus on more visible and profitable videos while still providing a base level of review. And, all of this is assuming that they do not have any automated mechanism to directly decide on cases that are trivially obvious, thus reducing their burden even lower to only the contentious cases where the automated systems can not create a clear-cut answer. Frankly, I could go on about various other potential mechanisms they could do to reduce costs while increasing efficiency and avoiding bad actors, but what I have already said should already be sufficient to handle the basic problem.
All in all, they could probably institute a reasonable human review process that could handle all reasonable cases for <5% of yearly revenue and frankly I would be shocked if it were anywhere near the high end of that estimate. Even without any automated systems I would put my median cost prediction at 1% of yearly revenue and with even basic automated systems probably close to 0.2%.
Youtubes revenue is not its profit. It has significant expenses in terms of paying creators, hosting and bandwidth costs and the cost of all the engineers it takes to run the systems.
In addition you do not factor in significant complexities like the ones below:
You are incorrect about your assumptions in many respects. Let me list out a few:
1) There is a big difference between watching a 15 min video at 2X for fun vs watching 8 hours of videos a day while having to follow laid down policy with complexity. Videos are not taken down only for one reason and there is a lot of complexity involved with edge cases. Humans are not robots and this is not a task we are inherently good at. Give it a shot yourself for a day and see how easy it is.
2) Your overhead does not consider any additional complexity in terms of languages, specialty, regional expertise etc. This is not a simple problem either. Sure maybe you can hire 1000 reviewers in lets say Indonesia but you cannot find a native Kswahili speaker there. Its not cheap to setup an office for 2 people in Kenya. Scale this to the world which has 193 countries and 6,500 languages.
3) People dont work without any management overhead. You need frontline reviewers. Then you need a layer of experts above them. The you need managers to take care of the operations. The managers to manage those managers. Then recruiters to hire those people. Then HR to deal with their issues.
4) Turnover is a big deal. Very few people in the world can watch beheadings 40 hours a week. An even smaller proportion can handle child safety material. What happens when those people need to take time off? You need additional people. If you want a humane operation then you might have to get them to work only 2 hours a day.
It continues to amaze me how some of the smartest and technically savvy people on HN either do not recognize or refuse to admit how complicated a problem planet scale moderation is. If it was a problem that could be solved by spending a few tens of million dollars a year, it would have been solved.
You actually want the algorithm to be a black box so that people don't know that you can trick it by only commenting between 5pm and 8pm on an IP in the Netherlands from Chrome, and avoid using words like "cactus."
The important part here is a transparent appeal process, not that ML is only pretty good at these problems.