I’ve noticed that my own Instagram feed was only showing me content from large accounts and female friends who posted bikini pics. And if I decided to keep scrolling without liking a picture it would continue showing me the same post on subsequent logins or even the second photo in the album of the post further down in the feed. The app was essentially telling me what posts to like while filtering out all the genuine content my friends were posting. So I deleted it. We need a good ole consumer boycott and government-initiated privacy probe.
I remember watching a YouTube video from The Wall Street Journal where they had configured 100 bot accounts to essentially crawl TikTok to study how it develops profiles for each user. For most users it may seem rather innocent, but in the video the user was modeled to be someone struggling with depression and towards the end of the simulation was being fed mostly depressing content.
I think recommendation engines can be useful for discovering content, but using gradient descent to do this can also be extremely dangerous. There is seemingly no control for when the recommended content is slowly pushing someone towards self-harm or other acts of violence.
Generally they optimize for engagement not informing you of your friends activities.
So it had likely pigeon holed you into a class (age, sex, relationship status, whatever sites the facebook button has said you were on) and it is showing you things that "similar" people have engaged with and lets be honest a good amount of the popular posts on social media are thirst traps...
I know you can download the information from Facebook about the profile it keeps on you but I am not sure you can do the same for Instagram. Regardless it was pretty interesting seeing what it had in there. For me some of it was correct and some was wildly off.
This raises a really interesting question. It’s quite possible that an algorithm tuned to predict which post most users will engage with learns to boost posts of people in bikinis without being explicitly programmed to do so.
This is slightly out of my league but is there not a human, or many humans, that write the algorithm? Is this so out of control that you can't make changes?
I guess it sort of depends on what you mean by the algorithm. I'm not an expert on this either, but the general idea is that you pick some sort of thing you want to promote (e.g. engagement), and then you use a really large data set with thousands of parameters to construct the `parameters -> engagement function`, some of those parameters will be user supplied, and some will be Facebook settings.
The thing that's opaque is this function that the program produces. You can really only know how the function works experimentally.
If you build a robot to help people and program that robot to do what its owner says and the owner tells it to shoot people, who's to blame? And if it's you for programming it, is it not then the gun manufacturer to blame when a user of a gun shoots someone? the car manufacturer to blame when the user of a car hits and kills someone?
I think this form of counter works if and only if users explicitly direct the algorithm.
If you buy a handgun, and sometimes that handgun goes off without the trigger being pulled due to faulty design, then that's the one time the handgun manufacturer is actually liable.
It's not as if you get to tell the algorithm "Hey, I want Facebook to be a positive experience" or opt in to content.
When you have an interest like religion or politics and the algorithm starts showing you anti-vaxx material, that isn't the user in control. It is more like that faulty firearm.
Your argument is similar to one person saying “stop showing me pictures of that,” and the response being “we can’t be held responsible for showing you this over and over, we can’t help ourselves, you have to forgive us, but it’s your fault I keep showing you this. Our spies say it affects you, so that makes it your fault”
Yeah this is what’s most sad about tiktok to me. It’s a great new format for media, but it allows, or is especially susceptible to, attractive people putting their goods on display and teasing their viewers. Some days it feels like tiktok promotes and validates this behavior, and other days it feels like the technology just scales up high school popularity dynamics and isn't really anything new. So does the tech drive the algorithm or do the users?
Youtube doesn't particularly have this type of problem, for example. It has others, sure, but the core content it breeds is not shorts of high schoolers in their bedroom dancing in their underwear. Snapchat and sending that content privately was one thing but now tiktok validates that behavior with a public audience. And then onlyfans exists to graduate your content and monetize your body directly. I’m pretty sex-positive so it’s not that I’m turned off by the concept of people wielding sexual power in society or making money off their body. But I’m not sure what society looks like when that’s the highlighted path for 16 year old girls to replicate in order to make life changing money or be immensely popular as their own brand in their 20s.
Back to the youtube example: on youtube it seems the content is important because you have to engage for more than 30s whereas on tiktok it feels like you’re following personal brands and the utility of the content is almost irrelevant.
I cannot make a sensible comment on algorithmwatch per se, but I think the old "we cannot reveal our algorithm otherwise bad people will game against it" has reached the end of its life.
These feed algorithms (and co-related moderation rules) do have an enormous effect on what information people globally receive. Studying them should be encouraged, and IMO the code itself open sourced or otherwise made available to study. If they take approaches that are game-able maybe we just have to dial it down and it have such sohisticated algorithms.
Modern social media platforms are highly profitable AI-accelerated mass manipulation systems, on steroids. These companies have little incentive and are very reluctant to changing their 'winning formula', obviously.
Misinformation is rampant and even rewarded as studies have shown repeatedly. Continuous exposure to these centralised feeds providing an infinite amount of content - managed by barely regulated for-profit companies - is worrying.
Also most platforms are based in the US, a country that has completely different social/cultural views (and laws) compared to the rest of the world. I'm glad I am not a US citizen for example. Still these platforms are being used worldwide, so I think that decisions made about the 'feed algorithm(s)' can have a big social/cultural impact on other countries. More studies (and regulation, taxes) are probably a good idea.
How useful would the algorithm being open source be without things like the ML models that feed its parameters or the data used to train the models? It would be like knowing PageRank without any of the data and asking why one site is ranked above another in search results.
We don't need to know the details of the algorithm or the models when we can see how it behaves. E.g. the example of one of the other posters where they cite that IG keeps pushing bikini pics over other content. Just knowing that the above anecdote is true empirically via whatever means is super useful and can shape our understanding and push back.
Its more about knowing whether the algorithm isnt politically rigged to engineer public opinion by amplifying certain truths and decreasing visibility of other truths.
Since I don't have any insight into their algorithm I'm going to use a naive hypothetical example: Say they did have a system to downrank negative/harmful/spammy content and it works by applying some kind of embedding model to posts and comparing their cosine similarity to a dictionary of known bad content. You have the code, but neither the trained model for doing embeddings nor the set of "bad" vectors. Can you answer the question: Is it politically rigged?
That's why I tend to be in agreement with the idea of treating it as a black box and observing what biases it exhibits.
The research is interesting in its data. The conclusions are very speculative and don't sound very well informed. The problem with nudity or clickbait in general is simply that it appeals to most people, statistically, and thus tends to emerge on its own as an apparent ranking factor even when there might be nothing in the algorithm itself to boost it specifically. The simpler models end up being the more clickbaity ones. If anything, when algorithms do take into account nudity explicitly, my guess is that it's primarily to prevent too much of it (eg porn).
Another problem in general with this sort of research is that it seems to take for granted a moral ground in which it is irresponsible to give people what they want to see most (statistically) if some people don't want to see it, or if it doesn't match what the people involved in production want to project. I'm not sure what grounds it. It seems to broadly describe media. I received a promotional copy of Sports Illustrated swimsuit edition and it's pretty transparent that it's being done with a mindset of "sex sells". I dislike it but I dislike media censorship more, so I just don't use Instagram or subscribe to lewd magazines. This group would be more convincing if they explored more of the moral underpinnings of their outrage and whether what they recommend is more morally motivated self-censorship by the platform (even though we know that users want to see skin, we think they shouldn't, so we're not going to), plain-old "white box" ranking algorithms like reverse chron (which is completely non biased from the point of view of the metric they were looking at in the nudity article, but just as susceptible to abuse in other ways), or something else...
EDIT: I don't mean this to defend the status quo. I'm just as outraged as the next person about exploitative social media. I just think it's not clear what the moral framing is and I think it's interesting to discuss it.
"irresponsible to give people what they want to see"
How do you know what people want? Because someone in charge told you so?
'Maximising engagement' is like how airports force you to go through a shopping mall on the way to your flight, through a windy path. Or imagine you design a really confusing shopping center where people get lost. They measure time wasted and call it 'engagement'
They waste your time in hopes it will push you to buy something / click on ads.
It's not the first time business leadership chases a fad on bad data, they do it all the time on bad data. Developers do the same 'fashion chasing' with their tech stack.
We have proof that'sex sells' is a myth, open plan offices are harmfull to productivity, working remotely is more productive,'saving' on IT equipment nets losses to the business, moving to microservices or agile or to NoSQL doesn't save money
Last time I condicted a poll, not a single person said their preferred content is misinformation and clickbait.
> How do you know what people want? Because someone in charge told you so?
No, I say this as an interpretation of people behaviors and assuming some degree of free agency. Users aren't coerced to use Instagram, so it likely means they use it because they want to.
> Last time I condicted a poll, not a single person said their preferred content is misinformation and clickbait.
I'm sure. There is a big difference between preferences as expressed in polls and actual preferences observed in fact. People say they consume educational content way more than they actually do. Neither kind of preference is the absolute truth, they're both valid. Should companies optimize for opinion polling outcomes instead? Is it really progress if people have a high opinion of a service but few people use it, and another exploitative service steps in to sweep users away?
What about regular old media doing rage-baiting of their own? Isn't Fox News or NYT also behaving the same way to some extent? I'm not sure people would answer a poll saying the reason they watch the news is that they're looking to be upset and outraged, but that's a lot of the apparent audience strategy.
IMO this is just natural capitalistic response to incentives and the systemic "fix" would be to regulate the media sector so that it be forced to stop feeding people's worst impulses. The only problem is that by American standards this would amount to censorship - the notion that "someone knows better" what is good for me. One's clickbait filtering strategy always ends up being someone else's muzzled speech, so it's easy for the cure to have unwanted side effects. It's a tough nut to crack especially in the US given the cultural and legal bent towards maximalist free speech.
You equate "what people are vulnerable to" with "what people want to see." "Engagement" isn't a measure of desire, there's little choice being made by the user to be shown what they respond to, but it's a measure of response. The person who you got to stare may not have wanted to see what you showed them in the first place.
> The person who you got to stare may not have wanted to see what you showed them in the first place.
Correct, but there is little ability long-term for a service like Instagram to repeatedly show things to people that they don't want to see.
> You equate "what people are vulnerable to" with "what people want to see."
TBH that sounds paternalistic to me. I may look at someone reading the people tabloids at the supermarket as a victim vulnerable to trashy content, but who am I to judge? They just like something else than I do. Nobody made them pick up the paper.
"Digital platforms play an ever-increasing role in structuring and influencing public debate. Civil society watchdogs, researchers and journalists need to be able to hold them to account. But Facebook is increasingly fighting those who try. It shut down New York University’s Ad Observatory last week, and went after AlgorithmWatch, too. "
What can be done to stop Facebook closing down these research projects?
The guy providing the data in the CA scandal was a credible scientist at the Universities of Toronto and Cambridge. His social media data collection back then was considered part of his legitimate research activities. It was funded by research grants, and nobody had a problem with that. That is, until some of the data he collected was potentially used for political microtargeting.
The thing is, Facebook and the other social media companies have never had any real interest to support research projects that collect and analyze their data. Why should they?
However, since the public outrage over the CA event they basically have carte blanche to deny all these requests - even from seemingly credible scientists - and just say: "Hey, we just want to prevent another CA."
Something you could consider is signing their open letter, they need 1000 supporters and have a bit more than 1 month remaining. That could bring this under the attention of the European Lawmakers. You can find it here:
> What can be done to stop Facebook closing down these research
projects?
Maybe I'm being naive, but what I would do is conduct my research
without having a facebook account myself and without directly
accessing or sending any messages through its servers. That way I
wouldn't be bound by facebook's terms of service (if they're legally
binding at all) and wouldn't incur any legal liability for
unauthorized access to someone else's computer. I hope the researchers
took those precautions, but nothing would surprise me.
It's unfortunate, yet not surprising, that Facebook's legal team would want to threaten and sue. Of course, Facebook's own behavior is most likely 1000x worse for users than the research that they are trying to prevent. As a consumer, if this bothers you, express your preference by uninstalling Instagram and whatever other product that belongs to Facebook.
All social platforms have one goal: keep you there, so you can see as much ads as possible. I seriously doubt that they sacrifice profit for who-knows-what conspiracy theory.
I've been thinking for a while that it should be mandated that any of the "infinite scroll" websites should have to be able to explain in plain English why they show you any particular item, including ads.
"The advertiser gave us a list that has the email address you used to register."
"You are a female in Orange county between 24 and 26."
"This is a post from your friend presented in chronological order."
"Tom from your uploaded contacts clicked on this ad."
And if they can't explain it, then they can't show it. So "our opaque neural network decided this" doesn't cut it.
Kind of like how you can click in the tip-right corner of a Facebook add and get a very obtuse explanation for each ad.
If I have to make a guess it’s much more likely that their deep learning algorithms learned nudity is more engaging (without them using that as a feature).
This obviously doesn’t mean the platform isn’t responsible. I agree we need to have regulations that require these companies to ensure their algorithms are “inclusive”.
