No matter which sort you use (except for new), content is recommended to you by activity. Depending on the sort (active, hot, top) it uses a slightly different mixture of votes/comments/time since post to determine the order.
The only exception is scaled, which boosts a little bit midsized communities, but still doesn’t manage to improve visibility of niche ones.
If lemmy is to truly start having active hobbyist communities instead of being 95% lefty US politics, Shitposts, and some tech stuff, it needs a sort that takes into account the user’s engagement.
For example, if I upvote / comment often in a community, there should be an option to have posts from the community be boosted in my feed, even if it’s a tiny community.
Let’s say I’m subscribed to !world@lemmy.world and !news@lemmy.world because I want to occasionally see news. However, I’m also subscribed to a couple hundred other communities, some of them who don’t manage to get more than a couple upvotes on their biggest posts. And whenever I see them I’m replying/upvoting because I’m passionate about that topic.
My feed shouldn’t be 95% c/news and c/world because those are the most upvoted and commented. I shouldn’t have to scroll down hundreds of posts to find “big” posts in small communities I interact with at any opportunity I get.
That’s why I think it would be beneficial to lemmy if the sort/algorithm took into account your engagement in a way.
It doesn’t have to be complicated, you can have a single number “engagement score” for every community calculated with a basic formula, and that number is used as a boost to the community.
I’m aware that there are some examples of successful niche communities on lemmy. But that’s mainly because either a significant chunk of the lemmy userbase is into that niche (let’s face it the lemmy community is not a representative sample of the world population, we tend to be very similar people), or because the posts on it are simplified image/video type posts which appeal to people who don’t know much about the subject.
I agree that fediverse needs a personalized “explore” page in general. For example, this is the only plus feature of Bluesky over Mastodon (in terms of technology). It is obvious how big difference it makes.
I generally avoid the evil algorithms found on other social networks, but I hope we see that in Lemmy.
Good algorithm should:
- be open-source for users
- be togglable
- boost engagement boost on positive emotions
- be personalized
- promote niche communities
- promote balanced political debate (probably the hardest)
What is quiblr?
Thats friggen neat! I really like my mobile client, but I wish it has that feature. I might have to give it a try
This looks cool, but I would prefer to not pass my credentials to a third party, especially for features that should probably be in the default client.
I think to trying meet all of these without compromises (such as privacy and performance) is basically impossible. How would one boost engagement on positive emotions or personalize without large data mining efforts, model building, and running text classification on every comment or post?
I agree they are good aspirations.
deleted by creator
Blackbox echo chamber generators really should be avoided. They add to the angst and anger of the Internet, and of society.
Community search could be improved. And people should learn to actually use it, rather than being spoon fed whatever some programmer they’ve never met thinks they should eat based on the last 3 things they clicked on.
Community search could be improved.
This isn’t a community search problem. I absolutely should NOT have to intentionally visit every single niche community that I am already subscribed too.
The USER needs a way to control their feed, either by throttling large communities or boosting smaller ones.
deleted by creator
The biggest problem with lemmy for me is the multiple “duplicate” communities.
There should be a feature to combine them at the client level. So the 3 different “privacy” communities could just be viewed as one on my lemmy client
The “duplicate” communities are housed on different websites. Websites that could very well have their own norms, rules, and culture. Lumping them together and treating them as the same thing is just kind of invasive to them, and promotes bad netiquette.
Just pick one that you like best.
Thats why i said client side view. Each servers community doesnt know i’m viewing 3 communities together on my phone and it doesnt affect them
Yeah, but you are still treating them as subsets of a singular whole.
Don’t do that. It’s actively bad for the ecosystem, and will trend things toward mega-community mono-spaces where people just snipe at each other for karma.
Nah. The different character of the communities and their history makes them unique and special, hiding that for broad appeal is unnecessary.
No need to muddy the waters with weird client-side obfuscations, one big one almost always wins and the other gets reposts, while subscribing to both is trivial if one wishes
The balkanization is a massive problem though because instead of one, active, community we have 3 or 4 dead ones. There needs to be a critical mass of users before communities can afford to start splintering, and that just isn’t here.
Piefed solves this with topics kind of neatly. You keep the unique communities but they are all in one place
How are comments handled? Are they combined as well, or are they still fractured?
A few apps have multi-community support where you can group whatever you want, how you want, in one stream. I’m using Summit, but I feel a few other of the bigger apps support it now. I group the AskLemmys, tv/movie communities and different art communities into groups so I can view by category.
deleted by creator
You rang?
I was considering doing a megathread about consolidating various communities hosted on an assortment of instances but it probably would’ve too much.
Multicommunities would also help with that. News communities would not flood your general feed anymore if you were able to have a specific feed for them
This sounds like the sort of thing that’s best solved with a ‘favourite’ option that pushes posts from favourited communities to the top of the feed. No need to get in there and over-complicate it with bespoke weightings or anything.
No matter which sort you use (except for new),
Yes, sorting by new is best. The rest of the post seems irrelevant.
I wish the web ui (and apps) could work like an old fashioned usenet reader, where it would list your subscribed communities and say how many unread posts each one had. I don’t like having all the communities jumbled together. That seems fixable.
Same, I stay away from /all and only follow my subscribed communities by new.
So almost forum like?
Yeah
Sorting by new is usually a waste of my time and has nothing of interest to me.
It works fine for me, everything else is worse.
Then try subscribing to communities that have something of interest to you.
Scaled sort was implemented in 0.19.0, is that not sufficient? https://join-lemmy.org/news/2023-12-15_-/_Lemmy_Release_v0.19.0_-_Instance_blocking,_Scaled_sort,_and_Federation_Queue
I mentioned scaled sort in my post. Yes it boosts communities with less activity (in practice this tends to be midsized communities as I mentioned in my post), but it does so generally. What my post is advocating for is a sort that boosts the communities you tend to engage with a lot, not every community that is less active.
Oh, you can also select to see only subscribed communities, and then apply scaled sort. This is my go-to sort after exploring top-6h for a while.
That’s what I do. As I mentioned in the example in my post.
I see you talked about using scaled, but not scaled in conjunction with subscribed. I reread your post and still don’t see it. Sorry.
Let’s say I’m subscribed to !world@lemmy.world and !news@lemmy.world because I want to occasionally see news.
I have this exact problem and it’s maddening. Fucking “news”, which is mostly just political posts about how shitty Republicans are completely drowns out all of my smaller niche communities!
I don’t know how to fix the problem but the USER needs some way to control their feed. We either need to be able to throttle the larger communities or boost the smaller ones.
Hi, I created the Lemmy client Quiblr which includes a For You feed which constantly evolves with the types of posts you interact with. 100% private and on-device (i.e. no data leaves your device).
On quiblr, you can use the “For You” sort like any other sort option
Nice. I will try it. Do I have to keep Quiblr cookies for the option to work?
Quiblr’s For You feed should still work without cookies. Let me know if it works for you
Maybe I have settings wrong or something. But is it possible to get a for you feed with only my subscribed communities?
Not at this time. I built the For You feed to be flexible for both signed in and not-signed in users. I can look into adding it as a feature, but it may take a while because Im working to finish the native mobile apps at the moment
cheers.
IMO would be a cool feature.
Because I often get posts I don’t have a clue about because I’m not subscribed to that community, take like 15 secs to understand what the hell is on my screen, and then the for you algorithm starts pushing that community even more because I stayed on it a while.
Ahh I see. If that happens in the future, you can also click the 3 dots on the post and press “Show Less”. It will add more weight so it rise up in the For You feed
Try Quiblr. It’s a lemmy client with exactly the features you ask for. It checks your engagement, and filters and sorts your feed based on what it learned from your habits.
Just replying here with the community link !quiblr@lemmy.world
because it doesn’t seem to be federated with my instance, it will once I subscribe though.
Can I use it on Android?
I haven’t tried it, but I think it’s on android. Might be a PWA, not sure.
Weird question, but does it not have a Subscribed view? I can’t find it anywhere
PieFed has some features that I find helpful in this regard.
One is the Categories of Communities. You’ll only see News in the News category or in the non-category search methods (standard Subscribed/All/New/etc.).
Another is the ability to follow - and arguably more important unfollow - everything. Including communities, posts, people, etc. I once made the mistake of replying to a comment in ChapoTrapHouse@hexbear.net, and another in Lemmygrad.ml, and the replies kept coming for WEEKS and WEEKS - I almost quit the Fediverse entirely at that point, and I hear that scenario repeated by others as well. But on PieFed, not only can I unfollow any conversation at any time, but unlike Lemmy it also allows a true blocking of all users from any instance you choose (edit: to clarify, I mean without requiring an admin to do it for you and everyone else on the same instance at the same time - a personal defederation that affects nobody else, just like a block, except that Lemmy doesn’t allow blocking users from instances, only communities from instances which is nowhere close to being the same thing).
Ofc nothing is perfect - e.g. I decided to unfollow poetry@lemmy.world bc I don’t want to read 5 of those in a row every morning, and rather would want to savor just one at a time. Also it would be nice to separate comment replies (that seem more urgent, for the sake of an active conversation) from e.g. a new post that you haven’t seen yet? But for a true niche community, with let’s say less than a handful of posts every day and you want to be notified about every single one? It seems perfect for that.
The UI for PieFed needs far more polish, especially outside of the narrow range of short comments on posts with few of those in number. e.g. far too often the existing notifications don’t work as the reply to your comment got buried away onto some other page entirely, in an effort to streamline reading but then that not interacting well with the newer (unfinished?) notifications feature. But while it lacks much polish that Lemmy’s UI and apps have, it also has so many features like those mentioned above that Lemmy lacks as well, and may move faster in its development due to using a more common programming language. It is so nice to have choices to pick from:-).
Otherwise on Lemmy you could make alts, like one subscribed to News communities, another for Memes and Shitposts, etc. Blocking users would get super annoying bc you’d have to do it multiple times. Blocking the largest news communities and the accounts that usually fill them, and then sorting by New can help, but it requires enormous curation efforts to get there and even then falls far short of what you asked - e.g. you also, still have to bookmark or otherwise check each one of your highly active communities one by one (edit: I mean niche ones here, bc the chances of seeing a post there on New can still be slim, if you are concerned about seeing EVERY post there rather than just find something to read from across hundreds of subscribed communities, so different solutions for different workflows).
Lemmy is great for checking memes, reading tech or politics news, and liking Linux - but for everything else it needs improvements to be made to support. I’m not going back to Reddit though:-).
I think that Lemmy would benefit from a tag system, one that allows both adding tags to communities but also to posts, the upshot is that these could be handled like hashtags on other federated platforms like Mastodon. Lemmy already does this with posts in a community, but it’s just a hashtag of the community name, would be good if users could add their own tags.
Well actually… I neglected to mention that PieFed has hashtags already. Here’s an example post showing them. Note the categories at the top and the hashtags below the post. Click them and they seem to work perfectly.
That’s great, hopefully Lemmy can support something like that soon. I know the devs said following tags is out of the question but I don’t think that means content shouldn’t have tags.
I mean, theoretically I could put hashtags right here and now. #seewhatimean? And if you copied and pasted them into the search bar you could theoretically use them to find other content that also used the same ones. But the whole entire point is to allow the submitter to make their content easier to link up with other content, so that you can simply see something, click it, and instantly find other similar things? It’s really not hard at all, and like PieFed already has it. It sounds like the Lemmy devs simply don’t care about that style, hence it won’t be done. Unless someone wants to donate their time to make it happen I suppose?
Yeah it would be so posts under those same tags get indexed together. Also probably wouldn’t be worth it to enable it in comments. It would probably be an extra field in the post where you add tags, not just typing a #hashtag.
Right, that’s how it works on PieFed now: only in posts but not comments, as separate fields not just the pound sign.
There’s plenty of communities where I have no interest in discussing things with the masses. I like smaller communities with thoughtful posters who care more about the subject matter than being heard. That’s the beauty of Lemmy. Not everything has to be perfect for everyone.
Engagement does not exclusively mean commenting or posting; voting is also engagement. If you just want to lurk, why have an account in the first place?
I didn’t say I wanted to lurk. I’m just saying I find beehaw’s Technology community more to my preference than LW’s
That’s fair, I guess I just don’t see the connection to OP. From how you phrased this I assumed you were disagreeing.
I’d rather be allowed to reverse sort on any of the algorithms. It would show content with less engagement and probably and up pushing up content from those niche communities you speak of.
I’d love custom feeds, it was the one of the things I really liked about reddit
Nice profile picture!
Isn’t that the difference between “active” and “hot?”
One is just upvotes; the other is votes and comments.
I don’t think it would help, to be honest. A dead community not being posted to still won’t show up on your feed if you sort by engagement.
read my post, not just the title.
Nah. Algorithms, especially personalized as a way of sorting a feed are just a shite idea. Maybe one of the iOS apps will add something like that but if anything Lemmy being different is a selling point. I got two friends on fedi by telling them that “it has no algorithm” which is a simplification of course but you get the gist. It also really hits home that this is not a corporate product.
When do you start counting it as an algoritm.
The current sorts (except new) are based on formulas, does suddenly adding a personal engagement variable into the formula make it an algorithm?
It’s arbitrary but something like
SELECT * FROM posts WHERE datePosted < ( currentDay() - 7) ORDER BY upvotes;doesn’t feel like an algorithm as it is now used in common parlance to me.A simple quantitative analysis of an existing metric and (upvotes in the above super simplified example) is just not really the same thing in practice as say: multiple linear regression of hidden backend engagement metrics gathered through things like cursor movements to pick a suggested video that is predicted to optimize the best for watch time and CTR from a list of videos on a balance of personalized and generalized (through tracking trends amongst demographics) favourites topics and other qualities classified and categorised by a whole other black box involving all sorts of classifier models from text to images and so on.
Idk, I didn’t take algorithms in CS at uni, so this is just a layman’s two cents. I’m happy to be explained to why this isn’t a valid perspective.
I did take Algorithms.
The definition we learned (let me know if I am wrong) is that an algorithm is a concrete set of steps to accomplish some goal in a finite amount of time given legitimate inputs.
Although in practice we use this more for stuff with a math formula and/or stuff you code. “Given the input of the world, if your eyes see it is raining outside, grab the umbrella from your closet. If you don’t see the umbrella, search for it. If the search takes 5+ minutes, just go to your destination” is an algorithm for trying to not get rained on, but in practice nobody’s going to be using that word that way.
I think the definition used online today is “some computer code that I can’t reliably determine the input/output of, that is used to my/society’s disadvantage in an exploitative way.”
Words evolve, and the word you learn in academia sometimes also gets used in real life and its usage changes in real life from what you would use in academia. And sometimes academia keeps using it that way, and real life keeps using it their different way, and so you use the same word while talking about slightly different concepts. And sometimes people in real life use it the academic way, others don’t, making things even more confusing… you just have to be aware people are using the same word to talk about two different things (or in this case, one group uses the word to talk about an unpleasant subset of the thing the other group uses the word to talk about) and clear up that misunderstanding.
Algorithms are fine when an algorithm is open, clear and optional. The default sort for many apps/UI is “Active”, that’s an algorithm. It may be a simple one, but it’s an algorithm.
