A reminder to move to smaller instances for a better experience
A reminder that this constant advice people blindly parrot to install and flock to smaller instance has now created something like 1000 new servers in 50 days that are poorly run and already going offline as quickly as they went online.
Github Issue 2910 is the kind of PostgreSQL problems that the developers ignored for months and people still defend the developer choices to have the code doing real-time counting of every single comment and post for numbers nobody needs to needs done in real-time.
PostgreSQL is voodoo to this project, they do everything they can to avoid going to !postgresql@lemmy.ml community and asking for help, learning 101 about how to fix their SQL TRIGGER logic like Github Issue 2910 spelled out June 4.
A reminder that this constant advice people blindly parrot to install and flock to smaller instance has now created something like 1000 new servers in 50 days that are poorly run and already going offline as quickly as they went online.
I am always advocating for any of the top 25 instances that are not Lemmy.world or Lemmy.ml
For the rest of your post, I don’t know what that has to do with people aggreating on LW.
For the rest of your post, I don’t know what that has to do with people aggreating on LW.
aggregation refers to the lemmy database tables, site_ aggregates, community, person. The SQL TRIGGER logic lemmy_server uses that has been the source of so many crashes the past 60+ days.
Even if the SQL was top notch, it would not be a good thing for 50% of the active users to be on a simple instance. Just makes it easier to take down by any potential attacker.
deleted by creator
I mean having a bunch of new servers is not a problem. Just choose one that’s been up for a while and more stable.
But each additional row in site_aggregates table was causing the instability itself. The SQL code had major flaws. Adding more servers actually made Lemmy crash more.
A reminder that this constant advice people blindly parrot to install and flock to smaller instance has now created something like 1000 new servers in 50 days that are poorly run and already going offline as quickly as they went online.
And this will always… always be the biggest problem in the FOSS community.
“I dont like X, so I’m going to leave and make my own version of X”
So userbases get spread thin, manpower gets spread thin, developers get spread thin, and the user experiences degrades for everyone until it pushes them back to the bullshit websites and products.
This is exactly what federation is meant to solve: power in numbers without the centralization. Is that so hard to understand?
Sometimes I question why people not in favor of the decentralization are commenting on a Fediverse platform. Why not go to Tildes, Squabbles or another centralized alternative? There is plenty of fish in the sea.
This is another big problem in the FOSS
“If you dare offer valid criticism, then why are you even here? get out and go somewhere else!”
Your criticism is nonsensical. It’s literally criticizing the purpose of the project.
Oh dear, are you gonna start another tissue shortage with all these tears?
Pot calling the kettle black. I offer criticism of your criticism and you throw a hissy fit. Poor child.
Wooosh
Your answer didn’t justify lemmy.world being treated the same as Lemmy as a whole. It’s just a bunch of people who don’t understand federation.
For the rest of your post, I don’t know what that has to do with people aggreating on LW.
And, factually, the project leaders telling everyone to create 1000 new instances and shutting down sign-up on Lemmy.ml caused more performance problems.
They had a bug in their PostgreSQL TRIGGER logic where 1500 instances were updating + 1 comment and +1 post counting instead of WHERE site_id = 1, a single database row. So each new Lemmy server that went online made the table larger and crashes more frequent on lemmy.ml
The amount of disk writing by lemmy was ignored
The developers of Lemmy seem to make every effort they can to avoid using Lemmy itself to discuss their !postgresql@lemmy.ml learning 101. They have made massive mistakes in SQL TRIGGER logic that they avoided to such a degree that their social motives are in question now. Github Issue 2910 was opened June 4, almost a month before the Reddit API deadline, and they ignored it. Just like they hang out on Matrix Chat and don’t use Lemmy their own self to discuss code.
They have cultivated a kind of voodoo attitude towards PostgreSQL where people using Lemmy won’t actually scrutinize the Rust code or PostgreSQL tuning parameters.
You can also just have multiple accounts. I have one on lemm.ee just for this reason.
Definitely!
Yeah, I have .world where I first signed up. It occasionally had some issues so I added another that I just mostly read on if the other isn’t working
It’d be nice to export/import subscriptions but I’m sure eventually the servers will get past it
You can use that script https://github.com/CMahaff/lasim
I’m afraid that as long as you have to go to github and run such scripts for these things, Lemmy will stay a nerd-only place.
Hopefully it gets integrated into the website at some point
It’s funny, at this point I don’t feel like I even have a main account. I just randomly pick an instance every time I go Lemmying
I’m trying to pick ones with meaningfully different aims so that I can try to keep some sort of separation in my head. Otherwise I’d quickly lose track of what conversation having where, with whom and why.
Same, I have accounts on lemmy.ml, lemmy.world, and Kbin.social. I’m just waiting for a good app to be compatible with Kbin. Right now all the apps coming out are geared toward Lemmy.
I don’t remember Kbin having an API, so that might explain why
I learnt my lesson to have multiple accounts after vlemmy
That was my main. The instance vanished faster than cotton candy being held by a raccoon standing in a pond.
Oddly specific, but very accurate
Is there a readily available tool that can export/import community subs? I have two accounts on different instances but would like them to have to same subscriptions
I’ve not used it myself but heard good things about https://github.com/CMahaff/lasim
Awesome, thank you!
Does this work on android?
I don’t think so, you might be out of luck there.
Lemmy.one, my instance of choice, has been down since Thursday - just a reminder that smaller instance isn’t always the solution. Having a few solid account choices on multiple instances is the way to go.
It’s a bit frustrating though. If I didn’t know better about the fediverse, I would’ve thought Lemmy in general was down, with no heads up or somewhere to see what’s going on.
Lemmy One still down by the way
Does anyone know why?
I have a few for this reason. Is there, or perhaps will there be a way for subscriptions to the shared across accounts?
Maybe this is already I thing and I’m oblivious.
I haven’t had a chance to try it yet but here is this:
Lemmy.dbzer0.com is pretty stable, both in terms of release timing/server uptime, and also in terms of not flooding your feed with “politically instable” instances on both sides. Avoid the echo chambers.
One might argue that such a selection is, itself, an echo chamber.
That’s my favorite instance as well but I’m not sure what you mean by politically instable on both sides
Quoting myself from a previous post:
First of all, it’s really fine to stay on LW for now, no need to rush anything. But if at some point you have some time for this, then read the following.
So, to pick your instance, you can have a look at https://lemmy.fediverse.observer/list, filter by “1m” to see what are the most popular ones. As you can see, with a 27433 monthly users, Lemmy.world is by far the most popular, which is why you might experience some issues from time to time.
You should have a look at the next instances on the list. Short story: lemm.ee, sh.itjust.works, lemmy.one, sopuli.xyz and reddthat.com are solid choices.
You are looking at instances with quite a lot of people (the more people help with filling your “All” feed), just not the most populous one (lemmy.world), the original one (lemmy.ml), and instances that are too specific, either due to country or specific focus.
Long story:
spoiler
- lemmy.ml is the original insance, also quite crowded, not really the best choice
- lemm.ee can be nice, you can have a look at it and see how fast it is for you. The admin communicates a lot and is very helpful.
- sh.itjust.works had some rough time in the last few days. You might also not like the name, that’s okay.
- beehaw.org does not federate with the big instances, so if you go there, you will be in their own space. It can a valid choice, but please have a look at their guidelines first, they tend to moderate a lot. Can work for you, or not.
- feddit.de, lemmy.ca, discuss.tchncs.de, feddit.uk, aussie.zone are country specific instances, so probably not interesting to you if you are not from there
- lemmynsfw is a NSFW instance, probably not the one you want to move to
- programming.dev is an instance focused on programming
- lemmy.blahaj.zone is a pro queer instance
.
To migrate your settings (including subscriptions and blocked instances), you can use that script: https://github.com/CMahaff/lasim
LASIM author here, ironically on my own alt: Just an FYI that support for Lemmy 0.18.3 is not yet out, but keep an eye out for it soon (I have it working on a branch but I need to test it more before release).
This is the first breaking API change since it’s creation, so here are the limitations:
- Old version (0.1.2) will only support API 0.18.1 and 0.18.2
- New version (0.2.0) will only support 0.18.3 (and above until there are more breaking API changes)
- Profiles downloaded with 0.1.2 (and below) will automatically be converted to work with 0.2.0.
So that all means:
- You can use the old LASIM to migrate between 0.18.2 Lemmy instances
- You can use the new LASIM to migrate between 0.18.3 Lemmy instances
- You can use the old LASIM to download from an 0.18.2 instance then use the new LASIM to upload to a 0.18.3 instance
- You cannot use the new LASIN to download from a 0.18.3 instance and then the old LASIM to upload to a 0.18.2 instance (unless you are comfortable doing some manual work editing the JSON file so “old LASIM” understands it).
This will be true of every release with breaking API changes.
EDIT: PR is out. Once it builds, I’ll publish a new release! https://github.com/CMahaff/lasim/pull/21
EDIT 2: Release is published! https://github.com/CMahaff/lasim/releases/tag/v0.2.0
Thanks for all your hard work on this. I’ve used lasim to transfer my subscriptions to my other accounts on other instances when the lemmy.world downtimes happened last week. I love the portability of the tool.
Thanks for your kind words, I’m glad it has helped you!
Thank you for the heads up! You might want to add that to the ReadMe on your Github
Removed “and above” from page and instead added a note to always get the latest version if your version isn’t listed as supported explicitly.
Great, thank you!
I think there is a huge misconception many people have that a larger instance is more likely to stay around, but due to the nonlinear costs involved in hosting fediverse instances this is not true.
Basically there is a sweet-spot around a few thousand (~2500) members where costs are low enough for a single admin paying things out of their own pocket long term is possible, but also enough members willing to occasionally donate or contribute otherwise to cover costs.
To be honest, I know it’s a controversial view, but I would almost like to see Ruud and the LW admins block registrations for a while, along with a communication “Have a look at those other instances, they are well managed, you can access all of Lemmy just as well from there”
I’m not looking forward to 5 years from now, where instances like this are the mastodon.social of federated reddit-likes. As much as they should block registrations, I don’t think they will. …but I have a hat on stand by, just in case I need to eat it.
@remindme@mstdn.social 5y
I mean, with 27k monthly active users, they are more prevalent that Mastodon.social. The following instances are so much smaller it’s almost comical.
If anyone’s looking, my instance is open for everyone and I plan on supporting it long-term. https://lemmings.world
geddit.social is a another good server, I have alt there. It’s run by @stux@geddit.social who has his own hosting company and also hosts mstdn.social.
Fwiw, sh.itjust.works is a horrible name, but a great instance lol.
What’s funny though, I’m getting beehaw posts in my all feed since yesterday. No idea if they’ve refederated with us, or if it’s an artifact of connect (my app of choice), or what,
Interesting. You are still on the list of their blocked instances, so I’m not sure: https://beehaw.org/instances
Must be some kind of weirdness with the app then. Thanks for looking at that!
You’re welcome, have a good one!
when I start writing this comment, the post is 47 minutes old. if I understand the linked page properly, lemmy.world has been functional (all green checkmarks) for the past 10 minutes which is the furthest back the data goes. All the other instances are all green except for lemmy.one which is all red. I am assuming that 47 minutes ago, lemmy.world had red boxes?
Maybe a different link would have explained the point better but I don’t really see how a 30 minute (??) server outage during an upgrade is compelling to avoid a large instance. Are you suggesting it’s better to use a server whos admins don’t upgrade? If not, is there really any size of server that would meaningfully avoid this kind of occasional disruption? Seems to me that the dynamism of the environment will inevitably lead to various problems. That’s part of the experience. TBH threadiverse uptime on the whole is pretty impressive for such a ragtag groups of admins and devs.
I have accounts on some smaller servers but they have their drawbacks too. Using a bigger server is more convenient because the people and content is already there. It’s easier. I didn’t plan to use lemmy.world but I ended up making account there to use sometimes.
I think in a year or so the situation might be different. I see the ideological point and I would like it to be true. Maybe the technology will catch up. I think it would be nice to be able to programmatically seed content, but maybe that would be obnoxious to admins.
I didn’t track the timing too closely, but in the last few weeks there have been quite long disruption of service due to DDoS attacks on the largest instances.
I am personally quite tolerant towards Lemmy as a platform in its very infancy, but some other users might want to quit it due to this kind of annoyances, hence my comment about moving to a smaller instance.
the people and content is already there. It’s easier
What do you mean? You can access all of the content in the Threadiverse from whatever instance, modulo defederation, but Lemmy.world defederates quite a few instances too, so that’s valid for both big and small instances. If you are talking about the “All” feed, which will indeed be empty if you are in a 10 people instances (communities need to be subscribed by an instance member for the instance to get the community content), then it’s a valid issue, and that’s why I suggest people to move to one of the 24 biggest instances that are not LW or Lemmy.ml
Lemm.ee, sh.itjust.works, sopuli.xyz, reddthat.com, lemmy.one, your country instance (if it’s big enough). I’m on sopuli since a while now, and I’m very happy with the experience. 680 monthly active users, so the All feed is pretty much identical to the one on Lemmy.world, except the vey niche community I either don’t care about or would already know by myself.
The biggest issue I see with having everyone on LW is that at some point the costs will be too high for the admins. It’s quite a big risk, and that why I’m advocating to use smaller instances.
@Blaze as I understand it, if you are user on a small server, you only see content from communities that others on your sever have previously subbed to previously, or if you do so yourself. And then you only seen content from the moment of subscription on. There is no way to see back prior.
So if you want to use a community like !fediverse it’s OK because its popular and there will be prior subs. but if you are interested in !rockingchairrepair you will miss all prior discussion. Am I incorrect?
Also in practice, from my experiments, there seem to be inconsistencies in how even this works.
When I mean a small server, I mean one of the top 20, which at least a few hundred users.
As I said elsewhere, with a few hundreds users, the chance of you stumbling upon a community that nobody else subscribes to is that either the community is dead, or it just started, or indeed it is very niche.
The only case where you want discussions from the start is the last case, which is very less likely to happen once your instance reaches a few thousands users (which is the scenario most of the people hope once we spread more evenly).
Of course I have one or two other accounts, but I personally like Lemmy.world. They serve as a necessary stress test that shows the devs and admins how to optimize further, and I just like learning admin practices at this scale of a userbase from a work perspective. Plus I don’t want to be on an instance so small they can’t or don’t know how to handle compliance stuff and evaporate if something like that comes up. Not saying I know how to handle all of those situations, that’s the job of someone else at work.
I think that’s one of the issues that the rest of the instances are facing to appear as trustful as LW. LW admins have a long established reputation and experience managing Fediverse services, and provide very good transparency and a large team.
Other instances are usually nowhere close to that (some will be in the future I hope). The question I usually raise when someone start promoting their instance is “how many admins do you have?´What happens if you run under a bus tomorrow (hopefully you’ll stay safe of course)? Is there a back up plan in place?”
Yeah, longevity and name recognition are why I went with sdf.org. They’ve been running many-user services for decades, even if the Lemmy service is pretty new.
ETA: they’ve been around since BBSes. I’m on a wicked nerdy old-school geek instance, and I love my local communities.
Best instance, I hope they make a firefish one.
Yes, very good instance indeed
you should see how many of these big servers were (and still are) running their instances as ‘root’…
I specifically stopped trying to grow my instance to focus on security and sysadmin back-end administrative infrastructure.
Lemm.ee is amazing with a top admin
💯 checking out the admin of a Fediverse instance you want to join is a good idea. Do they communicate with their users? Do they seem responsible ? Do they have any previous experience as a network admin? Lemm.ee was an obvious yes on all counts.
The admin for vlemmy.net was the complete opposite and look how that turned out.
Or just have 10 accounts so you only have downtime when your own internet is down.
And run 2 or 3 uplinks to different ISPs so that you’re never offline.
And setup your own transmission dish so you can instantaneously switch to satellite if all ISPs go dark
Yes, that’s called StarLink. Definitely a good option, if you make your own power.
I’m gonna make my own power! With black jack and hookers!
And their names are Deuterium and Tritium.
Nice. Where is power port on a hooker? Eh, nevermind.
I use photovoltaics, but not insular blackstart-capable battery buffered so when the grid is down I also have no power. Next year, though. Seems it’s 2000 EUR minimum.
As I said elsewhere
There are certain things that are memory intensive and CPU intensive. If you have 10k on one server doing that it really adds up. However having them across a wide range of smaller servers, its not such a big deal.
As a user, you literally lose out on nothing not being on lemmy.world. You can partake in all the same conversations, communities and everything. In fact when lemmy.world is down, you can still see everything and when it comes back up, your posts will synchronize. There’s genuinely no upside to being on lemmy.world. That’s the way the system was designed.
Not sure people will listen though. I will always talk up the amazing admin I have on lemmy.tf, but it’s also worth mentioning that I have a bunch of communities hosted on other instances and each and every one of them is amazing.
Definitely.
working fine for me , actually I have to say lemmy.world has been very stable
Perhaps I open Jerboa too often but in the last week lemmy.world has been down around 4 times for me, sometimes for a short time sometimes for much longer. When I was on lemmy.fmhy.ml it was only down once for me until their domain was seized and that was for the server upgrade.
You are not the only one, the last few weeks have been rough for LW. That’s why I suggest people to moving to other instances, I detailed the process in another comment in this thead (can’t get the comment link to work for some reason)
I’ve found it, thanks 👍
There are 63 instances according to Jerboa.
Do note that this doesn’t actually account for an accurate total of Lemmy instances across the Fediverse. It is a hardcoded list of instances.
Edit: Correction, its a generated list of instances (that gets copied into Jerboa) that have a MAU amount of more than 50 as pointed out below.
That list gets generated based on the lemmy instances tracked by fediverse.observer site that have a monthly active user amount greater than 50.
Fair point! I’ll update my comment to reflect that.
Oh, okay. So how many are there?
My favorite way of discovering new communities / instances on Lemmy is through Lemmyverse and it shows 1,378 instances at the time of my writing!
I think 10 is enough for me.
Wow.
I was on lemmy.world and had a really bad experience. The I moved to aussie.zone and its better than reddit. Never down, commenting, posts everything feels so fast and snappy.
This is why I love small instances as well. I’m on Lemmy.today and it flies.
People seem to be super worried about an instance disappearing for some reason. It has happened to like 3 instances, we are soon 1400 instances.
Also if it would happen, how hard is it to move to another and click subscribe on communities? Takes like 20 minutes if you have many of them.
I really think all this fear is silly. Enjoy Lemmy the way it was meant to be used. Use small instances.
Just waiting patiently to get approved on midwest.social
Hopefully they’ll get back to you soon!
Smaller instances are usually worse, in my experience
Where is this screenshot from? Link me plz :)
Danke!
