The domain https://fediverse.observer/ redirects to sedo now.
On October 11th these two commits where made to the repo:
this will be my final release, glad to have contributed since 2011 to some fediverse stats, https://fedidb.com/ is a great resource with more interest
this software is no longer developed
Sad to have lost such a great resource.
That is interesting to know. It doesn’t look like lemmy.world has opted out, judging by their robots.txt.
Oh, that’s a thought. I didn’t think of checking robots.txt.
Lemmy.today does have a blanket ban on all bots in its robots.txt, which I assume was put into place to help try to reduce (sporadically severe) scraping load. I’m not sure that the admin, @mrmanager@lemmy.today, realized that it might have cut it out of Lemmy statistics gathering.
Yeah I know, but its worth it. :)
I knew to check robots.txt because of fedidb’s Data Sources page, which specifies what data they get, their user agent, and how to opt-out via robots.txt.
One thing I’m noticing in regards to lemmy.world, I wonder if they ever disallowed access to the nodeinfo endpoint in their robots.txt in the past. They don’t currently, but the wording suggests that having that rule at any point will forever remove the instance from fedidb. Which could explain what we’re seeing for that instance.