Slow photo requests / inbox requests
Hi !Friendica Admins ,
I'm currently tracing down performance issues, I' currently suffering from on my nodes opensocial.at and friendica.me .
It seems like the main performance problems are
- delivering photos
- [code]/inbox/code] requests
You can see some peeks and some long runners here:
The long runners (10sec) are the /inbox requests, the peaks are delivering photos after opening conversations, network pane, ...
Do you have any hints how I could improve these two types of requests?
The database is currently using ~50GB RAM, I set them as high as possible.
You can enable the asynchronous with
'decoupled_receiver' => true, in your config. Then the inbox requests would be much faster. Then the processing is done via worker processes.For the photos you can define the avatar cache:
But that one is tricky to set up. Best is to store the photos in a folder outside the web folder and then you can define a rule in your webserver that will redirect requests to the avatar folder to that folder.
'avatar_cache' => true,
'avatar_cache_path' => '/path/to/a/folder',But that one is tricky to set up. Best is to store the photos in a folder outside the web folder and then you can define a rule in your webserver that will redirect requests to the avatar folder to that folder.
@Michael Vogel @Philipp Holzer Will these cached avatars be checked if they are outdated? I currently have trouble getting @Tobias 's avatar over here.
Friendica Admins reshared this.
@Michael Vogel @Philipp Holzer why is it necessary to have the folder outside of the web folder?
It will become extremely large over time. So storing in some separate folder might be better for backup purposes or for keeping your web root clean. Also there is an option for an alternate URL. So you could even store these files on some separate web server that only serves static content.
@Michael Vogel @Philipp Holzer is the avatar cache necessary for the workers too or just for the frontend app?
Friendica Admins reshared this.
These two don't depend on each other.
So I do need to redirect only the
/photo/contact route for the avatar cache, don't I?No, the setup process is completely different. You have to have a path (somewhere) that is both readable and writable by the frontend and backend process
This path needs to be reachable when you call
You you setup the file path from above in
The URL path is stored in the contact table fields. So once you defined that URL, you mustn't change it again, since all old requests would fail. This mechanism is powerful, but currently too easy to misconfigure, that's why it is only accessible via the config file.
This path needs to be reachable when you call
https://your.server.tld/avatar (you have to redirect /avatar) or you have to define avatar_cache_url.You you setup the file path from above in
avatar_cache_path and set avatar_cache to true. After you deactivate the caching of the avatars in the admin frontend, it should work.The URL path is stored in the contact table fields. So once you defined that URL, you mustn't change it again, since all old requests would fail. This mechanism is powerful, but currently too easy to misconfigure, that's why it is only accessible via the config file.
Friendica Admins reshared this.
I think I got it. I try it with my friendica.philipp.info node first and check it 
Friendica Admins reshared this.
Have a look at https://github.com/friendica/friendica/blob/develop/src/Contact/Avatar.php for a deep dive.
@Michael Vogel @Philipp Holzer U oh, writable and readable by the frontend, means web server? Do you mean here also through
https://? That's never a good idea.Friendica Admins reshared this.
seems to work : https://friendica.philipp.info/avatar/4c/a1/d7c/53/8b/f08b/cd3c2904ac0bfd56-300.png?ts=1682970014
This image is now served by a completely different nginx than the frontend nginx . Because it's stateless, I'm now able to start more nginx for "just" serving stateless static content!
Brilliant!
This image is now served by a completely different nginx than the frontend nginx . Because it's stateless, I'm now able to start more nginx for "just" serving stateless static content!
Brilliant!
Friendica Admins reshared this.
Old avatars still will be served the old way and will be replaced, once they are updated - which can take weeks. But I guess I wrote some script that speeds up this process.
currently, "movetoavatarcache" is running on my node 17000 from 59000 finished ;)
17000 from 59000 finished ;)@Hypolite Petovan @Philipp Holzer it's a XFS filesystem on a LVM
Hardware:
Hardware:
- SATA as main storage
- 2x NMVe RAID1 as read/write cache (https://manpages.ubuntu.com/manpages/impish/man7/lvmcache.7.html)
Hi, after moving all contact images to 
avatar/, there are still a lot of long running requests, see the image above. Do you have any idea? The overall CPU/RAM is on a normal state.
Friendica Admins reshared this.
xdebugand setmode=trace. Then use e.g.kcachegrindto analyze it, e.g. which method takes the most time. That might be a start! Yes, you normally want to havexdebugdisabled on live (no development) servers. But this is an exception as high load cannot be simulated properly to find the bottleneck.