Currently I have 103 laravel setups in the same server. The default supervisor conf file for every domain is:
[program:NAME]
command=/usr/bin/php7.4 /home/NAME/web/NAME.com/public_html//source//artisan horizon
autostart=true
autorestart=true
user=NAME
redirect_stderr=true
stdout_logfile=/home/NAME/web/NAME.com/public_html//source//storage/logs/horizon.log
stopwaitsecs=30
numprocs=2
process_name=%(program_name)s_%(process_num)02d
And every laravel .env file is configured to run in different times to avoid having all sites executing stuffs at the same time. for example:
SCHEDULE_SYNC_JOBS="33 * * * *"
SCHEDULE_SYNC_ARCHIVE_JOBS="33 2 * * *"
SCHEDULE_CATEGORIES="33 * * * *"
SCHEDULE_SYNC_USERS="*/5 * * * *"
SCHEDULE_JOBS_FEED="33 2 * * *"
SCHEDULE_ARCHIVE_JOBS="33 2 * * *"
SCHEDULE_ARCHIVE_MANUALLY_JOBS="33 2 * * *"
SCHEDULE_COUNT_JOBS="33 * * * *"
SCHEDULE_CITIES_STATISTIC="33 * * * *"
SCHEDULE_TITLES_STATISTIC="33 * * * *"
SCHEDULE_COMPANIES_STATISTIC="33 * * * *"
SCHEDULE_CITY_JOBS_STATISTIC="33 * * * *"
SCHEDULE_SITEMAP="33 * * * *"
SCHEDULE_SITEMAP_CONTENT_PAGES="33 * * * *"
SCHEDULE_SITEMAP_TITLE="33 1 * * *"
SCHEDULE_SITEMAP_BLOG="33 * * * *"
SCHEDULE_SITEMAP_PAGES="33 * * * *"
HORIZON_MAX_PROCESSES=4
HORIZON_TRIES=1
HORIZON_TIMEOUT=7200
The current memory usage just to run supervisor:
supervisor.service - Supervisor process control system for UNIX
Loaded: loaded (/lib/systemd/system/supervisor.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-02-25 16:27:23 UTC; 39min ago
Docs: http://supervisord.org
Main PID: 2813737 (supervisord)
Tasks: 1459 (limit: 108515)
Memory: 50.5G
This does not make sense to me at all. It does not seems right: 50GB and close to 1500 tasks just to run laravel queue? How can I reduce the supervisor memory usage?
This is an exampled of the loaded processes:
LOGIN 2814004 0.0 0.0 144748 71028 ? S 16:27 0:02 \_ /usr/bin/php7.4 /home/NAME/web/NAME.com/public_html//source//artisan horizon
LOGIN 2816048 0.0 0.0 144752 71104 ? S 16:27 0:02 | \_ /usr/bin/php7.4 artisan horizon:supervisor DOMAIN-nYm1:supervisor-1 redis --delay=0 --memory=128 --queue=commands,sitemap,default --sleep=3 --timeout=7200 --tries=1 --balance=auto --max-processes=4 --min-processes=1
LOGIN 2818638 0.0 0.0 142704 69932 ? S 16:27 0:00 | \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=commands --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-nYm1:supervisor-1
LOGIN 2818640 0.0 0.0 142704 69648 ? S 16:27 0:00 | \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=sitemap --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-nYm1:supervisor-1
LOGIN 2831583 0.0 0.0 144752 71740 ? S 16:35 0:00 | \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-nYm1:supervisor-1
LOGIN 2814008 0.0 0.0 144748 70844 ? S 16:27 0:02 \_ /usr/bin/php7.4 /home/NAME/web/NAME.com/public_html//source//artisan horizon
LOGIN 2816049 0.0 0.0 144752 71108 ? S 16:27 0:02 \_ /usr/bin/php7.4 artisan horizon:supervisor DOMAIN-JBzm:supervisor-1 redis --delay=0 --memory=128 --queue=commands,sitemap,default --sleep=3 --timeout=7200 --tries=1 --balance=auto --max-processes=4 --min-processes=1
LOGIN 2818349 0.0 0.0 145024 71772 ? S 16:27 0:00 \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=commands --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-JBzm:supervisor-1
LOGIN 2818354 0.0 0.0 144752 70608 ? S 16:27 0:00 \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=sitemap --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-JBzm:supervisor-1
LOGIN 2869705 0.0 0.0 144752 71824 ? S 17:00 0:00 \_ /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=default --sleep=3 --timeout=7200 --tries=1 --supervisor=DOMAIN-JBzm:supervisor-1
Someone have any idea about how to reduce the supervisor memory usage? Is related with
--memory=128
parameter?
You might try to lower the memory in your horizon
config but be careful because your jobs might run out of memory and get killed. I'd run some intensive tests before screwing with 103 installs at once. Fyi with a fresh supervisor install and 3 separate queues with a total of 13 tasks I get around 71MB per task (without any memory limit) while you get around 35, which is half.
Sign in to participate in this thread!
The Laravel portal for problem solving, knowledge sharing and community building.
The community