too many connections

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • c0bra
    Senior Member
    • Apr 2001
    • 294

    too many connections

    We're currently being inundated with email messages with mysql errors reporting "too many connections" error messages for users on our vbulletin board. Right now we have around 120 users constantly online our boards 24/7. This can reach around 250-280 during peak times.

    I've optimised our my.cnf file as follows (all recommendations taken from eva2000s MANY posts on this subject )

    [client]
    port = 3306
    socket = /tmp/mysql.sock

    [mysqld]
    port = 3306
    socket = /tmp/mysql.sock
    skip-locking
    set-variable = max_connections=1000
    set-variable = key_buffer=16M
    set-variable = myisam_sort_buffer_size=64M
    set-variable = join_buffer=3M
    set-variable = record_buffer=3M
    set-variable = sort_buffer=4M
    set-variable = table_cache=1024
    set-variable = thread_cache_size=512
    set-variable = wait_timeout=3600
    set-variable = connect_timeout=20

    [safe_mysqld]
    open-files-limit=8192

    [myisamchk]
    set-variable = key_buffer=384M
    set-variable = sort_buffer=384M
    set-variable = read_buffer=16M
    set-variable = write_buffer=16M

    ---
    <end paste>

    We've upped the maximum connections to 1000 because 500 wasn't apparently enough. Yesterday we had a major crash causing a corruption of our database - thankfully fixed once again from reading one of eva2000s earlier posts.

    We're using perisistent connections to the database because this seems to reduce high load average from users who use our forums to obtain live scores (click refresh every 10 seconds ARGH I know!) When we wernt using persistent, we had a load average reach 100.0!

    We have tried everything we can think of to fix this problem but.... some 3000 email floods later I'm throwing this out to you guys for suggestions.

    The server is a 1GHz AMD with 1GB RAM. Its hosting both the database and the PHP files. It also houses a few other relatively low traffic sites.
  • George L
    Former vBulletin Support
    • May 2000
    • 32996
    • 3.8.x

    #2
    please provide the following

    1. your server specs, such as mysql and php version
    2. if possible how mysql was compiled/installed
    3. your top stats
    4. your mysql configuration variables located at /etc/my.cnf or c:\my.cnf if on Windows server if you don't have that file you need to log into telnet and as root user type

    mysqladmin -u root -p variables

    copy and paste output here

    5. your mysql extended-status output either still telnet as root user type

    mysqladmin -u root -p extended-status

    copy and paste output here

    or preferred is to installed extended-status output script which is located at http://vbulletin.com/forum/showthread.php?threadid=3477

    and post url to that here

    6. oh and is your vB the only thing on the server? or other scripts? sites?

    7. how many average and max concurrent users on your vB forum ?

    upping max_connections to 1000 is pointless in your case as you wouldn't go near that figure and only end up using/allocating more total memory to mysql

    the answers to some of the above questions will help me determine what it is
    :: Always Back Up Forum Database + Attachments BEFORE upgrading !
    :: Nginx SPDY SSL - World Flags Demo [video results]
    :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

    Comment

    • c0bra
      Senior Member
      • Apr 2001
      • 294

      #3
      Sorry for taking so long to respond, here are the answers:

      1. MySQL is 3.23.36, Apache is 1.3.19 and PHP 4.06.
      2. It's a plesk installation.
      3.

      3:01am up 2 days, 11:02, 1 user, load average: 0.50, 0.66, 0.64
      334 processes: 332 sleeping, 2 running, 0 zombie, 0 stopped
      CPU states: 10.3% user, 13.8% system, 0.0% nice, 75.7% idle
      Mem: 1012356K av, 928456K used, 83900K free, 468K shrd, 259616K buff
      Swap: 265032K av, 4228K used, 260804K free 426428K cached

      Is that what you wanted?

      4:

      ansi_mode OFF
      back_log 50
      binlog_cache_size 32768
      character_set latin1
      character_sets latin1 dec8 dos german1 hp8 koi8_ru latin2 swe7 usa7 cp1251 danish hebrew win1251 estonia hungarian koi8 _ukr win1251ukr greek win1250 croat cp1257 latin5
      concurrent_insert ON
      connect_timeout 20
      delay_key_write ON
      delayed_insert_limit 100
      delayed_insert_timeout 300
      delayed_queue_size 1000
      flush OFF
      flush_time 0
      have_bdb NO
      have_gemini NO
      have_innobase NO
      have_isam YES
      have_raid NO
      have_ssl NO
      init_file
      interactive_timeout 28800
      join_buffer_size 3141632
      key_buffer_size 16773120
      large_files_support ON
      locked_in_memory OFF
      log OFF
      log_update OFF
      log_bin OFF
      log_slave_updates OFF
      long_query_time 10
      low_priority_updates OFF
      lower_case_table_names 0
      max_allowed_packet 1048576
      max_binlog_cache_size 4294967295
      max_binlog_size 1073741824
      max_connections 4000
      max_connect_errors 10
      max_delayed_threads 20
      max_heap_table_size 16777216
      max_join_size 4294967295
      max_sort_length 1024
      max_user_connections 0
      max_tmp_tables 32
      max_write_lock_count 4294967295
      myisam_recover_options OFF
      myisam_sort_buffer_size 67108864
      net_buffer_length 16384
      net_read_timeout 30
      net_retry_count 10
      net_write_timeout 60
      open_files_limit 0
      port 3306
      protocol_version 10
      record_buffer 3141632
      query_buffer_size 0
      safe_show_database OFF
      server_id 0
      skip_locking ON
      skip_networking OFF
      skip_show_database OFF
      slow_launch_time 2
      socket /tmp/mysql.sock
      sort_buffer 4194296
      table_cache 1024
      table_type MYISAM
      thread_cache_size 512
      thread_stack 65536
      transaction_isolation READ-COMMITTED
      timezone GMT
      tmp_table_size 1048576
      tmpdir /tmp/
      version 3.23.36
      wait_timeout 3600

      +--------------------------+------------+
      | Variable_name | Value |
      +--------------------------+------------+
      | Aborted_clients | 291 |
      | Aborted_connects | 2 |
      | Bytes_received | 589966400 |
      | Bytes_sent | 334274094 |
      | Connections | 25902 |
      | Created_tmp_disk_tables | 85 |
      | Created_tmp_tables | 43585 |
      | Created_tmp_files | 0 |
      | Delayed_insert_threads | 0 |
      | Delayed_writes | 0 |
      | Delayed_errors | 0 |
      | Flush_commands | 1 |
      | Handler_delete | 24715 |
      | Handler_read_first | 376343 |
      | Handler_read_key | 26166342 |
      | Handler_read_next | 58905072 |
      | Handler_read_prev | 0 |
      | Handler_read_rnd | 31412326 |
      | Handler_read_rnd_next | 1218391938 |
      | Handler_update | 459306 |
      | Handler_write | 4080466 |
      | Key_blocks_used | 15582 |
      | Key_read_requests | 112136434 |
      | Key_reads | 30041 |
      | Key_write_requests | 217311 |
      | Key_writes | 202786 |
      | Max_used_connections | 218 |
      | Not_flushed_key_blocks | 0 |
      | Not_flushed_delayed_rows | 0 |
      | Open_tables | 146 |
      | Open_files | 214 |
      | Open_streams | 0 |
      | Opened_tables | 152 |
      | Questions | 3579477 |
      | Select_full_join | 87 |
      | Select_full_range_join | 0 |
      | Select_range | 871447 |
      | Select_range_check | 0 |
      | Select_scan | 419014 |
      | Slave_running | OFF |
      | Slave_open_temp_tables | 0 |
      | Slow_launch_threads | 0 |
      | Slow_queries | 1 |
      | Sort_merge_passes | 0 |
      | Sort_range | 874985 |
      | Sort_rows | 32205826 |
      | Sort_scan | 267996 |
      | Table_locks_immediate | 3952053 |
      | Table_locks_waited | 1796 |
      | Threads_cached | 156 |
      | Threads_created | 219 |
      | Threads_connected | 63 |
      | Threads_running | 1 |
      | Uptime | 125846 |
      +--------------------------+------------+

      6. There are four other sites on the server. None of them have much traffic. There is a POLL script on the server too which uses mySQL too. This is called from the frontpage of all the sites. But as I say, the sites are not receiving many unique users at all.

      7. Average of around 120. At peak times this can increase to around 250 - which then causes our server to overload, too many connections, and a lot of database error emails being generated.

      I hope that little lot covers everything. Thanks so much for your time... I'm very grateful.

      Comment

      • George L
        Former vBulletin Support
        • May 2000
        • 32996
        • 3.8.x

        #4
        hmm you have your max_connections set at 4000 which is too high but it shouldn't be causing too many connections errors as you're only used 218 max_used_connections

        try replacing /etc/my.cnf contents or creating that file and add this to it and restart mysql

        [client]
        port = 3306
        socket = /var/lib/mysql/mysql.sock

        [mysqld]
        port = 3306
        socket = /var/lib/mysql/mysql.sock
        skip-locking
        set-variable = max_connections=600
        set-variable = key_buffer=16M
        set-variable = myisam_sort_buffer_size=64M
        set-variable = join_buffer=3M
        set-variable = record_buffer=3M
        set-variable = sort_buffer=4M
        set-variable = table_cache=1024
        set-variable = thread_cache_size=512
        set-variable = wait_timeout=3600
        set-variable = connect_timeout=10

        [safe_mysqld]
        open-files-limit=8192

        [mysqldump]
        quick
        set-variable = max_allowed_packet=128M

        [myisamchk]
        set-variable = key_buffer=128M
        set-variable = sort_buffer=128M
        set-variable = read_buffer=16M
        set-variable = write_buffer=16M
        :: Always Back Up Forum Database + Attachments BEFORE upgrading !
        :: Nginx SPDY SSL - World Flags Demo [video results]
        :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

        Comment

        • c0bra
          Senior Member
          • Apr 2001
          • 294

          #5
          I applied your suggestions to my.cnf, restarted, this morning we have 180 users online and the load average has shot up to 28.73... obviously the boards are veryyyyyyyyyyyyy slow...

          here is the output of top:
          1:01pm up 2 days, 21:02, 1 user, load average: 28.73, 30.24, 16.74
          542 processes: 529 sleeping, 13 running, 0 zombie, 0 stopped
          CPU states: 15.8% user, 4.3% system, 0.0% nice, 10.7% idle
          Mem: 1012356K av, 991812K used, 20544K free, 468K shrd, 221364K buff
          Swap: 265032K av, 12240K used, 252792K free 195696K cached

          here is the extended status:

          +--------------------------+------------+
          | Variable_name | Value |
          +--------------------------+------------+
          | Aborted_clients | 48 |
          | Aborted_connects | 0 |
          | Bytes_received | 108729037 |
          | Bytes_sent | 1695414721 |
          | Connections | 5242 |
          | Created_tmp_disk_tables | 2 |
          | Created_tmp_tables | 7378 |
          | Created_tmp_files | 0 |
          | Delayed_insert_threads | 0 |
          | Delayed_writes | 0 |
          | Delayed_errors | 0 |
          | Flush_commands | 1 |
          | Handler_delete | 4553 |
          | Handler_read_first | 69681 |
          | Handler_read_key | 4910749 |
          | Handler_read_next | 12672348 |
          | Handler_read_prev | 0 |
          | Handler_read_rnd | 6529209 |
          | Handler_read_rnd_next | 283048510 |
          | Handler_update | 55747 |
          | Handler_write | 633549 |
          | Key_blocks_used | 13161 |
          | Key_read_requests | 21059616 |
          | Key_reads | 12731 |
          | Key_write_requests | 39689 |
          | Key_writes | 37245 |
          | Max_used_connections | 209 |
          | Not_flushed_key_blocks | 0 |
          | Not_flushed_delayed_rows | 0 |
          | Open_tables | 185 |
          | Open_files | 243 |
          | Open_streams | 0 |
          | Opened_tables | 191 |
          | Questions | 663180 |
          | Select_full_join | 2 |
          | Select_full_range_join | 0 |
          | Select_range | 156025 |
          | Select_range_check | 0 |
          | Select_scan | 78957 |
          | Slave_running | OFF |
          | Slave_open_temp_tables | 0 |
          | Slow_launch_threads | 0 |
          | Slow_queries | 4 |
          | Sort_merge_passes | 0 |
          | Sort_range | 158137 |
          | Sort_rows | 6691656 |
          | Sort_scan | 49234 |
          | Table_locks_immediate | 733323 |
          | Table_locks_waited | 657 |
          | Threads_cached | 33 |
          | Threads_created | 210 |
          | Threads_connected | 177 |
          | Threads_running | 1 |
          | Uptime | 30229 |
          +--------------------------+------------+

          Is there anything more you suggest I do?

          Comment

          • George L
            Former vBulletin Support
            • May 2000
            • 32996
            • 3.8.x

            #6
            from the extended-status output i'm sure the server load is not related to mysql since those stats show mysql is performing well
            :: Always Back Up Forum Database + Attachments BEFORE upgrading !
            :: Nginx SPDY SSL - World Flags Demo [video results]
            :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

            Comment

            • ubbuser
              Senior Member
              • Jan 2002
              • 716

              #7
              Hi eva2000 I noticed he mentioned something about repairing database in case of crash, could you please tell me where that post of yours is? By the way, do you repair/optimize the database on regular basis & if so, how? thx as usual!

              Comment

              • George L
                Former vBulletin Support
                • May 2000
                • 32996
                • 3.8.x

                #8
                Originally posted by ubbuser
                Hi eva2000 I noticed he mentioned something about repairing database in case of crash, could you please tell me where that post of yours is? By the way, do you repair/optimize the database on regular basis & if so, how? thx as usual!
                come on it's obvious the info is in my sig











                in over 12 months i have had only 2 or 3 minor corruptions in my database

                but i have repair heaps of corrupted databases for vB licensed users alot of it due to their hosts improper shutdown or management of their server's mysql database
                :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                :: Nginx SPDY SSL - World Flags Demo [video results]
                :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                Comment

                • ubbuser
                  Senior Member
                  • Jan 2002
                  • 716

                  #9
                  Eva2000 thx!

                  c0bra I'm no expert but it seems that for 180 conc. users your # of 542 processes running on Linux is abnormal.

                  Comment

                  • George L
                    Former vBulletin Support
                    • May 2000
                    • 32996
                    • 3.8.x

                    #10
                    no probs

                    i think the 500+ processes alot would be http so maybe apache settings in http.conf need tweaking

                    what maxclient setting is the server set to in httpd.conf ?
                    :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                    :: Nginx SPDY SSL - World Flags Demo [video results]
                    :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                    Comment

                    • ubbuser
                      Senior Member
                      • Jan 2002
                      • 716

                      #11
                      Eva2000

                      You have open-files-limits=

                      But the show variables have open_files_limits as the variable (note the underscore vs. the hyphen). I've been using the open-files-limits in my MY.CNF file... curious is that any prob? I have used both versions and still the open_files_limits remains 0.

                      Bye.

                      Comment

                      • ubbuser
                        Senior Member
                        • Jan 2002
                        • 716

                        #12
                        Yep, from this URL



                        Quote:

                        open_files_limit If this is not 0, then mysqld will use this value to reserve file descriptors to use with setrlimit(). If this value is 0 then mysqld will reserve max_connections*5 or max_connections + table_cache*2 (whichever is larger) number of files.

                        You should try increasing this if mysqld gives you the error 'Too many open files'.
                        I'm no expert but it seems this variable name is a problem too... I'm not even sure if it's open_files_limit or open-files-limit

                        Comment

                        • ubbuser
                          Senior Member
                          • Jan 2002
                          • 716

                          #13
                          Heh, sorry the prob is too many connections, not too many open files Ignore plz.

                          Comment

                          • George L
                            Former vBulletin Support
                            • May 2000
                            • 32996
                            • 3.8.x

                            #14
                            Originally posted by ubbuser
                            Yep, from this URL



                            Quote:



                            I'm no expert but it seems this variable name is a problem too... I'm not even sure if it's open_files_limit or open-files-limit

                            doh.. try open_files_limit
                            :: Always Back Up Forum Database + Attachments BEFORE upgrading !
                            :: Nginx SPDY SSL - World Flags Demo [video results]
                            :: vBulletin hacked forums: Clean Up Guide for VPS/Dedicated hosting users [ vbulletin.com blog summary ]

                            Comment

                            • c0bra
                              Senior Member
                              • Apr 2001
                              • 294

                              #15
                              I was actually thinking the httpd.conf file might have something to do with our problem yesterday but never got a chance to have alook at the default setting. Then I just looked, and well.....

                              MaxClients 2048

                              What would you recommend I change this to?

                              Comment

                              widgetinstance 262 (Related Topics) skipped due to lack of content & hide_module_if_empty option.
                              Working...