14 August 2007

plesk won't start after tweaking mysql

i've had this happen to me again the other week and as always, felt like kicking myself for not remembering.

we do a lot of performance tuning on mysql, and as a result, we almost always manage to get plesk mangled. not its fault, it just can't run without mysql, doh.

not of preference, but plesk happens to be there to make handling domains easier, as a single editing point when you want to change something and not worry about missing something out.

so there's a custom /etc/my.cnf that we use, and with changes to limits done (both on mysql and via ulimit), you'd think that a reload of mysqld would do. (we use InnoDB btw)

normally, that's all you need to have mysqld read off the new settings. if you've grown so dependent on plesk (like i have), you would not notice that mysqld didn't actually start, and you will get an error when you load plesk, thinking that it's a plesk error:


ERROR: PleskMainDBException
MySQL query failed: Incorrect information in file: './psa/misc.frm'

0: /usr/local/psa/admin/plib/common_func.php3:218
db_query(string 'select param, val from misc')
1: /usr/local/psa/admin/plib/common_func.php3:497
get_param(string 'mysql41_compatible')
2: /usr/local/psa/admin/plib/common_func.php3:394
db_set_names()
3: /usr/local/psa/admin/plib/common_func.php3:373
db_connect_real(string 'localhost', string 'admin', string '********', string 'psa')
4: /usr/local/psa/admin/plib/common_func.php3:353
db_connect()
5: /usr/local/psa/admin/auto_prepend/auth.php3:82


even a restart of the plesk service will not work:


[root@jedi]# /etc/init.d/psa restart
Unable to query parameter by query select val from misc where param='mysql41_compatible': Incorrect information in file: './psa/misc.frm'
Unable to query parameter by query select val from misc where param='mysql41_compatible': Incorrect information in file: './psa/misc.frm'
Mysql error: Incorrect information in file: './psa/key_history.frm'
Processing config directory: /usr/local/psa/admin/conf/httpsd.*.include
Syntax OK
/usr/local/psa/admin/bin/httpsdctl restart: httpd restarted
ERROR 1033 (HY000) at line 1: Incorrect information in file: './psa/sessions.frm'


so this is a mysqld failing to start properly problem. to fix this, go to your mysql datadir and see if the sizes of your innodb log files match those in your .cnf. we usually set innodb_log_file_size = 256M and innodb_log_buffer_size = 256M in /etc/my.cnf.

if they don't match, stop mysqld, remove the ib_logfile0 and ib_logfile1 files, and start mysqld. be careful not to delete the ibdata1 or ibdata2 files or you lose your data. avoid using rm -r ib* (yea, wildcards and fast fingers give sysads their most stupefying moments)

innodb will recreate the log files to match your .cnf and mysqld now becomes available for plesk.

19 comments:

Ecropolis said...

My logs were not recreated. I added the lines specifying the log file size to the my.cnf file because they weren't even there at all.

MySql did restart on the 4th try. I got back up with plesk by restoring a dump file.

But those logs are still missing and I think that's causing a delay when I try to start or restart the server.

Any ideas? I'm on the latest 8.2 version of Plesk

Anonymous said...

Wow, that solved the problem. Thanks so much!

Anonymous said...

@ecropolan: what kind of tables do you use? (sorry, had to make sure we're talking about innodb). can you paste here tail entries of your mysql.err?

@montoya: glad i got to help!

Anonymous said...

We deactivated innodb in the my.cnf to free the RAM, but it seems that Plesk needs it. :-(

Anonymous said...

Any reason WHY this would have happened?

we are running alpha services (extra security audits and patches provided by the planet) and it seems a script they ran or something caused the exact error on our servers.

Anonymous said...

Plesk has a lot of quirkiness because it prefers to have its own configurations kept in its own MySQL database. Touch MySQL and you break Plesk.

I'm posting a more detailed response here very soon.

Anonymous said...

Hi there,

here's my problem:
########
ERROR: PleskMainDBException
MySQL query failed: Incorrect information in file: './psa/misc.frm'

0: /opt/psa/admin/plib/common_func.php3:218
db_query(string 'select param, val from misc')
1: /opt/psa/admin/plib/common_func.php3:515
get_param(string 'mysql41_compatible')
2: /opt/psa/admin/plib/common_func.php3:412
db_set_names()
3: /opt/psa/admin/plib/common_func.php3:391
db_connect_real(string 'localhost', string 'admin', string '***************', string 'psa')
4: /opt/psa/admin/plib/common_func.php3:371
db_connect()
5: /opt/psa/admin/auto_prepend/auth.php3:84
##########

This is the content of my /etc/mysql/my.cnf:
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

skip-bdb

set-variable = innodb_buffer_pool_size=2M
set-variable = innodb_additional_mem_pool_size=500K
set-variable = innodb_log_buffer_size=500K
set-variable = innodb_thread_concurrency=2
[mysqld]
set-variable=local-infile=0

user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-external-locking

key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 128K
thread_cache_size = 8
query_cache_limit = 1M
query_cache_size = 16M

log_bin = /var/log/mysql/mysql-bin.log
# WARNING: Using expire_logs_days without bin_log crashes the server! See README.Debian!
expire_logs_days = 10
max_binlog_size = 100M

skip-bdb
#
# * InnoDB
#


set-variable = innodb_buffer_pool_size=2M
set-variable = innodb_additional_mem_pool_size=500K
set-variable = innodb_log_buffer_size=500K
set-variable = innodb_thread_concurrency=2
[mysqldump]
quick
quote-names
max_allowed_packet = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer = 16M

!includedir /etc/mysql/conf.d/

And this is the content of /var/lib/mysql folder:
h1370310:~# ls -la /var/lib/mysql
insgesamt 28756
drwxr-xr-x 7 mysql mysql 4096 2008-11-08 16:45 .
drwxr-xr-x 31 root root 4096 2008-11-03 19:22 ..
-rw-r--r-- 1 mysql mysql 0 2008-11-03 19:00 debian-5.0.flag
drwx------ 2 mysql mysql 4096 2008-03-08 12:45 horde
-rw-rw---- 1 mysql mysql 18874368 2008-11-05 11:34 ibdata1
-rw-rw---- 1 mysql mysql 5242880 2008-11-05 11:34 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 2008-11-05 11:34 ib_logfile1
drwxr-xr-x 2 mysql mysql 4096 2008-11-08 16:41 mysql
-rw------- 1 mysql mysql 6 2008-02-26 20:07 mysql_upgrade_info
drwx------ 2 mysql mysql 4096 2008-02-26 20:11 phpmyadmin_wyagrwJ09YBN
drwx------ 2 mysql mysql 4096 2008-11-08 16:06 psa
drwx------ 2 mysql mysql 8192 2008-10-24 16:14 sitebuilder3

1. I have stoped mysql by /etc/init.d/mysql stop.
2. Then I renamed ib_logfile0 and ib_logfile1 files to ib_logfile0.orig and ib_logfile1.orig
3. restarted mysql with /etc/init.d/mysql start.

Unfortunately nothing happend. Mysql seems to be running but new ib_logfile0
and ib_logfile1 files have not been created and I still cannot login into plesk

Do you have any ideas? I have searched the internet for a solution and I allways and with your suggestions, but they somehow do not work for me.

Thanks in advance for any help,
zavjah

Unknown said...

thank you thank you thank you! This just saved me doing a full re-install of my whole server (not to mention the backing up of all the stuff on the server :P)

Thanks again

Andy

Anonymous said...

Yeah, this just saved me a re-install.

Thanks a million!

-Jeremy

Anonymous said...

Hi Jon,
Unfortunately I have the same issue.
After I tuned mysql that error occured and I followed your instructions.
But the problem has not been solved.

I guess It is about the innodb thing because my tuned mt.cnf file does not mention about any innodb settings.

Actually It is like this;

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-locking
skip-innodb
query_cache_limit=8M
query_cache_size=256M
query_cache_type=1
max_connections=500
max_user_connections=10
interactive_timeout=20
wait_timeout=20
connect_timeout=6
thread_cache_size=128
key_buffer=16M
join_buffer=1M
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
max_connect_errors=10
# Try number of CPU's*2 for thread_concurrency
thread_concurrency=4
myisam_sort_buffer_size=64M
#log-bin
server-id=1

[mysql.server]
user=mysql
basedir=/var/lib

[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/lib/mysql/mysql.pid
open_files_limit=8192

[mysqldump]
quick
max_allowed_packet=16M

[mysql]
no-auto-rehash
#safe-updates

[isamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M

[myisamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M

How do I have to tune my my.cnf file in order to make plesk work again ?

I appreciate any help!

Sina said...

Yavuz, try to comment the line skip-innodb in /etc/my.cnf.
This would solve your problem.

Sina

Anonymous said...

It worked (need to remove the 'skip-innodb' line too) Thank You!

Ahmet Kemal said...
This comment has been removed by the author.
Ahmet Kemal said...

Thank you for this post. It didn't helped me directly but it was inspirativ for me to do something about Innodb. Yes, I put # on line skip-innodb. That shows us "Plesk uses innodb" I think that is not wrong.

Thanks

Luis Gonzalez said...

Plesk now has an article on this.

http://kb.parallels.com/en/6586

micxz said...

Awesome Fixed my issue w/Plesk! Thanks'

Unknown said...

Great my friend! Thanks a lot!

Rento said...

Fantastic! I searched in google for this error and was happy to land on your site with solution, many many thanks - it solved my problem. It's great to have people sharing solutions for such a problems :)

Anonymous said...

Thank you man for saving my life!!!!