dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
1845

cowboyro
Premium Member
join:2000-10-11
CT

cowboyro

Premium Member

What happened to MySQL???

I remembered it being a fast, small-memory footprint server...
I installed the latest x64 version on a Server 2012.. it takes almost 450M or RAM with the same configuration as I had with an old version... that was only taking some 60M (ok, that was 32-bit but still...)
On comparison the full-blown MS SQL (x64) only takes 200M. Not only that, but it constantly beats the execution time of MySQL by 10-50% on the same data, on the same machine...
Have the roles changed?

Robert
Premium Member
join:2001-08-25
Miami, FL

Robert

Premium Member

It's better on a unix based system.
aguen
Premium Member
join:2003-07-16
Grants Pass, OR

aguen to cowboyro

Premium Member

to cowboyro
Don't know if you've looked into tweaking the config settings at all with regard to memory or not but you might want to. Also, since Oracle bought MySql a few years back, it hasn't been the same IMHO.

cdru
Go Colts
MVM
join:2003-05-14
Fort Wayne, IN

cdru to cowboyro

MVM

to cowboyro
It grew. Programs don't usuall get smaller when you all sorts of features (whether you use them or not). Windows now takes gobs of memory and a gazillionbytes of disk space? I remember when it ran with 640kB and a few MB of disk space...

ekiM5
Oh Well
join:2001-01-06
/usr/home

ekiM5 to cowboyro

Member

to cowboyro
Agreed unix makes for a better data server. Also mentioned was that the memory config/tuning may require some attention. In the past the base install memory allocations were too small to be of much use. I don't have a "server 2012" to look at so I can't validate the theory. If you want help with tuning - just ask.

cowboyro
Premium Member
join:2000-10-11
CT

cowboyro

Premium Member

It was configured as a "small server" by the installer. Same results as manually installing from the zip file and using the same ini file as I had before for the older version.
What is more annoying is that it consistently performs worse than MSSQL on anything that involves table updates and/or calculations (grouping, joins, filtering). However for plain data selection it is marginally faster when the proper indexes are created.

# Other default tuning values
# MySQL Server Instance Configuration File
# ----------------------------------------------------------------------
# Generated by the MySQL Server Instance Configuration Wizard
#
#
# Installation Instructions
# ----------------------------------------------------------------------
#
# On Linux you can copy this file to /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options
# (@localstatedir@ for this installation) or to
# ~/.my.cnf to set user-specific options.
#
# On Windows you should keep this file in the installation directory 
# of your server (e.g. C:\Program Files\MySQL\MySQL Server X.Y). To
# make sure the server reads the config file use the startup option 
# "--defaults-file". 
#
# To run run the server from the command line, execute this in a 
# command line shell, e.g.
# mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# To install the server as a Windows service manually, execute this in a 
# command line shell, e.g.
# mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# And then execute this in a command line shell to start the server, e.g.
# net start MySQLXY
#
#
# Guildlines for editing this file
# ----------------------------------------------------------------------
#
# In this file, you can use all long options that the program supports.
# If you want to know the options a program supports, start the program
# with the "--help" option.
#
# More detailed information about the individual options can also be
# found in the manual.
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
#
#
# CLIENT SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by MySQL client applications.
# Note that only client applications shipped by MySQL are guaranteed
# to read this section. If you want your own MySQL client program to
# honor these values, you need to specify it as an option during the
# MySQL client library initialization.
#
[client]
no-beep
 
# pipe
# socket=mysql
port=3306
 
[mysql]
 
default-character-set=utf8
 
# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this 
# file.
#
# server_type=3
[mysqld]
 
# The next three options are mutually exclusive to SERVER_PORT below.
# skip-networking
# enable-named-pipe
# The Pipe the MySQL Server will use
# socket=mysql
 
# The TCP/IP Port the MySQL Server will listen on
port=3306
 
# Path to installation directory. All paths are usually resolved relative to this.
# basedir="D:/Servers/MySQL/MySQL Server 5.6/"
 
# Path to the database root
datadir="D:/Servers/MySQL/data/data\"
 
# The default character set that will be used when a new schema or table is
# created and no character set is defined
character-set-server=utf8
 
# The default storage engine that will be used when create new tables when
default-storage-engine=INNODB
 
# Set the SQL mode to strict
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
 
# Enable Windows Authentication
# plugin-load=authentication_windows.dll
 
# General and Slow logging.
log-output=FILE
general-log=0
general_log_file="WIN-VA12QIAC4DT.log"
slow-query-log=1
slow_query_log_file="D:/Servers/logs/mysql/slow_query.log"
long_query_time=10
 
# Binary Logging.
# log-bin
 
# Error Logging.
log-error="D:/Servers/logs/mysql/error.log"
 
# The maximum amount of concurrent sessions the MySQL server will
# allow. One of these connections will be reserved for a user with
# SUPER privileges to allow the administrator to login even if the
# connection limit has been reached.
max_connections=151
 
# Query cache is used to cache SELECT results and later return them
# without actual executing the same query once again. Having the query
# cache enabled may result in significant speed improvements, if your
# have a lot of identical queries and rarely changing tables. See the
# "Qcache_lowmem_prunes" status variable to check if the current value
# is high enough for your load.
# Note: In case your tables change very often or if your queries are
# textually different every time, the query cache may result in a
# slowdown instead of a performance improvement.
query_cache_size=0
 
# The number of open tables for all threads. Increasing this value
# increases the number of file descriptors that mysqld requires.
# Therefore you have to make sure to set the amount of open files
# allowed to at least 4096 in the variable "open-files-limit" in
# section [mysqld_safe]
table_open_cache=2000
 
# Maximum size for internal (in-memory) temporary tables. If a table
# grows larger than this value, it is automatically converted to disk
# based table This limitation is for a single table. There can be many
# of them.
tmp_table_size=25M
 
# How many threads we should keep in a cache for reuse. When a client
# disconnects, the client's threads are put in the cache if there aren't
# more than thread_cache_size threads from before.  This greatly reduces
# the amount of thread creations needed if you have a lot of new
# connections. (Normally this doesn't give a notable performance
# improvement if you have a good thread implementation.)
thread_cache_size=10
 
#*** MyISAM Specific options
# The maximum size of the temporary file MySQL is allowed to use while
# recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE.
# If the file-size would be bigger than this, the index will be created
# through the key cache (which is slower).
myisam_max_sort_file_size=100G
 
# If the temporary file used for fast index creation would be bigger
# than using the key cache by the amount specified here, then prefer the
# key cache method.  This is mainly used to force long character keys in
# large tables to use the slower key cache method to create the index.
myisam_sort_buffer_size=41M
 
# Size of the Key Buffer, used to cache index blocks for MyISAM tables.
# Do not set it larger than 30% of your available memory, as some memory
# is also required by the OS to cache rows. Even if you're not using
# MyISAM tables, you should still set it to 8-64M as it will also be
# used for internal temporary disk tables.
key_buffer_size=8M
 
# Size of the buffer used for doing full table scans of MyISAM tables.
# Allocated per thread, if a full scan is needed.
read_buffer_size=64K
read_rnd_buffer_size=256K
 
# This buffer is allocated when MySQL needs to rebuild the index in
# REPAIR, OPTIMZE, ALTER table statements as well as in LOAD DATA INFILE
# into an empty table. It is allocated per thread so be careful with
# large settings.
sort_buffer_size=256K
 
#*** INNODB Specific options ***
# innodb_data_home_dir=0.0
 
# Use this option if you have a MySQL server with InnoDB support enabled
# but you do not plan to use it. This will save memory and disk space
# and speed up some things.
# skip-innodb
 
# Additional memory pool that is used by InnoDB to store metadata
# information.  If InnoDB requires more memory for this purpose it will
# start to allocate it from the OS.  As this is fast enough on most
# recent operating systems, you normally do not need to change this
# value. SHOW INNODB STATUS will display the current amount used.
innodb_additional_mem_pool_size=4M
 
# If set to 1, InnoDB will flush (fsync) the transaction logs to the
# disk at each commit, which offers full ACID behavior. If you are
# willing to compromise this safety, and you are running small
# transactions, you may set this to 0 or 2 to reduce disk I/O to the
# logs. Value 0 means that the log is only written to the log file and
# the log file flushed to disk approximately once per second. Value 2
# means the log is written to the log file at each commit, but the log
# file is only flushed to disk approximately once per second.
innodb_flush_log_at_trx_commit=1
 
# The size of the buffer InnoDB uses for buffering log data. As soon as
# it is full, InnoDB will have to flush it to disk. As it is flushed
# once per second anyway, it does not make sense to have it very large
# (even with long transactions).
innodb_log_buffer_size=2M
 
# InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and
# row data. The bigger you set this the less disk I/O is needed to
# access data in tables. On a dedicated database server you may set this
# parameter up to 80% of the machine physical memory size. Do not set it
# too large, though, because competition of the physical memory may
# cause paging in the operating system.  Note that on 32bit systems you
# might be limited to 2-3.5G of user level memory per process, so do not
# set it too high.
innodb_buffer_pool_size=123M
 
# Size of each log file in a log group. You should set the combined size
# of log files to about 25%-100% of your buffer pool size to avoid
# unneeded buffer pool flush activity on log file overwrite. However,
# note that a larger logfile size will increase the time needed for the
# recovery process.
innodb_log_file_size=48M
 
# Number of threads allowed inside the InnoDB kernel. The optimal value
# depends highly on the application, hardware as well as the OS
# scheduler properties. A too high value may lead to thread thrashing.
innodb_thread_concurrency=9
 
# The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full.
innodb_autoextend_increment=64
 
# The number of regions that the InnoDB buffer pool is divided into.
# For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency,
# by reducing contention as different threads read and write to cached pages.
innodb_buffer_pool_instances=8
 
# Determines the number of threads that can enter InnoDB concurrently.
innodb_concurrency_tickets=5000
 
# Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before
# it can be moved to the new sublist.
innodb_old_blocks_time=1000
 
# It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10.
innodb_open_files=300
 
# When this variable is enabled, InnoDB updates statistics during metadata statements.
innodb_stats_on_metadata=0
 
# When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table
# in a separate .ibd file, rather than in the system tablespace.
innodb_file_per_table=1
 
# Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none.
innodb_checksum_algorithm=0
 
# The number of outstanding connection requests MySQL can have.
# This option is useful when the main MySQL thread gets many connection requests in a very short time.
# It then takes some time (although very little) for the main thread to check the connection and start a new thread.
# The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily
# stops answering new requests.
# You need to increase this only if you expect a large number of connections in a short period of time.
back_log=80
 
# If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and
# synchronize unflushed data to disk.
# This option is best used only on systems with minimal resources.
flush_time=0
 
# The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use
# indexes and thus perform full table scans.
join_buffer_size=256K
 
# The maximum size of one packet or any generated or intermediate string, or any parameter sent by the
# mysql_stmt_send_long_data() C API function.
max_allowed_packet=4M
 
# If more than this many successive connection requests from a host are interrupted without a successful connection,
# the server blocks that host from performing further connections.
max_connect_errors=100
 
# Changes the number of file descriptors available to mysqld.
# You should try increasing the value of this option if mysqld gives you the error "Too many open files".
open_files_limit=4161
 
# Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND.
query_cache_type=0
 
# If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the
# sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization
# or improved indexing.
sort_buffer_size=256K
 
# The number of table definitions (from .frm files) that can be stored in the definition cache.
# If you use a large number of tables, you can create a large table definition cache to speed up opening of tables.
# The table definition cache takes less space and does not use file descriptors, unlike the normal table cache.
# The minimum and default values are both 400.
table_definition_cache=1400
 
# Specify the maximum size of a row-based binary log event, in bytes.
# Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256.
binlog_row_event_max_size=8K
 
# If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk.
# (using fdatasync()) after every sync_master_info events.
sync_master_info=10000
 
# If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk.
# (using fdatasync()) after every sync_relay_log writes to the relay log.
sync_relay_log=10000
 
# If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk.
# (using fdatasync()) after every sync_relay_log_info transactions.
sync_relay_log_info=10000
 

stray
join:2000-01-16
Warren, NJ

stray to cowboyro

Member

to cowboyro
The latest MySQL defaults to enabling Innodb, unlike earlier versions. Uncomment the # skip-innodb and your memory footprint will go way down.

You also might want to give MariaDB a try. API compatible with MySQL and considerably faster.

ekiM5
Oh Well
join:2001-01-06
/usr/home

ekiM5 to cowboyro

Member

to cowboyro
It needs tuning then...
How much ram on the box (and can it built as a db machine or is it doing other stuff too)?

I need to see the output from the following SQL query:

 select TABLE_SCHEMA "Data Base", TABLE_NAME, ENGINE, TABLE_ROWS, DATA_LENGTH, INDEX_LENGTH, ( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB", index_length/( data_length + index_length )* 10 "% indexes" from information_schema.tables;
 

cowboyro
Premium Member
join:2000-10-11
CT

cowboyro

Premium Member

There is some info I can not put here. I assume you want the db size and indexes... ABout 2.5GB of data and 1GB of indexes.
cowboyro

cowboyro to stray

Premium Member

to stray
said by stray:

The latest MySQL defaults to enabling Innodb, unlike earlier versions. Uncomment the # skip-innodb and your memory footprint will go way down.

You also might want to give MariaDB a try. API compatible with MySQL and considerably faster.

Some tables are using innodb so can't do that.
If I go another route I have no reason to not use MSSQL which is installed anyway.

ekiM5
Oh Well
join:2001-01-06
/usr/home

ekiM5 to cowboyro

Member

to cowboyro
Yes and...
1. the mix of table engines.
How much of the 2.5GB above is MyISAM vs Innodb?
- Numbers need to allocate ram per engine
2. is there 4GB of ram I can dedicate to mysql?

Im only interested in the real data database (not mysql or information schema etc)

An option might be to run the query with the table name removed from the SELECT to remove/exclude the nature of the content. Also if you only have 1 database of real data (yourdb) you can add a WHERE to the query.
ekiM5

ekiM5 to cowboyro

Member

to cowboyro
A quick additional comment...

Without knowing details, based upon your posts - these values are too low:

........................
key_buffer_size=8M

innodb_buffer_pool_size=123M
........................

The config file gives good explanations of their usage and config.

I asked for the details to better ballpark a recommendation.
I'm gonna venture that:

innodb_buffer_pool_size=2G
and key_buffer_size=500M

(and a service restart)
will take the lead weights from the server...

cowboyro
Premium Member
join:2000-10-11
CT

2 edits

cowboyro to ekiM5

Premium Member

to ekiM5
Virtually all MyIsam
Tried removing innodb but throws error although I can't see any table running innodb
2014-02-03 23:55:29 3388 [Note] Plugin 'FEDERATED' is disabled.
2014-02-03 23:55:29 3388 [Note] Plugin 'InnoDB' is disabled.
2014-02-03 23:55:29 3388 [ERROR] Unknown/unsupported storage engine: INNODB
2014-02-03 23:55:29 3388 [ERROR] Aborting
 

select
case when TABLE_SCHEMA not in ('information_schema','mysql','performance_schema') then 'OBFUSCATED_DB_NAME' else TABLE_SCHEMA end  "Data Base"
#,TABLE_NAME
,ENGINE, sum(TABLE_ROWS)
, sum(DATA_LENGTH), sum(INDEX_LENGTH)
, sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB"
,sum(index_length/( data_length + index_length ))* 10 "% indexes" from information_schema.tables
where engine is not null
group by table_schema,engine
 

'OBFUSCATED_DB_NAME', 'MEMORY', 0, 0, 0, 0.00000000, 
'OBFUSCATED_DB_NAME', 'MyISAM', 384710, 13666396, 5814272, 18.57821274, 37.7780
'OBFUSCATED_DB_NAME', 'MyISAM', 12699687, 194654753, 468421632, 632.35891819, 12.0580
'information_schema', 'MEMORY', , 0, 0, 0.00000000, 
'information_schema', 'MyISAM', , 0, 10240, 0.00976563, 100.0000
'OBFUSCATED_DB_NAME', 'MyISAM', 36092890, 1008165082, 719504384, 1647.63399696, 23.8540
'OBFUSCATED_DB_NAME', 'MyISAM', 5469, 267948, 124928, 0.37467575, 12.3060
'OBFUSCATED_DB_NAME', 'MyISAM', 49075, 2203699, 1282048, 3.32426739, 125.0820
'mysql', 'CSV', 4, 0, 0, 0.00000000, 
'mysql', 'MyISAM', 2353, 789468, 116736, 0.86422348, 214.3360
'performance_schema', 'PERFORMANCE_SCHEMA', 74026, 0, 0, 0.00000000, 
'OBFUSCATED_DB_NAME', 'MEMORY', 66, 61464, 61456, 0.11722565, 5.0000
'OBFUSCATED_DB_NAME', 'MyISAM', 746535, 63348018, 11645952, 71.51982307, 21.3770
'OBFUSCATED_DB_NAME', 'MyISAM', 1318715, 49273015, 23905280, 69.78826046, 24.5820
'OBFUSCATED_DB_NAME', 'MyISAM', 5302, 374048, 66560, 0.42019653, 5.3640
 

As you can see nothing crazy going on. Also right now only ONE client running a couple of storedprocs every few minutes.
I don't want to give Mysql too much memory... right now it's an annoyance with 420M and I just want to give it another chance before switching for good to MSSQL since the Express edition does all MySQL does and then some.

stray
join:2000-01-16
Warren, NJ

stray to cowboyro

Member

to cowboyro
If you are using MySql 5.5 or above,

ignore-builtin-innodb

and

default-storage-engine = myisam

under

[mysqld]
in /etc/my.cnf

cowboyro
Premium Member
join:2000-10-11
CT

cowboyro

Premium Member

said by stray:

If you are using MySql 5.5 or above,

ignore-builtin-innodb

and

428.9M with it, 428.9M without it.
cowboyro

cowboyro

Premium Member

Found others were complaining about high memory usage with 5.6.10+
One line of config cut the usage to 78M
table_definition_cache=400 ---- changed from 1400
No difference in performance (MSSQL is still a bit faster) but meh...

ekiM5
Oh Well
join:2001-01-06
/usr/home

ekiM5 to cowboyro

Member

to cowboyro
If your user data is all MyISAM then "key_buffer_size" is important.
What value is it set to?

The innodb value can be the default or perhaps less.

cowboyro
Premium Member
join:2000-10-11
CT

cowboyro

Premium Member

said by ekiM5:

If your user data is all MyISAM then "key_buffer_size" is important.
What value is it set to?

key_buffer_size=8M
I should be good now. Seems odd to set defaults that are gobbling memory.

ekiM5
Oh Well
join:2001-01-06
/usr/home

ekiM5

Member

You will see significant performance improvement if you kick that value up.
That low value is forcing the machine to do lots of i/o since you are not letting it cache data (and you are using MyISAM).

The stock my.cnf values are recommendations/starting points.
They ALWAYS require adjustment.
Not using InodDB goes against the grain of the new versions.
Mysql is meant to be enterprise class (like MS SQL Ent, Oracle etc). Your usage is counter to that design and requires adjustments to be made - like the setting you mentioned prior.

The high-end DBs prefer to run on a dedicated machine, thus all the tuning knobs and levers. You are driving your 6 figure sports car in commute traffic. You can but allowances/changes are required.

Set your key_buffer to at least 100M (more would be better) and performance will increase.