This is a discussion on why filesort? within the MySQL Database forums, part of the Database Forums category; Hi, I have a simple query like this: SELECT a FROM data ORDER BY a limit 10; Now when I ...
|
|||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
|
|||
|
Hi,
I have a simple query like this: SELECT a FROM data ORDER BY a limit 10; Now when I let mysql 5.0.24a "explain" this query, it gives: mysql> explain select abs_pressure from data order by abs_pressure limit 10; +----+-------------+-------+------+---------------+------+---------+------+-------+----------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+------+-------+----------------+ | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | NULL | 53912 | Using filesort | +----+-------------+-------+------+---------------+------+---------+------+-------+----------------+ 1 row in set (0.00 sec) So it seems it always does a filesort, even for simple queries like this. More details: - the system has 2GB of ram which 1.1GB free(!) - the table has 54k rows - the table uses 6.5MB diskspace - the table is fairly simple: mysql> describe data; +----------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+--------------+------+-----+---------+-------+ | timestamp | datetime | NO | PRI | NULL | | | temp_in | decimal(4,1) | NO | | 0.0 | | | temp_out | decimal(4,1) | NO | | 0.0 | | | dewpoint | decimal(4,1) | NO | | 0.0 | | | rel_hum_in | tinyint(3) | NO | | 0 | | | rel_hum_out | tinyint(3) | NO | | 0 | | | windspeed | decimal(4,1) | NO | | 0.0 | | | wind_direction | char(3) | NO | | NULL | | | wind_angle | decimal(4,1) | NO | | 0.0 | | | wind_chill | decimal(4,1) | NO | | 0.0 | | | rain_1h | decimal(6,1) | NO | | 0.0 | | | rain_24h | decimal(6,1) | NO | | 0.0 | | | rain_1w | decimal(7,1) | NO | | 0.0 | | | rain_1m | decimal(7,1) | NO | | 0.0 | | | rain_total | decimal(8,1) | NO | | 0.0 | | | rel_pressure | decimal(8,1) | NO | | 0.0 | | | abs_pressure | decimal(8,1) | NO | | 0.0 | | | tendency | varchar(10) | NO | | NULL | | | forecast | varchar(7) | NO | | NULL | | +----------------+--------------+------+-----+---------+-------+ 19 rows in set (0.01 sec) - sort buffer of 8MB: mysql> show variables like '%sort%'; +---------------------------+------------+ | Variable_name | Value | +---------------------------+------------+ | max_length_for_sort_data | 1024 | | max_sort_length | 1024 | | myisam_max_sort_file_size | 2147483647 | | myisam_sort_buffer_size | 8388608 | | sort_buffer_size | 2097144 | +---------------------------+------------+ 5 rows in set (0.00 sec) - database is MyISAM: ) ENGINE=MyISAM DEFAULT CHARSET=latin1; -- -------------------------------------------------------------------- Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com |
|
|||
|
On 22 Feb, 13:34, "Folkert van Heusden" <folk...@vanheusden.com>
wrote: > Hi, > > I have a simple query like this: > SELECT a FROM data ORDER BY a limit 10; > Now when I let mysql 5.0.24a "explain" this query, it gives: > > mysql> explain select abs_pressure from data order by abs_pressure limit 10; > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref | > rows | Extra | > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | NULL| > 53912 | Using filesort | > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > 1 row in set (0.00 sec) > > So it seems it always does a filesort, even for simple queries like this. > > More details: > - the system has 2GB of ram which 1.1GB free(!) > - the table has 54k rows > - the table uses 6.5MB diskspace > - the table is fairly simple: > mysql> describe data; > +----------------+--------------+------+-----+---------+-------+ > | Field | Type | Null | Key | Default | Extra | > +----------------+--------------+------+-----+---------+-------+ > | timestamp | datetime | NO | PRI | NULL | | > | temp_in | decimal(4,1) | NO | | 0.0 | | > | temp_out | decimal(4,1) | NO | | 0.0 | | > | dewpoint | decimal(4,1) | NO | | 0.0 | | > | rel_hum_in | tinyint(3) | NO | | 0 | | > | rel_hum_out | tinyint(3) | NO | | 0 | | > | windspeed | decimal(4,1) | NO | | 0.0 | | > | wind_direction | char(3) | NO | | NULL | | > | wind_angle | decimal(4,1) | NO | | 0.0 | | > | wind_chill | decimal(4,1) | NO | | 0.0 | | > | rain_1h | decimal(6,1) | NO | | 0.0 | | > | rain_24h | decimal(6,1) | NO | | 0.0 | | > | rain_1w | decimal(7,1) | NO | | 0.0 | | > | rain_1m | decimal(7,1) | NO | | 0.0 | | > | rain_total | decimal(8,1) | NO | | 0.0 | | > | rel_pressure | decimal(8,1) | NO | | 0.0 | | > | abs_pressure | decimal(8,1) | NO | | 0.0 | | > | tendency | varchar(10) | NO | | NULL | | > | forecast | varchar(7) | NO | | NULL | | > +----------------+--------------+------+-----+---------+-------+ > 19 rows in set (0.01 sec) > > - sort buffer of 8MB: > mysql> show variables like '%sort%'; > +---------------------------+------------+ > | Variable_name | Value | > +---------------------------+------------+ > | max_length_for_sort_data | 1024 | > | max_sort_length | 1024 | > | myisam_max_sort_file_size | 2147483647 | > | myisam_sort_buffer_size | 8388608 | > | sort_buffer_size | 2097144 | > +---------------------------+------------+ > 5 rows in set (0.00 sec) > > - database is MyISAM: > ) ENGINE=MyISAM DEFAULT CHARSET=latin1; > > -- > -------------------------------------------------------------------- > Phone: +31-6-41278122, PGP-key: 1F28D8AE,www.vanheusden.com Since `a` isn't a valid row in the table, I think MySQL is doing pretty good to give you any output at all! |
|
|||
|
Right.
You're kidding right? If not: that query was an example. To please you: SELECT temp_out FROM data ORDER BY temp_out limit 10; "Captain Paralytic" <paul_lautman@yahoo.com> wrote in message news:1172153890.007642.154900@a75g2000cwd.googlegr oups.com... On 22 Feb, 13:34, "Folkert van Heusden" <folk...@vanheusden.com> wrote: > Hi, > > I have a simple query like this: > SELECT a FROM data ORDER BY a limit 10; > Now when I let mysql 5.0.24a "explain" this query, it gives: > > mysql> explain select abs_pressure from data order by abs_pressure limit > 10; > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > | id | select_type | table | type | possible_keys | key | key_len | ref > | > rows | Extra | > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | NULL > | > 53912 | Using filesort | > +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ > 1 row in set (0.00 sec) > > So it seems it always does a filesort, even for simple queries like this. > > More details: > - the system has 2GB of ram which 1.1GB free(!) > - the table has 54k rows > - the table uses 6.5MB diskspace > - the table is fairly simple: > mysql> describe data; > +----------------+--------------+------+-----+---------+-------+ > | Field | Type | Null | Key | Default | Extra | > +----------------+--------------+------+-----+---------+-------+ > | timestamp | datetime | NO | PRI | NULL | | > | temp_in | decimal(4,1) | NO | | 0.0 | | > | temp_out | decimal(4,1) | NO | | 0.0 | | > | dewpoint | decimal(4,1) | NO | | 0.0 | | > | rel_hum_in | tinyint(3) | NO | | 0 | | > | rel_hum_out | tinyint(3) | NO | | 0 | | > | windspeed | decimal(4,1) | NO | | 0.0 | | > | wind_direction | char(3) | NO | | NULL | | > | wind_angle | decimal(4,1) | NO | | 0.0 | | > | wind_chill | decimal(4,1) | NO | | 0.0 | | > | rain_1h | decimal(6,1) | NO | | 0.0 | | > | rain_24h | decimal(6,1) | NO | | 0.0 | | > | rain_1w | decimal(7,1) | NO | | 0.0 | | > | rain_1m | decimal(7,1) | NO | | 0.0 | | > | rain_total | decimal(8,1) | NO | | 0.0 | | > | rel_pressure | decimal(8,1) | NO | | 0.0 | | > | abs_pressure | decimal(8,1) | NO | | 0.0 | | > | tendency | varchar(10) | NO | | NULL | | > | forecast | varchar(7) | NO | | NULL | | > +----------------+--------------+------+-----+---------+-------+ > 19 rows in set (0.01 sec) > > - sort buffer of 8MB: > mysql> show variables like '%sort%'; > +---------------------------+------------+ > | Variable_name | Value | > +---------------------------+------------+ > | max_length_for_sort_data | 1024 | > | max_sort_length | 1024 | > | myisam_max_sort_file_size | 2147483647 | > | myisam_sort_buffer_size | 8388608 | > | sort_buffer_size | 2097144 | > +---------------------------+------------+ > 5 rows in set (0.00 sec) > > - database is MyISAM: > ) ENGINE=MyISAM DEFAULT CHARSET=latin1; > > -- > -------------------------------------------------------------------- > Phone: +31-6-41278122, PGP-key: 1F28D8AE,www.vanheusden.com Since `a` isn't a valid row in the table, I think MySQL is doing pretty good to give you any output at all! |
|
|||
|
Folkert van Heusden wrote:
> Right. > You're kidding right? > If not: that query was an example. > To please you: > SELECT temp_out FROM data ORDER BY temp_out limit 10; > > "Captain Paralytic" <paul_lautman@yahoo.com> wrote in message > news:1172153890.007642.154900@a75g2000cwd.googlegr oups.com... > On 22 Feb, 13:34, "Folkert van Heusden" <folk...@vanheusden.com> > wrote: >> Hi, >> >> I have a simple query like this: >> SELECT a FROM data ORDER BY a limit 10; >> Now when I let mysql 5.0.24a "explain" this query, it gives: >> >> mysql> explain select abs_pressure from data order by abs_pressure limit >> 10; >> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >> | id | select_type | table | type | possible_keys | key | key_len | ref >> | >> rows | Extra | >> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >> | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | NULL >> | >> 53912 | Using filesort | >> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >> 1 row in set (0.00 sec) >> >> So it seems it always does a filesort, even for simple queries like this. >> >> More details: >> - the system has 2GB of ram which 1.1GB free(!) >> - the table has 54k rows >> - the table uses 6.5MB diskspace >> - the table is fairly simple: >> mysql> describe data; >> +----------------+--------------+------+-----+---------+-------+ >> | Field | Type | Null | Key | Default | Extra | >> +----------------+--------------+------+-----+---------+-------+ >> | timestamp | datetime | NO | PRI | NULL | | >> | temp_in | decimal(4,1) | NO | | 0.0 | | >> | temp_out | decimal(4,1) | NO | | 0.0 | | >> | dewpoint | decimal(4,1) | NO | | 0.0 | | >> | rel_hum_in | tinyint(3) | NO | | 0 | | >> | rel_hum_out | tinyint(3) | NO | | 0 | | >> | windspeed | decimal(4,1) | NO | | 0.0 | | >> | wind_direction | char(3) | NO | | NULL | | >> | wind_angle | decimal(4,1) | NO | | 0.0 | | >> | wind_chill | decimal(4,1) | NO | | 0.0 | | >> | rain_1h | decimal(6,1) | NO | | 0.0 | | >> | rain_24h | decimal(6,1) | NO | | 0.0 | | >> | rain_1w | decimal(7,1) | NO | | 0.0 | | >> | rain_1m | decimal(7,1) | NO | | 0.0 | | >> | rain_total | decimal(8,1) | NO | | 0.0 | | >> | rel_pressure | decimal(8,1) | NO | | 0.0 | | >> | abs_pressure | decimal(8,1) | NO | | 0.0 | | >> | tendency | varchar(10) | NO | | NULL | | >> | forecast | varchar(7) | NO | | NULL | | >> +----------------+--------------+------+-----+---------+-------+ >> 19 rows in set (0.01 sec) >> >> - sort buffer of 8MB: >> mysql> show variables like '%sort%'; >> +---------------------------+------------+ >> | Variable_name | Value | >> +---------------------------+------------+ >> | max_length_for_sort_data | 1024 | >> | max_sort_length | 1024 | >> | myisam_max_sort_file_size | 2147483647 | >> | myisam_sort_buffer_size | 8388608 | >> | sort_buffer_size | 2097144 | >> +---------------------------+------------+ >> 5 rows in set (0.00 sec) >> >> - database is MyISAM: >> ) ENGINE=MyISAM DEFAULT CHARSET=latin1; >> >> -- >> -------------------------------------------------------------------- >> Phone: +31-6-41278122, PGP-key: 1F28D8AE,www.vanheusden.com > Since `a` isn't a valid row in the table, I think MySQL is doing > pretty good to give you any output at all! > > No, he's not kidding. The actual column name makes a BIG difference. First question - do you have an index on temp_out? It looks like you don't. -- ================== Remove the "x" from my email address Jerry Stuckle JDS Computer Training Corp. jstucklex@attglobal.net ================== |
|
|||
|
I understood it and found it too obvious that's why I asked.
Ok no index on temp_out because a) the table gets updated way more then read (at least 250 times more often updated) and furthermore I would have to have an index for each column. Also I was wondering why it goes to file as there's not so much data in the table, it should easily fit in memory when sorting. > No, he's not kidding. The actual column name makes a BIG difference. > > First question - do you have an index on temp_out? It looks like you > don't. > "Jerry Stuckle" <jstucklex@attglobal.net> wrote in message news:LMGdnVBnUK47S0DYnZ2dnUVZ_orinZ2d@comcast.com. .. > Folkert van Heusden wrote: >> Right. >> You're kidding right? >> If not: that query was an example. >> To please you: >> SELECT temp_out FROM data ORDER BY temp_out limit 10; >> >> "Captain Paralytic" <paul_lautman@yahoo.com> wrote in message >> news:1172153890.007642.154900@a75g2000cwd.googlegr oups.com... >> On 22 Feb, 13:34, "Folkert van Heusden" <folk...@vanheusden.com> >> wrote: >>> Hi, >>> >>> I have a simple query like this: >>> SELECT a FROM data ORDER BY a limit 10; >>> Now when I let mysql 5.0.24a "explain" this query, it gives: >>> >>> mysql> explain select abs_pressure from data order by abs_pressure limit >>> 10; >>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>> | id | select_type | table | type | possible_keys | key | key_len | ref >>> | >>> rows | Extra | >>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>> | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | >>> NULL | >>> 53912 | Using filesort | >>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>> 1 row in set (0.00 sec) >>> >>> So it seems it always does a filesort, even for simple queries like >>> this. >>> >>> More details: >>> - the system has 2GB of ram which 1.1GB free(!) >>> - the table has 54k rows >>> - the table uses 6.5MB diskspace >>> - the table is fairly simple: >>> mysql> describe data; >>> +----------------+--------------+------+-----+---------+-------+ >>> | Field | Type | Null | Key | Default | Extra | >>> +----------------+--------------+------+-----+---------+-------+ >>> | timestamp | datetime | NO | PRI | NULL | | >>> | temp_in | decimal(4,1) | NO | | 0.0 | | >>> | temp_out | decimal(4,1) | NO | | 0.0 | | >>> | dewpoint | decimal(4,1) | NO | | 0.0 | | >>> | rel_hum_in | tinyint(3) | NO | | 0 | | >>> | rel_hum_out | tinyint(3) | NO | | 0 | | >>> | windspeed | decimal(4,1) | NO | | 0.0 | | >>> | wind_direction | char(3) | NO | | NULL | | >>> | wind_angle | decimal(4,1) | NO | | 0.0 | | >>> | wind_chill | decimal(4,1) | NO | | 0.0 | | >>> | rain_1h | decimal(6,1) | NO | | 0.0 | | >>> | rain_24h | decimal(6,1) | NO | | 0.0 | | >>> | rain_1w | decimal(7,1) | NO | | 0.0 | | >>> | rain_1m | decimal(7,1) | NO | | 0.0 | | >>> | rain_total | decimal(8,1) | NO | | 0.0 | | >>> | rel_pressure | decimal(8,1) | NO | | 0.0 | | >>> | abs_pressure | decimal(8,1) | NO | | 0.0 | | >>> | tendency | varchar(10) | NO | | NULL | | >>> | forecast | varchar(7) | NO | | NULL | | >>> +----------------+--------------+------+-----+---------+-------+ >>> 19 rows in set (0.01 sec) >>> >>> - sort buffer of 8MB: >>> mysql> show variables like '%sort%'; >>> +---------------------------+------------+ >>> | Variable_name | Value | >>> +---------------------------+------------+ >>> | max_length_for_sort_data | 1024 | >>> | max_sort_length | 1024 | >>> | myisam_max_sort_file_size | 2147483647 | >>> | myisam_sort_buffer_size | 8388608 | >>> | sort_buffer_size | 2097144 | >>> +---------------------------+------------+ >>> 5 rows in set (0.00 sec) >>> >>> - database is MyISAM: >>> ) ENGINE=MyISAM DEFAULT CHARSET=latin1; >>> >>> -- >>> -------------------------------------------------------------------- >>> Phone: +31-6-41278122, PGP-key: 1F28D8AE,www.vanheusden.com >> Since `a` isn't a valid row in the table, I think MySQL is doing >> pretty good to give you any output at all! >> >> > > > -- > ================== > Remove the "x" from my email address > Jerry Stuckle > JDS Computer Training Corp. > jstucklex@attglobal.net > ================== |
|
|||
|
Folkert van Heusden wrote:
> "Jerry Stuckle" <jstucklex@attglobal.net> wrote in message > news:LMGdnVBnUK47S0DYnZ2dnUVZ_orinZ2d@comcast.com. .. >> No, he's not kidding. The actual column name makes a BIG difference. >> >> First question - do you have an index on temp_out? It looks like you >> don't. >> >> Folkert van Heusden wrote: >>> Right. >>> You're kidding right? >>> If not: that query was an example. >>> To please you: >>> SELECT temp_out FROM data ORDER BY temp_out limit 10; >>> >>> "Captain Paralytic" <paul_lautman@yahoo.com> wrote in message >>> news:1172153890.007642.154900@a75g2000cwd.googlegr oups.com... >>> On 22 Feb, 13:34, "Folkert van Heusden" <folk...@vanheusden.com> >>> wrote: >>>> Hi, >>>> >>>> I have a simple query like this: >>>> SELECT a FROM data ORDER BY a limit 10; >>>> Now when I let mysql 5.0.24a "explain" this query, it gives: >>>> >>>> mysql> explain select abs_pressure from data order by abs_pressure limit >>>> 10; >>>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>>> | id | select_type | table | type | possible_keys | key | key_len | ref >>>> | >>>> rows | Extra | >>>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>>> | 1 | SIMPLE | data | ALL | NULL | NULL | NULL | >>>> NULL | >>>> 53912 | Using filesort | >>>> +----+-------------+-------+------+---------------+------+---------+------+*-------+----------------+ >>>> 1 row in set (0.00 sec) >>>> >>>> So it seems it always does a filesort, even for simple queries like >>>> this. >>>> >>>> More details: >>>> - the system has 2GB of ram which 1.1GB free(!) >>>> - the table has 54k rows >>>> - the table uses 6.5MB diskspace >>>> - the table is fairly simple: >>>> mysql> describe data; >>>> +----------------+--------------+------+-----+---------+-------+ >>>> | Field | Type | Null | Key | Default | Extra | >>>> +----------------+--------------+------+-----+---------+-------+ >>>> | timestamp | datetime | NO | PRI | NULL | | >>>> | temp_in | decimal(4,1) | NO | | 0.0 | | >>>> | temp_out | decimal(4,1) | NO | | 0.0 | | >>>> | dewpoint | decimal(4,1) | NO | | 0.0 | | >>>> | rel_hum_in | tinyint(3) | NO | | 0 | | >>>> | rel_hum_out | tinyint(3) | NO | | 0 | | >>>> | windspeed | decimal(4,1) | NO | | 0.0 | | >>>> | wind_direction | char(3) | NO | | NULL | | >>>> | wind_angle | decimal(4,1) | NO | | 0.0 | | >>>> | wind_chill | decimal(4,1) | NO | | 0.0 | | >>>> | rain_1h | decimal(6,1) | NO | | 0.0 | | >>>> | rain_24h | decimal(6,1) | NO | | 0.0 | | >>>> | rain_1w | decimal(7,1) | NO | | 0.0 | | >>>> | rain_1m | decimal(7,1) | NO | | 0.0 | | >>>> | rain_total | decimal(8,1) | NO | | 0.0 | | >>>> | rel_pressure | decimal(8,1) | NO | | 0.0 | | >>>> | abs_pressure | decimal(8,1) | NO | | 0.0 | | >>>> | tendency | varchar(10) | NO | | NULL | | >>>> | forecast | varchar(7) | NO | | NULL | | >>>> +----------------+--------------+------+-----+---------+-------+ >>>> 19 rows in set (0.01 sec) >>>> >>>> - sort buffer of 8MB: >>>> mysql> show variables like '%sort%'; >>>> +---------------------------+------------+ >>>> | Variable_name | Value | >>>> +---------------------------+------------+ >>>> | max_length_for_sort_data | 1024 | >>>> | max_sort_length | 1024 | >>>> | myisam_max_sort_file_size | 2147483647 | >>>> | myisam_sort_buffer_size | 8388608 | >>>> | sort_buffer_size | 2097144 | >>>> +---------------------------+------------+ >>>> 5 rows in set (0.00 sec) >>>> >>>> - database is MyISAM: >>>> ) ENGINE=MyISAM DEFAULT CHARSET=latin1; >>>> >>>> -- >>>> -------------------------------------------------------------------- >>>> Phone: +31-6-41278122, PGP-key: 1F28D8AE,www.vanheusden.com >>> Since `a` isn't a valid row in the table, I think MySQL is doing >>> pretty good to give you any output at all! >>> >>> >> >> -- >> ================== >> Remove the "x" from my email address >> Jerry Stuckle >> JDS Computer Training Corp. >> jstucklex@attglobal.net >> ================== > > > I understood it and found it too obvious that's why I asked. > > Ok no index on temp_out because a) the table gets updated way more then read > (at least 250 times more often updated) and furthermore I would have to have > an index for each column. > Also I was wondering why it goes to file as there's not so much data in the > table, it should easily fit in memory when sorting. > (Top posting fixed) MySQL may or may not actually write to a temp file. The EXPLAIN wouldn't know for sure how much data will be sorted (it doesn't actually retrieve the data). The temp file(s) will be written if the amount of data exceeds sort_buffer_size in your mysql configuration. See http://dev.mysql.com/doc/internals/en/filesort.html for a description on how MySQL does a filesort. P.S. Please don't top post. Thanks. -- ================== Remove the "x" from my email address Jerry Stuckle JDS Computer Training Corp. jstucklex@attglobal.net ================== |