Sample Header Ad - 728x90

Is there a limit to how many temporary tables MySQL can handle?

0 votes
0 answers
30 views
I manage a web service. The service has multiple endpoints, and most of these endpoints simply return some data representing an entity (such as a track, genre or composer). Each endpoint usually supports sorting and filtering through a number of parameters. Let's assume an endpoint which returns a list of tracks. A track can be associated with one or more genres and one or more composers. The endpoint will first issue a query such as : 1. select {fields} from track t where t.album_id = {album_id} order by t.number limit {offset}, {limit} The service also needs to load associated genres and composers. For this, the applications extracts the track ids from the first query and executes : 2. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id where gt.track_id in {track_ids} 3. select {fields} from composer_track ct inner join composer c on ct.composer_id = c.id where ct.track_id in {track_ids} This has worked well. The initial query takes care of filtering, sorting and range - and the subsequent queries loads additional data. However, I'm considering using temporary tables in the following way : 1. insert into temptable (id) select id from track t where t.album_id = {album_id} order by t.number limit {offset}, {limit} 2. select {fields} from temptable tmp left join track t on tmp.id = t.id 3. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id inner join temptable tmp on tmp.id = gt.track_id 4. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id inner join temptable tmp on tmp.id = ct.track_id I do know that this approach will result in one additional query (the query which does the initial select and insert resulting ids into a temporary table). I don't expect that to give a lot of overhead. The underlying assumption is, however, that a join on a temporary table with index will be faster than doing where id in {track_ids} on e.g 2000 track ids. A typical query will load 100 tracks, but often the number of tracks to be loaded is somewhere between 500 and 5000. Question : The number of temporary tables in use may become large. Could this potentially become a problem?
Asked by sbrattla (193 rep)
Feb 25, 2025, 02:57 PM
Last activity: Feb 25, 2025, 03:02 PM