Is there a limit to how many temporary tables MySQL can handle?
0
votes
0
answers
30
views
I manage a web service. The service has multiple endpoints, and most of these endpoints simply return some data representing an entity (such as a track, genre or composer). Each endpoint usually supports sorting and filtering through a number of parameters.
Let's assume an endpoint which returns a list of tracks. A track can be associated with one or more genres and one or more composers.
The endpoint will first issue a query such as :
1.
select {fields} from track t where t.album_id = {album_id} order by t.number limit {offset}, {limit}
The service also needs to load associated genres and composers. For this, the applications extracts the track ids from the first query and executes :
2. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id where gt.track_id in {track_ids}
3. select {fields} from composer_track ct inner join composer c on ct.composer_id = c.id where ct.track_id in {track_ids}
This has worked well. The initial query takes care of filtering, sorting and range - and the subsequent queries loads additional data.
However, I'm considering using temporary tables in the following way :
1. insert into temptable (id) select id from track t where t.album_id = {album_id} order by t.number limit {offset}, {limit}
2. select {fields} from temptable tmp left join track t on tmp.id = t.id
3. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id inner join temptable tmp on tmp.id = gt.track_id
4. select {fields} from genre_track gt inner join genre g on gt.genre_id = g.id inner join temptable tmp on tmp.id = ct.track_id
I do know that this approach will result in one additional query (the query which does the initial select and insert resulting ids into a temporary table). I don't expect that to give a lot of overhead.
The underlying assumption is, however, that a join on a temporary table with index will be faster than doing where id in {track_ids}
on e.g 2000 track ids.
A typical query will load 100 tracks, but often the number of tracks to be loaded is somewhere between 500 and 5000.
Question :
The number of temporary tables in use may become large. Could this potentially become a problem?
Asked by sbrattla
(193 rep)
Feb 25, 2025, 02:57 PM
Last activity: Feb 25, 2025, 03:02 PM
Last activity: Feb 25, 2025, 03:02 PM