hive - Can not create the managed table The associated location already exists

I'm trying to create a managed Hive table using Spark SQL with the following query: DROP TABLE IF EXISTS db.TMP_ARR; CREATE TABLE db.TMP_ARR AS SELECT ID, - more fields.. FROM some_source_table INT; However, the job fails with the following error: org.apache.spark.sql.AnalysisException: Can not create the managed table('db.tmp_arr'). The associated location ('hdfs://coreCluster/warehouse/tablespace/managed/hive/db.db/tmp_arr') already exists **What I understand:** I'm trying to create a managed table. Spark expects that the target location in HDFS does not already exist when creating a managed table. Apparently, that folder already exists, possibly due to a previous failed run or manual intervention. **My questions:** Why does Spark throw this error even though I used DROP TABLE IF EXISTS before CREATE TABLE? What's the correct way to ensure a managed table can be created without this conflict? Should I manually delete the path in HDFS before creating the table, or is there a safer/better approach? **Environment:** Spark version: 3.3.2 Hive metastore: enabled Storage: HDFS *1. It's important that the table is managed (not external), and that we don’t manually assign a LOCATION. 2. many similar jobs are running concurrently (creating/dropping managed tables in the same Hive schema).*