Sample Header Ad - 728x90

Is my queue table implementation race condition safe?

7 votes
4 answers
1108 views
Hello people smarter than me! I've created a sort-of-a-queue table system, but it seems too simple to be safe from race conditions. Am I missing something or is the following race condition safe? The Schema - I have a table, let's call it ProductQueue: CREATE TABLE dbo.ProductQueue ( SerialId BIGINT PRIMARY KEY, QueuedDateTime DATETIME NOT NULL -- Only using this for reference, no functionality is tied to it ); I have procedure for adding to the queue called AddToProductQueue: CREATE PROCEDURE dbo.AddToProductQueue (@SerialId BIGINT) AS BEGIN INSERT INTO dbo.ProductQueue (SerialId, QueuedDateTime) OUTPUT Inserted.SerialId SELECT @SerialId, GETDATE(); END I also have a procedure for removing from the queue called RemoveFromProductQueue: CREATE PROCEDURE dbo.RemoveFromProductQueue (@SerialId BIGINT) AS BEGIN DELETE FROM dbo.ProductQueue OUTPUT Deleted.SerialId WHERE SerialId = @SerialId; END Note, SerialId is globally unique for a Product in the source database / system. I.e. no two instances of a Product can ever have the same SerialId. That's the extent of it on the database side. The Workflow - - I have an application process that runs hourly. - That process gets a variable list of SerialIds from the source system. - It iteratively calls the AddToProductQueue procedure on each SerialId in its list. - If the procedure tries to insert a SerialId that exists in the ProductQueue table already, it throws a primary key violation error, and the application process catches that error and skips that SerialId. - Otherwise, the procedure successfully adds that SerialId to the ProductQueue table and returns it back to the application process. - The application process then adds that successfully queued SerialId to a separate list. - After the application process finishes iterating its list of all candidate SerialIds to enqueue, it then iterates its new list of successfully queued SerialIds and does external work on them, in a **separate thread** per SerialId. (This work is all unrelated to the database.) - Finally, as each thread finishes its external work, the last step in that asynchronous thread is to remove that SerialId from the ProductQueue table by calling the RemoveFromProductQueue procedure. (Note that a new database context object is instantiated and a new connection is created for each asynchronous call to this procedure, so that it is thread-safe on the application side.) Additional Information - - There aren't any indexes on the ProductQueue table, and it'll never have more than 1,000 rows in it at one time. (Actually, most times it'll literally only have a couple of rows.) - The same SerialId can become a candidate again to be re-added to the queue table on a future execution of the application process. - There are no safe guards from preventing a second instance of the application process from concurrently running, either by accident or if the first instance took more than 1 hour to run, etc. (This is the concurrent part I'm most concerned about.) - The transaction isolation level of the database (and connection being made) where the queue table and procedures live is the default isolation level of Read Committed. Potential Problems - - The running instance of the application process crashes in an unhandled way, leaving SerialIds stuck in the queue table. This is acceptable for the business needs, and we plan to have exception reports to help us manually remediate this case. - The application process gets executed multiple times concurrently and grabs some of the same SerialIds between instances in their initial source lists. I can't think of any negative ramifications of this case yet, since the enqueuing procedure is atomic, and the actual list of SerialIds that the application process will work on should be self-contained due to that atomic enqueuing procedure. We don't care which instance of the application process actually processes each SerialId as long as the same SerialId isn't processed concurrently by both process instances.
Asked by J.D. (40893 rep)
Mar 13, 2024, 09:46 PM
Last activity: Mar 16, 2024, 03:03 PM