Recursive grep matching only specific line number

How can I can I find files that contain a specific pattern **on a specific line number** ? Let's assume I have a directory with a bunch of text files containing 3 lines, such as:

Title A
Category X
Description Y

How can I grep / filter every files that have Category X on line 2? How can I find files that have Title A as line 1? I looked at the grep man page, ripgrep and alternative but not sure you can limit the search of a pattern to specific line numbers.