Why does the special character `?` need to be escaped in grep, but not `.` or `*`?

The only file in my current working directory is called test.txt and its content is simply:

This is a little test file.

* Executing

-sh
    grep -in * -e 'te.?t file'

gives **no match**. * But escaping ? works:

-sh
    grep -in * -e 'te.\?t file'

**gives match** **Crazy:** With the star *, things are the other way round! * Without escaping:

-sh
    grep -in * -e 'te.*t file'

**gives match** * and escaping * does not work:

-sh
    grep -in * -e 'te.\*t file'

**no match** Why are the meta-characters ? and * treated differently with respect to escaping?