How would you use your ext4 formatted thumb drive via OTG on Android?

Is there a workaround to use an ext4 formatted thumb drives on Android phones? Right now when I plug it (thumb drive has micro usb end and phone supports thumb drives) Android says it doesn't recognize the file system. Android formats the micro SD cards to exFat by default so I guess that's all it supports for thumb drives too? But I would prefer to have the same (ext4) file system across all my PCs and thumb drives. Any non-painful way to achieve that?